I am implementing an application which retrieves CSV data from COVID-19 info web.
I have made a parser which gets the cases per day of an specific place (Canary Island).
String url = "https://cnecovid.isciii.es/covid19/resources/datos_ccaas.csv";
String[] contents = HTTPFileDownloader.downloadFromURL(url).split("\n");
for(String i : contents) {
if(isCanaryIslandData(i)) {
String[] line = i.split(",");
String[] date = line[1].split("-"); // "YYYY-MM-DD"
int cases = Integer.parseInt(line[2]);
casesPerDay.add(cases);
}
}
Now I want to make a chart displaying the data. Something like this:
Currently I am storing the values in an ArrayList (just for testing). I know I will need to store the date and the number of cases, but I don't know which type of dataset should I use.
I want to make a line and a bar chart. I have managed to do it:
But as I have said, I want to find a way to display the dates as the x labels as shown in the example.
I tried with a CategoryDataset, but that prints every single x label so it won't be readable at all. I know XYSeries can do the trick as shown before, but I don't know how to insert a string as label instead of an integer.
Hope I explained it well.
I managed to do it using TimeSeriesCollection
TimeSeries series = new TimeSeries("Cases per Day");
String url = "https://cnecovid.isciii.es/covid19/resources/datos_ccaas.csv";
String[] contents = HTTPFileDownloader.downloadFromURL(url).split("\n");
for(String i : contents) {
if(isCanaryIslandData(i)) {
String[] line = i.split(",");
String[] date = line[1].split("-");
int year = Integer.parseInt(date[0]);
int month = Integer.parseInt(date[1]);
int day = Integer.parseInt(date[2]);
int cases = Integer.parseInt(line[2]);
series.add(new Day(day, month, year), cases);
}
}
Then on the chart class:
TimeSeriesCollection dataset = new TimeSeriesCollection();
dataset.addSeries(series);
JFreeChart chart = ChartFactory.createXYLineChart(
"Line Chart",
"x",
"y",
dataset,
PlotOrientation.VERTICAL,
true,
true,
false);
ChartPanel chartPanel = new ChartPanel(chart);
chartPanel.setMouseWheelEnabled(true);
chartPanel.setPreferredSize(new java.awt.Dimension(560,367));
XYPlot plot = (XYPlot) chart.getPlot();
DateAxis dateAxis = new DateAxis();
dateAxis.setDateFormatOverride(new SimpleDateFormat("dd-MM-yyyy"));
plot.setDomainAxis(dateAxis);
And output:
I don't know if it's the best solution, but it does the work.
I have a column with type Timestamp with the format yyyy-MM-dd HH:mm:ss in a dataframe.
The column is sorted by time where the earlier date is at the earlier row
When I ran this command
List<Row> timeRows = df.withColumn(ts, df.col(ts).cast("long")).select(ts).collectAsList();
I face a strange issue where the value of the later date is smaller than the earlier date. Example:
[670] : 1550967304 (2019-02-23 04:30:15)
[671] : 1420064100 (2019-02-24 08:15:04)
Is this the correct way to convert to Epoch or is there another way?
Try using unix_timestamp to convert the string date time to the timestamp. According to the document:
unix_timestamp(Column s, String p) Convert time string with given
pattern (see
[http://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html
]) to Unix time stamp (in seconds), return null if fail.
import org.apache.spark.functions._
val format = "yyyy-MM-dd HH:mm:ss"
df.withColumn("epoch_sec", unix_timestamp($"ts", format)).select("epoch_sec").collectAsList()
Also, see https://jaceklaskowski.gitbooks.io/mastering-spark-sql/spark-sql-functions-datetime.html
You should use the built in function unix_timestamp() in org.apache.spark.sql.functions
https://spark.apache.org/docs/1.6.0/api/java/org/apache/spark/sql/functions.html#unix_timestamp()
I think you are looking at using: unix_timestamp()
Which you can import from:
import static org.apache.spark.sql.functions.unix_timestamp;
And use like:
df = df.withColumn(
"epoch",
unix_timestamp(col("date")));
And here is a full example, where I tried to mimic your use-case:
package net.jgp.books.spark.ch12.lab990_others;
import static org.apache.spark.sql.functions.col;
import static org.apache.spark.sql.functions.from_unixtime;
import static org.apache.spark.sql.functions.unix_timestamp;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Row;
import org.apache.spark.sql.RowFactory;
import org.apache.spark.sql.SparkSession;
import org.apache.spark.sql.types.DataTypes;
import org.apache.spark.sql.types.StructField;
import org.apache.spark.sql.types.StructType;
/**
* Use of from_unixtime() and unix_timestamp().
*
* #author jgp
*/
public class EpochTimestampConversionApp {
/**
* main() is your entry point to the application.
*
* #param args
*/
public static void main(String[] args) {
EpochTimestampConversionApp app = new EpochTimestampConversionApp();
app.start();
}
/**
* The processing code.
*/
private void start() {
// Creates a session on a local master
SparkSession spark = SparkSession.builder()
.appName("expr()")
.master("local")
.getOrCreate();
StructType schema = DataTypes.createStructType(new StructField[] {
DataTypes.createStructField(
"event",
DataTypes.IntegerType,
false),
DataTypes.createStructField(
"original_ts",
DataTypes.StringType,
false) });
// Building a df with a sequence of chronological timestamps
List<Row> rows = new ArrayList<>();
long now = System.currentTimeMillis() / 1000;
for (int i = 0; i < 1000; i++) {
rows.add(RowFactory.create(i, String.valueOf(now)));
now += new Random().nextInt(3) + 1;
}
Dataset<Row> df = spark.createDataFrame(rows, schema);
df.show();
df.printSchema();
// Turning the timestamps to Timestamp datatype
df = df.withColumn(
"date",
from_unixtime(col("original_ts")).cast(DataTypes.TimestampType));
df.show();
df.printSchema();
// Turning back the timestamps to epoch
df = df.withColumn(
"epoch",
unix_timestamp(col("date")));
df.show();
df.printSchema();
// Collecting the result and printing out
List<Row> timeRows = df.collectAsList();
for (Row r : timeRows) {
System.out.printf("[%d] : %s (%s)\n",
r.getInt(0),
r.getAs("epoch"),
r.getAs("date"));
}
}
}
And the output should be:
...
[994] : 1551997326 (2019-03-07 14:22:06)
[995] : 1551997329 (2019-03-07 14:22:09)
[996] : 1551997330 (2019-03-07 14:22:10)
[997] : 1551997332 (2019-03-07 14:22:12)
[998] : 1551997333 (2019-03-07 14:22:13)
[999] : 1551997335 (2019-03-07 14:22:15)
Hopefully this helps.
Introduction
My goal is to input a set of products and choose an option for display such as:
System.out.print("Input number of products to represent: ");
prodNum = scan.nextInt();
String[] prodName = new String[prodNum];
System.out.print("Enter which products to represent: ");
for (int i = 0; i < prodName.length; i++)
{
prodName[i] = scan.next();
}
System.out.println("----Options----");
System.out.println("1. Cantidad");
System.out.println("2. Calidad");
System.out.println("3. RealmQ");
System.out.println("4. Coste");
option = scan.nextInt();
userOptionWas(option);
then by using JFreeCharts I wanted to be able to choose how many lines (for each product I input) and what option to display (quantity, quality and stuff) which is stored inside some obviously named arrays.
My approach
private XYDataset multipleLine(int prodNum, String[] prodName, int option){
XYSeriesCollection dataset = new XYSeriesCollection();
XYSeries product3 = new XYSeries("product3");
XYSeries product4 = new XYSeries("product4");
XYSeries product5 = new XYSeries("product5");
XYSeries product6 = new XYSeries("product6");
XYSeries product7 = new XYSeries("product7");
XYSeries product8 = new XYSeries("product8");
switch(prodNum)
{
case(2):
{
if (option == 1)
{
XYSeries product1 = new XYSeries(prodName[0]);
XYSeries product2 = new XYSeries(prodName[1]);
for (int i = 0; i < CSVinput.cantidad.length; i++)
{
try
{
if (CSVinput.nombre[i].equals(prodName[0]))
{
product1.add(i, CSVinput.cantidad[i]);
}
}catch(ArrayIndexOutOfBoundsException e){}
}
}
if (option == 2)
{
}
if (option == 3)
{
}
if (option == 4)
{
}
}
}
return dataset;
}
Halfway through implementing my idea I started to wonder if it is not possible to do it, or just I am in the wrong path to my goal.
What I expect
I want to be able to create different plots from a simple menu I showed before.
If it is possible, dynamic (if I input 2 products the program displays 2 lines referring to each product)
Extra Notes
I am new at JFreeCharts.
Take a moment here at product1.add(i, CSVinput.cantidad[i]); where I use (int)"i" instead of my (String)"date" format. Why I am not able to use a String? Is there any way to bypass this?
What I expect from this question
I would like to know more efficient and friendly ways to achieve this without too much unnecessary complexity.
Data Example
2018/12/29-Tejido,321 908,13.55,43.18,$15.98,
2018/12/29-Ropa,195 045,20.55,45.93,$123.01,
2018/12/29-Gorra de visera,126 561,17.43,42.32,$79.54,
2018/12/29-Cerveza,80 109,3.37,17.93,$12.38,
2018/12/29-Mercancías de playa,75 065,11.48,39.73,$105.93,
2018/12/29-Bebidas alcohólicas,31 215,4.84,27.90,$32.29,
2018/12/29-Artículos de cuero,19 098,23.13,44.09,$198.74,
2018/12/29-Bolsas y carteras,7 754,23.09,41.34,$1 176.54,
2018/12/30-Tejido,252 229,12.86,43.14,$18.87,
2018/12/30-Ropa,132 392,18.09,46.02,$177.58,
2018/12/30-Gorra de visera,87 676,14.42,42.46,$122.48,
2018/12/30-Cerveza,44 593,2.72,17.79,$18.71,
2018/12/30-Mercancías de playa,44 593,8.26,39.56,$200.78,
2018/12/30-Bebidas alcohólicas,27 306,4.30,23.88,$31.95,
2018/12/30-Artículos de cuero,16 147,21.08,43.91,$207.49,
2018/12/30-Bolsas y carteras,6 552,21.11,40.59,$1 195.41,
2019/01/02-Tejido,321 908,13.55,43.18,$15.98,
2019/01/02-Ropa,195 045,20.55,45.93,$123.01,
2019/01/02-Gorra de visera,126 561,17.43,42.32,$79.54,
2019/01/02-Cerveza,80 109,3.37,17.93,$12.38,
2019/01/02-Mercancías de playa,75 065,11.48,39.73,$105.93,
2019/01/02-Bebidas alcohólicas,31 215,4.84,27.90,$32.29,
2019/01/02-Artículos de cuero,19 098,23.13,44.09,$198.74,
2019/01/02-Bolsas y carteras,7 754,23.09,41.34,$1 176.54,
2019/01/03-Tejido,1 164 607,12.87,43.14,$15.54,
2019/01/03-Ropa,131 409,17.18,45.97,$161.17,
2019/01/03-Gorra de visera,79 242,13.54,43.17,$100.38,
2019/01/03-Cerveza,48 332,2.80,18.10,$17.48,
2019/01/03-Mercancías de playa,46 157,8.70,38.39,$180.54,
2019/01/03-Bebidas alcohólicas,25 210,4.04,23.72,$33.52,
2019/01/03-Artículos de cuero,14 321,19.56,39.92,$216.00,
2019/01/03-Bolsas y carteras,5 814,19.85,39.68,$1 227.29,
EDIT:
Thanks a lot for the help I think I have learnt but the way is never ending as by some reason, the instance which takes the dataset does not plot the dataset
What I am missing?.
PD: Let me know in the comments if you need CSVinput in order to provide a solution.
import java.awt.Color;
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.time.ZonedDateTime;
import java.util.Date;
import java.util.Scanner;
import org.jfree.chart.ChartFactory;
import org.jfree.chart.ChartPanel;
import org.jfree.chart.JFreeChart;
import org.jfree.chart.plot.XYPlot;
import org.jfree.data.general.SeriesException;
import org.jfree.data.time.Day;
import org.jfree.data.time.Second;
import org.jfree.data.time.TimeSeries;
import org.jfree.data.time.TimeSeriesCollection;
import org.jfree.data.xy.XYDataset;
import org.jfree.data.xy.XYSeries;
import org.jfree.data.xy.XYSeriesCollection;
import org.jfree.ui.ApplicationFrame;
import org.jfree.ui.RefineryUtilities;
import webscrapper.CSVinput;
/**
*
* #author Jonathan
*/
public class TimeSeriesChartExample extends ApplicationFrame{
//To parse dates below and be able to debug outside the loop
public static String[] dateSplit;
//Instace that declares properties and uses the dataset to create the chart
public TimeSeriesChartExample(String title)
{
super(title);
// Create dataset
XYDataset dataset = createDataset();
// Create chart
JFreeChart chart = ChartFactory.createTimeSeriesChart(
"Titulo", // Chart
"Date", // X-Axis Label
"Number", // Y-Axis Label
dataset);
//Changes background color
XYPlot plot = (XYPlot)chart.getPlot();
plot.setBackgroundPaint(new Color(255,228,196));
}
//Create the set of data to be plotted
private XYDataset createDataset()
{
//A collection of datasets
TimeSeriesCollection dataset = new TimeSeriesCollection();
//The object were this new series are added thus it's name
TimeSeries prod1 = new TimeSeries("Cantidad");
//"For each" name stored in the array
for (int i = 0; i < CSVinput.nombre.length; i++)
{
//If name equals
if ("Ropa".equals(CSVinput.nombre[i]))
{
//Pass the "i" position to work it
dateSplit = CSVinput.fecha[i].split("/");
//Parse the date
int day = Integer.parseInt(dateSplit[2]);
int month = Integer.parseInt(dateSplit[1]);
int year = Integer.parseInt(dateSplit[0]);
System.out.println("day is: " + day);
//Pass the parsed date and the value from CSV.cantidad at "i" position referring to the name "Ropa"
prod1.add(new Day(day, month, year), CSVinput.cantidad[i]);
System.out.println(CSVinput.cantidad[i]);//Debug stuff
}
}
//Add everything to the dataset
dataset.addSeries(prod1);
//Return it
return dataset;
}
private XYDataset createDataset2()
{
//A collection of datasets
TimeSeriesCollection dataset = new TimeSeriesCollection();
//The object were this new series are added thus it's name
TimeSeries prod2 = new TimeSeries("Cantidad");
//"For each" name stored in the array
for (int i = 0; i < CSVinput.nombre.length; i++)
{
//If name equals
if (CSVinput.nombre[i] == "Tejido")
{
//Pass the "i" position to work it
dateSplit = CSVinput.fecha[i].split("/");
//Parse the date
int day = Integer.parseInt(dateSplit[0]);
int month = Integer.parseInt(dateSplit[1]);
int year = Integer.parseInt(dateSplit[2]);
//Pass the parsed date and the value from CSV.cantidad at "i" position referring to the name "Ropa"
prod2.add(new Day(day, month, year), CSVinput.cantidad[i]);
System.out.println(CSVinput.cantidad[i]);//Debug stuff
}
}
//Add everything to the dataset
dataset.addSeries(prod2);
//Return it
return dataset;
}
public static void main(String[] args) throws FileNotFoundException
{
//Custom class to import data from the csv into arrays (TODO: make it dynamic)
CSVinput.ImportData("caca.csv");
/* //More debugg stuff
dateSplit = CSVinput.fecha[1].split("/");
System.out.println("year: " + dateSplit[0] + " month: " + dateSplit[1] + " day: " + dateSplit[2]);
*/
//Create an instance of the previous class
final TimeSeriesChartExample example = new TimeSeriesChartExample("ItWorks!");
example.pack(); //Part from the ApplicationFrame
RefineryUtilities.centerFrameOnScreen(example); //Render the graph at the center of screen
example.setVisible(true); //make it visible
}
}
First, focus on you program's data model:
If your series include a time domain, consider using a TimeSeriesCollection of TimeSeries rather than XYSeries; TimeSeriesChartDemo1, cited here, is a basic example; parse dates as shown here and here; for database access, consider a JDBCXYDataset.
As you read the data for each series, store the series in a List<TimeSeries>, rather than an array; if latency is a problem, do the parsing in the background, as shown here.
The exact details will depend on your application's design, but using the observer pattern will minimize complexity; the basic principle is to update the model (Dataset) and the listening view (JFreeChart) will update itself in response. Using Swing for example, you can control the displayed chart in several ways:
Use your List<TimeSeries> to construct a suitable ListModel<TimeSeries>; for the series menu, make your chosen model a DefaultComboBoxModel<TimeSeries> for a JComboBox or a DefaultListModel<TimeSeries> for a JList.
Add a suitable listener to your menu component; give your XYPlot a TimeSeriesCollection; use addSeries() or removeSeries() to alter the plot as a series is selected; alternatively, use setSeriesVisible() to toggle visibility, as shown here.
For a command-line interface, the interactions are less complex but also less versatile:
Use a List<TimeSeries> to build your TimeSeriesCollection.
Use the TimeSeriesCollection methods to control what the listening XYPlot displays.
This post is a suite to an answer I made to question: Transforming a shape
Here is the image I want:
Here is the image a simple program produces, as you can see the text is rotated. I want horizontal text:
The canvas is scaled, translated, rotated to do the drawing, so the text is not displayed horizontaly and the font size need to be extremely reduced (1.4). The program is wrote in Java (awt and JavaFX) but the problem is not language or technology relevant, so any suggestion is welcome.
Here is the simple program:
import javafx.application.Application;
import javafx.geometry.VPos;
import javafx.scene.Scene;
import javafx.scene.canvas.Canvas;
import javafx.scene.canvas.GraphicsContext;
import javafx.scene.layout.BorderPane;
import javafx.scene.paint.Color;
import javafx.scene.text.Font;
import javafx.scene.text.TextAlignment;
import javafx.stage.Stage;
public class TransRotScale extends Application {
private static void drawGraph( GraphicsContext g ) {
//---
g.scale( 10.0, 10.0 );
g.rotate( Math.toDegrees( Math.atan2( -15.0, 40.0 )));
g.translate( -8, -10 );
//---
g.setStroke( Color.DARKRED );
g.setLineWidth( LINE_WIDTH );
g.strokeLine( 10, 20, 10, 30 );
g.strokeLine( 10, 30, 50, 30 );
g.strokeLine( 50, 30, 50, 35 );
//---
g.setFill( Color.BLACK );
g.fillOval( 50-ENDPOINT_RADIUS, 35-ENDPOINT_RADIUS,
ENDPOINT_DIAMETER, ENDPOINT_DIAMETER );
g.fillOval( 10-ENDPOINT_RADIUS, 20-ENDPOINT_RADIUS,
ENDPOINT_DIAMETER, ENDPOINT_DIAMETER );
//---
g.setFill( Color.LIGHTSALMON );
g.fillOval( 10-ENDPOINT_RADIUS, 30-ENDPOINT_RADIUS,
ENDPOINT_DIAMETER, ENDPOINT_DIAMETER );
g.fillOval( 50-ENDPOINT_RADIUS, 30-ENDPOINT_RADIUS,
ENDPOINT_DIAMETER, ENDPOINT_DIAMETER );
//---
g.setStroke( Color.DARKGRAY );
g.setFont( Font.font( Font.getDefault().getFamily(), 1.4 ));
g.setLineWidth( 0.1 );
g.setTextAlign( TextAlignment.CENTER );
g.setTextBaseline( VPos.BOTTOM );
g.strokeText( "[10, 20]", 10, 20-ENDPOINT_RADIUS );
g.setTextBaseline( VPos.TOP );
g.strokeText( "[10, 30]", 10, 30+ENDPOINT_RADIUS );
g.setTextBaseline( VPos.BOTTOM );
g.strokeText( "[50, 30]", 50, 30-ENDPOINT_RADIUS );
g.setTextBaseline( VPos.TOP );
g.strokeText( "[50, 35]", 50, 35+ENDPOINT_RADIUS );
}
#Override
public void start( Stage primaryStage ) throws Exception {
BorderPane bp = new BorderPane();
Canvas canvas = new Canvas( 540, 240 );
bp.setCenter( canvas );
drawGraph( canvas.getGraphicsContext2D());
primaryStage.setScene( new Scene( bp ));
primaryStage.centerOnScreen();
primaryStage.show();
}
public static final double ENDPOINT_RADIUS = 2.0;
public static final double ENDPOINT_DIAMETER = 2.0*ENDPOINT_RADIUS;
public static final double LINE_WIDTH = 1.0;
public static void main( String[] args ) {
launch();
}
}
In the program used to display first image (the goal), I use two canvases, the first canvas is scaled, translated, rotated to do the drawing without any text and the second canvas is used only to draw labels horizontally, using java.awt.geom.AffineTransform to compute coordinates to match the item displayed in the first canvas. Both canvases are displayed superposed, they are transparent.
This is what suggest Alexander Kirov, if I understand well:
import javafx.application.Application;
import javafx.geometry.VPos;
import javafx.scene.Scene;
import javafx.scene.layout.Pane;
import javafx.scene.paint.Color;
import javafx.scene.shape.Circle;
import javafx.scene.shape.Polyline;
import javafx.scene.text.Font;
import javafx.scene.text.Text;
import javafx.scene.text.TextAlignment;
import javafx.stage.Stage;
public class TransRotScal extends Application {
#Override
public void start( Stage primaryStage ) throws Exception {
Pane pane = new Pane();
pane.setScaleX( 10.0 );
pane.setScaleY( 10.0 );
pane.setRotate( theta );
pane.setTranslateX( 468.0 );
pane.setTranslateY( 152.0 );
Polyline line = new Polyline( 10,20, 10,30, 50,30, 50,35 );
line.setStroke( Color.DARKRED );
Circle c0 = new Circle( 10, 20, 2, Color.BLACK );
Circle c1 = new Circle( 10, 30, 2, Color.LIGHTSALMON );
Circle c2 = new Circle( 50, 30, 2, Color.LIGHTSALMON );
Circle c3 = new Circle( 50, 35, 2, Color.BLACK );
Text t0 = createText( 10, 20, "[10,20]", VPos.BOTTOM );
Text t1 = createText( 10, 30, "[10,30]", VPos.TOP );
Text t2 = createText( 50, 30, "[50,30]", VPos.BOTTOM );
Text t3 = createText( 50, 35, "[50,35]", VPos.TOP );
pane.getChildren().addAll( line, c0, c1, c2, c3, t0, t1, t2, t3 );
primaryStage.setScene( new Scene( pane ));
primaryStage.centerOnScreen();
primaryStage.setWidth ( 580 );
primaryStage.setHeight( 280 );
primaryStage.show();
}
private Text createText( int x, int y, String label, VPos vPos ) {
Text text = new Text( x, y, label );
text.setFill( Color.DARKGRAY );
text.setFont( Font.font( Font.getDefault().getFamily(), 1.4 ));
text.rotateProperty().set( -theta );
text.textAlignmentProperty().setValue( TextAlignment.CENTER );
text.setX( text.getX() - text.getBoundsInLocal().getWidth()/2.0);
text.textOriginProperty().set( vPos );
if( vPos == VPos.BOTTOM ) {
text.setY( text.getY() - 2 );
}
else {
text.setY( text.getY() + 2 );
}
return text;
}
private final double theta = Math.toDegrees( Math.atan2( -15.0, 40.0 ));
public static void main( String[] args ) {
launch();
}
}
It works but it use Node in place of canvas and the values to adjust texts are obtained by iterative tries (a lot!); I don't know how to calculate them.
Alexander, you may edit this post or post your own to complete it, in the later case I'll delete this.
Here is the result, note the approximative placement of text around discs:
Instead of drawing the string directly onto the Graphics object can you create a GlyphVector from the string instead and draw that to the Graphics object? The advantage to this approach would be that the GlyphVector can have its own transform which you could use to effectively cancel the rotation of the canvas. I don't remember the exact details to creating the glyphs, but you need a Font and a FontRenderContext...both of which are already available to the Graphics context.