Develop Reference

Read and format data in java

I have an assignment where I have to read data from a text file. The data looks like this in the file:
1946-01-01;07:00:00;-1.8;G
1946-01-01;13:00:00;-1.0;G
1946-01-01;18:00:00;-1.9;G
1946-01-02;07:00:00;-1.7;G
I want to format the data and put it in an appropriate data structure so that I then can search for average temperature for all dates between two dates. What is the simplest way to format this data when my code is:
package algo.weatherdata;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.time.LocalDate;
import java.util.List;
/**
* Provides methods to retrieve temperature data from a weather station file.
*/
public class WeatherDataHandler {
/**
* Load weather data from file.
*
* #param filePath path to file with weather data
* #throws IOException if there is a problem while reading the file
*/
public void loadData(String filePath) throws IOException {
//Read all weather data
List<String> fileData = Files.readAllLines(Paths.get(filePath));
/**
* TODO: Format data and put it in appropriate data structure.
*/
}

You can iterate over your lines and separate each line by a delimiter (in your case ;):
public void loadData(String filePath) throws IOException {
List<String> fileData = Files.readAllLines(Paths.get(filePath));
String[] lineValues = str.split(";");
// line values are stored in this array
}
Then you can do with the array as you like (store them, process them).

Let's assume the following class that will contain weather data:
public class WeatherData {
private LocalDateTime dateTime;
private double temperature;
private String tag; // I don't really know what is 'G' mean
public WeatherData(LocalDateTime dateTime, double temperature, String tag) {
this.dateTime = dateTime;
this.temperature = temperature;
this.tag = tag;
}
}
Next, we analyze the data file into the current structure and collect them all into a list:
private List<WeatherData> weatherData = new ArrayList<>();
public void loadData(String filePath) throws IOException {
List<String> fileData = Files.readAllLines(Paths.get(filePath));
for(String str : fileData) {
List<String> parsed = parseData(str);
LocalDateTime dateTime = LocalDateTime.of(dateOf(parsed.get(0)),
timeOf(parsed.get(1)));
double temperature = Double.parseDouble(parsed.get(2));
String tag = parsed.get(3);
WeatherData weather = new WeatherData(dateTime, temperature, tag);
weatherData.add(weather);
}
}
private List<String> parseData(String s) {
return Arrays.asList(s.split(";"));
}
private LocalDate dateOf(String date) {
return LocalDate.parse(date);
}
private LocalTime timeOf(String time) {
return LocalTime.parse(time);
}
And then you could work with a list to search between dates and calculate the average temperature

I have tried above Dinar Zaripov piece of code in my work space with addition of generating getters methods for dateTime, temperature and tag.
Continuation after for loop
private static float averageTemperature(List<WeaterData> weatherData)
{
float sum = 0; // Variable to store the sum
float count = 0; // Variable to keep the count
if (weatherList != null) {
for (WeaterData averageWeather : weatherData) {
float value = averageWeather.getTemperature();
System.out.println(value);
if (value <= 0) { // Check if the number is negative
sum += value; // Add the value to the current sum
count++; // Increment the count by 1
}
}
}
return (count > 0) ? (sum /count) : 0; // Calculate the average if there were any negative numbers
}
But I am getting average as ** average::1.5999999 **
However as per the math it should be 1.6. could someone tell me how ? or this is correct ?
when I give below data , I am getting correct average :)
1946-01-01;07:00:00;-1.8;A
** 1946-01-01;13:00:00;-1.1; B**
1946-01-01;18:00:00;-1.9;C
1946-01-02;07:00:00;-1.7;D

validate ArrayList contents against specific set of data

I want to check and verify that all of the contents in the ArrayList are similar to the value of a String variable. If any of the value is not similar, the index number to be printed with an error message like (value at index 2 didn't match the value of expectedName variable).
After I run the code below, it will print all the three indexes with the error message, it will not print only the index number 1.
Please note that here I'm getting the data from CSV file, putting it into arraylist and then validating it against the expected data in String variable.
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVParser;
import org.apache.commons.csv.CSVRecord;
import java.io.IOException;
import java.io.Reader;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.ArrayList;
public class ValidateVideoDuration {
private static final String CSV_FILE_PATH = "C:\\Users\\videologs.csv";
public static void main(String[] args) throws IOException {
String expectedVideo1Duration = "00:00:30";
String expectedVideo2Duration = "00:00:10";
String expectedVideo3Duration = "00:00:16";
String actualVideo1Duration = "";
String actualVideo2Duration = "";
String actualVideo3Duration = "";
ArrayList<String> actualVideo1DurationList = new ArrayList<String>();
ArrayList<String> actualVideo2DurationList = new ArrayList<String>();
ArrayList<String> actualVideo3DurationList = new ArrayList<String>();
try (Reader reader = Files.newBufferedReader(Paths.get(CSV_FILE_PATH));
CSVParser csvParser = new CSVParser(reader,
CSVFormat.DEFAULT.withFirstRecordAsHeader().withIgnoreHeaderCase().withTrim());) {
for (CSVRecord csvRecord : csvParser) {
// Accessing values by Header names
actualVideo1Duration = csvRecord.get("Video 1 Duration");
actualVideo1DurationList.add(actualVideo1Duration);
actualVideo2Duration = csvRecord.get("Video 2 Duration");
actualVideo2DurationList.add(actualVideo2Duration);
actualVideo3Duration = csvRecord.get("Video 3 Duration");
actualVideo3DurationList.add(actualVideo3Duration);
}
}
for (int i = 0; i < actualVideo2DurationList.size(); i++) {
if (actualVideo2DurationList.get(i) != expectedVideo2Duration) {
System.out.println("Duration of Video 1 at index number " + Integer.toString(i)
+ " didn't match the expected duration");
}
}
The data inside my CSV file look like the following:
video 1 duration, video 2 duration, video 3 duration
00:00:30, 00:00:10, 00:00:16
00:00:30, 00:00:15, 00:00:15
00:00:25, 00:00:10, 00:00:16

Don't use == or != for string compare. == checks the referential equality of two Strings and not the equality of the values. Use the .equals() method instead.
Change your if condition to if (!actualVideo2DurationList.get(i).equals(expectedVideo2Duration))

Java Hash map / Array List Count distinct values

I am pretty new into programming and I have an assignment to make, but I got stuck.
I have to implement a program which will read a CSV file (1 million+ lines) and count how many clients ordered "x" distinct products on a specific day.
The CSV looks like this:
Product Name | Product ID | Client ID | Date
Name 544 86 10/12/2017
Name 545 86 10/12/2017
Name 644 87 10/12/2017
Name 644 87 10/12/2017
Name 9857 801 10/12/2017
Name 3022 801 10/12/2017
Name 3021 801 10/12/2017
The result from my code is:
801: 2 - incorrect
86: 2 - correct
87: 2 - incorrect
Desired output is:
Client 1 (801): 3 distinct products
Client 2 (86): 2 distinct products
Client 3 (87): 1 distinct product
Additionally,
If I want to know how many clients ordered 2 distinct products I would like a result to look like this:
Total: 1 client ordered 2 distinct products
If I want to know the maximum number of distinct products ordered in a day, I would like the result to look like this:
The maximum number of distinct products ordered is: 3
I tried to use a Hash Map and Multimap by Google Guava (my best guess here), but I couldn't wrap my head around it.
My code looks like this:
package Test;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;
import com.google.common.collect.ArrayListMultimap;
import com.google.common.collect.HashMultimap;
public class Test {
public static void main(String[] args) {
//HashMultimap<String, String> myMultimap = HashMultimap.create();
Map<String, MutableInteger> map = new HashMap<String, MutableInteger>();
ArrayList<String> linesList = new ArrayList<>();
// Input of file which needs to be parsed
String csvFile = "file.csv";
BufferedReader csvReader;
// Data split by 'TAB' in CSV file
String csvSplitBy = "\t";
try {
// Read the CSV file into an ArrayList array for easy processing.
String line;
csvReader = new BufferedReader(new FileReader(csvFile));
while ((line = csvReader.readLine()) !=null) {
linesList.add(line);
}
csvReader.close();
} catch (IOException e) {
e.printStackTrace();
}
// Process each CSV file line which is now contained within
// the linesList list Array
for (int i = 0; i < linesList.size(); i++) {
String[] data = linesList.get(i).split(csvSplitBy);
String col2 = data[1];
String col3 = data[2];
String col4 = data[3];
// Determine if Column 4 has the desired date
// and count the values
if (col4.contains("10/12/2017")) {
String key = col3;
if (map.containsKey(key)) {
MutableInteger count = map.get(key);
count.set(count.get() + 1);
} else {
map.put(key, new MutableInteger(1));
}
}
}
for (final String k : map.keySet()) {
if (map.get(k).get() == 2) {
System.out.println(k + ": " + map.get(k).get());
}
}
}
}
Any advise or suggestion on how this can be implemented would be greatly appreciated.
Thank you in advance guys.

You could store a Setof productIds per clientId, and just take the size of that.
As a Set does not allow duplicate values, this will effectively give you the distinct number of productIds.
Also, I recommend that you give your variables meaningful name instead of col2, k, map... This will make your code more readable.
Map<String, Set<String>> distinctProductsPerClient = new HashMap<String, Set<String>>();
// Process each CSV file line which is now contained within
// the linesList list Array
// Start from 1 to skip the first line
for (int i = 1; i < linesList.size(); i++) {
String line = linesList.get(i);
String[] data = line.split(csvSplitBy);
String productId = data[1];
String clientId = data[2];
String date = data[3];
// Determine if Column 4 has the desired date
// and count the values
if (date.contains("10/12/2017")) {
if (!distinctProductsPerClient.containsKey(clientId)) {
distinctProductsPerClient.put(clientId, new HashSet<>());
}
distinctProductsPerClient.get(clientId).add(productId);
}
}
for (final String clientId : distinctProductsPerClient.keySet()) {
System.out.println(clientId + ": " + distinctProductsPerClient.get(clientId).size());
}
More advanced solution using Stream API (requires Java 9)
If you introduce the class OrderData(that represents a single line in the CSV) like this:
private static class OrderData {
private final String productName;
private final String productId;
private final String clientId;
private final String date;
public OrderData(String csvLine) {
String[] data = csvLine.split("\t");
this.productName = data[0];
this.productId = data[1];
this.clientId = data[2];
this.date = data[3];
}
public String getProductName() {
return productName;
}
public String getProductId() {
return productId;
}
public String getClientId() {
return clientId;
}
public String getDate() {
return date;
}
}
you can replace the for loop with this:
Map<String, Set<String>> distinctProductsPerClient2 = linesList.stream()
.skip(1)
.map(OrderData::new)
.collect(groupingBy(OrderData::getClientId, mapping(OrderData::getProductId, toSet())));
But I reckon this might be a little bit to complex if you're new into programming (although it might be a good exercise if you would try to understand what the above code does).

Need an help to sort, Handle a tab seperated text file

Hi i have a text file(tab seperated).
I would like to open it, read it and filter the required columns just like we do in excel. Could someone help with a sample code.
I am stuck up with the concept on how to proceed further for the below steps.
Price has to sorted with DESC order before processing.
Always, First two column has to be printed in the output.
Other than the first two columns only the required column has to be printed based on the input given.
Input will be something like (Mango/purchased/top50). so it should pick only top50 "yet to buy" under "Mango" along with its respective first two columns.
Sample input file.
itemNumber Price Mango Apple Bannana
112201 purchased need to plan purchased
112202 55 yet to buy yet to buy purchased
112202 67 need to plan purchased purchased
112203 456 need to plan need to plan need to plan
112203 33 need to plan yet to buy need to plan
112204 456 need to plan yet to buy need to plan
112204 yet to buy purchased need to plan
112205 77 yet to buy purchased need to plan
112205 99 yet to buy purchased yet to buy
112206 0 yet to buy purchased yet to buy
The code is incomplete.
Here i am trying to add the heading of the file to an arraylist and adding the content to an another arraylist. Then trying to compare them using index number. Is this way correct ?
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Enumeration;
import java.util.List;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
public class main {
#SuppressWarnings({ "resource", "unused" })
public static void main(String[] args) throws IOException {
ZipFile zipFile = new ZipFile(
"filename.tsv.zip");
Enumeration<? extends ZipEntry> entries = zipFile.entries();
while (entries.hasMoreElements()) {
String fruit = "Mango";
String mappingstatus = "purchased";
// reading a file
ZipEntry entry = entries.nextElement();
InputStream stream = zipFile.getInputStream(entry);
InputStreamReader read = new InputStreamReader(stream);
BufferedReader br = new BufferedReader(read);
// creating a new list
List<String> heading = new ArrayList<String>();
String[] a = br.readLine().split("\t");
heading = Arrays.asList(a);
List<String> content = new ArrayList<String>();
String s;
while ((s = br.readLine()) != null) {
String[] b = br.readLine().split("\t");
content = Arrays.asList(b);
}
}
}
}

Try this
class Item implements Comparable<Item> {
int itemNumber;
int price;
String mango;
String apple;
String bannana;
public Item(int itemNumber, int price, String mango, String apple, String bannana) {
this.itemNumber = itemNumber;
this.price = price;
this.mango = mango;
this.apple = apple;
this.bannana = bannana;
}
//GETTERS
#Override
public int compareTo(Item compareItem) {
int comparePrice = ((Item) compareItem).getPrice();
//ascending order
//return this.price - comparePrice;
//descending order
return comparePrice - this.price;
}
}
public static void main(String[] args) {
List<Item> items = new ArrayList<>();
//populate the items list by creating an Item for every line you read.
//Handle null price values
Collections.sort(items);
//assuming input is some like 'Mango/purchased/top50'
String input = "Mango/purchased/top50";
String[] parts = input.split("/");
int max = Integer.parseInt(parts[2].substring(3));
List<Item> result = new ArrayList<>();
for (int i = 0; i < items.size() && result.size() < max; i++) {
Item item = items.get(i);
if ((parts[0].equals("Mango") && item.getMango().equals(parts[1]))
|| (parts[0].equals("Apple") && item.getApple().equals(parts[1]))
|| (parts[0].equals("Bannana") && item.getBannana().equals(parts[1]))) {
result.add(item);
}
}
}
Complete the commented sections and it must work. More references about sorting here: Sorting

How to group the sorting of a file?

I'm trying to sort delimited files with a Time stamp | level | sensor name | measurement value structure so that all data associated with the sensor having the smallest time stamp would first be listed in increasing timestamp, then all the data associated with the sensor having the second smallest time stamp would be listed in increasing timestamp, etc…
Here’s an example of delimited file to be sorted :
20140102073500|1|sensor5|0.188|
20140102073502|1|sensor2|0.193|
20140102073600|2|sensor5|0.577|
20140102073603|2|sensor2|0.585|
20140102073700|3|sensor5|1.207|
20140102073702|3|sensor2|1.183|
Here what I want :
20140102073500|1|sensor5|0.188|
20140102073600|2|sensor5|0.577|
20140102073700|3|sensor5|1.207|
20140102073502|1|sensor2|0.193|
20140102073603|2|sensor2|0.585|
20140102073702|3|sensor2|1.183|
(note that I cannot sort by "sensor name / time stamp" because the sensor with the smallest time stamp changes from one file to the other...)
Here the coding I’m trying to work from (which only sort in ascending time stamp) :
import java.io.*;
import java.util.*;
public class Sort8 {
public static void main(String[] args) throws Exception {
Map<String, String> map;
try (BufferedReader reader = new BufferedReader(new FileReader("C:\\Test\\test.txt"))) {
map = new TreeMap<>();
String line;
while((line=reader.readLine())!=null){
map.put(getField(line),line);
System.out.println(getField(line));
}
System.out.println(map.values());
}
try (FileWriter writer = new FileWriter("C:\\Test\\test_sorted.txt")) {
for(String val : map.values()){
// System.out.println(val);
writer.write(val) ;
writer.write("\r\n");
}
}
}
private static String getField(String line) {
return ((line.split("\\|")[1])+(line.split("\\|")[3]));
}
}
I'm new to Java so thank in advance for the help you can provide !

Put a comparator in the construction of your tree map.
map = new TreeMap<>(new Comparator<String>() {
#Override
public int compare(String s1, String s2) {
String[] a1 = s1.split("|");
String[] a2 = s2.split("|");
// First compare sensor
int sensor1 = Integer.parseInt(a1[2].replace("sensor", ""));
int sensor2 = Integer.parseInt(a2[2].replace("sensor", ""));
if(sensor1 != sensor2) {
return Integer.valueOf(sensor1).compareTo(sensor2);
}
// Second compare timestamp
long time1 = Long.parseLong(a1[0]);
long time2 = Long.parseLong(a2[0]);
return Long.valueOf(time1).compareTo(time2);
}
});