I am trying to read from one CSV file using OpenCSV. I then want to copy all the data from the input csv and output it to another csv file while adding a new column with information.
public void run_streets_tsv( String tsvIn, String tsvOut) throws Exception
{
CSVReader reader = null;
CSVWriter writer = null;
try
{
reader = new CSVReader((new FileReader(tsvIn)));
writer = new CSVWriter(new FileWriter(tsvOut), '\t');
String element [] = null;
List<String[]> a = new ArrayList<String[]>();
while((element = reader.readNext()) != null){
for(int i = 0; i<element.length; i++){
a.add(i, element);
//a.add("JSON"); need to add this json element at the end of each column
}
}
writer.writeAll(a);
}
catch(Exception e)
{
throw e;
}
finally
{
reader.close();
writer.close();
}
}
Another method I am trying is like this (changing the while loop, all other code remains the same):
String element [] = null;
while((element = reader.readNext()) != null){
ArrayList list = new ArrayList(Arrays.asList(reader));
list.add(element);
list.add("JSON");
writer.writeNext(element);
}
This does correctly print all the lines, but it just copies. I want to add that extra "JSON" column with its data.
The following "enlarges" the element-Array by one, enabling you to put something in the newly created last index. Then just save that array.
import java.util.Arrays;
String element[] = null;
while((element = reader.readNext()) != null){
element = Arrays.copyOf(element, element.length + 1);
element[element.length - 1] = "JSON";
writer.writeNext(element);
}
OK, you are close although I see a few errors.
'reader.readNext()' return a line from the input as a String array, we basically need to add an element to this for the output.
while((element = reader.readNext()) != null) {
String[] output = getExpandedArray(element);
a.add(output);
}
You will need to implement getExpandedArray, I will start it off.
private String[] getExpandedArray(String[] input) {
String[] output = null;
//Populate/create output from input, but with the array 1 bigger.
output[output.length -1] = "JSON";
return output;
}
Related
i'm doing some java coding and i have to import a file into a Jtable that has 4 columns while my file has 3 (separated by whitespaces, i need the first column of each line to be auto increment, here is my code:
try {
FileReader files = new FileReader(file);
BufferedReader buf = new BufferedReader(files);
String line = null;
String tokens[] = null;
while ((line = buf.readLine()) != null) {
tokens = line.split("\\p{javaWhitespace}+");
//System.out.println( Arrays.toString( tokens ));
model.addRow(tokens);
}
}
and this is what I'm getting :
and this is my file :
Simply add an additional token to the front of your data. This is easiest using a Vector and not an array. The first item in the vector is your row index, the next itmes are filled from your tokens array. For example:
try {
FileReader files = new FileReader(file);
BufferedReader buf = new BufferedReader(files);
String line = null;
String tokens[] = null;
int count = 0;
while ((line = buf.readLine()) != null) {
tokens = line.split("\\p{javaWhitespace}+");
Vector<Object> row = new Vector<>();
row.add(count);
count++;
for (String text: tokens) {
row.add(text);
}
model.addRow(row); // add the Vector, not the tokens array
}
}
There are other ways, including extending the table model such that it automatically does this, and these may need to be done, depending on your needs -- for example, are the rows to renumber if one row is deleted or added during the running of the program? If so the logic needs to be within the table model.
I'm currently working on a program that reads in a preset text file and then manipulates the data in various ways. I've got the data manipulation to work with some dummy data but I still need to get the text file read in correctly.
The test file looks like this for 120 lines:
Aberdeen,Scotland,57,9,N,2,9,W,5:00,p.m. Adelaide,Australia,34,55,S,138,36,E,2:30,a.m. Algiers,Algeria,36,50,N,3,0,E,6:00,p.m.(etc etc)
So each of these needs to be read into its own array, in order String[] CityName,String[] Country,int[] LatDeg,int[] LatMin,String[] NorthSouth,int[] LongDeg,int LongMin,String[] EastWest,int[] Time.String[] AMPM
So the problem is that while I'm reasonably comfortable with buffered readers, designing this particular function has proven difficult. In fact, I've been drawing a blank for the past few hours. It seems like it would need multiple loops and counters but I can't figure out the precisely how.
I am assuming that you have one city per line type of file structure. If it is not, it will require a bit of tweaking in the following solution:
I will do the following way if I am more comfortable with BufferReader as you say:
List<List<String>> addresses = new ArrayList<List<String>>();
try(BufferedReader br = new BufferedReader(new FileReader(file))) {
for(String line; (line = br.readLine()) != null; ) {
addresses.add(line.split(","));
}
}
Later, let's say you want to retrieve the country information of say 'Adelaid', you can try the following:
for (List<String> cityInfo : addresses) {
if("Adelaid".equals(cityInfo.get(0)) {
country = cityInfo.get(1);
}
}
Instead of creating different arrays (like String[] CityName,String[] Country, etc.,), try using a Domain Object.
Here, you can have a Domain object or Custom class Location with attributes
public class Location
{
private String cityName;
private String country;
private String latDeg;
etc
getters();
setters();
}`
Then you can write a file reader, each line item in the file will be a Location. So result will have
Location[] locations;
or
List locations;`
To carry out this task I should think the first thing you want to do is establish how many lines of data actually exist within the data file. You say it is 120 lines but what if it happens that it will be more or less? We would want to know exactly what it is so as to properly initialize all our different Arrays. We can use a simple method to accomplish this, let's call it the getFileLinesCount() method which will ulitmately return a Integer value that would be the number of text lines the data file holds:
private int getFileLinesCount(final String filePath) {
int lines = 0;
try{
File file =new File(filePath);
if(file.exists()){
FileReader fr = new FileReader(file);
try (LineNumberReader lnr = new LineNumberReader(fr)) {
while (lnr.readLine() != null){ lines++; }
}
}
else {
throw new IllegalArgumentException("GetFileLinesCount() Method Error!\n"
+ "The supplied file path does not exist!\n(" + filePath + ")");
}
}
catch(IOException e){ e.printStackTrace(); }
return lines;
}
Place this method somewhere within your main class. Now you need to Declare and initialize all your Arrays:
String filePath = "C:\\My Files\\MyDataFile.txt";
int lines = getFileLinesCount(filePath);
String[] CityName = new String[lines];
String[] Country = new String[lines];
int[] LatDeg = new int[lines];
int[] LatMin = new int[lines];
String[] NorthSouth = new String[lines];
int[] LongDeg = new int[lines];
int[] LongMin = new int[lines];
String[] EastWest = new String[lines];
int[] Time = new int[lines];
String[] AMPM = new String[lines];
Now to fill up all those Arrays:
public static void main(String args[]) {
loadUpArrays();
// Do whatever you want to do
// with all those Arrays.....
}
private void loadUpArrays() {
// Read in the data file.
try (BufferedReader br = new BufferedReader(new FileReader(filePath))) {
String sCurrentLine;
int x = 0;
// Read in one line at a time and Fill the Arrays...
while ((sCurrentLine = br.readLine()) != null) {
// Split each line read into an array upon itself.
String[] fileLine = sCurrentLine.split(",");
// Fill our required Arrays...
CityName[x] = fileLine[0];
Country[x] = fileLine[1];
LatDeg[x] = Integer.parseInt(fileLine[2]);
LatMin[x] = Integer.parseInt(fileLine[3]);
NorthSouth[x] = fileLine[4];
LongDeg[x] = Integer.parseInt(fileLine[5]);
LongMin[x] = Integer.parseInt(fileLine[6]);
EastWest[x] = fileLine[7];
Time[x] = Integer.parseInt(fileLine[8]);
AMPM[x] = fileLine[9];
x++;
}
br.close();
}
catch (IOException ex) { ex.printStackTrace(); }
}
Now, I haven't tested this, I just quickly punched it out but I think you can get the jest of it.
EDIT:
As #Mad Physicist has so graciously pointed out within his comment below, a List can be used to eliminate the need to count file lines therefore eliminating the need to read the data file twice. All the file lines can be placed into the List and the number of valid file lines can be determined by the size of the List. Filling of your desired arrays can now also be achieved by iterating through the List elements and processing the data accordingly. Everything can be achieved with a single method we'll call fillArrays(). Your Arrays declaration will be a little different however:
String[] CityName;
String[] Country;
int[] LatDeg;
int[] LatMin;
String[] NorthSouth;
int[] LongDeg;
int[] LongMin;
String[] EastWest;
String[] Time;
String[] AMPM;
public static void main(String args[]) {
fillArrays("C:\\My Files\\MyDataFile.txt");
// Whatever you want to do with all
// those Arrays...
}
private void fillArrays(final String filePath) {
List<String> fileLinesList = new ArrayList<>();
try{
File file = new File(filePath);
if(file.exists()){
try (BufferedReader br = new BufferedReader(new FileReader(file))) {
String strg;
while((strg = br.readLine()) != null){
// Make sure there is no blank line. If not
// then add line to List.
if (!strg.equals("")) { fileLinesList.add(strg); }
}
br.close();
}
}
else {
throw new IllegalArgumentException("GetFileLinesCount() Method Error!\n"
+ "The supplied file path does not exist!\n(" + filePath + ")");
}
// Initialize all the Arrays...
int lines = fileLinesList.size();
CityName = new String[lines];
Country = new String[lines];
LatDeg = new int[lines];
LatMin = new int[lines];
NorthSouth = new String[lines];
LongDeg = new int[lines];
LongMin = new int[lines];
EastWest = new String[lines];
Time = new String[lines];
AMPM = new String[lines];
// Fill all the Arrays...
for (int i = 0; i < fileLinesList.size(); i++) {
String[] lineArray = fileLinesList.get(i).split(",");
CityName[i] = lineArray[0];
Country[i] = lineArray[1];
LatDeg[i] = Integer.parseInt(lineArray[2]);
LatMin[i] = Integer.parseInt(lineArray[3]);
NorthSouth[i] = lineArray[4];
LongDeg[i] = Integer.parseInt(lineArray[5]);
LongMin[i] = Integer.parseInt(lineArray[6]);
EastWest[i] = lineArray[7];
Time[i] = lineArray[8];
AMPM[i] = lineArray[9];
}
}
catch(IOException e){ e.printStackTrace(); }
}
On another note...your Time Array can not be Integer since in data, what is considered the time contains a colon (:) which is a alpha character therefore (in case you haven't noticed) I have changed its declaration to String[]
I am a little stuck with a java exercise I am currently working on. I have a text file in this format:
Quio Kla,2221,3.6
Wow Pow,3332,9.3
Zou Tou,5556,9.7
Flo Po,8766,8.1
Andy Candy,3339,6.8
I now want to calculate the average of the whole third column, but I have to extract the data first I believe and store it in an array. I was able to read all the data with a buffered reader and print out the entire file in console, but that did not get me closer to get it into an array. Any suggestions on how I can read in a specific column of a text file with a buffered readder into an array would be highly appreciated.
Thank you very much in advance.
You can split your text file by using this portion of code:
BufferedReader in = null;
try {
in = new BufferedReader(new FileReader("textfile.txt"));
String read = null;
while ((read = in.readLine()) != null) {
String[] splited = read.split(",");
for (String part : splited) {
System.out.println(part);
}
}
} catch (IOException e) {
System.out.println("There was a problem: " + e);
e.printStackTrace();
} finally {
try {
in.close();
} catch (Exception e) {
}
}
And then you'll have all your columns in the array part.
It`s definitely not the best solution, but should be sufficient for you
BufferedReader input = new BufferedReader(new FileReader("/file"));
int numOfColumn = 2;
String line = "";
ArrayList<Integer>lines = new ArrayList<>();
while ((line = input.readLine()) != null) {
lines.add(Integer.valueOf(line.split(",")[numOfColumn-1]));
}
long sum =0L;
for(int j:lines){
sum+=j;
}
int avg = (int)sum/lines.size();
I'm going to assume each data set is separated by newline characters in your text file.
ArrayList<Double> thirdColumn = new ArrayList<>();
BufferedReader in = null;
String line=null;
//initialize your reader here
while ((line = in.readLine())!=null){
String[] split = line.split(",");
if (split.length>2)
thirdColumn.add(Double.parseDouble(split[2]));
}
By the end of the while loop, you should have the thirdColumn ArrayList ready and populated with the required data.
The assumption is made that your data set has the following standard format.
String,Integer,Double
So naturally a split by a comma should give a String array of length 3, Where the String at index 2 contains your third column data.
I am appending the data to the last row of a csv. I wanted to delete the existing row and then rewrite it with the appended element. Is there any way of deleting the row in csv? I am using opencsv to read and the write the file. I tried using CSVIterator class. However, it seems the iterator does not support the remove() operation.
Here is the code that I tried:
static String[] readLastRecord(File outputCSVFile) throws WAGException {
checkArgument(outputCSVFile != null, "Output CSV file cannot be null");
FileReader fileReader = null;
CSVReader csvFileReader = null;
CSVIterator csvIterator = null;
String[] csvLastRecord = null;
try {
fileReader = new FileReader(outputCSVFile);
csvFileReader = new CSVReader(fileReader, ',', '\'',
csvRowCount - 1);
csvIterator = new CSVIterator(csvFileReader);
while (csvIterator.hasNext()) {
csvLastRecord = csvIterator.next();
csvIterator.remove();
}
} catch (IOException ioEx) {
throw new WAGException(
WAGInputExceptionMessage.FILE_READ_ERR.getMessage());
} finally {
try {
if (csvFileReader != null)
csvFileReader.close();
} catch (IOException ioEx) {
throw new WAGException(
WAGInputExceptionMessage.FILE_CLOSE_ERR.getMessage());
}
}
return csvLastRecord;
}
i just found an answer. Hope it helps.
You need to read the csv, add elements to the list string, remove specific row from it with allelements.remove(rowNumber) and then write the list string back to the csv file.
The rowNumber is an int with row number.
CSVReader reader2 = new CSVReader(new FileReader(filelocation));
List<String[]> allElements = reader2.readAll();
allElements.remove(rowNumber);
FileWriter sw = new FileWriter(filelocation);
CSVWriter writer = new CSVWriter(sw);
writer.writeAll(allElements);
writer.close();
Look at this example from opencsv opencsv example
use unset to remove the row in csv
function readCSV($csvFile){
$file_handle = fopen($csvFile, 'r');
while (!feof($file_handle) ) {
$line_of_text[] = fgetcsv($file_handle, 1024);
}
fclose($file_handle);
return $line_of_text;
}
$csvFile1 = '../build/js/snowcem.csv';
$csv1 = readCSV($csvFile1);
//specified row number want to delete on place of $id
unset($csv1[$id]);
$file = fopen("../build/js/snowcem.csv","w");
foreach ($csv1 as $file1) {
$result = [];
array_walk_recursive($file1, function($item) use (&$result) {
$item = '"'.$item.'"';
$result[] = $item;
});
fputcsv($file, $result);
}
fclose($file);
I have a text file with 300 lines or so. And the format is like:
Name Amount Unit CountOfOrder
A 1 ml 5000
B 1 mgm 4500
C 4 gm 4200
// more data
I need to read the text file line by line because each line of data should be together for further processing.
Now I just use string array for each line and access the data by index.
for each line in file:
array[0] = {data from the 'Name' column}
array[1] = {data from the 'Amount' column}
array[2] = {data from the 'Unit' column}
array[3] = {data from the 'CountOfOrder' column}
....
someOtherMethods(array);
....
However, I realized that if the text file changes its format (e.g. switch two columns, or insert another column), it would break my program (accessing through index might be wrong or even cause exception).
So I would like to use the title as reference to access each column. Maybe HashMap is a good option, but since I have to keep each line of data together, if I build a HashMap for each line, that would be too expensive.
Does anyone have any thought on this? Please help!
you only need a single hash map to map your column names to the proper column index. you fill the arrays by indexing with integers as you did before, to retrieve a column by name you'd use array[hashmap.get("Amount")].
You can read the file using opencsv.
CSVReader reader = new CSVReader(new FileReader("yourfile.txt"), '\t');
List<String[]> lines = reader.readAll();
The fist line contains the headers.
you can read each line of the file and assuming that the first line of the file has the column header you can parse that line to get all the names of the columns.
String[] column_headers = firstline.split("\t");
This will give you the name of all the columns now you just read through splitting on tabs and they will all line up.
You could do something like this:
BufferedReader in = new BufferedReader(new InputStreamReader(
new FileInputStream(FILE)));
String line = null;
String[] headers = null;
String[] data = null;
Map<String, List<String>> contents = new HashMap<String, List<String>>();
if ((line = in.readLine()) != null) {
headers = line.split("\t");
}
for(String h : headers){
contents.put(h, new ArrayList<String>());
}
while ((line = in.readLine()) != null) {
data = line.split("\t");
if(data.length != headers.length){
throw new Exception();
}
for(int i = 0; i < data.length; i++){
contents.get(headers[i]).add(data[i]);
}
}
It would give you flexibility, and would only require making the map once. You can then get the data lists from the map, so it should be a convenient data structure for the rest of your program to use.
This will give you individual list of columns.
public static void main(String args[]) throws FileNotFoundException, IOException {
List<String> headerList = new ArrayList<String>();
List<String> column1 = new ArrayList<String>();
List<String> column2 = new ArrayList<String>();
List<String> column3 = new ArrayList<String>();
List<String> column4 = new ArrayList<String>();
int lineCount=0;
BufferedReader br = new BufferedReader(new FileReader("file.txt"));
try {
StringBuilder sb = new StringBuilder();
String line = br.readLine();
String tokens[];
while (line != null) {
tokens = line.split("\t");
if(lineCount != 0)
{
int count = 0;
column1.add(tokens[count]); ++count;
column2.add(tokens[count]); ++count;
column3.add(tokens[count]); ++count;
column4.add(tokens[count]); ++count;
continue;
}
if(lineCount==0){
for(int count=0; count<tokens.length; count++){
headerList.add(tokens[count]);
lineCount++;
}
}
}
} catch (IOException e) {
} finally {
br.close();
}
}
using standard java.util.Scanner
String aa = " asd 9 1 3 \n d -1 4 2";
Scanner ss = new Scanner(aa);
ss.useDelimiter("\n");
while ( ss.hasNext()){
String line = ss.next();
Scanner fs = new Scanner(line);
System.out.println( "1>"+ fs.next()+" " +fs.nextInt() +" " +fs.nextLong()+" " +fs.nextBigDecimal());
}
using a bunch of hashmap's is ok...i won't be afraid ;)
if you need to process a lot of data...then try to translate your problem into a dataprocessing transformation
for example:
read all of you data into a hashmap's, but store them in a database using some JPA implementation....then you can go round'a'round your data ;)\