I am trying to read a huge file where a new line is indicated by no space, comma, new line character, or anything.
Example: line1element1, line1element2, line1element3, line2element1, line2element2, and so on..
The file is a csv and I am reading it like following:
public static void main(String[] args) throws Exception {
ArrayList<String> list = new ArrayList<>();
String element;
String filename = "E:\\csv.csv";
Scanner scanner = new Scanner(new File(filename));
scanner.useDelimiter(",");
for (int i = 0; i < 50; i++) {
element = scanner.next();
list.add(element);
}
System.out.print(list);
}
This causes issues because the element50 in a line gets combined with element51, although it should be a new line.
Use a BufferedReader for this:
String filename = "E:\\csv.csv";
BufferedReader fileReader = null;
//Delimiter used in CSV file
final String DELIMITER = ",";
String line = "";
//Create the file reader
fileReader = new BufferedReader(new FileReader(filename ));
//Read the file line by line
while ((line = fileReader.readLine()) != null)
{
//Get all tokens available in line
String[] tokens = line.split(DELIMITER);
for(String token : tokens)
{
//Print all tokens
System.out.println(token);
}
}
USe a BufferedReader, not Scanner
File f= ...;
BufferedReader br = new BufferedReader(new FileReader(f));
String line;
while ((line = br.nextLine()) != null) {
String[] columns = line.split(",");
}
Why not use CSVParser from Apache Commons or OpenCSV?
Examples here:
OpenCSV Example
Apache Commons example
If you insist on doing this manually, use BufferedReader as the other comments mention.
From your description it seems your file does not have headers for each column. Use uniVocity-parsers to do this for you - it's 3 times faster than Commons CSV & OpenCSV and packed with features.
// you have many configuration options here - check the tutorial. By default values are trimmed and blank lines skipped.
CsvParserSettings settings = new CsvParserSettings();
CsvParser parser = new CsvParser(settings);
List<String[]> allRows = parser.parseAll(new FileReader(new File("/path/to/your/file.csv")));
Disclosure: I am the author of this library. It's open-source and free (Apache V2.0 license).
Related
Python has a method lines = f.read().splitlines() by which we can read a file into list. Do we have a similar method in Java?
You can use Scanner and then read line by line and insert the line into a list
Scanner fileScanner= new Scanner(new File("yourfile.txt");
List<String> lines=new ArrayList();
while(scanner.hasNext()){
String line = scanner.next();
lines.add(line);
}
File in= new File(new URI("file://server/folder/text.txt"));
BufferedReader br = new BufferedReader(in);
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
Note: The file path has to be a URI that can't contain spaces.
Files conains static metods https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html
new String(Files.readAllBytes(Paths.get(path)))
.split(System.lineSeparator);
I've got a Spring MVC app with a file upload capability. Files are passed to the controller as MultipartFile from which it's easy to get an InputStream. I'm uploading zip files that contain CSVs and I'm struggling to find a way to open the CSVs and read them a line at a time. There are plenty of examples on the 'net of reading into a fixed sizes buffer. I've tried this, but the buffers don't concatenate very well and it soon gets out of sync and uses a lot of memory:
ZipEntry entry = input.getNextEntry();
while(entry != null)
{
if (entry.getName().matches("Data/CSV/[a-z]{0,1}[a-z]{0,1}.csv"))
{
final String fullPath = entry.getName();
final String filename = fullPath.substring(fullPath.lastIndexOf('/') + 1);
visitor.startFile(filename);
final StringBuilder fileContent = new StringBuilder();
final byte[] buffer = new byte[1024];
while (input.read(buffer) > 0)
fileContent.append(new String(buffer));
final String[] lines = fileContent.toString().split("\n");
for(String line : lines)
{
final String[] columns = line.split(",");
final String postcode = columns[0].replace(" ", "").replace("\"", "");
if (columns.length > 3)
visitor.location(postcode, "", "");
}
visitor.endFile();
}
entry = input.getNextEntry();
}
There must be a better way that actually works.
Not clear if this suits your need, but have you tried opencsv (http://opencsv.sourceforge.net)? Their example is really intuitive:
CSVReader reader = new CSVReader(new FileReader("yourfile.csv"));
String [] nextLine;
while ((nextLine = reader.readNext()) != null) {
// nextLine[] is an array of values from the line
System.out.println(nextLine[0] + nextLine[1] + "etc...");
}
For your case, all you will need is to wrap the zipped file stream into a buffered reader and pass the reader to create a CSVReader and use it:
FileInputStream fis = new FileInputStream(file);
GZIPInputStream gis = new GZIPInputStream(fis);
InputStreamReader isr = new InputStreamReader(gis);
BufferedReader br = new BufferedReader(isr);
CSVReader reader = new CSVReader(br);
You could use a BufferedReader which includes the convenient readLine() method and wont load the entire contents of the file into memory e.g.
BufferedReader in = new BufferedReader(new InputStreamReader(input), 1024);
String line=null;
while((line=br.readLine())!=null) {
String[] columns = line.split(",");
//rest of your code
}
I have a text file like this:
Item 1
Item 2
Item 3
I need to be able to read each "Item X" into a string and ideally store all the strings as a vector / ArrayList.
I tried:
InputStream is = new FileInputStream("file.txt");
is.read(); //looped for every line of text
but that seems to only handle integers.
Thanks
You have several answers here, the easiest would be to us a Scanner (in java.util).
It has several convenience methods like nextLine() and next() and nextInt(), so you could simply do the following:
Scanner scanner = new Scanner(new File("file.txt"));
List<String> text = new ArrayList<String>();
while (scanner.hasNextLine()) {
text.add(scanner.nextLine());
}
Alternatively you could use a BufferedReader (in java.io):
BufferedReader reader = new BufferedReader(new FileReader("file.txt"));
List<String> text = new ArrayList<String>();
for (String line; (line = reader.readLine()) != null; ) {
text.add(line);
}
However Scanners are generally easier to work with.
You should use FileUtils to do this. It has a method named readLines
public static List<String> readLines(File file, Charset encoding) throws IOException
Reads the contents of a file line by line to a List of Strings. The file is always closed.
See #BackSlash's comment above to see how you're using InputStream.read() wrong.
#BackSlash also mentioned you can use java.nio.file.Files#readAllLines but only if you're using Java 1.7 or later.
You could use Java 7's Files#readAllLines. A short one-liner and no 3rd party library imports necessary :)
List<String> lines =
Files.readAllLines(Paths.get("file.txt"), StandardCharsets.UTF_8);
BufferedReader br = new BufferedReader(new FileReader("file.txt"));
try {
StringBuilder sb = new StringBuilder();
String line = br.readLine();
String [] tmp ;
while (line != null) {
sb.append(line);
tmp = line.Split(" ");
line = br.readLine();
}
String everything = sb.toString();
} finally {
br.close();
}
Scanner scan = new Scanner(new FileInputStream("file.txt"));
scan.nextLine();
I am trying to read a BufferedReader that reads in a file containing records separated by commas. I would like to split each string (or record) in between two commas, strip the double quotes, and put each of those into an index of a String array. For example:
say I have this line in the file:
("0001", "00203", "82409" (newline)
"0002", "00204", "82500" (newline)
etc.)
I want to put 0001 into a String array[1],
I want 00203 into String array[2],
and so on....
The following code traverses the file, putting all records in column two into String array[2]. This means, after I execute the code below, if I do System.out.println (arr[2]), it will print 00203 and 00204, whereas I would like array[2] to be 00203 and array[5] to be 00204.
Here is my code:
public String[] getArray(String source) {
FileInputStream fileinput = new FileInputStream(source);
GZIPInputStream gzip = new GZIPInputStream(fileinput);
InputStreamReader inputstream = new InputStreamReader(gzip);
BufferedReader bufr = new BufferedReader(inputstream);
String str = null;
String[] arr = null;
while((str = bufr.readLine()) != null) {
arr = str.replace("\"", "").split("\\s*,\\s*");
}
return arr;
Commons CSV was designed for your specific use case. Let's not reinvent the wheel, the code below will result in a GZipped CSV being parsed into fields and lines and seems to be what you're trying to do.
public String[][] getInfo() throws IOException {
final CSVParser parser = new CSVParser(new FileReader(new InputStreamReader(new GZIPInputStream(fileinput)), CSVFormat.DEFAULT.withIgnoreSurroundingSpaces(true));
String[][] result = parser.nextRecord().values();
return result;
}
Few of these modifications should work for you.
public String[] getArray(String source) {
FileInputStream fileinput = new FileInputStream(source);
GZIPInputStream gzip = new GZIPInputStream(fileinput);
InputStreamReader inputstream = new InputStreamReader(gzip);
BufferedReader bufr = new BufferedReader(inputstream);
String str = null;
List<String> numbers = new LinkedList<String>;
while((str = bufr.readLine()) != null) {
String[] localArr = str.split(",");
for(String intString : localArr){
numbers.add(intString.trim());
}
}
return arr;
Have you tried using the scanner class as well as scanner.nextInt(). you do not need to do striping then.
Scanner s = new Scanner(inputstream);
ArrayList<String> list = new ArrayList<String>();
while (s.hasNextInt())
list.add(s.nextInt());
String[] arr = list.toArray(new String[list.size()]);
Not tested:
arr = str.replaceAll("\"", "").replaceAll("(","").replaceAll(")","").split(",");
How would I read a .txt file in Java and put every line in an array when every lines contains integers, strings, and doubles? And every line has different amounts of words/numbers.
I'm a complete noob in Java so sorry if this question is a bit stupid.
Thanks
Try the Scanner class which no one knows about but can do almost anything with text.
To get a reader for a file, use
File file = new File ("...path...");
String encoding = "...."; // Encoding of your file
Reader reader = new BufferedReader (new InputStreamReader (
new FileInputStream (file), encoding));
... use reader ...
reader.close ();
You should really specify the encoding or else you will get strange results when you encounter umlauts, Unicode and the like.
Easiest option is to simply use the Apache Commons IO JAR and import the org.apache.commons.io.FileUtils class. There are many possibilities when using this class, but the most obvious would be as follows;
List<String> lines = FileUtils.readLines(new File("untitled.txt"));
It's that easy.
"Don't reinvent the wheel."
The best approach to read a file in Java is to open in, read line by line and process it and close the strea
// Open the file
FileInputStream fstream = new FileInputStream("textfile.txt");
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));
String strLine;
//Read File Line By Line
while ((strLine = br.readLine()) != null) {
// Print the content on the console - do what you want to do
System.out.println (strLine);
}
//Close the input stream
fstream.close();
To learn more about how to read file in Java, check out the article.
Your question is not very clear, so I'll only answer for the "read" part :
List<String> lines = new ArrayList<String>();
BufferedReader br = new BufferedReader(new FileReader("fileName"));
String line = br.readLine();
while (line != null)
{
lines.add(line);
line = br.readLine();
}
Common used:
String line = null;
File file = new File( "readme.txt" );
FileReader fr = null;
try
{
fr = new FileReader( file );
}
catch (FileNotFoundException e)
{
System.out.println( "File doesn't exists" );
e.printStackTrace();
}
BufferedReader br = new BufferedReader( fr );
try
{
while( (line = br.readLine()) != null )
{
System.out.println( line );
}
#user248921 first of all, you can store anything in string array , so you can make string array and store a line in array and use value in code whenever you want. you can use the below code to store heterogeneous(containing string, int, boolean,etc) lines in array.
public class user {
public static void main(String x[]) throws IOException{
BufferedReader b=new BufferedReader(new FileReader("<path to file>"));
String[] user=new String[500];
String line="";
while ((line = b.readLine()) != null) {
user[i]=line;
System.out.println(user[1]);
i++;
}
}
}
This is a nice way to work with Streams and Collectors.
List<String> myList;
try(BufferedReader reader = new BufferedReader(new FileReader("yourpath"))){
myList = reader.lines() // This will return a Stream<String>
.collect(Collectors.toList());
}catch(Exception e){
e.printStackTrace();
}
When working with Streams you have also multiple methods to filter, manipulate or reduce your input.
For Java 11 you could use the next short approach:
Path path = Path.of("file.txt");
try (var reader = Files.newBufferedReader(path)) {
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
}
Or:
var path = Path.of("file.txt");
List<String> lines = Files.readAllLines(path);
lines.forEach(System.out::println);
Or:
Files.lines(Path.of("file.txt")).forEach(System.out::println);