I have written a java code which is working perfectly when I am executing locally. But gives out of memory exception when invoked from informatica. I am reading a file and splitting the contents and replacing the contents of the same file.
public static void main(String[] args) throws IOException {
String delims = "#####";
StringBuilder sb=null;
String filepath = "FILE_PATH";
try {
FileReader in = new FileReader(filepath);
BufferedReader br = new BufferedReader(in);
sb= new StringBuilder();
sb.append(br.readLine());
in.close();
}
catch (Exception ex) {
}
StringTokenizer st = new StringTokenizer(sb.toString(), delims);
try {
FileWriter fw = new FileWriter(filepath, false);
while (st.hasMoreElements()) {
fw.write(st.nextToken() + System.getProperty("line.separator"));
}
fw.flush();
fw.close();
} catch (Exception ex) {
}
}
Now, the exception occurs while reading the contents of a big data file. For small data files, it is working fine.
Related
I coded up a method which reads a text file, when I run it it works, but when I execute the jar, I get the NoSuchFileException error.
Here is my code:
private static final String textFile = "textFile.txt";
public readText() {
try {
regex = new String(Files.readAllBytes(Paths
.get(textFile)))
.replaceAll("\\r", "").split("\n");
} catch (final IOException e) {
e.printStackTrace();
}
}
Could someone point out where am I going wrong or what changes I'm supposed to make?
I figured out a way:
try {
InputStream in = getClass().getResourceAsStream(textFile);
StringBuilder content = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
String line = reader.readLine();
while(line!=null){
content.append(line).append(System.lineSeparator());
line = reader.readLine();
}
regex = content.toString()
.replaceAll("\\r", "").split("\n");
} catch (final IOException e) {
e.printStackTrace();
}
Below I have the following code to read in a file and go through it line by line.. This is using java's BufferedReader class. That I am fine with.
String filename = "C:\\test.txt"
String line = null;
FileReader fileReader = new FileReader(filename);
BufferedReader bufferedReader = new BufferedReader(fileReader);
try {
while (((line = bufferedReader.readLine()) != null)) {
//do the following....
}
} catch (IOException) {
e.printStackTrace();
}
However I want to now start using InputStreamReader in Spring / Java. I have the below code written but I am unsure how I can step through my file line by line. Really confused over this part. Anyone have any ideas or know how this can be done?
String filepath= "C:\\test.txt"
File filename= new File(filepath);
try {
InputStream fileInputStream = new BOMInputStream(new fileInputStream(filename));
// now want to step through the file, line by line..
} catch (IOException) {
e.printStackTrace();
}
Thanks
This is how you can read your input file byte by byte using InputStreamReader.
char[] chars = new char[100];
try {
InputStream inputStream = new FileInputStream("C:\\test.txt");
InputStreamReader inputStreamReader = new InputStreamReader(inputStream,"UTF-8");
inputStreamReader.read(chars);
System.out.println(new String(chars).trim());
} catch (IOException e) {
e.printStackTrace();
}
Check this out -
String filename = "C:\\test.txt"
String line = null;
FileInputStream fileInputStream = new FileInputStream(filename);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(fileInputStream));
try {
while (((line = bufferedReader.readLine()) != null)) {
//do the following....
}
} catch (IOException) {
e.printStackTrace();
}
public static void main(String[] args) {
try (BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream("c:\\test.txt")))) {
reader.lines().forEach(line -> {
// do what you want with the line
});
} catch (IOException e) {
throw new RuntimeException(e);
}
}
I'm trying to webget some bz2 files from Wikipedia, I don't care whether they are save as bz2 or unpacked, since I can unzip them locally.
When I call:
public static void getZip(String theUrl, String filename) throws IOException {
URL gotoUrl = new URL(theUrl);
try (InputStreamReader isr = new InputStreamReader(new BZip2CompressorInputStream(gotoUrl.openStream())); BufferedReader in = new BufferedReader(isr)) {
StringBuffer sb = new StringBuffer();
String inputLine;
// grab the contents at the URL
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine + "\r\n");
}
// write it locally
Wget.createAFile(filename, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
I get a part of the unzipped file, never more than +- 883K.
When I don't use the BZip2CompressorInputStream, like:
public static void get(String theUrl, String filename) throws IOException {
try {
URL gotoUrl = new URL(theUrl);
InputStreamReader isr = new InputStreamReader(gotoUrl.openStream());
BufferedReader in = new BufferedReader(isr);
StringBuffer sb = new StringBuffer();
String inputLine;
// grab the contents at the URL
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine);// + "\r\n");
}
// write it locally
Statics.writeOut(filename, false, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
I get a file of which the size is the same as it suppose to (compared to the KB not B). But also a message that that the zipped file is damaged, also when using byte [] instead of readLine(), like:
public static void getBytes(String theUrl, String filename) throws IOException {
try {
char [] cc = new char[1024];
URL gotoUrl = new URL(theUrl);
InputStreamReader isr = new InputStreamReader(gotoUrl.openStream());
BufferedReader in = new BufferedReader(isr);
StringBuffer sb = new StringBuffer();
// grab the contents at the URL
int n = 0;
while (-1 != (n = in.read(cc))) {
sb.append(cc);// + "\r\n");
}
// write it locally
Statics.writeOut(filename, false, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
Finally, when I bzip2 the inputstream and outputstream, I get a valid bzip2 file, but of the size like the first one, using:
public static void getWriteForBZ2File(String urlIn, final String filename) throws CompressorException, IOException {
URL gotoUrl = new URL(urlIn);
try (final FileOutputStream out = new FileOutputStream(filename);
final BZip2CompressorOutputStream dataOutputStream = new BZip2CompressorOutputStream(out);
final BufferedInputStream bis = new BufferedInputStream(gotoUrl.openStream());
final CompressorInputStream input = new CompressorStreamFactory().createCompressorInputStream(bis);
final BufferedReader br2 = new BufferedReader(new InputStreamReader(input))) {
String line = null;
while ((line = br2.readLine()) != null) {
dataOutputStream.write(line.getBytes());
}
}
}
So, how do I get the entire bz2 file, in either bz2 format or unzipped?
A bz2 file contains bytes, not characters. You can't read it as if it contained characters, with a Reader.
Since all you want to do is download the file and save it locally, all you need is
Files.copy(gotoUrl.openStream(), Paths.get(fileName));
I am trying to read values from CSV file which is present in package com.example.
But when i run code with the following syntax:
DataModel model = new FileDataModel(new File("Dataset.csv"));
It says:
java.io.FileNotFoundException:Dataset.csv
I have also tried using:
DataModel model = new FileDataModel(new File("/com/example/Dataset.csv"));
Still not working.
Any help would be helpful.
Thanks.
If this is the FileDataModel from org.apache.mahout.cf.taste.impl.model.file then it can't take an input stream and needs just a file. The problem is you can't assume the file is available to you that easily (see answer to this question).
It might be better to read the contents of the file and save it to a temp file, then pass that temp file to FileDataModel.
InputStream initStream = getClass().getClasLoader().getResourceAsStream("Dataset.csv");
//simplistic approach is to put all the contents of the file stream into memory at once
// but it would be smarter to buffer and do it in chunks
byte[] buffer = new byte[initStream.available()];
initStream.read(buffer);
//now save the file contents in memory to a temporary file on the disk
//choose your own temporary location - this one is typical for linux
String tempFilePath = "/tmp/Dataset.csv";
File tempFile = new File(tempFilePath);
OutputStream outStream = new FileOutputStream(tempFile);
outStream.write(buffer);
DataModel model = new FileDataModel(new File(tempFilePath));
...
public class ReadCVS {
public static void main(String[] args) {
ReadCVS obj = new ReadCVS();
obj.run();
}
public void run() {
String csvFile = "file path of csv";
BufferedReader br = null;
String line = "";
String cvsSplitBy = ",";
try {
br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
// Do stuff here
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
System.out.println("Done");
}
}
CSV file which is present in package com.example
You can use getResource() or getResourceAsStream() to access the resource from within the package. For example
InputStream is = getClass().getResourceAsStream("/com/example/Dataset.csv");//uses absolute (package root) path
BufferedReader br = new BufferedReader(new InputStreamReader(is));
//read from BufferedReader
(note exception handling and file closing are omitted above for brevity)
Ok, forgive my beginner-ness and please tell me how I can output my text from "before.txt" into a fresh new file called "after". Obviously I have altered the text along the way to make it lower-case and eliminate non alphabetic characters.
import java.io.*;
public class TextReader {
public void openFile() throws IOException {
try {
// Read in the file
BufferedReader br = new BufferedReader(
new FileReader(
new File("before.txt")));
String currentLine = br.readLine();
currentLine = currentLine.toLowerCase();
currentLine = currentLine.replaceAll("[A-Z]", "");
br.close(); // Close br to prevent resource leak
}
// Exception if the file is not in the path specified
catch (Exception e) {
System.out.println("Error: File not found");
}
}
public void writeFile() throws IOException {
BufferedWriter output = new BufferedWriter(new FileWriter("/WS3Ex3/after.txt"));
output.write("before.txt");
output.close();
}
}
What about this
public void openFile() throws IOException {
try {
// Read in the file
BufferedReader br = new BufferedReader(
new FileReader(
new File("before.txt")));
String currentLine = br.readLine();
currentLine = currentLine.toLowerCase();
currentLine = currentLine.replaceAll("[A-Z]", "");
br.close(); // Close br to prevent resource leak
writeFile(currentLine);
}
// Exception if the file is not in the path specified
catch (Exception e) {
System.out.println("Error: File not found");
}
}
public void writeFile(String text) throws IOException {
BufferedWriter output = new BufferedWriter(new FileWriter("/WS3Ex3/after.txt"));
output.write(text);
output.close();
}
}
Let me guess, is this a school assignment?
Try this:
public void ReadAndWrite() throws IOException {
try {
// Read in the file
BufferedWriter output = new BufferedWriter(new FileWriter("/WS3Ex3/after.txt"));
BufferedReader br = new BufferedReader(
new FileReader(
new File("before.txt")));
String currentLine;
while((currentLine = br.readLine()) != NULL){
currentLine = currentLine.toLowerCase();
currentLine = currentLine.replaceAll("[A-Z]", "");
output.write(currentLine);
}
br.close(); // Close br to prevent resource leak
output.close();
}
// Exception if the file is not in the path specified
catch (Exception e) {
System.out.println("Error: File not found");
}
}