[Update]Problem is different, nothing about csv file format. Question
would be "While using File Writer to write a csv file last few records
are missing.
In my java application i need to append more than 65535 rows in a csv file. but it only writes 65535 rows in a sheet. I haven't used any libraries. some final records missing. how to resolve this..........
public void writeSubmission(){
try {
writer = new FileWriter("res/sample.csv");
writer.append("PhraseId");
writer.append(',');
writer.append("Sentiment");
writer.append('\n');
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public void readTestData(){
String path="res/test.tsv";
Calculation cal=new Calculation();
int counter=0;
try {
BufferedReader bReader = new BufferedReader(new FileReader(path));
String line;
writeSubmission();
bReader.readLine();
while ((line = bReader.readLine()) != null) {
String datavalue[] = line.split("\t");
writer.append(datavalue[0]);
writer.append(',');
try {
double value=cal.calculate(datavalue[2]);
System.out.println(value);
String val;
if(value<-0.4)
{
val="0";
}
else if(value>-0.4 && value<-0.1)
{
val="1";
}
else if(value>-0.1 && value<+0.1)
{
val="2";
}
else if(value>+0.1 && value<+0.40)
{
val="3";
}
else{
val="4";
}
counter++;
writer.append(val);
} catch (Exception e) {
// TODO Auto-generated catch block
System.out.println(e);
}
writer.append('\n');
}
System.out.println(counter);
bReader.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
System.out.println(e);
} catch (IOException e){
e.printStackTrace();
System.out.println(e);
}
}
I think your issue is probably that the tool you're opening up the CSVs with on the other end doesn't want more than 65535 rows, not that Java's doing anything wrong. It's a bug on the other end, not your Java code, almost certainly. (FileWriter wouldn't care at all about 65535 lines, for example.)
If you're using Excel 2003, for example, you'd see this issue: How to get around 64k row limit in Excel
Finally i fixed the error.File Writer should be flush after used in the code push existing stream. OR should be close the writer it automatically flush and close
The flush method flushes the output stream and forces any buffered
output bytes to be written out. The general contract of flush is that
calling it is an indication that, if any bytes previously written have
been buffered by the implementation of the output stream, such bytes
should immediately be written to their intended destination.
writer = new FileWriter("res/sample.csv");
writer.flush();
writer.close();
Related
I am trying to convert English words from a text file to a new file that translates the words into pig Latin. Everything translates the way it should when it is simply printed to the console but the issue I am having is that only the last line from the initial file appears on the new one.
public static void newFile(String pigLatin) {
OutputStream os = null;
try {
os = new FileOutputStream(new File("/Users/amie/Documents/inputnewnew.pig.txt"));
os.write(pigLatin.getBytes(), 0, pigLatin.length());
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
os.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
By default FileOutputStream is overriding the existing file. What you need to do is to use another constructor with append parameter
FileOutputStream(String name, boolean append)
like
os = new FileOutputStream(new File("/Users/...", true))
Take a look at the reference
class HelloWorld {
public static void main(String args[]) {
File file = new File("d://1.mp4");
FileInputStream fr = null;
FileOutputStream fw = null;
byte a[] = new byte[(int) file.length()];
try {
fr = new FileInputStream(file);
fw = new FileOutputStream("d://2.mp4");
fr.read(a);
fw.write(a);
fw.write(a);
fw.write(a);
fw.write(a);
fw.write(a);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
fr.close();
fw.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
Here i write fw.write(a) five times, the size of the file increases to 5x but the original 1.mp4 and copy 2.mp4 both have same length i.e. 3:30 minutes ?
Simply duplicating the bytes of certain files does not necessarily mean it simply duplicates things when inspecting them with software. For example, the video player might read the data until some terminal is encountered and not look forward. This terminal would then exist at the end of the first file data block.
You could open the new file with a hex editor and check if you can see the data of the original video file five times in a row.
FileOutputStream fooStream = new FileOutputStream("FilePath", false);
This will overwrite the content and the size of the file created will be same size as of original file.
I created a new file roomChecker which is empty. Now when I read it, it throws me an EOFException which is undesirable. Instead I want it to see that, if file is empty then it would run other two functions that are in if(roomFeed.size() == 0) condition. I could write this statement in EOFException catch clause; but that's not what I want to do because then every time when the file will be read and reaches end of file it will execute those functions. Instead when the file has some data it should do what is specified in else.
File fileChecker = new File("roomChecker.ser");
if(!fileChecker.exists()) {
try {
fileChecker.createNewFile();
} catch (IOException e) {
e.printStackTrace();
System.out.println("Unable to create new File");
}
}
try(FileInputStream fis = new FileInputStream("roomChecker.ser"); ObjectInputStream ois = new ObjectInputStream(fis)) {
roomFeed = (List<roomChecker>) ois.readObject();
System.out.println("End of read");
if(roomFeed.size() == 0) {
System.out.println("your in null if statement");
defaultRoomList();
uploadAvailableRooms();
} else {
for(int i=0; i<roomNumber.size(); i++) {
for(int j=0; j<roomFeed.size(); i++) {
if((roomNumber.get(i)).equals(roomFeed.get(i).getRoomNumSearch())){
System.out.println("Reach Dead End for now");
} else {
defaultRoomList();
uploadAvailableRooms();
}
}
}
}
} catch (IOException ioe) {
ioe.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
All this:
if(!fileChecker.exists()) {
try {
fileChecker.createNewFile();
} catch (IOException e) {
e.printStackTrace();
System.out.println("Unable to create new File");
}
}
is a complete waste of time, and it is one of two possible causes for your empty file problem. Creating a file just so you can open it and then get a different problem instead of coping correctly with the original problem of the file not being there isn't a rational strategy. Instead, you should do this:
if (fileChecker.isFile() && fileChecker.length() == 0) {
// file is zero length: bail out
}
and, in the following code, this:
try(FileInputStream fis = new FileInputStream(fileChecker); ObjectInputStream ois = new ObjectInputStream(fis)) {
// ...
}
catch (FileNotFoundException exc) {
// no such file ...
}
// other catch blocks as before.
Of course you can still get EOFException if you read the file to its end, or if the file is incomplete, and you still need to handle that.
i am getting out of memory error in asynctask which loop to stringbuilder . My target for using this to download image from server and store inside my sd card.My code as below :
HttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
HttpPost httppost = new HttpPost(severPath);
httppost.setEntity(params[0]);
System.out.println("executing request " + httppost.getRequestLine());
HttpResponse response = null;
try {
response = httpclient.execute(httppost);
} catch (ClientProtocolException e6) {
// TODO Auto-generated catch block
e6.printStackTrace();
} catch (IOException e6) {
// TODO Auto-generated catch block
e6.printStackTrace();
}
String output;
System.out.println("Output from Server .... \n");
BufferedReader br = null;
try {
br = new BufferedReader(
new InputStreamReader((response.getEntity().getContent())));
} catch (IllegalStateException e5) {
// TODO Auto-generated catch block
e5.printStackTrace();
} catch (IOException e5) {
// TODO Auto-generated catch block
e5.printStackTrace();
}
OutputStreamWriter outputStreamWriter = null;
try {
outputStreamWriter = new OutputStreamWriter(context.openFileOutput("LargeImages.txt", context.MODE_PRIVATE));
} catch (FileNotFoundException e6) {
// TODO Auto-generated catch block
e6.printStackTrace();
}
int i = 0;
StringBuilder builder = new StringBuilder();
String Result = "";
try {
for (String line = null; (line = br.readLine()) != null ; ) {
builder.append(line.toString());
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
outputStreamWriter.close();
i am getting out of memory allocation error. please help. i try many method but also not getting the right.
if you are downloading an image, then you should not use Reader/Writer/StringBuilder to store it's content. Because the file is binary content will be scrambled because of the character encoding used by Reader/Writer classes.
Try using InputStream/OutputStream and store the content directly to sdcard without storing it in memory.
Try out the below code:
InputStream in = response.getEntity().getContent();
OutputStream out = context.openFileOutput("LargeImages.txt", context.MODE_PRIVATE);
byte b[] = new byte[4096];
int i;
while ((i = in.read(b)) >= 0) {
out.write(b, 0, i);
}
There may be two problems.
The first - the cycle for (String line = null; (line = br.readLine()) != null ; ) is not terminated properly. Try to find it out by opening a small file(e.g. with 10 lines total).
The second - it's actually a memory insufficient case. Probably it's not the best idea to get image via strings as images may be very heavy and creating a plenty of Strings causes natural memory error. Try to find another approach.
I don't see code that is actually writing to the output stream. Shouldn't there be a line before the close, that is like outputStreamWriter.print(builder)?
About your question. Instead of collecting the whole data in memory in a StringBuilder and than write it at once, you should write directly each line you get within your for-loop. You don't need the StringBuilder at all. Here's a code snippet:
try {
for (String line = br.readLine(); line != null; line = br.readLine()) {
outputStreamWriter.append(line);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return;
}
Three more remarks:
When you get an Exception you should also stop the action, e.g. return from your method. Your code above would print the Stacktrace (which is definitely helpful) but would then continue, which would be not so helpful. Just add return after each printstackTrace.
There's still a chance that one line is too long for memory, but the risk is minimized.
Is the data you download binary image or text? You name it image but you download text. Please be aware that there's a difference between bytes and characters (encoded with character set) and stay within what you actually receive.
I want to know the offset of every line present in a text file.
For now I have tried,
path=FileSystems.getDefault().getPath(".",filename);
br=Files.newBufferedReader(path_doc_title_index_path, Charset.defaultCharset());
int offset=0; //offset of first line.
String strline=br.readline();
offset+=strline.length()+1; //offset of second line
In this way I can loop through entire file to know offset of begining of lines in entire text file. But if I use RandomAccessFile to seek through file and access a line using offset calulated by above method then I found myself in the middle of some line. That is it seems that offset are not correct.
What's wrong? Is this method incorrect to calculate offset? Any better and fast methods please?
Your code will only work for ASCII encoded text. Since some characters need more than one byte, you have to change following line
offset += strline.length() + 1;
to
offset += strline.getBytes(Charset.defaultCharset()).length + 1;
As stated in my comments below your question, you have to specifiy the correct encoding of your file. E.g. Charset.forName("UTF-8") here and also where you initialize your BufferedReader.
Apparently, this gives me the expected result. In the following program I print out each line of a file through a set of offsets that I collect through the BufferedReader. Is this your case?
public static void main(String[] args) {
File readFile = new File("/your/file/here");
BufferedReader reader = null;
try
{
reader = new BufferedReader( new FileReader(readFile) );
}
catch (IOException ioe)
{
System.err.println("Error: " + ioe.getMessage());
}
List<Integer> offsets=new ArrayList<Integer>(); //offset of first line.
String strline;
try {
strline = reader.readLine();
while(strline!=null){
offsets.add(strline.length()+System.getProperty("line.separator").length()); //offset of second line
strline = reader.readLine();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
RandomAccessFile raf = new RandomAccessFile(readFile, "rw");
for(Integer offset : offsets){
try {
raf.seek(offset);
System.out.println(raf.readLine());
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}