Would you please tell me why does this character " ÿ " appears at the end of my output file. (I use try/catch)
File f1 = new File("C:/Users/NetBeansProjects/QuestionOne/input.txt");
File f2 = new File("C:/Users/NetBeansProjects/QuestionOne/output.txt");
fin = new FileInputStream(f1);
fout = new FileOutputStream(f2);
do {
i = fin.read();
fout.write(i);
} while (i != -1);
The code copies the file content but it ends it with this strange character.
Do I have to include the entire code?
Thanks.
The method fin.read() returns -1 when there's nothing left to read; but you're actually writing that -1 to fout, even though it didn't occur in fin. It shows up as that ÿ character.
One way to write your loop to avoid this problem is
while((i = fin.read()) != -1 ){
fout.write(i);
}
try to use the new Files class introduced in Java 7
public static void copyFile( File from, File to ) throws IOException {
Files.copy( from.toPath(), to.toPath() );
}
Because the last fin.read() will not read anything. According to JavaDoc it will return -1, because of this your fout.write(i) will write that -1. You would do something like this, to correct this behavior:
do {
i = fin.read();
if (i>-1) //note the extra line
fout.write(i);
} while (i != -1);
Or change the do .. while into a while .. do call.
I suggest you should also look at the new NIO API that will perform much better then transferring one character at a time.
File sourceFile = new File("C:/Users/NetBeansProjects/QuestionOne/input.txt");
File destFile = new File("C:/Users/NetBeansProjects/QuestionOne/output.txt");
FileChannel source = null;
FileChannel destination = null;
try {
if (!destFile.exists()) {
destFile.createNewFile();
}
source = new FileInputStream(sourceFile).getChannel();
destination = new FileOutputStream(destFile).getChannel();
destination.transferFrom(source, 0, source.size());
} catch (IOException e) {
System.err.println("Error while trying to transfer content");
//e.printStackTrace();
} finally {
try{
if (source != null)
source.close();
if (destination != null)
destination.close();
}catch(IOException e){
System.err.println("Not able to close the channels");
//e.printStackTrace();
}
}
or you can simply check if(i != -1) before fout
do {
i = fin.read();
if(i != -1)
fout.write(i);
} while (i != -1);
Related
My application streams twitter data and writes them to files.
while(true){
Status status = queue.poll();
if (status == null) {
Thread.sleep(100);
}
if(status!=null){
list.add(status);
}
if(list.size()==10){
FileOutputStream fos = null;
ObjectOutputStream out = null;
try {
String uuid = UUID.randomUUID().toString();
String filename = "C:/path/"+topic+"-"+uuid+".ser";
fos = new FileOutputStream(filename);
out = new ObjectOutputStream(fos);
out.writeObject(list);
tweetsDownloaded += list.size();
if(tweetsDownloaded % 100==0)
System.out.println(tweetsDownloaded+" tweets downloaded");
// System.out.println("File: "+filename+" written.");
out.close();
} catch (IOException e) {
e.printStackTrace();
}
list.clear();
}
I have this code which gets data from files.
while(true){
File[] files = folder.listFiles();
if(files != null){
Arrays.sort(//sorting...);
//Here we manage each single file, from data-load until the deletion
for(int i = 0; i<files.length; i++){
loadTweets(files[i].getAbsolutePath());
//TODO manageStatuses
files[i].delete();
statusList.clear();
}
}
}
The method loadTweets() does the following operations:
private static void loadTweets(String filename) {
FileInputStream fis = null;
ObjectInputStream in = null;
try{
fis = new FileInputStream(filename);
in = new ObjectInputStream(fis);
statusList = (List<Status>) in.readObject();
in.close();
}
catch(IOException | ClassNotFoundException ex){
ex.printStackTrace();
}
}
Unfortunately, I don't know why sometimes it throws a
EOFException
when running this line
statusList = (List<Status>) in.readObject();
Anybody knows how I can solve this? Thank you.
I've seen that you're passing the file correctly with the getAbsolutePath() based on a previous question of yours
From what I've read that can be a couple of things, one of them the file being null.
Explaining this idea, you might have written the file but something caused the file to have nothing inside, this might cause an EOFException. The file in fact exists it's just empty
EDIT
Try to enclose the code in while(in.available() > 0)
It would look like this
private static void loadTweets(String filename) {
FileInputStream fis = null;
ObjectInputStream in = null;
try{
fis = new FileInputStream(filename);
in = new ObjectInputStream(fis);
while(in.available() > 0) {
statusList = (List<Status>) in.readObject();
}
in.close();
}
catch(IOException | ClassNotFoundException ex){
ex.printStackTrace();
}
}
Found out what was necessary to solve this. Thanks to #VGR's comment, I thought to pause the executing thread for 0.2 seconds if the file has been created less than a second ago.
if(System.currentTimeMillis()-files[i].lastModified()<1000){
Thread.sleep(200);
This prevents the exception and the application works now fine.
I am using some sort of tail -f implementation to tail a file for changes (pretty much like this ). For this I am using a RandomAccessFile, periodically check if the file length has increased and if so, seek and read the new lines (everything happening in a separate thread of the FileTailer).
Now, everything is working as expected on Windows, but I tested my program on Linux and it does not work as expected. Here is the run()-method of the FileTailer class. Specifically where it fails on linux is the part where file.seek(filePointer) gets called and then file.readLine(), of which the latter surprisingly returns NULL (although the filePointer gets incremented correctly if I append content to the file getting tailed at runtime).
public void run() {
// The file pointer keeps track of where we are in the file
long filePointer = 0;
// Determine start point
if(startAtBeginning){
filePointer = 0;
}
else {
filePointer = logfile.length();
}
try {
// Start tailing
tailing = true;
RandomAccessFile file = new RandomAccessFile(logfile, "r");
while(tailing) {
// Compare the length of the file to the file pointer
long fileLength = logfile.length();
System.out.println("filePointer = " + filePointer + " | fileLength = " + fileLength);
if(fileLength < filePointer) {
// Log file must have been rotated or deleted;
// reopen the file and reset the file pointer
file = new RandomAccessFile(logfile, "r");
filePointer = 0;
}
if(fileLength > filePointer) {
// There is data to read
file.seek(filePointer);
String line = file.readLine();
System.out.println("new line = " + line);
while(line != null){
if(!line.isEmpty())
try {
fireNewFileLine(line);
} catch (ParseException e) {
e.printStackTrace();
}
line = file.readLine();
}
filePointer = file.getFilePointer();
}
// Sleep for the specified interval
sleep(sampleInterval);
}
// Close the file that we are tailing
file.close();
}
catch(InterruptedException | IOException e){
e.printStackTrace();
}
}
Like I said, everything is working as it should on Windows, but on Linux the String variable "line" is NULL after it should have been filled with the newly appended line, so fireNewLine gets called on NULL and everything goes to crap.
Does anyone have an idea why this happens on Linux Systems?
You don't need all this, or RandomAccessFile. You are always at the end of the file. All you need is this:
public void run() {
try {
// Start tailing
tailing = true;
BufferedReader reader = new BufferedReader(new FileReader(logfile));
String line;
while (tailing) {
while ((line = reader.readLine() != null) {
System.out.println("new line = " + line);
if(!line.isEmpty()) {
try {
fireNewFileLine(line);
} catch (ParseException e) {
e.printStackTrace();
}
}
}
// Sleep for the specified interval
sleep(sampleInterval);
}
// Close the file that we are tailing
reader.close();
} catch(InterruptedException | IOException e) {
e.printStackTrace();
}
}
with maybe some provision for reopening the file.
E&OE
I have implement following code to copy file(binary file)
code
private void copyFileWithChannels(File aSourceFile, File aTargetFile) {
log("Copying files with channels.");
FileChannel inChannel = null;
FileChannel outChannel = null;
FileInputStream inStream = null;
FileOutputStream outStream = null;
try {
inStream = new FileInputStream(aSourceFile);
inChannel = inStream.getChannel();
outStream = new FileOutputStream(aTargetFile);
outChannel = outStream.getChannel();
long bytesTransferred = 0;
while(bytesTransferred < inChannel.size()){
bytesTransferred += inChannel.transferTo(0, inChannel.size(), outChannel);
}
}
catch(FileNotFoundException e){
log.error("FileNotFoundException in copyFileWithChannels()",e);
}
catch (IOException e) {
log.error("IOException in copyFileWithChannels()",e);
}
catch (Exception e) {
log.error("Exception in copyFileWithChannels()",e);
}
finally {
try{
if (inChannel != null) inChannel.close();
if (outChannel != null) outChannel.close();
if (inStream != null) inStream.close();
if (outStream != null) outStream.close();
}catch(Exception e){
log.error("Exception in copyFileWithChannels() while closing the stream",e);
}
}
}
I have test code with one zip file. when i verify file I found that file which generated is corrupt(size was increased).
Source zip file is about 9GB.
Try this:
while(bytesTransferred < inChannel.size()){
bytesTransferred += inChannel.transferTo(bytesTransferred, inChannel.size() - bytesTransferred, outChannel);
}
Also, I would refer to IOUtils implementation, as a reference
https://github.com/apache/commons-io/blob/master/src/main/java/org/apache/commons/io/FileUtils.java
specifically
private static void doCopyFile(final File srcFile, final File destFile, final boolean preserveFileDate)
The transferTo method's first argument gives the position from which to transfer, not relative to where the stream left off, but relative to the start of the file. Since you put 0 there it will always transfer from the start of the file. So that line needs to be
bytesTransferred += inChannel.transferTo(bytesTransferred , inChannel.size(), outChannel);
mavarazy mentioned in his answer he's not sure if you need a loop when using inChannel.size(), since the expectation is that if you supply the whole size it will copy the entire file. However, the actual transfer might be less than the requested number of bytes if the output channel's buffer has less room available. So you do need the loop as in his second code snippet.
Unless you have a good reason best to use Files.copy(Path, Path, CopyOption...).
the first one is from a book which looks very cryptic/complex to me ,second one is the way i have seen people around me write including me :) ,also for the first style eclipse is showing that the catch "IOException openx" block is handing the exception for part where read and write is happening viz
while ((len = is.read(buf)) >= 0)
out.write(buf, 0, len);
.Does it mean catch "IOException iox" is useless code?
first style.
File file = new File("hsjdhsaj");
InputStream is = null;
try {
URL url = new URL("");
is = url.openStream();
OutputStream out = new FileOutputStream(file);
try {
byte[] buf = new byte[4096];
int len;
while ((len = is.read(buf)) >= 0)
out.write(buf, 0, len);
} catch (IOException iox) {
} finally {
try {
out.close();
} catch (IOException closeOutx) {
}
}
} catch (FileNotFoundException fnfx) {
} catch (IOException openx) {
} finally {
try {
if (is != null)
is.close();
} catch (IOException closeInx) {
}
}
second style.
File file = new File("hsjdhsaj");
InputStream is = null;
OutputStream out = null;
try {
URL url = new URL("");
is = url.openStream();
out = new FileOutputStream(file);
byte[] buf = new byte[4096];
int len;
while ((len = is.read(buf)) >= 0)
out.write(buf, 0, len);
} catch (FileNotFoundException fnfx) {
} catch (IOException openx) {
} finally {
try {
if (out != null)
out.close();
if (is != null)
is.close();
} catch (IOException closeInx) {
}
}
if i put
try {
if (is != null) is.close();
} catch (IOException closeInx) { }
try {
if (out != null) out.close();
} catch (IOException closeInx) { }
in finally block for second style then are they both same
With the second style is is not closed when out.close() throws an exception. The first style does not have this problem.
In both code snippets often exceptions are silently swallowed. This can cause maintenance nightmares. Something does not work and you have no idea why.
The first approach is more correct. Your second approach has a bug if an exception is thrown when calling out.close, because you'll never call is.close().
Of course, both of them are ugly. You should be using a utility method like IOUtils.closeQuietly() to close your streams. And you shouldn't be swallowing exceptions.
Yes, the first one is more correct, but also very ugly. That's why Java 7 improved a lot exception handling. In you case you can use Try-with-Resources :
The new syntax allows you to declare resources that are part of the try block. What this means is that you define the resources ahead of time and the runtime automatically closes those resources (if they are not already closed) after the execution of the try block.
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(
new URL("http://www.yoursimpledate.server/").openStream())))
{
String line = reader.readLine();
SimpleDateFormat format = new SimpleDateFormat("MM/DD/YY");
Date date = format.parse(line);
} catch (ParseException | IOException exception) {
// handle I/O problems.
}
Take a look at Working with Java SE 7 Exception Changes
How we check whether zip file corrupted or valid Zip file before going to extract it
my code`
import java.io.IOException;
import java.io.OutputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
public void unzip() {
FileInputStream fin = null;
ZipInputStream zin = null;
OutputStream fout = null;
File outputDir = new File(_location);
File tmp = null;
try {
fin = new FileInputStream(_zipFile);
zin = new ZipInputStream(fin);
ZipEntry ze = null;
while ((ze = zin.getNextEntry()) != null) {
Log.d("Decompress", "Unzipping " + ze.getName());
if (ze.isDirectory()) {
dirChecker(ze.getName());
} else {
tmp = File.createTempFile( "decomp", ".tmp", outputDir );
fout = new BufferedOutputStream(new FileOutputStream(tmp));
DownloadFile.copyStream( zin, fout, _buffer, BUFFER_SIZE );
zin.closeEntry();
fout.close();
fout = null;
tmp.renameTo( new File(_location + ze.getName()) );
tmp = null;
}
}
zin.close();
zin = null;
} catch (IOException e) {
throw new RuntimeException(e);
} finally {
if ( tmp != null ) { try { tmp.delete(); } catch (Exception ignore) {;} }
if ( fout != null ) { try { fout.close(); } catch (Exception ignore) {;} }
if ( zin != null ) { try { zin.closeEntry(); } catch (Exception ignore) {;} }
if ( fin != null ) { try { fin.close(); } catch (Exception ignore) {;} }
}
}
`
this work fine with valid zipfile, but invalid zipfile it doesen't throw any exception not produce anything, but i need to confirm the validity of zip file before going to unzip it
I think it's pretty much useless for checking if the zip file is corrupted for two reasons:
Some zip files contain more bytes than just the zip part. For example, self-extracting archives have an executable part yet they're still valid zip.
The file can be corrupted without changing its size.
So, I suggest calculating the CRC for a guaranteed method of checking for corruption.
A Zip file will be valid as long as it has its zip entry catalog present. If you take the zip command, it will allow you to browse so long as the catalog is present. A parameter used for testing actually performs an extraction and CRC check.
What you can do is extract and do a CRC check on a temporary folder using the temp dir creation facility of Java. Then if it is all successful, commit the extract by copying the files from the temp dir to the final destination.