I have the following code which successfully copies a file. However, there are two problems with it:
The System.out.println() immediately after the progressBar.setValue() does not print intervals between 0 and 100 (just prints "0" till the end where it prints "100")
Besides the fact that the value for the progress bar might be wrong somehow due to question #1, in the actual code I am making other visual changes too, but they don't show until the entire file is processed. I thought the FileInputStream/FileOutputStream functions were non-blocking. How can I change the following code so that the progress bar is in fact updated during the operation?
startJob method:
private void startJob(File inFile, File outFile) {
long offset = 0;
int numRead = 0;
byte[] bytes = new byte[8192];
long fileLength = inFile.length();
Boolean keepGoing = true;
progressBar.setValue(0);
try {
inputStream = new FileInputStream(inFile);
outputStream = new FileOutputStream(outFile, false);
System.out.println("Total file size to read (in bytes) : " + inputStream.available());
} catch (FileNotFoundException err) {
inputStream = null;
outputStream = null;
err.printStackTrace();
} catch (IOException err) {
inputStream = null;
outputStream = null;
err.printStackTrace();
}
if (inputStream != null && outputStream != null) {
while (keepGoing) {
try {
numRead = inputStream.read(bytes);
outputStream.write(bytes, 0, numRead);
} catch (IOException err) {
keepGoing = false;
err.printStackTrace();
}
if (numRead > 0) {
offset += numRead;
}
if (offset >= fileLength) {
keepGoing = false;
}
progressBar.setValue(Math.round(offset / fileLength) * 100);
System.out.println(Integer.toString(Math.round(offset / fileLength) * 100));
}
}
if (offset < fileLength) {
//error
} else {
//success
}
try {
inputStream.close();
outputStream.close();
} catch (IOException err) {
err.printStackTrace();
}
}
I suspect you are calling your lengthy method from the EDT. Remove your operation from the EDT by placing it in it's own Runnable for instance and then call
SwingUtilities.invokeLater(new Runnable() {
#Override
public void run() {
progressBar.setValue(value);
// or any other GUI changes you want to make
}
});
Otherwise, your operation blocks the EDT until it is done, and with the EDT blocked no events like repaint etc will can be processed -> no GUI changes visible until the end.
The value of expression Math.round(offset / fileLength) will always equal 0 (zero), because offset < fileLength.
UPD:
If you want to do this calculation correctly, you have to change it to:
Math.round(((double)offset / (double)fileLength) * 100)
Related
Question at the bottom
I'm using netty to transfer a file to another server.
I limit my file-chunks to 1024*64 bytes (64KB) because of the WebSocket protocol. The following method is a local example what will happen to the file:
public static void rechunck(File file1, File file2) {
FileInputStream is = null;
FileOutputStream os = null;
try {
byte[] buf = new byte[1024*64];
is = new FileInputStream(file1);
os = new FileOutputStream(file2);
while(is.read(buf) > 0) {
os.write(buf);
}
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
} finally {
try {
if(is != null && os != null) {
is.close();
os.close();
}
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
}
}
}
The file is loaded by the InputStream into a ByteBuffer and directly written to the OutputStream.
The content of the file cannot change while this process.
To get the md5-hashes of the file I've wrote the following method:
public static String checksum(File file) {
InputStream is = null;
try {
is = new FileInputStream(file);
MessageDigest digest = MessageDigest.getInstance("MD5");
byte[] buffer = new byte[8192];
int read = 0;
while((read = is.read(buffer)) > 0) {
digest.update(buffer, 0, read);
}
return new BigInteger(1, digest.digest()).toString(16);
} catch(IOException | NoSuchAlgorithmException e) {
Controller.handleException(Thread.currentThread(), e);
} finally {
try {
is.close();
} catch(IOException e) {
Controller.handleException(Thread.currentThread(), e);
}
}
return null;
}
So: just in theory it should return the same hash, shouldn't it? The problem is that it returns two different hashes that do not differ with every run.. file size stays the same and the content either.
When I run the method once for in: file-1, out: file-2 and again with in: file-2 and out: file-3 the hashes of file-2 and file-3 are the same! This means the method will properly change the file every time the same way.
1. 58a4a9fbe349a9e0af172f9cf3e6050a
2. 7b3f343fa1b8c4e1160add4c48322373
3. 7b3f343fa1b8c4e1160add4c48322373
Here is a little test that compares all buffers if they are equivalent. Test is positive. So there aren't any differences.
File file1 = new File("controller/templates/Example.zip");
File file2 = new File("controller/templates2/Example.zip");
try {
byte[] buf1 = new byte[1024*64];
byte[] buf2 = new byte[1024*64];
FileInputStream is1 = new FileInputStream(file1);
FileInputStream is2 = new FileInputStream(file2);
boolean run = true;
while(run) {
int read1 = is1.read(buf1), read2 = is2.read(buf2);
String result1 = Arrays.toString(buf1), result2 = Arrays.toString(buf2);
boolean test = result1.equals(result2);
System.out.println("1: " + result1);
System.out.println("2: " + result2);
System.out.println("--- TEST RESULT: " + test + " ----------------------------------------------------");
if(!(read1 > 0 && read2 > 0) || !test) run = false;
}
} catch (IOException e) {
e.printStackTrace();
}
Question: Can you help me chunking the file without changing the hash?
while(is.read(buf) > 0) {
os.write(buf);
}
The read() method with the array argument will return the number of files read from the stream. When the file doesn't end exactly as a multiple of the byte array length, this return value will be smaller than the byte array length because you reached the file end.
However your os.write(buf); call will write the whole byte array to the stream, including the remaining bytes after the file end. This means the written file gets bigger in the end, therefore the hash changed.
Interestingly you didn't make the mistake when you updated the message digest:
while((read = is.read(buffer)) > 0) {
digest.update(buffer, 0, read);
}
You just have to do the same when you "rechunk" your files.
Your rechunk method has a bug in it. Since you have a fixed buffer in there, your file is split into ByteArray-parts. but the last part of the file can be smaller than the buffer, which is why you write too many bytes in the new file. and that's why you do not have the same checksum anymore. the error can be fixed like this:
public static void rechunck(File file1, File file2) {
FileInputStream is = null;
FileOutputStream os = null;
try {
byte[] buf = new byte[1024*64];
is = new FileInputStream(file1);
os = new FileOutputStream(file2);
int length;
while((length = is.read(buf)) > 0) {
os.write(buf, 0, length);
}
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
} finally {
try {
if(is != null)
is.close();
if(os != null)
os.close();
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
}
}
}
Due to the length variable, the write method knows that until byte x of the byte array, only the file is off, then there are still old bytes in it that no longer belong to the file.
I read data from a source location via BufferedInputStream and I pass the data to a destination using BufferedOutputStream. The problem I'm having is that sometimes my thread never exits the while loop because of starving on the bandwidth. Any ideas? Here's the code:
BufferedInputStream bis = new BufferedInputStream(sourceConnection.getInputStream());
BufferedOutputStream request = new BufferedOutputStream(destConnection.getOutputStream());
request.write(content.getBytes("UTF-8"));
boolean eof = false;
byte[] input = new byte[4096];
while ((length = bis.read(input)) != -1) {
request.write(input, 0, length);
request.flush();
}
request.close();
bis.close();
So to fix the issue I did a few things. I set the entire transfer process in a separate thread using an executor with a timeout
ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
final Future<Boolean> handler = executor.submit(new Callable<Boolean>() {
#Override
public Boolean call() throws Exception {
return processTransfer();
}
});
success = handler.get(10, TimeUnit.MINUTES);
That way if the transfer takes longer than 10 minutes, it exits with an exeption. The second thing was change the original code to detect starvation:
long lastDataRecvTime = System.currentTimeMillis();
byte[] input;
while (true) {
if(System.currentTimeMillis() - lastDataRecvTime >= 5 * 60 * 1000) {
throw new RuntimeException("Nothing received for 5 minutes. Transfer starved. Exiting");
}
int availableBuf = request.getAvailableBufferSize();
if(availableBuf == 0) {
request.flush();
continue;
}
input = new byte[Math.min(4096, availableBuf)];
int length = bis.read(input);
if (length == -1)
break;
if(length == 0) {
try { Thread.sleep(1); } catch (Exception ignored){}
continue;
}
lastDataRecvTime = System.currentTimeMillis();
request.write(input, 0, length);
}
request.flush();
request.close();
bis.close();
Thanks for the help
I have a thread that is constantly reading data from an InputStream. The InputStream data is coming from a Bluetooth socket. Previously, I wasn't using the if(mmInStream.available() > 0) around the InputStream read statement and when the bluetooth socket went away (someone turned off the device), the mmInStream.read would throw a IOException and then I could process my disconnection logic. What is the best way to determine when a disconnect has occurred?
First byte of 0xEE tells me its the leader of the data packet and the second tells me the length to read.
public void run() {
byte[] tempBuffer = new byte[1024];
byte[] buffer = null;
int byteRead=0;
long timeout=0;
long wait=100;
while (true) {
try {
timeout = System.currentTimeMillis() + wait;
if(mmInStream.available() > 0) {
while((mmInStream.available() > 0) && (tempBuffer[0] != (byte) 0xEE) && (System.currentTimeMillis() < timeout)){
byteRead = mmInStream.read(tempBuffer, 0, 1);
}
if(tempBuffer[0] == (byte) 0xEE){
timeout = System.currentTimeMillis() + wait;
while(byteRead<2 && (System.currentTimeMillis() < timeout)){
byteRead += mmInStream.read(tempBuffer, 1, 1);
}
}
timeout = System.currentTimeMillis() + wait;
while((byteRead<tempBuffer[1]) && (System.currentTimeMillis() < timeout)){
byteRead += mmInStream.read(tempBuffer, byteRead, tempBuffer[1]-byteRead);
}
}
if(byteRead > 0){
//do something with the bytes read in
}
}
catch (IOException e) {
bluetoothConnectionLost();
break;
}
}
}
You don't need all this malarkey with available(). Just set a read timeout with setSoTimeout, read, detect read returning -1, use the count returned by read if > 0 rather than assuming the buffer got filled, catch SocketTimeoutException to detect read timeouts, and catch IOException to detect other breakages.
After a look at the documentation, I think it's like this:
public void run() {
byte[] tempBuffer = new byte[1024];
int byteRead = 0;
while (true) {
try {
bytesRead = mmInStream.read(tempBuffer, 0, tempBuffer.length);
if (bytesRead < 0)
// End of stream.
break;
// Do something with the bytes read in. There are bytesRead bytes in tempBuffer.
} catch (IOException e) {
bluetoothConnectionLost();
break;
}
}
}
I think it's like this:
void fun(){
isOpen = true;
try{
InputStream stream = socket.getInputStream();
while(isOpen){
byte[] buf = new byte[8];
int pos = stream.read(buf);
if (pos < 0) {
throw new IOException();
}
//dosomething...
}
}catch(IOException e) {
isOpen = false;
}finally{
//do dispose here
}
}
I have troubles with my program when i need to send Strings from my server bluetooth-socket to my client bluetooth-socket.
Everything works fine as long as I am only sending one String at a time (for example chatting) but if I need to write more Strings at a short period of time (to interchange informations), the Strings will not get seperated from the client code. For example if I'm sending "FirstUser" and right after that "SecondUser" the client does not read "FirstUser" and then "SecondUser". It will read "FirstUserSecondUser". How can I avoid this behaviour?
Edit: If I let the Thread sleep before it is able to send a new message, it reads the right strings but this solution is not working fine for my need.
Server-Code: sending to all clients(edited)
public synchronized void sendToAll(String message)
{
try {
Thread.sleep(100);
} catch (InterruptedException e1) {
e1.printStackTrace();
}
publishProgress(message);
for(OutputStream writer:outputList) {
try {
writer.write(message.getBytes());
writer.flush();
} catch (IOException e) {
System.out.println("Some-Error-Code");
}
}
}
Server-Code: reading from a client:
public void run() {
String nachricht;
int numRead;
byte[] buffer = new byte[1024];
while (runningFlag)
{
try {
if((numRead = inputStream.read(buffer)) >= 0) {
nachricht = new String(buffer, 0, numRead);
serverThread.handleMessage(nachricht);
}
}
catch (IOException e) {
this.cancel();
e.printStackTrace();
}
}
}
Client-Code: reading from server(edited)
#Override
protected Void doInBackground(Integer... ints) {
String nachricht = new String();
byte[] buffer = new byte[1024];
int numRead;
while (runningFlag)
{
try {
if(((numRead = inputStream.read(buffer)) >= 0)) {
nachricht = new String(buffer, 0, numRead);
publishProgress(nachricht);
}
}
catch (IOException e) {
clientGame.finish();
e.printStackTrace();
}
}
return null;
}
Client-Code: writing to server
public synchronized void write(String nachricht)
{
try {
Thread.sleep(100);
} catch (InterruptedException e1) {
e1.printStackTrace();
}
try {
outputStream.write(nachricht.getBytes());
outputStream.flush();
} catch (IOException e) {
this.cancel();
e.printStackTrace();
}
}
I appreciate every little help :) .
You need to encapsulate your data item to avoid concatenation.
It means that you have to write and read a whole data item before continuing.
You should have some utility methods to do that instead of directly using methods of OutputStream and InputStream :
public static void writeItem(OutputStream out, String s) throws IOException
{
// Get the array of bytes for the string item:
byte[] bs = s.getBytes(); // as bytes
// Encapsulate by sending first the total length on 4 bytes :
// - bits 7..0 of length
out.write(bs.length); // modulo 256 done by write method
// - bits 15..8 of length
out.write(bs.length>>>8); // modulo 256 done by write method
// - bits 23..16 of length
out.write(bs.length>>>16); // modulo 256 done by write method
// - bits 31..24 of length
out.write(bs.length>>>24); // modulo 256 done by write method
// Write the array content now:
out.write(bs); // Send the bytes
out.flush();
}
public static String readItem(InputStream in) throws IOException
{
// first, read the total length on 4 bytes
// - if first byte is missing, end of stream reached
int len = in.read(); // 1 byte
if (len<0) throw new IOException("end of stream");
// - the other 3 bytes of length are mandatory
for(int i=1;i<4;i++) // need 3 more bytes:
{
int n = in.read();
if (n<0) throw new IOException("partial data");
len |= n << (i<<3); // shift by 8,16,24
}
// Create the array to receive len bytes:
byte[] bs = new byte[len];
// Read the len bytes into the created array
int ofs = 0;
while (len>0) // while there is some byte to read
{
int n = in.read(bs, ofs, len); // number of bytes actually read
if (n<0) throw new IOException("partial data");
ofs += n; // update offset
len -= n; // update remaining number of bytes to read
}
// Transform bytes into String item:
return new String(bs);
}
Then you use these methods both for server & client to read and write your String items.
What I currently have
I'm currently trying to create a little download manager in Java and I have a problem with writing the loaded bytes in a file. I'm using a DataOutputStream to write the byte-array which I read from a DataInputStream. Here is the class I created to do that:
public class DownloadThread extends Thread{
private String url_s;
private File datei;
public DownloadThread(String url_s, File datei){
this.url_s = url_s;
this.datei = datei;
}
public void run(){
// Connect:
int size = 0;
URLConnection con = null;
try {
URL url = new URL(url_s);
con = url.openConnection();
size = con.getContentLength();
// Set Min and Max of the JProgressBar
prog.setMinimum(0);
prog.setMaximum(size);
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
// Download:
if (con != null || size != 0){
byte[] buffer = new byte[4096];
DataInputStream down_reader = null;
// Output:
DataOutputStream out = null;
try {
out = new DataOutputStream(new FileOutputStream(datei));
} catch (FileNotFoundException e1) {
e1.printStackTrace();
}
// Load:
try {
down_reader = new DataInputStream(con.getInputStream());
int byte_counter = 0;
int tmp = 0;
int progress = 0;
// Read:
while (true){
tmp = down_reader.read(buffer);
// Check for EOF
if (tmp == -1){
break;
}
out.write(buffer);
out.flush();
// Set Progress:
byte_counter += tmp;
progress = (byte_counter * 100) / size;
prog.setValue( byte_counter );
prog.setString(progress+"% - "+byte_counter+"/"+size+" Bytes");
}
// Check Filesize:
prog.setString("Checking Integrity...");
if (size == out.size()){
prog.setString("Integrity Check passed!");
} else {
prog.setString("Integrity Check failed!");
System.out.println("Size: "+size+"\n"+
"Read: "+byte_counter+"\n"+
"Written: "+out.size() );
}
// ENDE
} catch (IOException e) {
e.printStackTrace();
} finally{
try {
out.close();
down_reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
// Clean Up...
load.setEnabled(true);
try {
this.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
This is currently an inner-class and the prog-Object is a JProgressBar from it's mother-class, so it can be accessed directly.
Example
I'm trying to download the Windows .exe Version of the TinyCC, which should be 281KB size. The file i downloaded with my download manager is 376KB big.
The Output from the Script looks like this:
Size: 287181
Read: 287181
Written: 385024
So it seems that the read bytes match the file-size but there are more bytes written. What am I missing here?
This is wrong:
out.write(buffer);
It should be
out.write(buffer, 0, tmp);
You need to specify how many bytes to write, a read doesn't always read a full buffer.
Memorize this. It is the canonical way to copy a stream in Java.
int count;
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}