I have been trying to understand how Java ByteBuffer works. My aim is to write a string to ByteBuffer and read it back. I want to understand how ByteBuffer properties like Limit, Capacity, Remaining, Position gets affected due to read/write operations.
Below is my test program (removed import statements for brevity).
public class TestBuffer {
private ByteBuffer bytes;
private String testStr = "Stackoverflow is a great place to discuss tech stuff!";
public TestBuffer() {
bytes = ByteBuffer.allocate(1000);
System.out.println("init: " + printBuffer());
}
public static void main(String a[]) {
TestBuffer buf = new TestBuffer();
try {
buf.writeBuffer();
} catch (IOException e) {
e.printStackTrace();
}
buf.readBuffer();
}
// write testStr to buffer
private void writeBuffer() throws IOException {
byte[] b = testStr.getBytes();
BufferedInputStream in = new BufferedInputStream(new ByteArrayInputStream(b));
in.read(bytes.array());
in.close();
System.out.println("write: " + printBuffer());
}
// read buffer data back to byte array and print
private void readBuffer() {
bytes.flip();
byte[] b = new byte[bytes.position()];
bytes.position(0);
bytes.get(b);
System.out.println("data read: " + new String(b));
System.out.println("read: " + printBuffer());
}
public String printBuffer() {
return "ByteBuffer [limit=" + bytes.limit() + ", capacity=" + bytes.capacity() + ", position="
+ bytes.position() + ", remaining=" + bytes.remaining() + "]";
}
}
Output
init: ByteBuffer [limit=1000, capacity=1000, position=0, remaining=1000]
write: ByteBuffer [limit=1000, capacity=1000, position=0, remaining=1000]
data read:
read: ByteBuffer [limit=0, capacity=1000, position=0, remaining=0]
As you can see, there is no data after calling readBuffer() and no change in value if various fields after write and read operations.
Update
Below is the working piece of code from Android Screen Library which I was originally trying to understand
// retrieve the screenshot
// (this method - via ByteBuffer - seems to be the fastest)
ByteBuffer bytes = ByteBuffer.allocate (ss.width * ss.height * ss.bpp / 8);
is = new BufferedInputStream(is); // buffering is very important apparently
is.read(bytes.array()); // reading all at once for speed
bytes.position(0); // reset position to the beginning of ByteBuffer
Please help me to understand this.
Thanks
Your buffer is never filled. bytes.array() simply retrieves the backing byte array. If you write anything to this then the ByteBuffer fields - except the array itself of course - are unaffected. So the position stays at zero.
What you are doing in in.read(bytes.array()) is identical to byte[] tmp = bytes.array() followed by in.read(tmp). Changes to the tmp variable cannot be reflected in the bytes instance. The backing array is changed which may mean that the contents of the ByteBuffer is changed as well. But the offsets into the backing byte array - including the position and limit - aren't.
You should only fill the ByteBuffer using any of the put methods (that do not take an index) such as put(byte[]).
I'll provide a code fragment that may get you thinking on how to handle strings, encodings and character and byte buffers:
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.CharsetEncoder;
import java.nio.charset.CoderResult;
import java.nio.charset.StandardCharsets;
public class TestBuffer {
private static final String testStr = "Stackoverflow is a great place to discuss tech stuff!";
private static final boolean END_OF_INPUT = true;
private ByteBuffer bytes = ByteBuffer.allocate(1000);
public TestBuffer() {
System.out.println("init : " + bytes.toString());
}
public static void main(String a[]) {
TestBuffer buf = new TestBuffer();
buf.writeBuffer();
buf.readBuffer();
}
// write testStr to buffer
private void writeBuffer() {
CharBuffer testBuffer = CharBuffer.wrap(testStr);
CharsetEncoder utf8Encoder = StandardCharsets.UTF_8.newEncoder();
CoderResult result = utf8Encoder.encode(testBuffer, bytes, END_OF_INPUT);
if (result.isError()) {
bytes.clear();
throw new IllegalArgumentException("That didn't go right because " + result.toString());
}
if (result.isOverflow()) {
bytes.clear();
throw new IllegalArgumentException("Well, too little buffer space.");
}
System.out.println("written: " + bytes.toString());
bytes.flip();
}
// read buffer data back to byte array and print
private void readBuffer() {
byte[] b = new byte[bytes.remaining()];
bytes.get(b);
System.out.println("data : " + new String(b, StandardCharsets.UTF_8));
System.out.println("read : " + bytes.toString());
bytes.clear();
}
}
Note that buffers and streams are really two separate ways of handling sequential data. If you are trying to use both of them at the same time you may be trying to be too clever.
You could also solve this without CharBuffer and ByteBuffer using a byte[] buffer and a StringReader wrapped by a ReaderInputStream.
That Android piece of code completely abuses the ByteBuffer. It should just have created a byte[] and wrapped that, setting the limit to the capacity. Whatever you do, do not use it as an example on ByteBuffer handling. It made my eyes water in disgust. Code like that is a bug waiting to happen.
You are not writting anything in the writeBuffer() method.
You may use something like bytes.put(b).
Although this question has been answered long time ago, let me also supplement some info. here.
There are 2 problems in the writeBuffer() and readBuffer() method separately which leads you failed to get your expected result.
1) writeBuffer() method
As explained by Maarten Bodewes above regarding the nature of bytebuffer's array, you cannot use byteBuffer.array() directly for reading from a Stream in your
Alternatively, if you want to keep testing the relationship between InputStream and ByteBuffer as your sample (which is also a common practice on server side application for handling incoming messages), you would require an additional byte array.
2) readBuffer() method
The original code is good of using an extra byte array for retrieving the context in bytebuffer for printing it.
However, the problem here is the improper use of flip() and position() method.
The flip() method should only be called right before you want to change the state of the bytebuffer from storing context to exporting context. So this method here should appear right before the line of bytes.get(b); instead. In the provided sample, it was too early to call this method before the line byte[] b = new byte[bytes.position()]; , as flip() method would change the position flag of the bytebuffer to 0 while setting the limit flag to current position.
There are no points of setting the bytebuffer's position to 0 explicitly in the sample codes. In case you want to keep storing context to the bytebuffer again at some later time by starting from current position (i.e. without overwriting existing context in it), you should follow this workflow:
2.1 store bytebuffer's current position: i.e. int pos = bytebuffer.position();
2.2 process with the bytebuffer which may affect the position flag of it: e.g. bytebuffer.get(byte[] dst) etc.
2.3 restore bytebuffer's position flag to original value: i.e. bytebuffer.position(pos);
Here I have slightly modified your sample code for achieving what you want to do:
import java.io.BufferedInputStream;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
public class TestBuffer {
private ByteBuffer bytes;
private String testStr = "Stackoverflow is a great place to discuss tech stuff!";
public TestBuffer() {
bytes = ByteBuffer.allocate(1000);
System.out.println("init: " + printBuffer());
}
public static void main(String a[]) {
TestBuffer buf = new TestBuffer();
try {
buf.writeBuffer();
} catch (IOException e) {
e.printStackTrace();
}
buf.readBuffer();
}
// write testStr to buffer
private void writeBuffer() throws IOException {
byte[] b = testStr.getBytes();
BufferedInputStream in = new BufferedInputStream(new ByteArrayInputStream(b));
// in.read(bytes.array());
byte[] dst = new byte[b.length];
in.read(dst);
bytes.put(dst);
in.close();
System.out.println("write: " + printBuffer());
}
// read buffer data back to byte array and print
private void readBuffer() {
//bytes.flip();
byte[] b = new byte[bytes.position()];
//bytes.position(0);
int pos = bytes.position();
bytes.flip(); // bytes.rewind(); could achieve the same result here, use which one depends on whether:
// (1) reading to this bytebuffer is finished and fine to overwrite the current context in this bytebuffer afterwards: can use flip(); or
// (2) just want to tentatively traverse this bytebuffer from the begining to current position,
// and keep writing to this bytebuffer again later from current position.
bytes.get(b);
bytes.position(pos);
System.out.println("data read: " + new String(b));
System.out.println("read: " + printBuffer());
}
public String printBuffer() {
return "ByteBuffer [limit=" + bytes.limit() + ", capacity=" + bytes.capacity() + ", position="
+ bytes.position() + ", remaining=" + bytes.remaining() + "]";
}
}
Related
I am struggling with finding a solution to write my bytes array to a playable AAC audio file.
From my Flutter.io front-end, I am encoding my .aac audio files as a list of UInt8List and sending it to my Spring-Boot server. Then I am able to convert them to a proper bytes array where I then attempt to write it back to a .aac file as seen below:
public void writeToAudioFile(ArrayList<Double> audioData) {
byte[] byteArray = new byte[1024];
Iterator<Double> iterator = audioData.iterator();
System.out.println(byteArray);
while (iterator.hasNext()) {
// for some reason my list came in as a list of doubles
// so I am making sure to get these values back to an int
Integer i = iterator.next().intValue();
byteArray[i] = i.byteValue();
}
try {
File someFile = new File("test.aac");
FileOutputStream fos = new FileOutputStream(someFile);
fos.write(byteArray);
fos.flush();
fos.close();
System.out.println("File created");
} catch (Exception e) {
// TODO: handle exception
System.out.println("Error: " + e);
}
I am able to write my bytes array back to an audio file, however, it is unplayable. So I am wondering if this approach is possible and If my issue does lie in Java.
I have been doing extraneous research and I think that I need to say that this file is a specific type of media file? Or maybe the encoded audio file is corrupt when reaching my server?
Your conversion loop
while (iterator.hasNext()) {
// for some reason my list came in as a list of doubles
// so I am making sure to get these values back to an int
Integer i = iterator.next().intValue();
byteArray[i] = i.byteValue();
}
gets the value i from the iterator, and then tries to write it at the position i in the byteArray, which kind of jumbles your audio bytes in a weird way.
A working function that converts List<Double> to byte[] would look something like this
byte[] inputToBytes(List<Double> audioData) {
byte[] result = new byte[audioData.size()];
for (int i = 0; i < audioData.size(); i++) {
result[i] = audioData.get(i).byteValue();
}
return result;
}
then you could use it in the writeToAudioFile():
void writeToAudioFile(ArrayList<Double> audioData) {
try (FileOutputStream fos = new FileOutputStream("test.aac")) {
fos.write(inputToBytes(audioData));
System.out.println("File created");
} catch (Exception e) {
// TODO: handle exception
System.out.println("Error: " + e);
}
}
This certainly produces the playable file if you have the valid bytes in the audioData. The contents and the extension should be enough for the OS/player to recognize the format.
If this doesn’t work, I would look into the data received to see if it is correct.
I am working a project in which I have to play with some file reading writing tasks. I have to read 8 bytes from a file at one time and perform some operations on that block and then write that block to second file, then repeat the cycle until first file is completely read in chuncks of 8 bytes everytime and the after manipulation the data should be added/appended to the second. However, in doing so, I am facing some problems. Following is what I am trying:
private File readFromFile1(File file1) {
int offset = 0;
long message= 0;
try {
FileInputStream fis = new FileInputStream(file1);
byte[] data = new byte[8];
file2 = new File("file2.txt");
FileOutputStream fos = new FileOutputStream(file2.getAbsolutePath(), true);
DataOutputStream dos = new DataOutputStream(fos);
while(fis.read(data, offset, 8) != -1)
{
message = someOperation(data); // operation according to business logic
dos.writeLong(message);
}
fos.close();
dos.close();
fis.close();
} catch (IOException e) {
System.out.println("Some error occurred while reading from File:" + e);
}
return file2;
}
I am not getting the desired output this way. Any help is appreciated.
Consider the following code:
private File readFromFile1(File file1) {
int offset = 0;
long message = 0;
File file2 = null;
try {
FileInputStream fis = new FileInputStream(file1);
byte[] data = new byte[8]; //Read buffer
byte[] tmpbuf = new byte[8]; //Temporary chunk buffer
file2 = new File("file2.txt");
FileOutputStream fos = new FileOutputStream(file2.getAbsolutePath(), true);
DataOutputStream dos = new DataOutputStream(fos);
int readcnt; //Read count
int chunk; //Chunk size to write to tmpbuf
while ((readcnt = fis.read(data, 0, 8)) != -1) {
//// POINT A ////
//Skip chunking system if an 8 byte octet is read directly.
if(readcnt == 8 && offset == 0){
message = someOperation(tmpbuf); // operation according to business logic
dos.writeLong(message);
continue;
}
//// POINT B ////
chunk = Math.min(tmpbuf.length - offset, readcnt); //Determine how much to add to the temp buf.
System.arraycopy(data, 0, tmpbuf, offset, chunk); //Copy bytes to temp buf
offset = offset + chunk; //Sets the offset to temp buf
if (offset == 8) {
message = someOperation(tmpbuf); // operation according to business logic
dos.writeLong(message);
if (chunk < readcnt) {
System.arraycopy(data, chunk, tmpbuf, 0, readcnt - chunk);
offset = readcnt - chunk;
} else {
offset = 0;
}
}
}
//// POINT C ////
//Process remaining bytes here...
//message = foo(tmpbuf);
//dos.writeLong(message);
fos.close();
dos.close();
fis.close();
} catch (IOException e) {
System.out.println("Some error occurred while reading from File:" + e);
}
return file2;
}
In this excerpt of code, what I did was:
Modify your reading code to include the amount of bytes actually read from the read() method (noted readcnt).
Added a byte chunking system (the processing does not happen until there are at least 8 bytes in the chunking buffer).
Allowed for separate processing of the final bytes (that do not make up a 8 byte octet).
As you can see from the code, the data being read is first stored in a chunking buffer (denoted tmpbuf) until at least 8 bytes are available. This will happen only if 8 bytes are not always available (If 8 bytes are available directly and nothing is chunked, directly process. See "Point A" in code). This is done as a form of optimization to prevent excess array copies.
The chunking system uses offsets which increment every time bytes are written to tmpbuf until it reaches a value of 8 (it will not go over as the Math.min() method used in the assignment of 'chunk' will limit the value). Upon offset == 8, proceed to execute the processing code.
If that particular read produced more bytes than actually processed, continue writing them to tmpbuf, from the beginning again, whilst setting offset appropriately, otherwise set offset to 0.
Repeat cycle.
The code will leave the last few bytes of data that do not fit in an octet in the array tmpbuf with the offset variable indicating how much has actually been written. This data can then be processed separately at point C.
Seems a lot more complicating than it should be, and there probably is a better solution (possibly using existing java library methods), but off the top of my head, this is what I got. Hope this is clear enough for you to understand.
You could use the following, it uses NIO and especially the ByteBuffer class for the long handling. You can of course implement it the standard java way, but since i am a NIO fan, here is a possible solution.
The major problem in your code is that while(fis.read(data, offset, 8) != -1) will read up to 8 bytes, and not always 8 bytes, plus reading in such small portions is not very efficient.
I have put some comments in my code, if something is unclear please leave a comment. My someOperation(...) function just copies the next long value from the buffer.
Update:
added finally block to close the files.
import java.io.File;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.file.StandardOpenOption;
public class TestFile {
static final int IN_BUFFER_SIZE = 1024 * 8;
static final int OUT_BUFFER_SIZE = 1024 *9; // make the out-buffer > in-buffer, i am lazy and don't want to check for overruns
static final int MIN_READ_BYTES = 8;
static final int MIN_WRITE_BYTES = 8;
private File readFromFile1(File inFile) {
final File outFile = new File("file2.txt");
final ByteBuffer inBuffer = ByteBuffer.allocate(IN_BUFFER_SIZE);
final ByteBuffer outBuffer = ByteBuffer.allocate(OUT_BUFFER_SIZE);
FileChannel readChannel = null;
FileChannel writeChannel = null;
try {
// open a file channel for reading and writing
readChannel = FileChannel.open(inFile.toPath(), StandardOpenOption.READ);
writeChannel = FileChannel.open(outFile.toPath(), StandardOpenOption.CREATE, StandardOpenOption.WRITE);
long totalReadByteCount = 0L;
long totalWriteByteCount = 0L;
boolean readMore = true;
while (readMore) {
// read some bytes into the in-buffer
int readOp = 0;
while ((readOp = readChannel.read(inBuffer)) != -1) {
totalReadByteCount += readOp;
} // while
// prepare the in-buffer to be consumed
inBuffer.flip();
// check if there where errors
if (readOp == -1) {
// end of file reached, read no more
readMore = false;
} // if
// now consume the in-buffer until there are at least MIN_READ_BYTES in the buffer
while (inBuffer.remaining() >= MIN_READ_BYTES) {
// add data to the write buffer
outBuffer.putLong(someOperation(inBuffer));
} // while
// compact the in-buffer and prepare for the next read, if we need to read more.
// that way the possible remaining bytes of the in-buffer can be consumed after leaving the loop
if (readMore) inBuffer.compact();
// prepare the out-buffer to be consumed
outBuffer.flip();
// write the out-buffer until the buffer is empty
while (outBuffer.hasRemaining())
totalWriteByteCount += writeChannel.write(outBuffer);
// prepare the out-buffer for writing again
outBuffer.flip();
} // while
// error handling
if (inBuffer.hasRemaining()) {
System.err.println("Truncated data! Not a long value! bytes remaining: " + inBuffer.remaining());
} // if
System.out.println("read total: " + totalReadByteCount + " bytes.");
System.out.println("write total: " + totalWriteByteCount + " bytes.");
} catch (IOException e) {
System.out.println("Some error occurred while reading from File: " + e);
} finally {
if (readChannel != null) {
try {
readChannel.close();
} catch (IOException e) {
System.out.println("Could not close read channel: " + e);
} // catch
} // if
if (writeChannel != null) {
try {
writeChannel.close();
} catch (IOException e) {
System.out.println("Could not close write channel: " + e);
} // catch
} // if
} // finally
return outFile;
}
private long someOperation(ByteBuffer bb) {
// consume the buffer, do whatever you want with the buffer.
return bb.getLong(); // consumes 8 bytes of the buffer.
}
public static void main(String[] args) {
TestFile testFile = new TestFile();
File source = new File("input.txt");
testFile.readFromFile1(source);
}
}
I have been given a task of copying data from a server. I am using BufferedInputStream and output stream to copy the data and I am doing it byte by byte. Even though it is running but It is taking ages to copy the data as some of them are in 100's MBs, so definitely it is not gonna work. Can anyone suggest me any alternate of Byte by Byte copy so that my code can copy file that are in few Hundred MBs.
Buffer is 2048.
Here is how my code look like:
static void copyFiles(SmbFile[] files, String parent) throws IOException {
SmbFileInputStream input = null;
FileOutputStream output = null;
BufferedInputStream buf_input = null;
try {
for (SmbFile f : files) {
System.out.println("Working on files :" + f.getName());
if (f.isDirectory()) {
File folderToBeCreated = new File(parent+f.getName());
if (!folderToBeCreated.exists()) {
folderToBeCreated.mkdir();
System.out.println("Folder name " + parent
+ f.getName() + "has been created");
} else {
System.out.println("exists");
}
copyFiles(f.listFiles(), parent + f.getName());
} else {
input = (SmbFileInputStream) f.getInputStream();
buf_input = new BufferedInputStream(input, BUFFER);
File t = new File(parent + f.getName());
if (!t.exists()) {
t.createNewFile();
}
output = new FileOutputStream(t);
int c;
int count;
byte data[] = new byte[BUFFER];
while ((count = buf_input.read(data, 0, BUFFER)) != -1) {
output.write(data, 0, count);
}
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (input != null) {
input.close();
}
if (output != null) {
output.close();
}
}
}
Here is a link to an excellent post explaining how to use nio channels to make copies of streams. It introduces a helper method ChannelTools.fastChannelCopy that lets you copy streams like this:
final InputStream input = new FileInputStream(inputFile);
final OutputStream output = new FileOutputStream(outputFile);
final ReadableByteChannel inputChannel = Channels.newChannel(input);
final WriteableByteChannel outputChannel = Channels.newChannel(output);
ChannelTools.fastChannelCopy(inputChannel, outputChannel);
inputChannel.close();
outputChannel.close()
Well since you're using a BufferedInputStream, you aren't reading byte by byte, but rather the size of the buffer. You could just try increasing the buffer size.
Reading/writing byte-by-byte is definitely going to be slow, even though the actual reading/writing is done by chunks of the buffer size. One way to speed it up is to read/write by blocks. Have a look at read(byte[] b, int off, int len) method of BufferedInputStream. However it probably won't give you enough of the improvement.
What would be much better is to use nio package (new IO) to copy data using nio channels. Have a look at nio documentation for more info.
I would suggest to use FileUtils from org.apache.commons.io. It has enough utility methods to perform file operations.
org.apache.commons.io.FileUtils API Here
I have just started learning java. I modified the client side code for a server/client communication program, by creating two threads for the client side, main thread for receiving user's input, and inputThread for receiving server's response. I am sure that server has sent the response to client, however, no response message is obtain at client.
Here is my code. Can anyone help me to figure it out? Thanks
package clientnio;
import java.net.*;
import java.nio.*;
import java.io.*;
import java.nio.channels.*;
import java.util.Scanner;
public class ClientNIO {
public static int bufferLen = 50;
public static SocketChannel client;
public static ByteBuffer writeBuffer;
public static ByteBuffer readBuffer;
public static void main(String[] args) {
writeBuffer = ByteBuffer.allocate(bufferLen);
readBuffer = ByteBuffer.allocate(bufferLen);
try {
SocketAddress address = new InetSocketAddress("localhost",5505);
System.out.println("Local address: "+ address);
client=SocketChannel.open(address);
client.configureBlocking(false);
//readBuffer.flip();
new inputThread(readBuffer);
/*
String a="asdasdasdasddffasfas";
writeBuffer.put(a.getBytes());
writeBuffer.clear();
int d=client.write(writeBuffer);
writeBuffer.flip();
*/
while (true) {
InputStream inStream = System.in;
Scanner scan = new Scanner(inStream);
if (scan.hasNext()==true) {
String inputLine = scan.nextLine();
writeBuffer.put(inputLine.getBytes());
//writeBuffer.clear();
System.out.println(writeBuffer.remaining());
client.write(writeBuffer);
System.out.println("Sending data: "+new String(writeBuffer.array()));
writeBuffer.flip();
Thread.sleep(300);
}
}
}
catch(Exception e) {
System.out.println(e);
}
}
}
class inputThread extends Thread {
private ByteBuffer readBuffer;
public inputThread(ByteBuffer readBuffer1) {
System.out.println("Receiving thread starts.");
this.readBuffer = readBuffer1;
start();
}
#Override
public void run() {
try {
while (true) {
readBuffer.flip();
int i=ClientNIO.client.read(readBuffer);
if(i>0) {
byte[] b=readBuffer.array();
System.out.println("Receiving data: "+new String(b));
//client.close();
//System.out.println("Connection closed.");
//break;
}
Thread.sleep(100);
}
}
catch (Exception e) {
System.out.println(e);
}
}
}
Disclaimer: I'm not an active user of Java. (I only used it in school.)
Advice: I think it will greatly simplify the debugging process if you use blocking mode, at least until your code example is working correctly. (Currently your code does not seem to benefit from the non-blocking mode.)
I have identified two issues, culminating into four possible lines of code that may require changing:
When a ByteBuffer allocates its backing array, it sets itself ready to write by setting position to zero and limit to the capacity of that array. Your two uses of ByteBuffer.flip() (in the writing loop and the reading loop respectively) seem to be contrary to the convention.
Calling the ByteBuffer.array() method always returns the whole backing array, thus it always has size bufferLen. Because of this, a String constructed from the full-size array may contain junk from a previous transmission.
Typically, the array needs to be trimmed to the transmission size, and the conversion between a String and a byte array must use the same encoding as the server.
My suggested changes for first issue: (Note: I don't know how to fix the array trimming and encoding issue.)
writeBuffer.put(inputLine.getBytes());
writeBuffer.flip(); // <--here
client.write(writeBuffer);
...
writeBuffer.clear(); // <-- should be clear() instead of flip()
Thread.sleep(300);
// readBuffer.flip(); // <-- remove this line
int i=ClientNIO.client.read(readBuffer);
if(i>0) {
readBuffer.flip(); // <-- move it here
byte[] b=readBuffer.array();
System.out.println("Receiving data: "+new String(b));
...
}
References
http://docs.oracle.com/javase/1.4.2/docs/api/java/nio/ByteBuffer.html
http://docs.oracle.com/javase/1.4.2/docs/api/java/nio/channels/SocketChannel.html
Socketchannel always null
http://www.exampledepot.com/egs/java.nio.charset/ConvertChar.html
Calling flip() on a buffer prior to reading it is wrong. Don't do that. You need to flip it prior to writing from it, or getting from it, and compact() afterwards.
I received this python script that generates a file checksum:
import sys,os
if __name__=="__main__":
#filename=os.path.abspath(sys.argv[1])
#filename=r"H:\Javier Ortiz\559-7 From Pump.bin"
cksum=0
offset=0
pfi=open(filename,'rb')
while 1:
icks=0
chunk=pfi.read(256)
if not chunk: break #if EOF exit loop
for iter in chunk:
icks+=ord(iter)
print ord(iter)
cksum=(cksum+icks) & 0xffff
pfi.close()
print "cksum=0x%4.4x"%cksum
And I'm trying to convert it to Java but I'm not geting the same results.
Here's my Java code:
import java.io.BufferedInputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
public class ChecksumCalculator {
private ChecksumCalculator() {
}
public static int getChecksum(File file) {
int cksum = 0;
FileInputStream fis = null;
BufferedInputStream bis = null;
DataInputStream dis = null;
try {
fis = new FileInputStream(file);
// Here BufferedInputStream is added for fast reading.
bis = new BufferedInputStream(fis);
dis = new DataInputStream(bis);
byte[] buffer = new byte[256];
// dis.available() returns 0 if the file does not have more lines.
while (dis.read(buffer) != -1) {
int icks = 0;
for (byte b : buffer) {
icks += b & 0xff;
System.out.println(b & 0xff);
}
cksum = (cksum + icks) & 0xffff;
System.out.println("Checksum: " + cksum);
}
// dispose all the resources after using them.
fis.close();
bis.close();
dis.close();
return cksum;
} catch (FileNotFoundException e) {
e.printStackTrace();
return -1;
} catch (IOException e) {
e.printStackTrace();
return -1;
}
}
static public void main(String[] s) {
System.out.println("0x" + getChecksum(new File("H:\\Javier Ortiz\\559-7 From Pump.bin")));
}
}
But I get different results on a file. For example if I run it on a plain txt file containing only the word test it gives out the following result:
python: cksum=0x01c0
java: cksum=0x448
Any idea?
Your Python version prints the checksum in hex, while your Java version prints it in decimal. You should make your Java version print in hex, too. 0x1c0 == 448.
To use the cksum=0x%4.4x format string as you had in your Python version, use this:
System.out.printf("cksum=0x%4.4x%n", ...);
or even better
System.out.printf("cksum=%#04x%n", ...);
Also, you don't need a DataInputStream for this. Just use bis.read(buffer) instead of dis.read(buffer).
1C016 = 44810
I think that's your problem.
dis.read(buffer) returns the number of bytes that was actually read. For the last chunk, it will probably be less than 256. So the for loop shouldn't always be performed 256 times - it should be performed as many times as the actual byte count that was read from the stream.
I'm not a Python developer, but it doesn't look like ord(icks) in Python does the same as b & 0xff in Java.
Keep in mind that all Java types are signed; this might affect the calculation.
Also, although this doesn't affect correctness - it's a good practice to clean all the resources (i.e. to close the streams) in a finally block.