I'm trying to read the last 128 bytes from a file (the signature) and then trying to read until those bytes but the first part (reading the last 128 bytes) is returning an ArrayIndexOutOfBoundsException:
byte[] signature = new byte[128];
FileInputStream sigFis = new FileInputStream(f);
sigFis.read(signature, (int)f.length()-128, 128);
sigFis.close();
And then the last part doesn't seem to be working either, I'm using an offset that i increase gradually:
CipherInputStream cis = new CipherInputStream(fis, c);
FileOutputStream fos = new FileOutputStream(destFile);
int i = cis.read(data);
int offset = 0, maxsize = (int)f.length()-128;
while((i != -1) && offset<maxsize){
fos.write(data, 0, i);
sig.update(data);
fos.flush();
i = cis.read(data);
offset+=1024;
}
I get an EOFExcpetion with the RAF I used to do my ops...
byte[] signature = new byte[128];
int offset = (int)f.length()-128;
raf.seek(offset);
raf.readFully(signature, 0, 128);
I would use File or FileChannel to get the file size. This is how to read until the last 128 bytes
FileInputStream is = new FileInputStream("1.txt");
FileChannel ch = is.getChannel();
long len = ch.size() - 128;
BufferedInputStream bis = new BufferedInputStream(is);
for(long i = 0; i < len; i++) {
int b = bis.read();
...
}
if we continue reading we will get the last 128 bytes
ByteArrayOutputStream bout128 = new ByteArrayOutputStream();
for(int b; (b=bis.read() != -1);) {
bout128.write(b);
}
byte[] last128 = bout128.toByteArray();
I think you got confused with the read method parameters..
FileInputStream sigFis = new FileInputStream(f);
sigFis.read(signature, (int)f.length()-128, 128);
//This doesn't give you last 128 bits.
// The offset is offset of the byte array 'signature
// Thats the reason you see ArrayIndexOutOfBoundsException
sigFis.close();
replace your read() method with
sigFis.read(signature);
//But now signature cannot be just 128 array but length of file. And read the last 128 bytes
InputStream read method signature looks as below:
int java.io.FileInputStream.read(byte[] b, int off, int len)
Parameters:
b the buffer into which the data is read.
off the start offset in the destination array b
len the maximum number of bytes read.
Hope this helps!
Related
I've got a simple test case that fails to compress a stream of data. I generate a byte[] of some random bytes, compress it via DeflaterOutputStream, flush() the stream, then reverse those operations to retrieve the original array. At byte 505 the reconstructed stream starts to consist entirely of 0x00 bytes, and I don't understand why:
//
// create some random bytes
//
Random rng = new Random();
int len = 5000;
byte[] data = new byte[len];
for (int i = 0; i < len; ++i)
data[i] = (byte) rng.nextInt(0xff);
//
// write to byte[] via a deflater stream
//
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DeflaterOutputStream os = new DeflaterOutputStream(baos, true);
os.write(data);
os.flush();
//
// read back into byte[] via an inflater stream
//
ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
InflaterInputStream is = new InflaterInputStream(bais);
byte[] readbytes = new byte[len];
is.read(readbytes);
//
// check they match (they don't, at byte 505)
//
for (int i = 0; i < len; ++i)
if (data[i] != readbytes[i])
throw new RuntimeException("Mismatch at position " + i);
It doesn't seem to matter what's in the source array, it's always at position 505 it fails.
Here's what the two byte[] arrays look like around the region they differ:
?\m·g··gWNLErZ···,··-··=·;n=··F?···13·{·rw·······\`3···f····{/····t·1·WK$·······WZ······x
?\m·g··gWNLErZ···,··-····································································
^byte 505
All those unprintable chars are 0x00 from that point on. Why is this happening? I feel like I must be misunderstanding something fundamental about how the Deflate/Inflate streams work. The real-world use case here is a stream over a network that I thought I could easily improve the performance of by inserting Deflate/Inflate streams into
When I test this, is.read(readBytes) returns 505, the length of bytes read. The other single-argument-array stream methods return void and guarantee that the entire array is read or written, but is.read() is a different API and requires that you check the amount of bytes actually read.
ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
System.err.println( "bais size = " + bais.available() );
InflaterInputStream is = new InflaterInputStream(bais);
byte[] readbytes = new byte[len];
System.err.println( "read = " + is.read(readbytes) ); // 505
This runs without throwing an error for me:
ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
System.err.println( "bais size = " + bais.available() );
InflaterInputStream is = new InflaterInputStream(bais);
byte[] readbytes = new byte[len];
for( int total = 0, result = 0; (result = is.read(readbytes, total, len-total )) != -1; )
{
total += result;
System.err.println( "reading : " + total );
if( total == len ) break;
}
Consider this code:
BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream("/home/gala/falken_test/test_file"));
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(new FileOutputStream("/home/gala/falken_test/out"));
int b;
byte[] buff = new byte[8096];
while ((b = bufferedInputStream.read(buff)) > -1) {
bufferedOutputStream.write(buff, 0, b);
System.out.println("Writing " + b);
}
bufferedInputStream.close();
bufferedOutputStream.close();
This copies a file from point A (bufferedInputStream) to point B (bufferedOutputStream), 8096 bytes at a time, untill the last try is larger than the file itself, then it trims the buffer and reads the remaining data. This looks like this, this is the output from System.out.println("Writing " + b);
Writing 8096
Writing 8096
...
Writing 8096
Writing 8096
Writing 5681
This took 14 ms
Now the problem rises that, how can I copy a certain range of bytes?
For example, from 1012 bytes to 2134, if the file is 10000 bytes long for example.
What I an writing is a tar implementation, where a lot of files get concatenated one to other with a 512 byte header between them, which describes the bytes following the header.
And I read the header, skip 512 bytes of the header and start writing the file, until n-bytes of data, which are specified in the header.
How can I solve this without copying byte-by-byte?
You can use BufferedInputStream.skip(long) to skip until right pos, and the read and write the amount of bytes from 1012 (startPos) to 2134 (endPos), that is 2134-1012=1122 bytes.
Note: I changed the buffer size to 512 instead to show how it works with your example positions.
BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream("test_file"));
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(new FileOutputStream("test_out"));
int CHUNK_SIZE = 512; // change to 8096
int startPos = 1012;
int endPos = 2134;
int bytesToRead = endPos - startPos;
int b;
byte[] buff = new byte[CHUNK_SIZE];
bufferedInputStream.skip(startPos - 1);
int currentChunkSize = Math.min(CHUNK_SIZE, bytesToRead);
while ((b = bufferedInputStream.read(buff, 0, currentChunkSize)) != -1) {
bufferedOutputStream.write(buff, 0, b);
System.out.println("Writing " + b);
bytesToRead -= b;
if (bytesToRead == 0) {
break;
}
currentChunkSize = Math.min(CHUNK_SIZE, bytesToRead);
}
bufferedInputStream.close();
bufferedOutputStream.close();
The output from this is:
$ java -cp ReadWriteTest
Writing 512
Writing 512
Writing 98
I want to read bytes from a wave file into an array. Since the number of bytes read depends upon the size of the wave file, I'm creating a byte array with a maximum size of 1000000. But this is resulting in empty values at the end of the array. So, I wanted to create a dynamically increasing array and I found that ArrayList is the solution. But the read() function of the AudioInputStream class reads bytes only into a byte array! How do I pass the values into an ArrayList instead?
ArrayList isn't the solution, ByteArrayOutputStream is the solution. Create a ByteArrayOutputStream write your bytes to it, and then invoke toByteArray() to get the bytes.
Example of what your code should look like:
in = new BufferedInputStream(inputStream, 1024*32);
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] dataBuffer = new byte[1024 * 16];
int size = 0;
while ((size = in.read(dataBuffer)) != -1) {
out.write(dataBuffer, 0, size);
}
byte[] bytes = out.toByteArray();
You can have an array of byte like:
List<Byte> arrays = new ArrayList<Byte>();
To convert it back to arrays
Byte[] soundBytes = arrays.toArray(new Byte[arrays.size()]);
(Then, you will have to write a converter to transform Byte[] to byte[]).
EDIT: You are using List<Byte> wrong, I'll just show you how to read AudioInputStream simply with ByteArrayOutputStream.
AudioInputStream ais = ....;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int read;
while((read = ais.read()) != -1) {
baos.write(read);
}
byte[] soundBytes = baos.toByteArray();
PS An IOException is thrown if frameSize is not equal to 1. Hence use a byte buffer to read data, like so:
AudioInputStream ais = ....;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int bytesRead = 0;
while((bytesRead = ais.read(buffer)) != -1) {
baos.write(buffer, 0, bytesRead);
}
byte[] soundBytes = baos.toByteArray();
Something like this should do:
List<Byte> myBytes = new ArrayList<Byte>();
//assuming your javax.sound.sampled.AudioInputStream is called ais
while(true) {
Byte b = ais.read();
if (b != -1) { //read() returns -1 when the end of the stream is reached
myBytes.add(b);
} else {
break;
}
}
Sorry if the code is a bit wrong. I haven't done Java for a while.
Also, be careful if you do implement it as a while(true) loop :)
Edit: And here's an alternative way of doing it that reads more bytes each time:
int arrayLength = 1024;
List<Byte> myBytes = new ArrayList<Byte>();
while(true) {
Byte[] aBytes = new Byte[arrayLength];
int length = ais.read(aBytes); //length is the number of bytes read
if (length == -1) { //read() returns -1 when the end of the stream is reached
break; //or return if you implement this as a method
} else if (length == arrayLength) { //Array is full
myBytes.addAll(aBytes);
} else { //Array has been filled up to length
for (int i = 0; i < length; i++) {
myBytes.add(aBytes[i]);
}
}
}
Note that both read() methods throw an IOException - handling this is left as an exercise for the reader!
How do I append a portion of byte array to a StringBuilder object under Java? I have a segment of a function that reads from an InputStream into a byte array. I then want to append whatever I read into a StringBuilder object:
byte[] buffer = new byte[4096];
InputStream is;
//
//some setup code
//
while (is.available() > 0)
{
int len = is.read(buffer);
//I want to append buffer[0] to buffer[len] into StringBuilder at this point
}
You should not use a StringBuilder for this, since this can cause encoding errors for variable-width encodings. You can use a java.io.ByteArrayOutputStream instead, and convert it to a string when all data has been read:
byte[] buffer = new byte[4096];
ByteArrayOutputStream out = new ByteArrayOutputStream();
InputStream is;
//
//some setup code
//
while (is.available() > 0) {
int len = is.read(buffer);
out.write(buffer, 0, len);
}
String result = out.toString("UTF-8"); // for instance
If the encoding is known not to contain multi-byte sequences (you are working with ASCII data, for instance), then using a StringBuilder will work.
You could just create a String out of your buffer:
String s = new String(buffer, 0, len);
Then if you need to you can just append it to a StringBuilder.
Something like below should do the trick for you.
byte[] buffer = new byte[3];
buffer[0] = 'a';
buffer[1] = 'b';
buffer[2] = 'c';
StringBuilder sb = new StringBuilder(new String(buffer,0,buffer.length-1));
System.out.println("buffer has:"+sb.toString()); //prints ab
I have a bin file that I need to convert to a byte array. Can anyone tell me how to do this?
Here is what I have so far:
File f = new File("notification.bin");
is = new FileInputStream(f);
long length = f.length();
/*if (length > Integer.MAX_VALUE) {
// File is too large
}*/
// Create the byte array to hold the data
byte[] bytes = new byte[(int)length];
// Read in the bytes
int offset = 0;
int numRead = 0;
while (offset < bytes.length && (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) {
offset += numRead;
}
// Ensure all the bytes have been read in
if (offset < bytes.length) {
throw new IOException("Could not completely read file "+f.getName());
}
But it's not working...
Kaddy
try using this
public byte[] readFromStream(InputStream inputStream) throws Exception
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(baos);
byte[] data = new byte[4096];
int count = inputStream.read(data);
while(count != -1)
{
dos.write(data, 0, count);
count = inputStream.read(data);
}
return baos.toByteArray();
}
Btw, do you want a Java code or C++ code. Seeing the code in your question, I assumed it to be a java code and hence gave a java answer to it
You're probably better off using a memory mapped file. See this question
In Java, a simple solution is:
InputStream is = ...
ByteArrayOutputStream os = new ByteArrayOutputStream();
byte[] data = new byte[4096]; // A larger buffer size would probably help
int count;
while ((count = is.read(data)) != -1) {
os.write(data, 0, count);
}
byte[] result = os.toByteArray();
If the input is a file, we can preallocate a byte array of the right size:
File f = ...
long fileSize = f.length();
if (fileSize > Integer.MAX_VALUE) {
// file too big
}
InputStream is = new FileInputStream(f);
byte[] data = new byte[fileSize];
if (is.read(data)) != data.length) {
// file truncated while we were reading it???
}
However, there is probably a more efficient way to do this task using NIO.
Unless you really need to do it just that way, maybe simplify what you're doing.
Doing everything in the for loop may seem like a very slick way of doing it, but it's shooting yourself in the foot when you need to debug and don't immediately see the solution.
In this answer I read from an URL
You could modify it so the InputStream is from a File instead of a URLConnection.
Something like:
FileInputStream inputStream = new FileInputStream("your.binary.file");
ByteArrayOutputStream output = new ByteArrayOutputStream();
byte [] buffer = new byte[ 1024 ];
int n = 0;
while (-1 != (n = inputStream.read(buffer))) {
output.write(buffer, 0, n);
}
inputStream.close();
etc
Try open source library apache commons-io
IOUtils.toByteArray(inputStream)
You are not the first and not the last developer who needs to read a file, no need to reinvent it each time.