Compressing byte[] to byte[] with GZIPOutputStream? Unexpected end of ZLIB input stream - java

I am trying to compress and array of bytes into another array of bytes using GZIPOutputStream (in Java).
This is my code:
#Test
public void testCompressBytes() throws IOException {
final byte[] uncompressed = RandomStringUtils.randomAlphanumeric(100000 /* 100 kb */).getBytes();
// compress
byte[] compressed;
try (InputStream is = new ByteArrayInputStream(uncompressed);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
OutputStream os = new GZIPOutputStream(baos)) {
IOUtils.copy(is, os); // org.apache.commons.io
os.flush();
compressed = baos.toByteArray();
}
System.out.println("Size before compression = " + uncompressed.length + ", after = " + compressed.length);
// decompress back
byte[] decompressedBack;
try (InputStream is = new GZIPInputStream(new ByteArrayInputStream(compressed));
ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
IOUtils.copy(is, baos); // EXCEPTION THROWN HERE
baos.flush();
decompressedBack = baos.toByteArray();
}
assertArrayEquals(uncompressed, decompressedBack);
}
And this is the output I'm getting:
Size before compression = 100000, after = 63920
java.io.EOFException: Unexpected end of ZLIB input stream
What could I be doing wrong?

You need to call GZIPOutputStream::close before calling ByteArrayOutputStream::toByteArray, so that GZIPOutputStream writes all the end bits.
In your current code you are calling ByteArrayOutputStream::toByteArray before GZIPOutputStream::close (via try-with-resources) that's why it doesn't work.

Thanks, everybody!
Although calling GZIPOutputStream::finish() before ByteArrayOutputStream::toByteArray() seems to do the trick, I believe it's better to completely close the GZIP stream first, which in turn forces us to keep ByteArrayOutputStream outside the try-with-resources clause.
So, my reworked compression part looks like that now:
final ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (InputStream is = new ByteArrayInputStream(uncompressed);
GZIPOutputStream gzos = new GZIPOutputStream(baos)) {
IOUtils.copy(is, gzos);
} catch (final IOException e) {
throw new RuntimeException(e);
}
IOUtils.closeQuietly(baos);
final byte[] compressed = baos.toByteArray();

Related

Android converting stream to byte array Out of Memory Exceptions

I have ran into a situation where I am getting out of memory exceptions when trying to convert an input stream into a byte array. On newer Android phones it's no problem, but some of these cheaper models are experiencing it. Here is the method I'm using. Is there a more efficient way to do this?
public byte[] convertStreamToByteArray(InputStream is) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buff = new byte[1024];
int i = 0;
while ((i = is.read(buff, 0, buff.length)) > 0) {
baos.write(buff, 0, i);
}
return baos.toByteArray();
}
There is, do not copy all the data from the stream to memory.
Why do you need to do so ?
Can't you read and consume the stream data as it comes without holding the whole data in memory ?
If you need some buffering, buffer as small as you can for it to be consumable/processed and throw that buffer away and create a new one so that you have as less memory used at a time as possible.
See if IOUtils can help you out here.
Import it
import org.apache.commons.io.IOUtils;
Then try:
public byte[] convertStreamToByteArray(InputStream is) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
IOUtils.copy(is, baos);
return baos.toByteArray();
}
Try to create an byte array with the size of InputStream like this :
public byte[] convertStreamToByteArray(InputStream is) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buff = new byte[is.available()];
int i = 0;
while ((i = is.read(buff, 0, buff.length)) > 0) {
baos.write(buff, 0, i);
}
return baos.toByteArray();
}

only first few hundred samples are read from input stream into byte array(the rest are zeros)

i'm trying to convert audio(mp3/wav etc.) to byte array. i did it using inputStream to byte array conversion.
the problem is that after few hundred samples i recieve only zeroes.
at first i thought the problem was with the file so i tried debugging with another file and had the same problem.
I thought the problem was with the code so i tried using IOUtils and got the exact same resualts.
can anyone tell me what i'm doing wrong?
the code i used:
File file = new File(path);
final InputStream inputStream = new FileInputStream(file);
byte[] byteSamples = inputStreamToByteArray(inputStream);
public byte[] inputStreamToByteArray(InputStream inStream) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = inStream.read(buffer)) > 0) {
baos.write(buffer, 0, bytesRead);
}
return baos.toByteArray();
}
the alternate version using IOUtils:
byte[] byteSamples = IOUtils.toByteArray(inputStream);
update : i tried doing it using BufferedInputStream, still the exact same results.
byte[] byteSamples = new byte[(int)file.length()];
try {
BufferedInputStream buf = new BufferedInputStream(new FileInputStream(file));
buf.read(byteSamples, 0, byteSamples.length);
buf.close();
} catch (FileNotFoundException e) {
e.printStackTrace();}
You need to close the streams when done.

Java ObjectOutputStream Not Writing to ZipEntry

I am trying to serialize an object into a ZipEntry using an ObjectOutputStream, however it doesn't appear to be writing anything because when I print the byte array produced, it shows null. I tried writing a string with the ZipOutputStream, and upon printing the resulting byte array got a sizeable result. SO my question is: why is the objectoutput stream not correctly writing into the ZipEntry. (ConfigEntry does implement Serializable).
String s = "Tired, Exhausted";
ConfigEntry con = new ConfigEntry("rand", "random", 3);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try {
ZipOutputStream zos = new ZipOutputStream(baos);
ZipEntry entry = new ZipEntry("test.txt");
ObjectOutputStream obs = new ObjectOutputStream(zos);
zos.putNextEntry(entry);
obs.writeObject(con);
obs.close();
zos.closeEntry();
zos.close();
} catch(IOException ioe) {
ioe.printStackTrace();
}
os = bs.getOutputStream();
byte[] result = baos.toByteArray();
String test = new String(result, "UTF-8");
Log.v("Mac Address", test);
Log.v("Mac Address", Arrays.toString(result));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
This baos goes out of scope after the try block. You are writing to one baos and you are looking into another baos declared in an outer scope, probably an instance member of the class.

Zipping files with ZipOutputStream gives inconsistent results

I want to zip a text file using the java.util.ZipOutputStream class. I found two examples on the internet explaining on how to do that. This led me to the two possible implementations shown below. While both methods produce 'healthy zip files', my problem is that on every run the binary content of the file is slightly different (around the 10th byte). Does someone know if
This is intended behaviour
There is a way to always produce exactly the same result
Here is my current code:
public byte[] getZipByteArray(String fileName) throws IOException
{
byte[] result = new byte[0];
byte[] buffer = new byte[1024];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ZipOutputStream zos = new ZipOutputStream(baos);
ZipEntry ze = new ZipEntry(fileName);
zos.putNextEntry(ze);
InputStream inputStream = ZipCompression.class.getResourceAsStream(fileName);
int len;
while ((len = inputStream.read(buffer)) > 0)
{
zos.write(buffer, 0, len);
}
zos.closeEntry();
zos.close();
result = baos.toByteArray();
return result;
}
public byte[] ZipByteArrayBuffered(String fileName) throws IOException
{
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(byteArrayOutputStream);
ZipOutputStream zipOutputStream = new ZipOutputStream(bufferedOutputStream);
File file = new File(fileName);
InputStream fileInputStream = ZipCompression.class.getResourceAsStream(file.getName());
zipOutputStream.putNextEntry(new ZipEntry(file.getName()));
IOUtils.copy(fileInputStream, zipOutputStream);
fileInputStream.close();
zipOutputStream.closeEntry();
if (zipOutputStream != null)
{
zipOutputStream.finish();
zipOutputStream.flush();
IOUtils.closeQuietly(zipOutputStream);
}
IOUtils.closeQuietly(bufferedOutputStream);
IOUtils.closeQuietly(byteArrayOutputStream);
return byteArrayOutputStream.toByteArray();
}
Byte 10 starts the file modification date and so this will always differ. See Wikipedia for the details of the zip file format.

how to convert image to byte array in java?(With out using buffered image)

Hi could anyone please explain me how to convert the image data to byte array in java I am trying like this.I do not need to use buffered image here.
File file = new File("D:/img.jpg");
FileInputStream imageInFile = new FileInputStream(file);
byte imageData[] = new byte[(int) file.length()];
imageInFile.read(imageData);
You can convert your image data using FileInputStream also.
File file = new File("D:\\img.jpg");
FileInputStream fis = new FileInputStream(file);
//Now try to create FileInputStream which obtains input bytes from a file.
//FileInputStream is meant for reading streams of raw bytes,in this case its image data.
//For reading streams of characters, consider using FileReader.
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for (int readNum; (readNum = fis.read(buf)) != -1;) {
//Now Write to this byte array output stream
bos.write(buf, 0, readNum);
System.out.println("read " + readNum + " bytes,");
}
} catch (IOException ex) {
Logger.getLogger(ConvertImage.class.getName()).log(Level.SEVERE, null, ex);
}
byte[] bytes = bos.toByteArray();
or you could use:
Image image = Toolkit.getDefaultToolkit().getImage("D:/img.jpg");
byte[] imageBytes = getImageBytes(image);
private byte[] getImageBytes(Image image) throws IOException {
try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
ImageIO.write(image, "bmp", baos);
baos.flush();
return baos.toByteArray();
}
}

Categories

Resources