I have read quite a few posts on stack overflow and it seems gzip can compress streams(ie. I am assuming they mean compress X bytes of the stream at a time). What I don't get is how to do this properly so browsers will decompress it? Let's say I have a 1GB file people need to download and I don't want to bring that all into memory at the same time. How do I compress each chunk and send it on?
closely related question gzip compression of chunked encoding response?
though java code would be great if anyone knows of any. I looked at old playframework 1.2.x code but that doesn't appear to do any compression on writing out chunks :(.
thanks,
Dean
Related
I have a webservice sending a huge JSON text to an Android app. There's about 20000 ID numbers. Unfortunately but perhaps not surprisingly it's timing out.
What options do I have? The easiest one that comes to mind is somehow compressing this data. Is there any way I can do this (PHP webservice, Java Android app) effectively?
Failing that, is there some technique to send JSON in parts? If so, how does that work? At what point is JSON considered too big to send in one part? Thank you
You can use GZIP in php and send as stream to client and then decode data with java in android
you can use this for gzip in php: GZIP
and Gzip in Android : GZIP
You could compress data with ob_gzhandler(). Put this call in your script before any output:
ob_start('ob_gzhandler');
After that, output will be compressed with gzip.
This is not a good solution, indeed. You should split JSON and send it as sequential smaller pieces. Otherwise, what will you do when even compressed data is too big?
I was trying to open jpeg files in a Java program and noticed that neither ImageIO nor the Apache commons imaging library tools could open the images. The commons library showed me this error:
"Only sequential, baseline JPEGs are supported at the moment"
So, my image files are compressed in a way both libraries aren't able to read. I could make an ImageJ macro and transform all the images first but I would like to just use my program and not something extra.
Is there a way to find the compression mode in jpeg or even a java library that can read jpegs in several modes?
Thanks in advance
Suddenly it works with the ImageIO standard Class. Don't ask me why. Thanks for your help guys.
You need to get some tool that will allow you to dump a JPEG stream. There are a number of them out there.
What you are looking for is the start of frame marker.
FFC0 indicates baseline sequential.
FFC1 indicates extended sequential. It doesn't take much more code to do extended sequential than baseline. It is puzzling why a decoder would limit itself to baseline these day.
FFC2 is progressive.
There are others but those are the only ones you are likely to encounter and are widely supported.
You just need to find a tool that will save in baseline format. Finding one to read the other two formats is easy.
I have a database which contains picture data stored as a binary blob. The documentation says the data is encoded using LZW. I thought that I could decode it using the Zip or GZip input streams found in the Java library, but it didn't work - I got an exception that said the format of the data is not correct.
From what I've read, the library uses DEFLATE, which is not LZW. Also, I've read about some licensing problems for using the LZW algorithm.
What can I use to decode the data? Is there a library? Do I have to implement it myself? What about the licensing problems?
I know the question is old, but I just wanted to add a great resource about LZW:
http://www.matthewflickinger.com/lab/whatsinagif/lzw_image_data.asp
It's more specifically about the use of LZW in GIF images, but it explains the compression and decompression algorithms pretty well.
Here are a couple of links:
http://www.cs.sfu.ca/CC/365/li/squeeze/LZW.html
http://u.cs.biu.ac.il/~freskom1/AlgProg1/Progs/LZW.java
http://www.codeproject.com/KB/java/lzw.aspx
And there are others.
Indeed if the images are LZW compressed TIFF files, The Java Advanced Imaging API apparently supports decoding directly (though not encoding it seems).
You can also try with 7-Zip JBinding which uses the 7zip library internally. It's quite easy to use.
I went through a surprising amount of LZW implementations before finding one that worked for my case.
UncompressedInputStream from the BioJava project worked for me, when I needed to unpack a .pax file.
I am using the JSpeex API to convert a .wav file into .spx file. Everything goes perfect when tested on desktop; it took only 2 seconds.
Android developer used the same code but it took around 3 minutes to encode the same file on their simulator & phone. Is there any way to reduce this time for encoding? Code used to convert is as follows:
new JSpeexEnc().encode(new File("source.wav"), new File("dest.spx"));
Compression takes time. The better the compression, the longer it takes, and Speex is pretty good compression.
2 seconds of desktop computer time is absolutely ages.
JSpeex is a java implementation. Use a native implementation, ideally use the platform codecs, instead.
On phones, speech is best compressed using AMR - not necessarily the best quality/compression, but most likely hardware accelerated since its the format used by GSM. You can usually get AMR straight from the microphone.
How do you get large WAV files onto an Android device in the first place? If its actually the output of the microphone, consider using AMR as outlined above.
If you need Speex and you have a wav file, then consider sending it to a server for compression.
If i compress and decompress data using java then it works fine
My if friend uses C#.NET to compress data while i am using java SO inthis case I can not decompress the same data.
I am using inflatter and deflater in java.
Thanks
Bapi
Use GZIPInputStream and GZIPOutputStream which are compatible with gzip format.
Note: each compression format is different (though similar in approach) and they have to be the same to work.
Inflator and Deflator are a cut down version of the GZIP format and I wouldn't expect it to work with anything another other than Java.
The java.util Deflator and Inflator support are the basic zip compression and decompression format implementations - they do not create a free standing archive file. Rather they create/read the data stream which corresponds to the data portion of one entry in an archive file.
To create an archive file readable by general compression utilities use either the java.util.GZIPXxx or java.util.ZipXxx classes.
Thanks for your support.
I changed my compression logic. I used GZIPInputStream both in java and in C#.NET
after this only. my problem is solved
Thanks
Bapi