I'm a bit of a Java noob, and I have read some basics about sockets and I can successfully send images over socket using ImageIO, but I want to reduce the amount of data that is sent. Ultimately I want the image (screen capture) to be send as fast as possible with the smallest possible file size.
Right now, I have imageIO set up as such;
DataInputStream in=new DataInputStream(client.getInputStream());
DataOutputStream out = new DataOutputStream(client.getOutputStream());
ImageIO.write(captureImg(),"JPG",client.getOutputStream());
And the receiver:
BufferedImage img=ImageIO.read(ImageIO.createImageInputStream(server.getInputStream()));
File outputfile = new File("Screen"+(date.toString())+".jpg");
ImageIO.write(img, "jpg", outputfile);
In case you're wondering, this is my method that is used to take the image.
Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
BufferedImage capture = new Robot().createScreenCapture(screenRect);
I have heard about Byte arrays, where you can send the bytes then draw the image at the other end. However I'm not sure if this is more efficient.
Any help would be greatly appreciated, please ask if you would like me to add any extra info or code for the byte array!
Thanks.
EDIT: Patrick:
ByteArrayOutputStream bScrn = new ByteArrayOutputStream();
ImageIO.write(captureImg(), "JPG", bScrn);
byte imgBytes[] = bScrn.toByteArray();
out.write((Integer.toString(imgBytes.length)).getBytes());
out.write(imgBytes,0,imgBytes.length);
There already has been an extensive discussion in the comments, but to summarize a few points that I find important:
You have a trade-off between several criteria:
Minimize network traffic
Minimize CPU load
Maximize image quality
You can reduce the network traffic with a high image compression. But this will increase the CPU load and might reduce the image quality.
Whether it reduces the image quality depends on the compression type: For JPG, you can make the image arbitrarily small, but the quality of the image will then be ... well, arbitrarily bad. For PNG, the image quality will stay the same (since it is a lossless compression), but the CPU load and the resulting image size may be greater.
The option of ZIPping the image data was also mentioned. It is true that ZIPping the JPG or PNG data of an image will hardly reduce the amount of data (because the data already is compressed). But compressing the raw image data can be a feasible option, as an alternative to JPG or PNG.
Which compression technique (JPG, PNG or ZIP) is appropriate also depends on the image content: JPG is more suitable for "natural" images, like photos or rendered images. These can withstand a high compression without causing artefacts. For artifical images (like line drawings), it will quickly cause undesirable artefacts, particularly at sharp edges or when the image contains texts. In contrast to that: When the image contains large areas with a single color, then a compression like PNG (or ZIP) can reduce the image size due to the "run length compression" nature of these compression methods.
I already made some experiments for such an image transfer quite a while ago, and implemented it in a way that easily allowed tweaking and tuning these parameters and switching between the different methods, and comparing the speed for different application cases. But from the tip of my head, I can not give a profound summary of the results.
BTW: Depending on what you actually want to transfer, you could consider obtaining the image data with a different technique than Robot#createScreenCapture(Rectangle). This method is well-known to be distressingly slow. For example, when you want to transfer a Swing application, you could let your application directly paint into an image. Roughly with a pattern like
BufferedImage image = new BufferedImage(w,h,type);
Graphics g = image.getGraphics();
myMainFrame.paint(g);
g.dispose();
(This is only a sketch, to show the basic idea!)
Additionally, you could consider further options for increasing the "percieved speed" of such an image transfer. For example, you could divide your image into tiles, and transfer these tiles one after another. The receiver will possibly appreciate it if the image would at least be partially visible as quickly as possible. This idea could be extended further. For example, by detecting which tiles have really changed between two frames, and only transfer these changed tiles. (This approach could be extended and implemented in a rather sophisticated way, by detecting the "minimum regions" that have to be transferred)
However, for the case that you first want to play around with the most obvious tuning parameter: Here is a method that allows writing a JPG image with a quality value between 0.0 and 1.0 into an output stream:
public static void writeJPG(
BufferedImage bufferedImage,
OutputStream outputStream,
float quality) throws IOException
{
Iterator<ImageWriter> iterator =
ImageIO.getImageWritersByFormatName("jpg");
ImageWriter imageWriter = iterator.next();
ImageWriteParam imageWriteParam = imageWriter.getDefaultWriteParam();
imageWriteParam.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
imageWriteParam.setCompressionQuality(quality);
ImageOutputStream imageOutputStream =
new MemoryCacheImageOutputStream(outputStream);
imageWriter.setOutput(imageOutputStream);
IIOImage iioimage = new IIOImage(bufferedImage, null, null);
imageWriter.write(null, iioimage, imageWriteParam);
imageOutputStream.flush();
}
Related
I'm loading a jpeg-file via BitmapFactory and try to save it again (later I want to do some calculation on the pixel data before I save it again).
But if I try to save it with
FileOutputStream fos = new FileOutputStream(new File("/sdcard/test.jpg"));
originalImage.compress(Bitmap.CompressFormat.JPEG, 100, fos);
then it is not exactly the same result as in the original picture. Some pixel have got different color values and this ist not useful for my later calculation.
Is there a possibility to safe it lossless? Or is the problem already when I load the picture with
Bitmap originalImage = BitmapFactory.decodeFile("/sdcard/input.jpg");
few lines before?
Is there a possibility to safe it lossless?
No. The JPEG format uses a lossy compression. It makes no formal guarantees even if you set the quality to 100.
Or is the problem already when I load the picture with [...]
No, bitmaps are... maps of bits, i.e. they represent the exact bits of the image data.
Apologies for any ignorance, but I have never worked with jpeg images (let alone any types of images) in Java before.
Supposing I want to send a jpeg image from a web service to a client. Is there any way that I can reduce the jpeg file size by manipulating the colour profile of the image in some way?
I have already been able to reduce the image size by scaling it using a neat tool for BufferedImages called imgscalr. See here.
I would also like a jpeg that has less colours than a high quality jpeg image. For example, I would like to be able to use 8bit colour in my jpeg instead of say 16bit colour.
What exactly would I need to change if I have a BufferedImage from Java's 2D package?
Another way to reduce image size is to change compression level. You can do that using ImageWriter.
ImageWriter writer = null;
Iterator<ImageWriter> iwi = ImageIO.getImageWritersByFormatName("jpg");
if (!iwi.hasNext())
return;
writer = (ImageWriter) iwi.next();
ImageWriteParam iwp = writer.getDefaultWriteParam();
iwp.setCompressionMode(ImageWriteParam.MODE_EXPLICIT) ;
iwp.setCompressionQuality(compressionQuality);
writer.setOutput(...);
writer.write(null, image, iwp);
The easiest way to do this is to decompress the byte stream into a Java Image, optionally resize it (which makes it smaller) and then regenerate a JPEG image from this with the desired quality setting.
This new image is then what is sent to the client.
Have a look at the ImageIO class. As for reducing file size: since the image would already be a JPEG the only things you could do is reduce the quality or the image size.
Another thing to keep in mind: if the image is a CMYK jpeg it might be bigger. Unfortunately ImageIO can't handle those, but you can try JAI ImageIO to convert from CMYK to RGB (which should be much smaller).
Two of the possible solutions are downscaling the image, here's how you'd do it:
BufferedImage original = //your image here
scaled = original.getScaledInstance(finalWidth, finalHeight, Image.SCALE_SMOOTH); // scale the image to a smaller one
BufferedImage result = new BufferedImage(finalWidth, finalHeight, original.getType());
Graphics2D g = result.createGraphics();
g.drawImage(scaled, 0, 0, null); //draw the smaller image
g.dispose();
Obviously, you have to calculate the scaled width and height so the image stays by the same aspect ratio.
Once you have drawn it smaller, you can now turn this image into a JPEG file:
BufferedImage image = // this is the final scaled down image
JPEGImageEncoder jpegEncoder = JPEGCodec.createJPEGEncoder(output);
JPEGEncodeParam jpegEncodeParam = jpegEncoder.getDefaultJPEGEncodeParam(image);
jpegEncodeParam.setDensityUnit(JPEGEncodeParam.DENSITY_UNIT_DOTS_INCH);
jpegEncodeParam.setXDensity(92);
jpegEncodeParam.setYDensity(92);
jpegEncodeParam.setQuality( 0.8F , false);
jpegEncoder.encode(image, jpegEncodeParam);
These classes are from the JAI package (more exactly com.sun.image.codec.jpeg) and the JVM might complain that they should not be used directly, but you can ignore that.
You can possibly download JAI from here, if it does not work I have github mirrors setup for the two libraries, JAI core and JAI ImageIO.
I have a large swing component to write to TIFF. The component is too large to load the TIFF in memory, so I either need to make a big BufferedImage which is backed by a disk-based WritableRaster (as mentioned here) or use JAI.
JAI seems like the better answer, aside from the utter confusion of the project.
Given that, can someone outline steps for writing my swing component to a tiled TIFF without running out of Memory?
Image size will be maybe 10000x700
Ideally I would create some sort of disk-based image, and write parts of the component to it, each write being flushed to disk.
EDIT
I think I could do this with an ImageWriter, however I'm getting a NoSuchElementException when I call:
ImageWriter imageWriter = ImageIO.getImageWritersByFormatName("tif").next();
I have the jai_code.jar and jai_core.jar jars on my classpath, is there something else I need to do?
EDIT
I can create a very large TIFF using JAI, but JAI doesn't support TIFF compression, so the file is 92 MB.
If I install JAI-ImageIO, I can create a compressed TIFF Using an ImageWriter, but only from a Raster or BufferedImage, which I don't have enough memory for.
Is there some way to do a two-step approach, use JAI to create the large TIFF, then compress the large TIFF without loading the whole thing into memory?
I had to load and store a large tiff (59392x40192px) with JAI. My solution is: TiledImages.
I have used a TiledImage because I need tiles and subimages.
To use the TiledImage efficient you should construct it with your prefered tile size. JAI uses a TileCache so not the whole Image will be in memory, when it's not needed.
To write the TiledImage in a File use the option "writeTiled" (avoid OutOfMemory because it writes tile by tile):
public void storeImage(TiledImage img, String filepath) {
TIFFEncodeParam tep = new TIFFEncodeParam();
//important to avoid OutOfMemory
tep.setTileSize(256, 256);
tep.setWriteTiled(true);
//fast compression
tep.setCompression(TIFFEncodeParam.COMPRESSION_PACKBITS);
//write file
JAI.create("filestore", img, filepath, "TIFF", tep);
}
It works fine with images up to 690mb (compressed), for larger images i haven't tested yet.
But if you are working on WinXP 32-bit you may not able to have more as 1280m HeapSpace size, this is still a limit of Java VM.
My TiledImage is build with a IndexedColorModel from my image-source data:
//here you create a ColorModel for your Image
ColorModel cm = source.createColorModel();
//then create a compatible SampleModel, with the tilesize
SampleModel sm = cm.createCompatibleSampleModel(tileWidth,tileHeight);
TiledImage image = new TiledImage(0, 0, imageWidth, imageHeight, 0, 0, sm, cm);
I had the same situation and I used these steps:
Load as BufferedImage with JAI
Resize BufferedImage size to preferable size (600x600px) maintaining aspect-ratio using Image#getScaledInstance(int w, int h, Image.SCALE_SMOOTH)
Draw image using Graphics2d.drawImage(..) method in JComponent#paintComponent(java.awt.Graphics) method
That helped me with showing and manipulating TIFF images ~50MB (5000x5000px).
I have java program that reads a jpegfile from the harddrive and uses it as the background image for various other things. The image itself is stored in a BufferImage object like so:
BufferedImage background
background = ImageIO.read(file)
This works great - the problem is that the BufferedImage object itself is enormous. For example, a 215k jpeg file becomes a BufferedImage object that's 4 megs and change. The app in question can have some fairly large background images loaded, but whereas the jpegs are never more than a meg or two, the memory used to store the BufferedImage can quickly exceed 100s of megabytes.
I assume all this is because the image is being stored in ram as raw RGB data, not compressed or optimized in any way.
Is there a way to have it store the image in ram in a smaller format? I'm in a situation where I have more slack on the CPU side than RAM, so a slight performance hit to get the image object's size back down towards the jpeg compression would be well worth it.
One of my projects I just down-sample the image as it is being read from an ImageStream on the fly. The down-sampling reduces the dimensions of the image to a required width & height whilst not requiring expensive resizing computations or modification of the image on disk.
Because I down-sample the image to a smaller size, it also significantly reduces the processing power and RAM required to display it. For extra optimization, I render the buffered image in tiles also... But that's a bit outside the scope of this discussion. Try the following:
public static BufferedImage subsampleImage(
ImageInputStream inputStream,
int x,
int y,
IIOReadProgressListener progressListener) throws IOException {
BufferedImage resampledImage = null;
Iterator<ImageReader> readers = ImageIO.getImageReaders(inputStream);
if(!readers.hasNext()) {
throw new IOException("No reader available for supplied image stream.");
}
ImageReader reader = readers.next();
ImageReadParam imageReaderParams = reader.getDefaultReadParam();
reader.setInput(inputStream);
Dimension d1 = new Dimension(reader.getWidth(0), reader.getHeight(0));
Dimension d2 = new Dimension(x, y);
int subsampling = (int)scaleSubsamplingMaintainAspectRatio(d1, d2);
imageReaderParams.setSourceSubsampling(subsampling, subsampling, 0, 0);
reader.addIIOReadProgressListener(progressListener);
resampledImage = reader.read(0, imageReaderParams);
reader.removeAllIIOReadProgressListeners();
return resampledImage;
}
public static long scaleSubsamplingMaintainAspectRatio(Dimension d1, Dimension d2) {
long subsampling = 1;
if(d1.getWidth() > d2.getWidth()) {
subsampling = Math.round(d1.getWidth() / d2.getWidth());
} else if(d1.getHeight() > d2.getHeight()) {
subsampling = Math.round(d1.getHeight() / d2.getHeight());
}
return subsampling;
}
To get the ImageInputStream from a File, use:
ImageIO.createImageInputStream(new File("C:\\image.jpeg"));
As you can see, this implementation respects the images original aspect ratio as well. You can optionally register an IIOReadProgressListener so that you can keep track of how much of the image has been read so far. This is useful for showing a progress bar if the image is being read over a network for instance... Not required though, you can just specify null.
Why is this of particular relevance to your situation? It never reads the entire image into memory, just as much as you need it to so that it can be displayed at the desired resolution. Works really well for huge images, even those that are 10's of MB on disk.
I assume all this is because the image
is being stored in ram as raw RGB
data, not compressed or optimized in
any way.
Exactly... Say a 1920x1200 JPG can fit in, say, 300 KB while in memory, in a (typical) RGB + alpha, 8 bits per component (hence 32 bits per pixel) it shall occupy, in memory:
1920 x 1200 x 32 / 8 = 9 216 000 bytes
so your 300 KB file becomes a picture needing nearly 9 MB of RAM (note that depending on the type of images you're using from Java and depending on the JVM and OS this may sometimes be GFX-card RAM).
If you want to use a picture as a background of a 1920x1200 desktop, you probably don't need to have a picture bigger than that in memory (unless you want to some special effect, like sub-rgb decimation / color anti-aliasing / etc.).
So you have to choices:
makes your files less wide and less tall (in pixels) on disk
reduce the image size on the fly
I typically go with number 2 because reducing file size on hard disk means you're losing details (a 1920x1200 picture is less detailed than the "same" at 3940x2400: you'd be "losing information" by downscaling it).
Now, Java kinda sucks big times at manipulating pictures that big (both from a performance point of view, a memory usage point of view, and a quality point of view [*]). Back in the days I'd call ImageMagick from Java to resize the picture on disk first, and then load the resized image (say fitting my screen's size).
Nowadays there are Java bridges / APIs to interface directly with ImageMagick.
[*] There is NO WAY you're downsizing an image using Java's built-in API as fast and with a quality as good as the one provided by ImageMagick, for a start.
Do you have to use BufferedImage? Could you write your own Image implementation that stores the jpg bytes in memory, and coverts to a BufferedImage as necessary and then discards?
This applied with some display aware logic (rescale the image using JAI before storing in your byte array as jpg), will make it faster than decoding the large jpg every time, and a smaller footprint than what you currently have (processing memory requirements excepted).
Use imgscalr:
http://www.thebuzzmedia.com/software/imgscalr-java-image-scaling-library/
Why?
Follows best practices
Stupid simple
Interpolation, Anti-aliasing support
So you aren't rolling your own scaling library
Code:
BufferedImage thumbnail = Scalr.resize(image, 150);
or
BufferedImage thumbnail = Scalr.resize(image, Scalr.Method.SPEED, Scalr.Mode.FIT_TO_WIDTH, 150, 100, Scalr.OP_ANTIALIAS);
Also, use image.flush() on your larger image after conversion to help with the memory utilization.
File size of the JPG on disk is completely irrelevant.
The pixel dimensions of the file are. If your image is 15 Megapixels expect it to require crap load of RAM to load a raw uncompressed version.
Re-size your image dimensions to be just what you need and that is the best you can do without going to a less rich colorspace representation.
You could copy the pixels of the image to another buffer and see if that occupies less memory then the BufferedImage object. Probably something like this:
BufferedImage background = new BufferedImage(
width,
height,
BufferedImage.TYPE_INT_RGB
);
int[] pixels = background.getRaster().getPixels(
0,
0,
imageBuffer.getWidth(),
imageBuffer.getHeight(),
(int[]) null
);
I am generating lots of images in java and saving them through the ImageIO.write method like this:
final BufferedImage img = createSomeImage();
ImageIO.write( img, "png", new File( "/some/file.png" );
I was happy with the results until Google's firefox addon 'Page Speed' told me that i can save up to 60% of the size if i optimize the images. The images are QR codes, their size is around 900B each and the firefox-plugin optimized versions are around 300B.
I'd like to save such optimized 300B Images directly from java.
So here my question again: How to save optimized png images with java's ImageIO?
Use PngEncoderB to convert your BufferedImage into a PNG encoded byte array.
You can apply a filter to it, which helps prepare the image for better optimization. This is what OptiPNG does, only OptiPNG calculates which filter will get you the best compression.
You might have to try applying each filter to see which one is consistently better for you. With 2 bit color, I think the only filter that might help is "up", so I'm guessing that's the one to use.
Once you get the image to a PNG encoded byte array, you can write that directly to a file.