Develop Reference

Maintaining LSB through JPG compilation - is it possible?

This is one of those "pretty sure we found the answer, but hoping we're wrong" questions. We are looking at a steganography problem and it's not pretty.
Situation:
We have a series of images. We want to mark them (watermark) so the watermarks survive a series of conditions. The kicker is, we are using a lossfull format, JPG, rather than lossless such as PNG. Our watermarks need to survive screenshotting and, furthermore, need to be invisible to the naked eye. Finally, they need contain at least 32 bytes of data (we expect them to be repeating patterns across an image, of course). Due to the above, we need to hide the information in the pixels themselves. I am trying a Least Significant Bit change, including using large blocks per "bit" (I tried both increments of 16 as these are the jpg compression algorithms size chunks from what we understand, as well as various prime numbers) and reading the average of the resulting block. This sort of leads to requirements:
Must be .jpg
Must survive the jpg compression algorithm
Must survive screenshotting (assume screenshots are saved losslessly)
Problem:
JPG compression, even 100% "minimum loss" changes the pixel values. EG if we draw a huge band across an image setting the Red channel to 255 in a block 64 pixels high, more than half are not 255 in the compiled image. This means that even using an average of the blocks yields the LSB to be random, rather than what we "encoded". Our current prototype can take a random image, compress the message into a bit-endoded string and convert it to a XbyX array which is then superimposed on the image using the LSB of one of the three color-channels. This works and is detectable while it remains a BufferedImage, but once we convert to a JPG the compression destroys the message.
Question:
Is there a way to better control a jpg compression's pixel values? Or are we simply SOOL here and need to drop this avenue, either shifting to PNG output (unlikely) or need to understand the JPG compression's algorithm at length and use it to somehow determine LSB pattern outcomes? Preferably java, but we are open to looking at alternative languages if there are any that can solve our problem (our current PoC is in java)

How to joing to jpegs without loading them in memory

I've read something about jpeg Lossless editing, with it is possible to rotate, crop and a few other things without the need to really load the full image into memory, is there by any means a way of joining two jpegs together using this kind of technique? I was thinking something like copying the MCUs, quantization tables and huffman tables from one image into the other, in another user's question someone told it could be done but didn't tell how, the person said it would need reprocessing the edge MCUs or something like that

Stream image format conversions and resizing in Java

So, lets say I want to recode some PNG to JPEG in Java. The image has extreme resolution, lets say for example 10 000 x 10 000px. Using "standard" Java image API Writers and Reader, you need at some point to have entire image decoded in RAM, which takes extreme amount of RAM space (hundreds of MB). I have been looking how other tools do this, and I found that ImageMagick uses disk pixel storage, but this seems to by way too slower for my needs. So what I need is tru streaming recoder. And by true streaming I mean read and process data by chuncks or bins, not just give stream as input but decode it whole beforehand.
Now, first the theory behind - is it even possible, given JPEG and PNG algorithms, to do this using streams, or lets say in bins of data? So there is no need to have entire image encoded in memory(or other storage)? In JPEG compression, first few stages could be done in streams, but I believe Huffman encoding needs to build entire tree of value probabilities after quantization, therefore it needs to analyze whole image - so whole image needs to be decoded beforehand, or somehow on demand by regions.
And the golden question, if above could be achieved, is there any Java library that can actually work in this way? And save large amount of RAM?

If I create a 10,000 x 10,000 PNG file, full of incompressible noise, with ImageMagick like this:
convert -size 10000x10000 xc:gray +noise random image.png
I see ImageMagick uses 675M of RAM to create the resulting 572MB file.
I can convert it to a JPEG with vips like this:
vips im_copy image.png output.jpg
and vips uses no more than 100MB of RAM while converting, and takes 7 seconds on a reasonable spec iMac around 4 years old - albeit with SSD.

I have thought about this for a while, and I would really like to implement such a library. Unfortunately, it's not that easy. Different image formats store pixels in different ways. PNG or GIFs may be interlaced. JPEGs may be progressive (multiple scans). TIFFs are often striped or tiled. BMPs are usually stored bottom up. PSDs are channeled. Etc.
Because of this, the minimum amount of data you have to read to recode to a different format, may in worst case be the entire image (or maybe not, if the format supports random access and you can live with a lot of seeking back and forth)... Resampling (scaling) the image to a new file using the same format would probably work in most cases though (probably not so good for progressive JPEGs, unless you can resample each scan separately).
If you can live with disk buffer though, as the second best option, I have created some classes that allows for BufferedImages to be backed by nio MappedByteBuffers (memory-mapped file Buffers, kind of like virtual memory). While performance isn't really like in-memory images, it's also not entirely useless. Have a look at MappedImageFactory and MappedFileBuffer.

I've written a PNG encoder/decoder that does that (read and write progressively, which only requires to store a row in memory) for PNG format: PNGJ
I don't know if there is something similar with JPEG

Split and Merge Image wthout loss in image quality in Java

I want split an image in number of chunks and merge again to create original image
I used this method to spit image http://kalanir.blogspot.in/2010/02/how-to-split-image-into-chunks-java.html and for merge used this http://kalanir.blogspot.in/2010/02/how-to-merge-multiple-images-into-one.html
But after merging i fount that my new image size is reduce compare to original.
How i do split and merging of image without loosing any information

Not sure if you are:
reading files into memory RGB data
splitting
joining them
writing a new file
But in that case:
When reading the JPEGs you are obtaining unencoded, uncompressed, lossless data.
When you join them you are encoding again (lossing data) to generate the file.
If the second encoding doesn't do what the original software that encoded the image did, it'll be different.
A starting point is using the same JPEG quality. You can indicate a number from 1 to 100 indicating how much quality you want to keep at a size cost. That number should be at least equal to the number used to encode the original image.
I don't know how exactly to get that or to write that using the libraries you've used, but it should be available.

You can use different image processing java api's like imgscalr, ImageJ, Marvin...etc

The right way to manage a big matrix in Java

I'm working with a big matrix (not sparse), it contains about 10^10 double.
Of course I cannot keep it in memory, and I need just 1 row at time.
I thought to split it in files, every file 1 row (it requires a lot of files) and just read a file every time I need a row. do you know any more efficient way?

Why do you want to store it in different files? Can't u use a single file?
You could use functions inside RandomAccessFile class to perform the reading from that File.

So, 800KB per file, sounds like a good division. Nothing really stops you from using one giant file, of course. A matrix, at least one like yours that isn't sparse, can be considered a file of fixed length records, making random access a trivial matter.
If you do store it one file per row, I might suggest making a directory tree corresponding to decimal digits, so 0/0/0/0 through 9/9/9/9.
Considerations one way or the other...
is it being backed up? Do you have high-capacity backup media or something ordinary?
does this file ever change?
if it does change and it is backed up, does it change all at once or are changes localized?

It depends on the algorithms you want to execute, but I guess that in most cases a representation where each file contains some square or rectangular region would be better.
For example, matrix multiplication can be done recursively by breaking a matrix into submatrices.

If you are going to be saving it in a file, I believe serializing it will save space/time over storing it as text.
Serializing the doubles will store them as 2 bytes (plus serialization overhead) and means that you will not have to convert these doubles back and forth to and from Strings when saving or loading the file.

I'd suggest to use a disk-persistent cache like Ehcache. Just configure it to keep as many fragments of your matrix in memory as you like and it will take care of the serialization. All you have to do is decide on the way of fragmentation.
Another approach that comes to my mind is using Terracotta (which recently bought Ehache by the way). It's great to get a large network-attached heap that can easily manage your 10^10 double values without caring about it in code at all.