Develop Reference

Extract then stack the high and low bytes from an array of image bytes

I'm relatively new to doing image compression on the byte level, and am currently working on a java image preprocessor that will take a bmp image, convert it to an 8-bit unsigned grayscale, then stack its bytes according to high and low before exporting and compressing it. After some extensive research and testing various methods of byte extraction, I'm still not seeing the results I need. Before I continue, it should be noted that all of these images are originally in DICOM format, and I'm using the ij.plugin.DICOM package to extract the pixel data as a bmp image.
The following description is represented by code bellow. Currently, I'm reading in the original image as a buffered image, converting it to grayscale, then getting the image bytes from the Raster. Then I take those bytes, and using some other code I found on stackoverflow and "converting" them to a String representation of binary bits. I then send that string to a character array. The next step might be extraneous, but I wanted to get your input before I removed it (since I'm new at this). I make a Bitset and iterate through the "binary" character array. If the character value is "1", I set that position in the BitSet to true. Then I send the BitSet to another byte array.
Then I make two new byte arrays, one for the high and one for the low byte. Using a for loop, I'm iterating over the "bit" array and storing every 4 "bits" in the high or low byte, depending on where we are in the array.
Lastly, I take the DICOM tag data, make a byte array from it, and then stack the tag array, the high byte array, and the low byte array together. My intended result is to have the image matrix be "split" with the top half containing all the high bytes and the bottom half containing all of the low bytes. I've been told that the tag bytes will be so small, they shouldn't affect the final outcome (I've tested the image without them, just to be sure, and there was no visible difference).
Below is the code. Please let me know if you have any questions, and I will modify my post accordingly. I've tried to include all relevant data. Let me know if you need more.
BufferedImage originalImage = getGrayScale(img.getBufferedImage());//returns an 8-bit unsigned grayscale conversion of the original image
byte[] imageInByte = ((DataBufferByte) originalImage.getRaster().getDataBuffer()).getData();
String binary = toBinary(imageInByte); //converts to a String representation of the binary bits
char[] binCharArray = binary.toCharArray();
BitSet bits = new BitSet(binCharArray.length);
for (int i = 0; i < binCharArray.length; i++) {
if (binCharArray[i] == '1') {
bits.set(i);
}
}
imageInByte = bits.toByteArray();
byte[] high = new byte[(int) imageInByte.length/2];
byte[] low = new byte[(int) imageInByte.length/2];
int highC = 0;
int lowC = 0;
boolean change = false; //start out storing in the high bit
//change will = true on very first run. While true, load in the high byte array. Else low byte
for(int i = 0; i < imageInByte.length; i++){
if(i % 4 == 0){
change = !change;
}
if(change){
high[highC] = imageInByte[i];
highC++;
} else {
low[lowC] = imageInByte[i];
lowC++;
}
}
//old code from a previous attempt.
// for (int j = 0; j < imageInByte.length; j++) {
// byte h = (byte) (imageInByte[j] & 0xFF);
// byte l = (byte) ((imageInByte[j] >> 8) & 0xFF);
// high[j] = h;
// low[j] = l;
// }
OutputStream out = null;
//add this array to the image array. It goes at the beginning.
byte[] tagBytes = dicomTags.getBytes();
currProcessingImageTagLength = tagBytes.length;
imageInByte = new byte[high.length + low.length + tagBytes.length];
System.arraycopy(tagBytes, 0, imageInByte, 0, tagBytes.length);
System.arraycopy(high, 0, imageInByte, tagBytes.length, high.length);
System.arraycopy(low, 0, imageInByte, tagBytes.length + high.length, low.length);
BufferedImage bImageFromConvert = new BufferedImage(dimWidth, dimHeight, BufferedImage.TYPE_BYTE_GRAY);//dimWidth and dimHeight are the image dimensions, stored much earlier in this function
byte[] bufferHolder = ((DataBufferByte) bImageFromConvert.getRaster().getDataBuffer()).getData();
System.arraycopy(imageInByte, 0, bufferHolder, 0, bufferHolder.length);
//This is where I try and write the final image before sending it off to an image compressor
ImageIO.write(bImageFromConvert, "bmp", new File(
directory + fileName + "_Compressed.bmp"));
return new File(directory + fileName + "_Compressed.bmp");
And below is the toBinary function in case you were interested:
private static String toBinary(byte[] bytes) {
StringBuilder sb = new StringBuilder(bytes.length * Byte.SIZE);
for (int i = 0; i < Byte.SIZE * bytes.length; i++) {
sb.append((bytes[i / Byte.SIZE] << i % Byte.SIZE & 0x80) == 0 ? '0' : '1');
}
return sb.toString();
}
Thank you so much for your help! I've spent nearly 20 hours now trying to solve this one problem. It's been a huge headache, and any insight you have would be appreciated.
EDIT: Here's the getGreyScale function:
public static BufferedImage getGrayScale(BufferedImage inputImage) {
BufferedImage img = new BufferedImage(inputImage.getWidth(), inputImage.getHeight(), BufferedImage.TYPE_BYTE_GRAY);
Graphics g = img.getGraphics();
g.drawImage(inputImage, 0, 0, null);
g.dispose();
return img;
}
EDIT 2: I've added some images upon request.
Current output:
current image
Note, I can't post the images with the "expected" high byte and low byte outcome due to my reputation being lower than 10.

This says every 4 bytes change; thats not what you intend:
for(int i = 0; i < imageInByte.length; i++){
if(i % 4 == 0){
change = !change;
}
if(change){
high[highC] = imageInByte[i];
highC++;
} else {
low[lowC] = imageInByte[i];
lowC++;
}
}
I would replace it with this, from your earlier attempt
for (int j = 0; j < imageInByte.length; j+=2) {
byte h = (byte) (imageInByte[j] & 0xF0);
byte h2 = (byte) (imageInByte[j+1] & 0xF0);
byte l = (byte) (imageInByte[j] & 0x0f);
byte l2 = (byte) (imageInByte[j+1] & 0x0f);
high[j/2] = h|(h2>>4);
low[j/2] = (l<<4)|l2;
}

Calculating CRC for int32 data format using the Modbus protocol

I am connecting to a device using the modbus protocol. I need to obtain 3 values from the machine. The first value is of the data format int16 and when I send an example byte array:
static byte[] hz = new byte[] { (byte) 0x01, (byte) 0x03, (byte) 0x00,
(byte) 0x33, (byte) 0x00, (byte) 0x01 };
and use a CRC calculation method I obtained from a previous question I asked on the subject.
// Compute the MODBUS RTU CRC
private static int ModRTU_CRC(byte[] buf, int len)
{
int crc = 0xFFFF;
for (int pos = 0; pos < len; pos++) {
crc ^= (int)buf[pos]; // XOR byte into least sig. byte of crc
for (int i = 8; i != 0; i--) { // Loop over each bit
if ((crc & 0x0001) != 0) { // If the LSB is set
crc >>= 1; // Shift right and XOR 0xA001
crc ^= 0xA001;
}
else // Else LSB is not set
crc >>= 1; // Just shift right
}
}
// Note, this number has low and high bytes swapped, so use it accordingly (or swap bytes)
return crc;
}
I can recieve a response. However, the other two values are of the int32 data format and do not return a reply when I use this method. To help troubleshoot I am using a program called Realterm. to fire off the commands as well. I use it to append a Modbus 16 CRC to the end of the byte stream and send it, this works for all three and returns the desired reply. Is this a case of the data format not working with this specific calculation formula? Whats the difference between CRC16 and Modbus16?

Modbus16 is a CRC16. CRC calculations have several parameters:
the bit width, in this case 16
the polynomial, in this case 0xA001
the initial value,in this case 0xFFFF
the bit order
whether the final CRC is inverted with an XOR.
There are quite a number of CRC16s defined, with different values for these parameters, and this appears to be one of them. See the Wikipedia article on Cyclic Redundancy Checks for more informaton.

class Obliczenia {
short POLYNOM = (short) 0x0A001;
short[] TAB = {2,3,8,0x13,0x88,1,0x90,0,0x3c,2,0};
short crc = (short) 0xffff;
short CRC_LByte,CRC_HByte;
public Obliczenia() {
for (short dana : TAB) {
crc= CRC16( crc, dana);
}
System.out.println("KOD CRC="+Integer.toHexString(crc&0xffff));
CRC_LByte=(short)(crc & 0x00ff);
CRC_HByte=(short)((crc & 0xff00)/256);
System.out.println(" W ramce CRC_LByte="+Integer.toHexString(CRC_LByte)+ " CRC_HByte "+Integer.toHexString(CRC_HByte));
}
short CRC16(short crct, short data) {
crct = (short) (((crct ^ data) | 0xff00) & (crct | 0x00ff));
for (int i = 0; i < 8; i++) {
boolean LSB = ((short) (crct & 1)) == 1;
crct=(short) ((crct >>> 1)&0x7fff);
if (LSB) {
crct = (short) (crct ^ POLYNOM);
}
}
return crct;
}
}

Java 16bit CRC packet validation

I have an Avl Packet that I'm recieving through GPRS from a Device.
The protocol manual says the packet has a 16bit CRC on the last 4 bytes and a source code for CRC calculation is given:
public static int getCrc16(Byte[] buffer) {
return getCrc16(buffer, 0, buffer.length, 0xA001, 0);
}
public synchronized static int getCrc16(Byte[] buffer, int offset, int bufLen, int polynom, int preset) {
preset &= 0xFFFF;
polynom &= 0xFFFF;
int crc = preset;
for (int i = 0; i < bufLen; i++) {
int data = buffer[i + offset] & 0xFF;
crc ^= data;
for (int j = 0; j < 8; j++) {
if ((crc & 0x0001) != 0) {
crc = (crc >> 1) ^ polynom;
} else {
crc = crc >> 1;
}
}
}
return crc & 0xFFFF;
}
So I get the CRC number the packet sends me,then I call getCrc16 for the Byte array in which I have stored the packet and then compare the two numbers right??
Here is the code I use inside my program:
public static String toBinaryString(byte n) {
StringBuilder sb = new StringBuilder("00000000");
for (int bit = 0; bit < 8; bit++) {
if (((n >> bit) & 1) > 0) {
sb.setCharAt(7 - bit, '1');
}
}
return sb.toString();
}
int CalculatedCRC = getCrc16(AvlPacket);
System.out.println("Calculated CRC= " + CalculatedCRC);
int index = (AvlPacket.length)-4;
String BinaryRecievedCRC = "";
for (int j = 0; j < 4; j++) {
BinaryRecievedCRC+= toBinaryString(AvlPacket[index]);
index+=1;
}
int RecievedCRC = Integer.parseInt(BinaryRecievedCRC, 2);
System.out.println("Recieved CRC= " + RecievedCRC);
toBinaryString() converts a byte to it's binary from and puts it into a string!
So I calculate the CRC through getCrc16() given to me from the manual.Then take an index 4 bytes before the end of the packet so I can read the last 4 bytes and get the CRC sent with the packet!
The for loop takes each of the last bytes and with toBinaryString() combines all them in binary form and into a String!So I got something like 0000000000000000101011011101001 (The manual states that first two bytes are always zeroes because its a 16bit CRC)
So I just parse the Binary String into a signed int and Compare the two CRCs...!
Yet I get Results like :
Calculated CRC= 21395
-----Recieved CRC= 30416
or
Calculated CRC= 56084
-----Recieved CRC= 10504
I've tested with many packets and they can't all have loss of data..And I'm parsing the data too so I know that the data I get is correct!
What am I missing in all this??

There is probably something wrong with the documentation wording (or your understanding of it). If there is a 16-Bit CRC in a packet, its most likely occupying two bytes, not four (in binary form). If it were a decimal, even four bytes wouldn't suffice (you would need 5 didgts to store it as unsigned decimal string).
Your code shows you do conversions (but I can't see what kind of conversion its supposed to do):
BinaryRecievedCRC+= toBinaryString(AvlPacket[index]);
I would expect the CRC to be store in the data in binary form, so I assume the only thing you need to figure out are which endianess is used and where the CRC is stored in the data.
Edit: Judging from your comment you would need to extract the CRC like this:
public int getCRC(byte[] data, int index) {
return ((data[index] & 0xFF) << 8)) | (data[index + 1] & 0xFF);
}

So I get the CRC number the packet sends me,then I call getCrc16 for
the Byte array in which I have stored the packet and then compare the
two numbers right??
Wrong. You calculate the CRC over the entire message, including the CRC bytes, and the result should be zero.

Problem Solved!
The problem was that the Packet had 8 other bytes before getting into the Data part!
So I had to exclude those first 8 bytes along with the last 4 bytes of the sent CRC before calculating the CRC!
Now the numbers agree and the above code is correct with the exception that the for loop in the getCrc16 function starts from i=8 (so as to skip the first 8 bytes of the packet which do no belong to the Data part!)
Thank you all for your time!

Java byte arrays and copying ints into them

What's the best way to put an int at a certain point in a byte[] array?
Say you have a byte array:
byte[] bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
int someInt = 12355; //0x43, 0x30
How can I do like bytes[4] = someInt; so that now bytes[4] will equal 0x43 and bytes[5] will be equal to 0x30?
I'm used to just using memcpy with C++ and don't know the alternatives in Java.
Thanks

If you want also high 0-bytes of the int put into the byte[]:
void place(int num, byte[] store, int where){
for(int i = 0; i < 4; ++i){
store[where+i] = (byte)(num & 0xFF);
num >>= 8;
}
}
If you only want the bytes to the highest nonzero byte:
void place(int num, byte[] store, int where){
while(num != 0){
store[where++] = (byte)(num & 0xFF);
num >>>= 8;
}
}
If you want the bytes big-endian (highest byte at lowest index), the version storing all four bytes is very easy, the other one slightly more difficult:
void placeBigEndian(int num , byte[] store, int where){
for(int i = 3; i >= 0; --i){
store[where+i] = (byte)(num & 0xFF);
num >>= 8;
}
}
void placeBigEndian(int num, byte[] store, int where){
in mask = 0xFF000000, shift = 24;
while((mask & num) == 0){
mask >>>= 8;
shift -= 8;
}
while(shift > 0){
store[where++] = (byte)((num & mask) >>> shift);
mask >>>= 8;
shift -= 8;
}
}

Note, you assume a big endian ordering! x86 is little endian... What's more, your int is 32bits long, hence 0x00004330 in big endian.
If this is what you want, use a ByteBuffer (which uses big endian ordering by default):
ByteBuffer buf = ByteBuffer.allocate(8);
// then use buf.putInt(yourint, index)
// buf.get(index) will read byte at index index, starting from 0

I don't see the problem, it looks like you solved it your own way:
public static void putShort(bytes[] array, int position, short value)
{
byte leftByte = (byte) (value >>> 8);
byte rightByte = (byte) (value & 0xFF);
array[position] = leftByte;
array[position + 1] = rightByte;
}
Note that an int is 4 bytes and a short is 2 bytes.

First of all, in Java you don't need to initialize byte arrays to zeroes. All arrays are initialized on construction time to 0/false/null.
Second, ints are signed 32-bit big-endian integers, so 12355 is actually 0x00003043. If you want to use 16-bit integers, use the short type.
Then, to get the individual bytes in your integer, you could do:
bytes[ i ] = (byte) (someInt >> 24);
bytes[ i+1 ] = (byte) (someInt >> 16);
bytes[ i+2 ] = (byte) (someInt >> 8);
bytes[ i+3 ] = (byte) (someInt);
The conversion to byte truncates the remaining bits, so no & 0xFF mask is needed. I'm assuming i is the index of the array. To change the endianness, swap the offsets at the indices.

One approach would be to use a DataOutputStream and it's writeInt() method, wrapped around a ByteArrayOutputStream. e.g. (with no error-handling)
public byte[] writeIntAtPositionX(int position, int iVal) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(baos);
// now, advancing to a specific spot is awkward.
// presumably you are actually writing other stuff out before the integer
// but if you really want to advance to a specific position
for (int i = 0; i < position; i++)
dos.writeByte(0);
dos.writeInt(iVal);
dos.flush();
dos.close();
return baos.toByteArray();
}
The big advantage of this method is that the guys who wrote Java figured out the byte ordering and the masking with 0xFF and all that stuff. Plus, if you ever envision writing doubles, shorts, longs or Strings etc. to your buffer you won't need to add all those methods, the work is already done.

byte array to short array and back again in java

I'm having some issues taking audio data stored in a byte array, converting it to a big-endian short array, encoding it, then changing it back into a byte array. Here is what I have. The original audio data is stored in audioBytes2. I am using the same format for decode with a minus on the cos function instead. Unfortunately, changing the byte and short data types is non-negotiable.
short[] audioData = null;
int nlengthInSamples = audioBytes2.length / 2;
audioData = new short[nlengthInSamples];
for (int i = 0; i < nlengthInSamples; i++) {
short MSB = (short) audioBytes2[2*i+1];
short LSB = (short) audioBytes2[2*i];
audioData[i] = (short) (MSB << 8 | (255 & LSB));
}
int i = 0;
while (i < audioData.length) {
audioData[i] = (short)(audioData[i] + (short)5*Math.cos(2*Math.PI*i/(((Number)EncodeBox.getValue()).intValue())));
i++;
}
short x = 0;
i = 0;
while (i < audioData.length) {
x = audioData[i];
audioBytes2[2*i+1] = (byte)(x >>> 0);
audioBytes2[2*i] = (byte)(x >>> 8);
i++;
}
I have done everything that I can think of to make this work, but the closest I've come is getting it to work every other encode/decode and I have no idea why. Thanks for any help.

I also suggest you try ByteBuffer.
byte[] bytes = {};
short[] shorts = new short[bytes.length/2];
// to turn bytes to shorts as either big endian or little endian.
ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(shorts);
// to turn shorts back to bytes.
byte[] bytes2 = new byte[shortsA.length * 2];
ByteBuffer.wrap(bytes2).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().put(shortsA);

public short bytesToShort(byte[] bytes) {
return ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN).getShort();
}
public byte[] shortToBytes(short value) {
return ByteBuffer.allocate(2).order(ByteOrder.LITTLE_ENDIAN).putShort(value).array();
}

How about some ByteBuffers?
byte[] payload = new byte[]{0x7F,0x1B,0x10,0x11};
ByteBuffer bb = ByteBuffer.wrap(payload).order(ByteOrder.BIG_ENDIAN);
ShortBuffer sb = bb.asShortBuffer();
while(sb.hasRemaining()){
System.out.println(sb.get());
}

byte[2] bytes;
int r = bytes[1] & 0xFF;
r = (r << 8) | (bytes[0] & 0xFF);
short s = (short)r;

Your code is doing little-endian shorts, not big. You've the indexing for MSB and LSB swapped.
Since you are using big-endian shorts, you could be using a DataInputStream wrapped around a ByteArrayInputStream (and DataOutputStream/ByteArrayOutputStream) on the other end, rather than doing your own decoding.
If you're getting every other decode working, I'd guess you've got an odd number of bytes, or an off-by-one error elsewhere which is causing your mistake to get fixed on every other pass.
Finally, I'd step through the array with i+=2 and use MSB= arr[i] and LSB=arr[i+1] rather than multiplying by 2, but that's just me.

It looks like you are swapping the byte order between reading the bytes in and writing them back out (unsure if this is intentional or not).