I am working with some low capacity module and I need to compress the data as much as possible. The data will look like this:
DeviceEvent:
1 byte:
2 bits for status (00 each time)
6 bits for rgb color (3 x 2 bits)
2 bytes: number of minutes from now to a certain datetime
I need to create a constructor (preferably 2 constructors) for conversion from/to:
Event:
byte[] color (rgb, colors will get simplified to only 64 available)
some datetime (but I will get the integer for difference in minutes and it will be small enough to fit in two bits)
So basically I need:
byte[3] color <-> 1 byte status and color
int minutes <-> byte[2]
minutes
I will be thankful for any help
I'm not very sure what is your problem, probably this will help:
final byte red = 1; // 01 binary
final byte green = 2; // 10 binary
final byte blue = 3; // 11 binary
final byte finalColor = (byte) ((red & 0x3) << 4) | ((green & 0x3) << 2) | (blue & 0x3);
System.out.println(finalColor);// finalColor is 011011 = 27 decimal
final int minutes = 0x1234; // first byte is 0x12, second byte is 0x34
final byte[] bytes = {(byte) (((minutes) >>> 8) & 0xff), (byte) (minutes & 0xff)};
System.out.println(bytes[0]); // 0x12 = 18 decimal
System.out.println(bytes[1]); // 0x34 = 52 decimal
I am not sure what the second problem is. So I made these two functions that might help you:
public static int convertToInt(int a, int b, int c, int d) {
a = Math.min(a, 255);
b = Math.min(b, 255);
c = Math.min(c, 255);
d = Math.min(d, 255);
return ((a & 0xFF) << 24) | ((b & 0xFF) << 16) | ((c & 0xFF) << 8) | (d & 0xFF);
}
public static int[] extractComponents(int data) {
int a = (data >> 24) & 0xFF;
int b = (data >> 16) & 0xFF;
int c = (data >> 8) & 0xFF;
int d = data & 0xFF;
return new int[] {a, b, c, d};
}
The convertToInt function takes four numbers(that are less than 255) and puts them all in one int.
The extractComponents function does the opposite.
This is an example:
int data = 0xC8E0601B;
int[] dataA = extractComponents(data);
for(int i = 0 ; i < dataA.length; i++) System.out.printf("%x\n", dataA[i]);
System.out.printf("%x\n", convertToInt(dataA[0], dataA[1], dataA[2], dataA[3]));
This is probably a basic question for out more experienced programmers out there. I'm a bit of a noob and can't work this one out. I'm trying to unpack a binary file and the doco is not too clear on how floats are stored. I have found a routine that does this, but it will only work if I pass an integer array of the bytes. The correct answer is -1865.0. I need to be able to pass the byte array and get the correct answer. How do I need to change the code to make float4byte return -1865.0. Thanks in advance.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
public class HelloWorld {
public static void main(String[] args) {
byte[] bytes = {(byte) 0xC3,(byte) 0X74,(byte) 0X90,(byte) 0X00 };
int[] ints = {(int) 0xC3,(int) 0X74,(int) 0X90,(int) 0X00 };
// This give the wrong answer
float f = ByteBuffer.wrap(bytes).order(ByteOrder.BIG_ENDIAN).getFloat();
System.out.println("VAL ByteBuffer BI: " + f);
// This give the wrong answer
f = ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN).getFloat();
System.out.println("VAL ByteBuffer LI: " + f);
//This gives the RIGHT answer
f = float4int (ints[0], ints[1], ints[2], ints[3]);
System.out.println("VAL Integer : " + f);
// This gives the wrong answer
f = float4byte (bytes[0], bytes[1], bytes[2], bytes[3]);
System.out.println("VAL Bytes : " + f);
}
private static float float4int(int a, int b, int c, int d)
{
int sgn, mant, exp;
System.out.println ("IN Int: "+String.format("%02X ", a)+
String.format("%02X ", b)+String.format("%02X ", c)+String.format("%02X ", d));
mant = b << 16 | c << 8 | d;
if (mant == 0) return 0.0f;
sgn = -(((a & 128) >> 6) - 1);
exp = (a & 127) - 64;
return (float) (sgn * Math.pow(16.0, exp - 6) * mant);
}
private static float float4byte(byte a, byte b, byte c, byte d)
{
int sgn, mant, exp;
System.out.println ("IN Byte : "+String.format("%02X ", a)+
String.format("%02X ", b)+String.format("%02X ", c)+String.format("%02X ", d));
mant = b << 16 | c << 8 | d;
if (mant == 0) return 0.0f;
sgn = -(((a & 128) >> 6) - 1);
exp = (a & 127) - 64;
return (float) (sgn * Math.pow(16.0, exp - 6) * mant);
}
}
The reason why your solution with ByteBuffer doesn't work: the bytes do not match the (Java) internal representation of the float value.
The Java representation is
System.out.println(Integer.toHexString(Float.floatToIntBits(-1865.0f)));
which gives c4e92000
bytes are signed in Java. When calculating the mantissa mant, the bytes are implicitly converted from bytes to ints - with the sign "extended", i.e. (byte)0x90 (decimal -112) gets converted 0xFFFFFF90 (32 bits int). However what you want is just the original bytes' 8 bits (0x00000090).
In order to compensate for the effect of sign extension, it suffices to change one line:
mant = (b & 0xFF) << 16 | (c & 0xFF) << 8 | (d & 0xFF)
Here, in (c & 0xFF), the 1-bits caused by sign extension are stripped after (implicit) conversion to int.
Edit:
The repacking of floats could be done via the IEEE 754 representation which can be obtained by Float.floatToIntBits (which avoids using slow logarithms). Some complexity in the code is caused by the change of base from 2 to 16:
private static byte[] byte4float(float f) {
assert !Float.isNaN(f);
// see also JavaDoc of Float.intBitsToFloat(int)
int bits = Float.floatToIntBits(f);
int s = (bits >> 31) == 0 ? 1 : -1;
int e = (bits >> 23) & 0xFF;
int m = (e == 0) ? (bits & 0x7FFFFF) << 1 : (bits& 0x7FFFFF) | 0x800000;
int exp = (e - 150) / 4 + 6;
int mant;
int mantissaShift = (e - 150) % 4; // compensate for base 16
if (mantissaShift >= 0) mant = m << mantissaShift;
else { mant = m << (mantissaShift + 4); exp--; }
if (mant > 0xFFFFFFF) { mant >>= 4; exp++; } // loose of precision
byte a = (byte) ((1 - s) << 6 | (exp + 64));
return new byte[]{ a, (byte) (mant >> 16), (byte) (mant >> 8), (byte) mant };
}
The code does not take into account any rules that may exist for the packaging, e.g. for representing zero or normalization of the mantissa. But it might serve as a starting point.
Thanks to #halfbit and a bit of testing and minor changes, this routine appears convert IEEE 754 float into IBM float.
public static byte[] byte4float(float f) {
assert !Float.isNaN(f);
// see also JavaDoc of Float.intBitsToFloat(int)
int bits = Float.floatToIntBits(f);
int s = (bits >> 31) == 0 ? 1 : -1;
int e = (bits >> 23) & 0xFF;
int m = (e == 0) ? (bits & 0x7FFFFF) << 1 : (bits& 0x7FFFFF) | 0x800000;
int exp = (e - 150) / 4 + 6;
int mant;
int mantissaShift = (e - 150) % 4; // compensate for base 16
if (mantissaShift >= 0) mant = m >> mantissaShift;
else mant = m >> (Math.abs(mantissaShift));
if (mant > 0xFFFFFFF) { mant >>= 4; exp++; } // loose of precision */
byte a = (byte) ((1 - s) << 6 | (exp + 64));
return new byte[]{ a, (byte) (mant >> 16), (byte) (mant >> 8), (byte) mant };
}
I think this is right and appears to be working.
I need hash over pretty large files which is stored on distributed FS. I'm able to process parts of file with much more better performance than whole file so I'd like to be able to calculate hash over parts and then sum it.
I'm thinking about CRC64 as hashing algorithm but I have no clue how to use its theoretical 'linear function' property so I can sum CRC over parts of file. Any recommendation? Anything I missed here?
Additional notes why I'm looking at CRC64:
I can control file blocks but because of application nature they need to have different size (up to 1 byte, no any fixed blocks are possible).
I know about CRC32 implementation (zlib) which includes way to sum CRC over parts but I'd like something more wider. 8 bytes look nice for me.
I know CRC is pretty fast. I'd like to get profit from this as file can be really huge (up to few Gb).
Decided that this was generally useful enough to write and make available:
/* crc64.c -- compute CRC-64
* Copyright (C) 2013 Mark Adler
* Version 1.4 16 Dec 2013 Mark Adler
*/
/*
This software is provided 'as-is', without any express or implied
warranty. In no event will the author be held liable for any damages
arising from the use of this software.
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
Mark Adler
madler#alumni.caltech.edu
*/
/* Compute CRC-64 in the manner of xz, using the ECMA-182 polynomial,
bit-reversed, with one's complement pre and post processing. Provide a
means to combine separately computed CRC-64's. */
/* Version history:
1.0 13 Dec 2013 First version
1.1 13 Dec 2013 Fix comments in test code
1.2 14 Dec 2013 Determine endianess at run time
1.3 15 Dec 2013 Add eight-byte processing for big endian as well
Make use of the pthread library optional
1.4 16 Dec 2013 Make once variable volatile for limited thread protection
*/
#include <stdio.h>
#include <inttypes.h>
#include <assert.h>
/* The include of pthread.h below can be commented out in order to not use the
pthread library for table initialization. In that case, the initialization
will not be thread-safe. That's fine, so long as it can be assured that
there is only one thread using crc64(). */
#include <pthread.h> /* link with -lpthread */
/* 64-bit CRC polynomial with these coefficients, but reversed:
64, 62, 57, 55, 54, 53, 52, 47, 46, 45, 40, 39, 38, 37, 35, 33, 32,
31, 29, 27, 24, 23, 22, 21, 19, 17, 13, 12, 10, 9, 7, 4, 1, 0 */
#define POLY UINT64_C(0xc96c5795d7870f42)
/* Tables for CRC calculation -- filled in by initialization functions that are
called once. These could be replaced by constant tables generated in the
same way. There are two tables, one for each endianess. Since these are
static, i.e. local, one should be compiled out of existence if the compiler
can evaluate the endianess check in crc64() at compile time. */
static uint64_t crc64_little_table[8][256];
static uint64_t crc64_big_table[8][256];
/* Fill in the CRC-64 constants table. */
static void crc64_init(uint64_t table[][256])
{
unsigned n, k;
uint64_t crc;
/* generate CRC-64's for all single byte sequences */
for (n = 0; n < 256; n++) {
crc = n;
for (k = 0; k < 8; k++)
crc = crc & 1 ? POLY ^ (crc >> 1) : crc >> 1;
table[0][n] = crc;
}
/* generate CRC-64's for those followed by 1 to 7 zeros */
for (n = 0; n < 256; n++) {
crc = table[0][n];
for (k = 1; k < 8; k++) {
crc = table[0][crc & 0xff] ^ (crc >> 8);
table[k][n] = crc;
}
}
}
/* This function is called once to initialize the CRC-64 table for use on a
little-endian architecture. */
static void crc64_little_init(void)
{
crc64_init(crc64_little_table);
}
/* Reverse the bytes in a 64-bit word. */
static inline uint64_t rev8(uint64_t a)
{
uint64_t m;
m = UINT64_C(0xff00ff00ff00ff);
a = ((a >> 8) & m) | (a & m) << 8;
m = UINT64_C(0xffff0000ffff);
a = ((a >> 16) & m) | (a & m) << 16;
return a >> 32 | a << 32;
}
/* This function is called once to initialize the CRC-64 table for use on a
big-endian architecture. */
static void crc64_big_init(void)
{
unsigned k, n;
crc64_init(crc64_big_table);
for (k = 0; k < 8; k++)
for (n = 0; n < 256; n++)
crc64_big_table[k][n] = rev8(crc64_big_table[k][n]);
}
/* Run the init() function exactly once. If pthread.h is not included, then
this macro will use a simple static state variable for the purpose, which is
not thread-safe. The init function must be of the type void init(void). */
#ifdef PTHREAD_ONCE_INIT
# define ONCE(init) \
do { \
static pthread_once_t once = PTHREAD_ONCE_INIT; \
pthread_once(&once, init); \
} while (0)
#else
# define ONCE(init) \
do { \
static volatile int once = 1; \
if (once) { \
if (once++ == 1) { \
init(); \
once = 0; \
} \
else \
while (once) \
; \
} \
} while (0)
#endif
/* Calculate a CRC-64 eight bytes at a time on a little-endian architecture. */
static inline uint64_t crc64_little(uint64_t crc, void *buf, size_t len)
{
unsigned char *next = buf;
ONCE(crc64_little_init);
crc = ~crc;
while (len && ((uintptr_t)next & 7) != 0) {
crc = crc64_little_table[0][(crc ^ *next++) & 0xff] ^ (crc >> 8);
len--;
}
while (len >= 8) {
crc ^= *(uint64_t *)next;
crc = crc64_little_table[7][crc & 0xff] ^
crc64_little_table[6][(crc >> 8) & 0xff] ^
crc64_little_table[5][(crc >> 16) & 0xff] ^
crc64_little_table[4][(crc >> 24) & 0xff] ^
crc64_little_table[3][(crc >> 32) & 0xff] ^
crc64_little_table[2][(crc >> 40) & 0xff] ^
crc64_little_table[1][(crc >> 48) & 0xff] ^
crc64_little_table[0][crc >> 56];
next += 8;
len -= 8;
}
while (len) {
crc = crc64_little_table[0][(crc ^ *next++) & 0xff] ^ (crc >> 8);
len--;
}
return ~crc;
}
/* Calculate a CRC-64 eight bytes at a time on a big-endian architecture. */
static inline uint64_t crc64_big(uint64_t crc, void *buf, size_t len)
{
unsigned char *next = buf;
ONCE(crc64_big_init);
crc = ~rev8(crc);
while (len && ((uintptr_t)next & 7) != 0) {
crc = crc64_big_table[0][(crc >> 56) ^ *next++] ^ (crc << 8);
len--;
}
while (len >= 8) {
crc ^= *(uint64_t *)next;
crc = crc64_big_table[0][crc & 0xff] ^
crc64_big_table[1][(crc >> 8) & 0xff] ^
crc64_big_table[2][(crc >> 16) & 0xff] ^
crc64_big_table[3][(crc >> 24) & 0xff] ^
crc64_big_table[4][(crc >> 32) & 0xff] ^
crc64_big_table[5][(crc >> 40) & 0xff] ^
crc64_big_table[6][(crc >> 48) & 0xff] ^
crc64_big_table[7][crc >> 56];
next += 8;
len -= 8;
}
while (len) {
crc = crc64_big_table[0][(crc >> 56) ^ *next++] ^ (crc << 8);
len--;
}
return ~rev8(crc);
}
/* Return the CRC-64 of buf[0..len-1] with initial crc, processing eight bytes
at a time. This selects one of two routines depending on the endianess of
the architecture. A good optimizing compiler will determine the endianess
at compile time if it can, and get rid of the unused code and table. If the
endianess can be changed at run time, then this code will handle that as
well, initializing and using two tables, if called upon to do so. */
uint64_t crc64(uint64_t crc, void *buf, size_t len)
{
uint64_t n = 1;
return *(char *)&n ? crc64_little(crc, buf, len) :
crc64_big(crc, buf, len);
}
#define GF2_DIM 64 /* dimension of GF(2) vectors (length of CRC) */
static uint64_t gf2_matrix_times(uint64_t *mat, uint64_t vec)
{
uint64_t sum;
sum = 0;
while (vec) {
if (vec & 1)
sum ^= *mat;
vec >>= 1;
mat++;
}
return sum;
}
static void gf2_matrix_square(uint64_t *square, uint64_t *mat)
{
unsigned n;
for (n = 0; n < GF2_DIM; n++)
square[n] = gf2_matrix_times(mat, mat[n]);
}
/* Return the CRC-64 of two sequential blocks, where crc1 is the CRC-64 of the
first block, crc2 is the CRC-64 of the second block, and len2 is the length
of the second block. */
uint64_t crc64_combine(uint64_t crc1, uint64_t crc2, uintmax_t len2)
{
unsigned n;
uint64_t row;
uint64_t even[GF2_DIM]; /* even-power-of-two zeros operator */
uint64_t odd[GF2_DIM]; /* odd-power-of-two zeros operator */
/* degenerate case */
if (len2 == 0)
return crc1;
/* put operator for one zero bit in odd */
odd[0] = POLY; /* CRC-64 polynomial */
row = 1;
for (n = 1; n < GF2_DIM; n++) {
odd[n] = row;
row <<= 1;
}
/* put operator for two zero bits in even */
gf2_matrix_square(even, odd);
/* put operator for four zero bits in odd */
gf2_matrix_square(odd, even);
/* apply len2 zeros to crc1 (first square will put the operator for one
zero byte, eight zero bits, in even) */
do {
/* apply zeros operator for this bit of len2 */
gf2_matrix_square(even, odd);
if (len2 & 1)
crc1 = gf2_matrix_times(even, crc1);
len2 >>= 1;
/* if no more bits set, then done */
if (len2 == 0)
break;
/* another iteration of the loop with odd and even swapped */
gf2_matrix_square(odd, even);
if (len2 & 1)
crc1 = gf2_matrix_times(odd, crc1);
len2 >>= 1;
/* if no more bits set, then done */
} while (len2 != 0);
/* return combined crc */
crc1 ^= crc2;
return crc1;
}
/* Test crc64() on vector[0..len-1] which should have CRC-64 crc. Also test
crc64_combine() on vector[] split in two. */
static void crc64_test(void *vector, size_t len, uint64_t crc)
{
uint64_t crc1, crc2;
/* test crc64() */
crc1 = crc64(0, vector, len);
if (crc1 ^ crc)
printf("mismatch: %" PRIx64 ", should be %" PRIx64 "\n", crc1, crc);
/* test crc64_combine() */
crc1 = crc64(0, vector, (len + 1) >> 1);
crc2 = crc64(0, vector + ((len + 1) >> 1), len >> 1);
crc1 = crc64_combine(crc1, crc2, len >> 1);
if (crc1 ^ crc)
printf("mismatch: %" PRIx64 ", should be %" PRIx64 "\n", crc1, crc);
}
/* Test vectors. */
#define TEST1 "123456789"
#define TESTLEN1 9
#define TESTCRC1 UINT64_C(0x995dc9bbdf1939fa)
#define TEST2 "This is a test of the emergency broadcast system."
#define TESTLEN2 49
#define TESTCRC2 UINT64_C(0x27db187fc15bbc72)
int main(void)
{
crc64_test(TEST1, TESTLEN1, TESTCRC1);
crc64_test(TEST2, TESTLEN2, TESTCRC2);
return 0;
}
OK, my contribution to this. Ported to Java.
I cannot win from 8-byte blocks without doing unsafe thing so I removed block calculation.
I stay with ECMA polynom - ISO one looks too transparent as for me.
Of course in final version I will move test code under JUnit.
So here is code:
package com.test;
import java.util.Arrays;
/**
* CRC-64 implementation with ability to combine checksums calculated over different blocks of data.
**/
public class CRC64 {
private final static long POLY = (long) 0xc96c5795d7870f42L; // ECMA-182
/* CRC64 calculation table. */
private final static long[] table;
/* Current CRC value. */
private long value;
static {
table = new long[256];
for (int n = 0; n < 256; n++) {
long crc = n;
for (int k = 0; k < 8; k++) {
if ((crc & 1) == 1) {
crc = (crc >>> 1) ^ POLY;
} else {
crc = (crc >>> 1);
}
}
table[n] = crc;
}
}
public CRC64() {
this.value = 0;
}
public CRC64(long value) {
this.value = value;
}
public CRC64(byte [] b, int len) {
this.value = 0;
update(b, len);
}
/**
* Construct new CRC64 instance from byte array.
**/
public static CRC64 fromBytes(byte [] b) {
long l = 0;
for (int i = 0; i < 4; i++) {
l <<= 8;
l ^= (long) b[i] & 0xFF;
}
return new CRC64(l);
}
/**
* Get 8 byte representation of current CRC64 value.
**/
public byte[] getBytes() {
byte [] b = new byte[8];
for (int i = 0; i < 8; i++) {
b[7 - i] = (byte) (this.value >>> (i * 8));
}
return b;
}
/**
* Get long representation of current CRC64 value.
**/
public long getValue() {
return this.value;
}
/**
* Update CRC64 with new byte block.
**/
public void update(byte [] b, int len) {
int idx = 0;
this.value = ~this.value;
while (len > 0) {
this.value = table[((int) (this.value ^ b[idx])) & 0xff] ^ (this.value >>> 8);
idx++;
len--;
}
this.value = ~this.value;
}
private static final int GF2_DIM = 64; /* dimension of GF(2) vectors (length of CRC) */
private static long gf2MatrixTimes(long [] mat, long vec)
{
long sum = 0;
int idx = 0;
while (vec != 0) {
if ((vec & 1) == 1)
sum ^= mat[idx];
vec >>>= 1;
idx++;
}
return sum;
}
private static void gf2MatrixSquare(long [] square, long [] mat)
{
for (int n = 0; n < GF2_DIM; n++)
square[n] = gf2MatrixTimes(mat, mat[n]);
}
/*
* Return the CRC-64 of two sequential blocks, where summ1 is the CRC-64 of the
* first block, summ2 is the CRC-64 of the second block, and len2 is the length
* of the second block.
*/
static public CRC64 combine(CRC64 summ1, CRC64 summ2, long len2)
{
// degenerate case.
if (len2 == 0)
return new CRC64(summ1.getValue());
int n;
long row;
long [] even = new long[GF2_DIM]; // even-power-of-two zeros operator
long [] odd = new long[GF2_DIM]; // odd-power-of-two zeros operator
// put operator for one zero bit in odd
odd[0] = POLY; // CRC-64 polynomial
row = 1;
for (n = 1; n < GF2_DIM; n++) {
odd[n] = row;
row <<= 1;
}
// put operator for two zero bits in even
gf2MatrixSquare(even, odd);
// put operator for four zero bits in odd
gf2MatrixSquare(odd, even);
// apply len2 zeros to crc1 (first square will put the operator for one
// zero byte, eight zero bits, in even)
long crc1 = summ1.getValue();
long crc2 = summ2.getValue();
do {
// apply zeros operator for this bit of len2
gf2MatrixSquare(even, odd);
if ((len2 & 1) == 1)
crc1 = gf2MatrixTimes(even, crc1);
len2 >>>= 1;
// if no more bits set, then done
if (len2 == 0)
break;
// another iteration of the loop with odd and even swapped
gf2MatrixSquare(odd, even);
if ((len2 & 1) == 1)
crc1 = gf2MatrixTimes(odd, crc1);
len2 >>>= 1;
// if no more bits set, then done
} while (len2 != 0);
// return combined crc.
crc1 ^= crc2;
return new CRC64(crc1);
}
private static void test(byte [] b, int len, long crcValue) throws Exception {
/* Test CRC64 default calculation. */
CRC64 crc = new CRC64(b, len);
if (crc.getValue() != crcValue) {
throw new Exception("mismatch: " + String.format("%016x", crc.getValue())
+ " should be " + String.format("%016x", crcValue));
}
/* test combine() */
CRC64 crc1 = new CRC64(b, (len + 1) >>> 1);
CRC64 crc2 = new CRC64(Arrays.copyOfRange(b, (len + 1) >>> 1, b.length), len >>> 1);
crc = CRC64.combine(crc1, crc2, len >>> 1);
if (crc.getValue() != crcValue) {
throw new Exception("mismatch: " + String.format("%016x", crc.getValue())
+ " should be " + String.format("%016x", crcValue));
}
}
public static void main(String [] args) throws Exception {
final byte[] TEST1 = "123456789".getBytes();
final int TESTLEN1 = 9;
final long TESTCRC1 = 0x995dc9bbdf1939faL; // ECMA.
test(TEST1, TESTLEN1, TESTCRC1);
final byte[] TEST2 = "This is a test of the emergency broadcast system.".getBytes();
final int TESTLEN2 = 49;
final long TESTCRC2 = 0x27db187fc15bbc72L; // ECMA.
test(TEST2, TESTLEN2, TESTCRC2);
final byte[] TEST3 = "IHATEMATH".getBytes();
final int TESTLEN3 = 9;
final long TESTCRC3 = 0x3920e0f66b6ee0c8L; // ECMA.
test(TEST3, TESTLEN3, TESTCRC3);
}
}
In below code, where and what exactly I am doing wrong? I am getting unexpected values when rotating data back to left. What is the fix for this?
public class RotateExample {
public static byte rotateRight(byte bits, int shift) {
return (byte)((bits >>> shift) | (bits << (8 - shift)));
}
public static byte rotateLeft(byte bits, int shift) {
return (byte)((bits << shift) | (bits >>> (8 - shift)));
}
public static void main(String[] args) {
//test 1 failed
byte a = (byte)1;
byte b = rotateRight(a,1);
byte c = rotateLeft(b,1);
System.out.println(a+" "+b+" "+c);
//test 2 passed
a = (byte)1;
b = rotateRight(a,2);
c = rotateLeft(b,2);
System.out.println(a+" "+b+" "+c);
//test 3 failed
a = (byte)2;
b = rotateRight(a,2);
c = rotateLeft(b,2);
System.out.println(a+" "+b+" "+c);
//test 4 passed
a = (byte)2;
b = rotateRight(a,3);
c = rotateLeft(b,3);
System.out.println(a+" "+b+" "+c);
}
}
The following works.
public static byte rotateRight(byte bits, int shift)
{
return (byte)(((bits & 0xff) >>> shift) | ((bits & 0xff) << (8 - shift)));
}
public static byte rotateLeft(byte bits, int shift)
{
return (byte)(((bits & 0xff) << shift) | ((bits & 0xff) >>> (8 - shift)));
}
Refer to this question. Behaviour of unsigned right shift applied to byte variable
This happens because the bytes are converted to signed int before the shift operations takes place.
The current answer didn't work for me, and I realized it's because the righthand side of the operator should be shifting by the size of an Integer (32) rather than 8. This is because the byte is promoted to an Integer prior to any shifting operations taking place on it. Here is my solution:
/**
* Performs a circular bitwise rotation on a byte, rotating right by shift positions.
*
* #param bits the byte to rotate
* #param shift the number of positions to rotate by
* #return the rotated byte
*/
private static byte rotateRight(byte bits, int shift) {
return (byte) (((bits & 0xff) >>> shift) | ((bits & 0xff) << (Integer.SIZE - shift)));
}
/**
* Performs a circular bitwise rotation on a byte, rotating left by shift positions.
*
* #param bits the byte to rotate
* #param shift the number of positions to rotate by
* #return the rotated byte
*/
private static byte rotateLeft(byte bits, int shift)
{
return (byte)(((bits & 0xff) << shift) | ((bits & 0xff) >>> (Integer.SIZE - shift)));
}
It's an old thread, but hopefully someone can find this answer useful!
I'm trying to convert date of birth (three ints) to byte and convert it back but I'm having an issue. I have to convert it by using bit operations and send data over multicast server and receive it and change back to int. Server works fine, but bit operations are hard for me. What's the matter with the code:
Convert:
int D=12;
int M=9;
int Y=1983;
short DMY=0;
DMY = (short)(DMY | (D << 19));
DMY = (short)(DMY | (M << 15));
DMY = (short)(DMY | Y);
byte[] data = new byte[3];
data[0] = (byte)(DMY >>> 8 );
data[1] = (byte)(DMY >>> 16 );
data[2] = (byte)(DMY & 0xffff);
Convert back:
byte[] rec_data = new byte[3];
rec_data = dp.getData();
short Rec_dmy;
Rec_dmy = (short)(rec_data[0] & 0xff);
Rec_dmy = (short) (Rec_dmy << 8);
Rec_dmy = (short)(Rec_dmy | (rec_data[1] & 0xff));
Rec_dmy = (short) (Rec_dmy << 8);
Rec_dmy = (short)(Rec_dmy | (rec_data[2] & 0xffff));
byte tmp = (byte) ((Rec_dmy >>> 19) & 0x1F);
byte tmp2 = (byte) ((Rec_dmy >>> 15) & 0x1FF);
byte tmp3 = (byte) (Rec_dmy & 0x7F);
System.out.println(tmp);
System.out.println(tmp2);
System.out.println(tmp3);
Println gives following answer:
31
-1
63
It's not near original 12 9 1983
Shorts can only hold 16 bits; you are trying to pack more than that (e.g. shifting day left by 19, which will result in an all-zero value once casted to a short). You need to use an int or a long to hold all the fields.
Indeed, you've got several things going on with the bit operations that aren't right.
My suggestion would be to ditch the bit operations and just send the day, month and year as separate fields: one byte for each of the day and month, and two (a short) for the year. That takes 4 bytes (only one extra byte) but requires a lot less fiddling to get right.
Its not easy, but you have to work systematically to ensure your operations don't a) lose information b) decode the reverse of how you have encoded.
int D = 12;
int M = 9;
int Y = 1983;
int DMY = (D << 19) | (M << 15) | Y;
byte[] data = new byte[3];
data[0] = (byte) (DMY >>> 16);
data[1] = (byte) (DMY >>> 8);
data[2] = (byte) DMY;
int DMY2 = ((data[0]&0xFF) << 16) | ((data[1]&0xFF) << 8) | (data[2]&0xFF);
int D2 = DMY2 >>> 19; // no mask required
int M2 = (DMY2 >>> 15) & 0x0F; // 4 bits mask
int Y2 = DMY2 & 0x7FFF; // 15 bit mask
System.out.println(D2 + "/" + M2 + "/" + Y2);
prints
12/9/1983
First, you need at least 14 bits to represent the year with max value 9999 because of 2^14 > 9999 && 2 ^ 13 < 9999. And the least number of bits for month is 4(12 at max), day is 5(31 at max). So you can use a short int(16 bits) to represent year and byte (8 bits) for each day and month. So you get a 32-bits int.
public int encoded(short year, byte month, byte day){
int data =0;
data = year & 0xFFFF;
data =(data << 8)|(month & 0xFF)
data =(data << 8)|(day & 0xFF)
return data;
}
public void decode(int data){
int day = data & 0xFF;
int month = (data >> 8) & 0xFF;
int year = (data >> 16) & 0xFFFF;
}