Develop Reference

Bit shifting exercise

The task is to read an integer from keyboard, convert it into 8 groups of 4 bits, then convert each bit into a hex number and output them one after one. This must be done by using bit shifting, no other solution counts.
My idea was to use a mask with 4 ones to select the group of bits, then shift that group right, removing preceding zeroes, and output the number in hex.
Here's how I tried to approach this:
public static void main(String args[]) {
Scanner input = new Scanner(System.in);
System.out.print("Enter an integer: ");
int x = input.nextInt();
System.out.println("Binary representation: " + Integer.toBinaryString(x));
System.out.println("Hexadecimal representatio1n: " + Integer.toHexString(x));
int mask = 15 << 28;
System.out.println(Integer.toBinaryString(mask));
int k = 28;
for (int i = 1; i <= 8; i++)
{
int result = mask & x;
//System.out.println(Integer.toBinaryString(result));
result = x >>> k ;
mask = mask >> 4;
k = k - 4;
System.out.println(Integer.toHexString(result));
}
}
Sample output:
Enter an integer: 324234234
Binary representation: 10011010100110110101111111010
Hexadecimal representatio1n: 13536bfa
11110000000000000000000000000000
1
13
135
1353
13536
13536b
13536bf
13536bfa
Why is it not working correctly?
Thanks.

Your when you shift the result variable, you aren't shifting by the result, you are shifting it by x
if you change it to
result = result >>> k;
it should work
on a side note, this problem is much easier done the other way(from the least significant bit to the most significant bit)
like
int x = 0xaabcdabcd;
int mask = 0x0fffffff;
for(int i =0;i < 8; i ++){
System.out.println(x & mask);
x = x >>> 4;
}

Because there are no unsigned types in Java, on line int mask = 15 << 28; you practically make your mask negative by moving bits up to sign field. Later you move it left with mask = mask >> 4; and thus carry down the sign bit. After the shift, your mask is not 00001111000000000000000000000000 but 11111111000000000000000000000000. Use logical right shift operator >>> instead.
PS. This seems like a homework. In such cases, one who asks question should mark the question with homework tag.

How to find the next lower integer (with the same number of 1s)

How to find the next lower binary number for an integer (same number of 1s)? For example: if given input number n = 10 (1010), the function should return 9 (1001), or n = 14 (1110) then return 13 (1101), or n = 22 (10110) then return 21 (10101), n = 25 (11001) then return 22 (10110)... etc.

You can do this.
static int nextLower(int n) {
int bc = Integer.bitCount(n);
for (int i = n - 1; i > 0; i--)
if (Integer.bitCount(i) == bc)
return i;
throw new RuntimeException(n+" is the lowest with a bit count of "+bc);
}
Of course if this is homework you are going to have trouble convincing someone you wrote this ;)

For the sake of clarity, in this answer I will use the term 'cardinality' to indicate the number of 1s in the binary representation of a number.
One (obvious) way is to run a downwards loop, and seek for the first number with the same cardinality as your input (just like Peter Lawrey suggested).
I don't think this is inefficient, because I guess the output number is always pretty close to the input. More precisely, all you have to do is to find the rightmost '10' bit sequence, and change it to '01'. Then replace the right part with a number having all 1s at its left, as many as you can, without breaking the postcondition. This brings us to another solution, which consists in converting the number to a binary string (like user2573153 showed you), performing the replacement (with a regular expression, maybe), and then converting back to int.
A slightly faster version of Peter's algorithm should be the following, which performs on integers the manipulation I proposed you for strings:
static int nextLower(int n) {
int fixPart = 0;
int shiftCount = 0;
while ((n & 3) != 2) {
if (n == 0) {
throw new IllegalArgumentException(
fixPart + " is the lowest number with its cardinality");
}
fixPart |= (n & 1) << shiftCount;
shiftCount += 1;
n /= 2;
}
int fixZeros = shiftCount - Integer.bitCount(fixPart);
return ((n ^ 3) << shiftCount) | (((1 << shiftCount) - 1) & ~((1 << fixZeros) - 1));
}
which is O(log n) rather than O(n), but it's definitely harder to understand, and may also be practically slower, due to its complexity. Anyway, you could only notice a difference if you try with some huge difficult number.
EDIT I tried a little benchmark, and found that this code is 67% faster than Peter Lawrey's when applied consecutively to all numbers from 2 to 100,000,000. I don't think this is enough to justify the increased code complexity.

I like such binary task, so to find next lower number you should find right most 1 followed by 0 and exchange them,. UPDATE: you need to "reorder" the rest part of number with 1s at left and 0s at right
10 1010 ->
9 1001
14 1110 ->
13 1101
25 11001 ->
22 10110
here is sample code:
int originalValue = 25;
int maskToCheck = 2; // in binary 10b
int clearingMask = 1;
int settingMask = 0;
int zeroCount = 0;
while (maskToCheck > 0)
{
if ( (originalValue&(maskToCheck|(maskToCheck>>1))) == maskToCheck ) // we found such
{
int newValue = originalValue&(~maskToCheck); // set 1 with 0
newValue = newValue&(~clearingMask)|(settingMask<<zeroCount); // clear all the rest bits, and set most valuable ones
newValue = newValue|(maskToCheck>>1); // set 0 with 1
System.out.println("for " + originalValue + " we found " + newValue);
break;
}
else
{
if ( (originalValue&(maskToCheck>>1)) > 0) // we have 1 bit in cleared part
settingMask = (settingMask<<1) | 1;
else
zeroCount++;
maskToCheck = maskToCheck<<1; // try next left bits
clearingMask = (clearingMask<<1)|1;
}
}

Inverse function of Java's Random function

Java's Random function takes a seed and produces the a sequence of 'psuedo-random' numbers.
(It is implemented based on some algorithm discussed in Donald Knuth, The Art of Computer Programming, Volume 3, Section 3.2.1.), but the article is too technical for me to understand)
Is there an inverse function of it?
That is, given a sequence of numbers, would it be possible to mathematically determine what the seed would be?
(, which means, brute-forcing doesn't count as a valid method)
[Edit]
There seems to be quite a number of comments here... I thought I'd clarify what I am looking for.
So for instance, the function y = f(x) = 3x has an inverse function, which is y = g(x) = x/3.
But the function z = f(x, y) = x * y does not have an inverse function, because (I could give a full mathematical proof here, but I don't want to sidetrack my main question), intuitively speaking, there are more than one pair of (x, y) such that (x * y) == z.
Now back to my question, if you say the function is not inversible, please explain why.
(And I am hoping to get answers from those who have really read to article and understand it. Answers like "It's just not possible" aren't really helping)

If we're talking about the Oracle (née Sun) implementation of java.util.Random, then yes, it is possible once you know enough bits.
Random uses a 48-bit seed and a linear congruential generator. These are not cryptographically safe generators, because of the tiny state size (bruteforceable!) and the fact that the output just isn't that random (many generators will exhibit small cycle length in certain bits, meaning that those bits can be easily predicted even if the other bits seem random).
Random's seed update is as follows:
nextseed = (seed * 0x5DEECE66DL + 0xBL) & ((1L << 48) - 1)
This is a very simple function, and it can be inverted if you know all the bits of the seed by calculating
seed = ((nextseed - 0xBL) * 0xdfe05bcb1365L) & ((1L << 48) - 1)
since 0x5DEECE66DL * 0xdfe05bcb1365L = 1 mod 248. With this, a single seed value at any point in time suffices to recover all past and future seeds.
Random has no functions that reveal the whole seed, though, so we'll have to be a bit clever.
Now, obviously, with a 48-bit seed, you have to observe at least 48 bits of output or you clearly don't have an injective (and thus invertible) function to work with. We're in luck: nextLong returns ((long)(next(32)) << 32) + next(32);, so it produces 64 bits of output (more than we need). Indeed, we could probably make do with nextDouble (which produces 53 bits), or just repeated calls of any other function. Note that these functions cannot output more than 248 unique values because of the seed's limited size (hence, for example, there are 264-248 longs that nextLong will never produce).
Let's specifically look at nextLong. It returns a number (a << 32) + b where a and b are both 32-bit quantities. Let s be the seed before nextLong is called. Then, let t = s * 0x5DEECE66DL + 0xBL, so that a is the high 32 bits of t, and let u = t * 0x5DEECE66DL + 0xBL so that b is the high 32 bits of u. Let c and d be the low 16 bits of t and u respectively.
Note that since c and d are 16-bit quantities, we can just bruteforce them (since we only need one) and be done with it. That's pretty cheap, since 216 is only 65536 -- tiny for a computer. But let's be a bit more clever and see if there's a faster way.
We have (b << 16) + d = ((a << 16) + c) * 0x5DEECE66DL + 11. Thus, doing some algebra, we obtain (b << 16) - 11 - (a << 16)*0x5DEECE66DL = c*0x5DEECE66DL - d, mod 248. Since c and d are both 16-bit quantities, c*0x5DEECE66DL has at most 51 bits. This usefully means that
(b << 16) - 11 - (a << 16)*0x5DEECE66DL + (k<<48)
is equal to c*0x5DEECE66DL - d for some k at most 6. (There are more sophisticated ways to compute c and d, but because the bound on k is so tiny, it's easier to just bruteforce).
We can just test all the possible values for k until we get a value whos negated remainder mod 0x5DEECE66DL is 16 bits (mod 248 again), so that we recover the lower 16 bits of both t and u. At that point, we have a full seed, so we can either find future seeds using the first equation, or past seeds using the second equation.
Code demonstrating the approach:
import java.util.Random;
public class randhack {
public static long calcSeed(long nextLong) {
final long x = 0x5DEECE66DL;
final long xinv = 0xdfe05bcb1365L;
final long y = 0xBL;
final long mask = ((1L << 48)-1);
long a = nextLong >>> 32;
long b = nextLong & ((1L<<32)-1);
if((b & 0x80000000) != 0)
a++; // b had a sign bit, so we need to restore a
long q = ((b << 16) - y - (a << 16)*x) & mask;
for(long k=0; k<=5; k++) {
long rem = (x - (q + (k<<48))) % x;
long d = (rem + x)%x; // force positive
if(d < 65536) {
long c = ((q + d) * xinv) & mask;
if(c < 65536) {
return ((((a << 16) + c) - y) * xinv) & mask;
}
}
}
throw new RuntimeException("Failed!!");
}
public static void main(String[] args) {
Random r = new Random();
long next = r.nextLong();
System.out.println("Next long value: " + next);
long seed = calcSeed(next);
System.out.println("Seed " + seed);
// setSeed mangles the input, so demangle it here to get the right output
Random r2 = new Random((seed ^ 0x5DEECE66DL) & ((1L << 48)-1));
System.out.println("Next long value from seed: " + r2.nextLong());
}
}

I normally wouldn't just link articles... But I found a site where someone looks into this in some depth and thought it was worth posting. http://jazzy.id.au/default/2010/09/20/cracking_random_number_generators_part_1.html
It seems that you can calculate a seed this way:
seed = (seed * multiplier + addend) mod (2 ^ precision)
where multiplier is 25214903917, addend is 11, and precision is 48 (bits). You can't calculate what the seed was with only 1 number, but you can with 2.
EDIT: As nhahtdh said there's a part 2 where he delves into more of the math behind the seeds.

I would like to present an implementation to reverse a sequence of integers generated by nextInt().
The program will brute force on the lower 16-bit discarded by nextInt(), use the algorithm provided in the blog by James Roper to find previous seed, then check that upper 32 bit of the 48-bit seed are the same as the previous number. We need at least 2 integers to derive the previous seed. Otherwise, there will be 216 possibilities for the previous seed, and all of them are equally valid until we have at least one more number.
It can be extended for nextLong() easily, and 1 long number is enough to find the seed, since we have 2 pieces of upper 32-bit of the seed in one long, due to the way it is generated.
Note that there are cases where the result is not the same as what you set as secret seed in the SEED variable. If the number you set as secret seed occupies more than 48-bit (which is the number of bits used for generating random numbers internally), then the upper 16 bits of 64 bit of long will be removed in the setSeed() method. In such cases, the result returned will not be the same as what you have set initially, it is likely that the lower 48-bit will be the same.
I would like to give most the credit to James Roper, the author of this blog article which makes the sample code below possible:
import java.util.Random;
import java.util.Arrays;
class TestRandomReverse {
// The secret seed that we want to find
private static long SEED = 782634283105L;
// Number of random numbers to be generated
private static int NUM_GEN = 5;
private static int[] genNum(long seed) {
Random rand = new Random(seed);
int arr[] = new int[NUM_GEN];
for (int i = 0; i < arr.length; i++) {
arr[i] = rand.nextInt();
}
return arr;
}
public static void main(String args[]) {
int arr[] = genNum(SEED);
System.out.println(Arrays.toString(arr));
Long result = reverse(arr);
if (result != null) {
System.out.println(Arrays.toString(genNum(result)));
} else {
System.out.println("Seed not found");
}
}
private static long combine(int rand, int suffix) {
return (unsignedIntToLong(rand) << 16) | (suffix & ((1L << 16) - 1));
}
private static long unsignedIntToLong(int num) {
return num & ((1L << 32) - 1);
}
// This function finds the seed of a sequence of integer,
// generated by nextInt()
// Can be easily modified to find the seed of a sequence
// of long, generated by nextLong()
private static Long reverse(int arr[]) {
// Need at least 2 numbers.
assert (arr.length > 1);
int end = arr.length - 1;
// Brute force lower 16 bits, then compare
// upper 32 bit of the previous seed generated
// to the previous number.
for (int i = 0; i < (1 << 16); i++) {
long candidateSeed = combine(arr[end], i);
long previousSeed = getPreviousSeed(candidateSeed);
if ((previousSeed >>> 16) == unsignedIntToLong(arr[end - 1])) {
System.out.println("Testing seed: " +
previousSeed + " --> " + candidateSeed);
for (int j = end - 1; j >= 0; j--) {
candidateSeed = previousSeed;
previousSeed = getPreviousSeed(candidateSeed);
if (j > 0 &&
(previousSeed >>> 16) == unsignedIntToLong(arr[j - 1])) {
System.out.println("Verifying: " +
previousSeed + " --> " + candidateSeed);
} else if (j == 0) {
// The XOR is done when the seed is set, need to reverse it
System.out.println("Seed found: " + (previousSeed ^ MULTIPLIER));
return previousSeed ^ MULTIPLIER;
} else {
System.out.println("Failed");
break;
}
}
}
}
return null;
}
private static long ADDEND = 0xBL;
private static long MULTIPLIER = 0x5DEECE66DL;
// Credit to James Roper
// http://jazzy.id.au/default/2010/09/21/cracking_random_number_generators_part_2.html
private static long getPreviousSeed(long currentSeed) {
long seed = currentSeed;
// reverse the addend from the seed
seed -= ADDEND; // reverse the addend
long result = 0;
// iterate through the seeds bits
for (int i = 0; i < 48; i++)
{
long mask = 1L << i;
// find the next bit
long bit = seed & mask;
// add it to the result
result |= bit;
if (bit == mask)
{
// if the bit was 1, subtract its effects from the seed
seed -= MULTIPLIER << i;
}
}
return result & ((1L << 48) - 1);
}
}

How to count the number of 1's a number will have in binary? [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Best algorithm to count the number of set bits in a 32-bit integer?
How do I count the number of 1's a number will have in binary?
So let's say I have the number 45, which is equal to 101101 in binary and has 4 1's in it. What's the most efficient way to write an algorithm to do this?

Instead of writing an algorithm to do this its best to use the built in function. Integer.bitCount()
What makes this especially efficient is that the JVM can treat this as an intrinsic. i.e. recognise and replace the whole thing with a single machine code instruction on a platform which supports it e.g. Intel/AMD
To demonstrate how effective this optimisation is
public static void main(String... args) {
perfTestIntrinsic();
perfTestACopy();
}
private static void perfTestIntrinsic() {
long start = System.nanoTime();
long countBits = 0;
for (int i = 0; i < Integer.MAX_VALUE; i++)
countBits += Integer.bitCount(i);
long time = System.nanoTime() - start;
System.out.printf("Intrinsic: Each bit count took %.1f ns, countBits=%d%n", (double) time / Integer.MAX_VALUE, countBits);
}
private static void perfTestACopy() {
long start2 = System.nanoTime();
long countBits2 = 0;
for (int i = 0; i < Integer.MAX_VALUE; i++)
countBits2 += myBitCount(i);
long time2 = System.nanoTime() - start2;
System.out.printf("Copy of same code: Each bit count took %.1f ns, countBits=%d%n", (double) time2 / Integer.MAX_VALUE, countBits2);
}
// Copied from Integer.bitCount()
public static int myBitCount(int i) {
// HD, Figure 5-2
i = i - ((i >>> 1) & 0x55555555);
i = (i & 0x33333333) + ((i >>> 2) & 0x33333333);
i = (i + (i >>> 4)) & 0x0f0f0f0f;
i = i + (i >>> 8);
i = i + (i >>> 16);
return i & 0x3f;
}
prints
Intrinsic: Each bit count took 0.4 ns, countBits=33285996513
Copy of same code: Each bit count took 2.4 ns, countBits=33285996513
Each bit count using the intrinsic version and loop takes just 0.4 nano-second on average. Using a copy of the same code takes 6x longer (gets the same result)

The most efficient way to count the number of 1's in a 32-bit variable v I know of is:
v = v - ((v >> 1) & 0x55555555);
v = (v & 0x33333333) + ((v >> 2) & 0x33333333);
c = ((v + (v >> 4) & 0xF0F0F0F) * 0x1010101) >> 24; // c is the result
Updated: I want to make clear that it's not my code, actually it's older than me. According to Donald Knuth (The Art of Computer Programming Vol IV, p 11), the code first appeared in the first textbook on programming, The Preparation of Programs for an Electronic Digital Computer by Wilkes, Wheeler and Gill (2nd Ed 1957, reprinted 1984). Pages 191–193 of the 2nd edition of the book presented Nifty Parallel Count by D B Gillies and J C P Miller.

See Bit Twidling Hacks and study all the 'counting bits set' algorithms. In particular, Brian Kernighan's way is simple and quite fast if you expect a small answer. If you expect an evenly distributed answer, lookup table might be better.

This is called Hamming weight. It is also called the population count, popcount or sideways sum.

The following is either from "Bit Twiddling Hacks" page or Knuth's books (I don't remember). It is adapted to unsigned 64 bit integers and works on C#. I don't know if the lack of unsigned values in Java creates a problem.
By the way, I write the code only for reference; the best answer is using Integer.bitCount() as #Lawrey said; since there is a specific machine code operation for this operation in some (but not all) CPUs.
const UInt64 m1 = 0x5555555555555555;
const UInt64 m2 = 0x3333333333333333;
const UInt64 m4 = 0x0f0f0f0f0f0f0f0f;
const UInt64 h01 = 0x0101010101010101;
public int Count(UInt64 x)
{
x -= (x >> 1) & m1;
x = (x & m2) + ((x >> 2) & m2);
x = (x + (x >> 4)) & m4;
return (int) ((x * h01) >> 56);
}

public int f(int n)
{
int result = 0;
for(;n > 0; n = n >> 1)
result += ((n & 1) == 1 ? 1 : 0);
return result;
}

The following Ruby code works for positive numbers.
count = 0
while num > 1
count = (num % 2 == 1) ? count + 1 : count
num = num >> 1
end
count += 1
return count

The fastest I have used and also seen in a practical implementation (in the open source Sphinx Search Engine) is the MIT HAKMEM algorithm. It runs superfast over a very large stream of 1's and 0's.

How do I determine number of bytes needed to represent a given integer?

I need a function in Java to give me number of bytes needed to represent a given integer. When I pass 2 it should return 1, 400 -> 2, 822222 -> 3, etc.
#Edit: For now I'm stuck with this:
numOfBytes = Integer.highestOneBit(integer) / 8
Don't know exactly what highestOneBit() does, but have also tried this:
numOfBytes = (int) (Math.floor(Math.log(integer)) + 1);
Which I found on some website.

static int byteSize(long x) {
if (x < 0) throw new IllegalArgumentException();
int s = 1;
while (s < 8 && x >= (1L << (s * 8))) s++;
return s;
}

Integer.highestOneBit(arg) returns only the highest set bit, in the original place. For example, Integer.highestOneBit(12) is 8, not 3. So you probably want to use Integer.numberOfTrailingZeros(Integer.highestOneBit(12)), which does return 3. Here is the Integer API
Some sample code:
numOfBytes = (Integer.numberOfTrailingZeroes(Integer.highestOneBit(integer)) + 8) / 8;
The + 8 is for proper rounding.

The lazy/inefficient way to do this is with Integer#toBinaryString. It will remove all leading zeros from positive numbers for you, all you have to do is call String#length and divide by 8.

Think about how to solve the same problem using normal decimal numbers. Then apply the same principle to binary / byte representation i.e. use 256 where you would use 10 for decimal numbers.

static int byteSize(long number, int bitsPerByte) {
int maxNumberSaveByBitsPerByte = // get max number can be saved by bits in value bitsPerByte
int returnValue = getFloor(number/maxNumberSaveByBitsPerByte); // use Math lib
if(number % maxNumberSaveByBitsPerByte != 0)
returnValue++;
return returnValue;
}

For positive values: 0 and 1 need 1 digit, with 2 digits you get the doubled max value, and for every digit it is 2 times that value. So a recursive solution is to divide:
public static int binaryLength (long l) {
if (l < 2) return 1;
else 1 + binaryLength (l /2L);
}
but shifting works too:
public static int binaryLength (long l) {
if (l < 2) return 1;
else 1 + binaryLength (l >> 1);
}
Negative values have a leading 1, so it doesn't make much sense for the question. If we assume binary1 is decimal1, binary1 can't be -1. But what shall it be? b11? That is 3.

Why you wouldn't do something simple like this:
private static int byteSize(int val) {
int size = 0;
while (val > 0) {
val = val >> 8;
size++;
}
return size;
}

int numOfBytes = (Integer.SIZE >> 3) - (Integer.numberOfLeadingZeros(n) >> 3);
This implementation is compact enough while performance-friendly, since it doesn't involve any floating point operation nor any loop.
It is derived from the form:
int numOfBytes = Math.ceil((Integer.SIZE - Integer.numberOfLeadingZeros(n)) / Byte.SIZE);
The magic number 3 in the optimized form comes from the assumption: Byte.SIZE equals 8