Develop Reference

Java BigInteger pow with an int base

I'm probably missing something and kinda rusted out this sort of thing, I was wondering what is the best way to implement 2^Biginteger into a variable? I'm sure it's a simple question. This is to aid in figuring out a 2-pseudoprime. (Basically to see if 2^N-1 = 1 mod N or not).

So if I understand you, you want to do something like this:
BigInteger bigExp = ... some really large value
BigInteger pow = BigInteger.valueOf(2).pow(bigExp);
Unfortunately, that won't work.
As you noted, there is no pow overload that takes a BigInteger argument. And when you think about it, such a method would be problematic.
The value of 2N is a value with N significant bits when represented in binary. If is larger than will fit into an int, then that means N is 231 or more. Or converting to bytes, that is 228 bytes or 0.25 Gigabytes. For a single number.
That isn't impossibly large. It should be possible to represent numbers that big, even in a 32 bit JVM. The problem is that any BigInteger arithmetic operation is liable to generate another one of these monsters. Just creating a number of this size is going to copy 0.25 Gigabytes, and operations like multiplication and division ... which are O(N^2) for N bit numbers ... are going to take "forever".
Having said that, there would be ways to generate numbers that size. For example, you could allocate a huge byte array (which default initializes to zero), set the appropriate byte to contain a 1 bit, and then use BigInteger(byte[]) to construct the monster number.

As #StephenC points out, these numbers will be too unwieldy.
But
to see if 2^N-1 = 1 mod N or not
You don't need to calculate 2^X. You can calculate 2^X mod N. A much smaller number.
BigInteger.valueOf(2).modPow(n.subtract(ONE), n);

Why does Random.nextLong not generate all possible long values in Java?

The Javadoc of the nextLong() method of the Random class states that
Because class Random uses a seed with only 48 bits, this algorithm will not return all possible long values. (Random javadoc)
The implementation is:
return ((long)next(32) << 32) + next(32);
The way I see it is as follows: to create any possible long, we should generate any possible bit pattern of 64 bits with equal likelihood. Assuming the calls to next(int) give us 32 random bits, then the concatenation of these bits will be a sequence of 64 random bits and hence we generate each 64 bit pattern with equal likelihood. And therefore all possible long values.
I suppose that the person who wrote the javadoc knows better and that my reasoning is flaw somehow. Can anyone explain where my reasoning is incorrect and what kind of longs will be returned then?

Since Random is pseudo-random we know that given the same seed it will return the same values. Taking the docs at their word there are 48 bits of seed. This means there are at most 2^48 unique values that can be printed out. If there were more that would mean that some value that we used before in position < 2^48 gives us a different value this time than it did last time.
If we try to join up two results what do we see?
|a|b|c|d|e|f|...|(2^48)-1|
Above are some values. How many pairs are there? a-b, b-c, c-d,... (2^48)-1-a. There are also 2^48 pairs. We can't fill all values of 2^64 with only the 2^48 pairs.

Pseudo-Random Number Generators are like giant rings of numbers. You start somewhere, and then move around the ring step by step, as you pull numbers out. This means that with a given seed - an initial internal state - all subsequent numbers are predetermined. Therefor, since the internal state is only 48 bits wide, only 2 to the power 48 random numbers are possible. So since the next number is given by the previous number, it is now clear why that implementation of nextLong will not generate all possible long values.

Let's say a perfect pseudo random K-bit generator is one that creates all possible 2^K seed values in 2^K trys. We can't do better, as there are only 2^K states, and every state is completly determined by the previous state and determines itself the next state.
Assume we write the output of the 48-bit generator down in binary. We get 2^48 * 48 bits that way.
And now we can say exactly how many 64-bit sequences we can get by going through the list and noting the next 64 bits (wrapping to the start when needed). It is exactly the number of bits we have: 13510798882111488.
Even if we assume that all those 64-bit sequences are pairwise different (which is not at all obvious), we have a long way to go until 2^64: 18446744073709551616.
I write the numbers again:
18446744073709551616 pairwise different 64 bit sequences we need
13510798882111488 64 bit sequences we can get with a 48 bit seed.
This proves that the javadoc writer was right. Only 1/1844th of all long values can be produced with the random generator

How to turn a division into a bitwise shift when power of two?

I have the following division that I need to do often:
int index = pos / 64;
Division can be expensive in the cpu level. I am hoping there is a way to do that with bitwise shift. I would also like to understand how you can go from division to shift, in other words, I don't want to just memorize the bitwise expression.

int index = pos >> 6 will do it, but this is unnecessary. Any reasonable compiler will do this sort of thing for you. Certainly the Sun/Oracle compiler will.
The general rule is that i/(2^n) can be implemented with i >> n. Similarly i*(2^n) is i << n.
You need to be concerned with negative number representation if i is signed. E.g. twos-complement produces reasonable results (if right shift is arithmetic--sign bit copied). Signed magnitude does not.

The compiler will implement it for you in the most efficient way, as long you understand what you need and ask the compiler to do exactly that. If shift is the most efficient way in this case, the compiler will use shift.
Keep in mind though that if you are performing signed division (i.e pos is signed), then it cannot be fully implemented by a shift alone. Shift by itself will generate invalid results for negative values of pos. If the compiler decides to use shifts for this operations, it will also have to perform some post-shift corrections on the intermediate result to make it agree with the requirements of the language specification.
For this reason, if you are really looking for maximum possible efficiency of your division operations, you have to remember not to use signed types thoughtlessly. Prefer to use unsigned types whenever possible, and use signed types only when you have to.
P.S. AFAIK, Java implements Euclidean division, meaning that the above remarks do not apply to Java. Euclidean division is performed correctly by a shift on a negative divisor in 2's-complement representation. The above remarks would apply to C/C++.

http://www.java-samples.com/showtutorial.php?tutorialid=58
For each power of 2 you want to divide by, right shift it once. So to divide by 4 you would right shift twice. To divide by 8 right shift 3 times. Divide by 16 right shift 4 times. 32 -> 5 times. 64 -> 6 times. So to divide by 64 you can right shift 6 times. myvalue = myvalue >> 6;

Packing many bounded integers into a large single integer

I have to store millions of entries in a database. Each entry is identified by a set of unique integer identifiers. For example a value may be identified by a set of 10 integer identifiers, each of which are less than 100 million.
In order to reduce the size of the database, I thought of the following encoding using a single 32 bit integer value.
Identifier 1: 0 - 100,000,000
Identifier 2: 100,000,001 - 200,000,000
.
.
.
Identifier 10: 900,000,001 - 1,000,000,000
I am using Java. I can write a simple method to encode/decode. The user code does not have to know that I am encoding/decoding during fetch/store.
What I want to know is: what is the most efficient (fastest) and recommended way to implement such encoding/decoding. A simple implementation will perform a large number of multiplications/subtractions.
Is it possible to use shifts (or bitwise operations) and choose different partition size (the size of each segment still has to be close to 100 million)?
I am open to any suggestions, ideas, or even a totally different scheme. I want to exploit the fact that the integer identifiers are bounded to drastically reduce the storage size without noticeably compromising performance.
Edit: I just wanted to add that I went through some of the answers posted on this forum. A common solution was to split the bits for each identifier. If I use 2 bits for each identifier for a total of 10 identifiers, then my range of identifiers gets severely limited.

It sounds like you want to pack multiple integer values of 0...100m into a single 32bit Integer? Unless you are omitting important information that would allow to store these 0...100m values more efficiently, there is simply no way to do it.
ceil(log2(100m)) = 27bit, which means you have only 5 "spare bits".

You can make the segmentation size 27 bits which gives you 32 * 128 M segements. instead of 42 * 100 M
int value =
int high = value >>> 27;
int low = value & ((1L << 27) -1);
It is worth nothing this calculation is likely to be trivial compared to the cost of using a database.

It's unclear what you actually want to do, but it sounds like you want an integer value, each bit representing having a particular attribute, and applying a bitmask.
A 32-bit integer can save 32 different attributes, 64-bit 64 etc. To have more, you'll need multiple integer columns.
If that's not it, I don't know what you mean by "encode".

Why is BIGInteger in JAVA not responding for higher powers?

When I try to find the value of a BigInteger data type for 223,000, I am not able to see the value.
However, for calculations up to 222,000, I could display the BigInteger value without any problems.
Is there any solution or reason for this?

I tried the following in order to make a BigInteger representation of 2^23000:
BigInteger bi = new BigInteger("2");
bi = bi.pow(23000);
System.out.println(bi);
And the number displayed was a very large number spanning 6925 digits. (I won't paste it here as it will span over 100 lines.)
This is with Java 6 SE version 1.6.0_12 in Windows XP.
According the API Specification, BigInteger is an arbitrary-precision integer value which means it should be able to cope with very large integer values.

It works fine for me on GNU/Linux. What do you mean you can't "display" it? What's your code and what error/problem do you get?

this limit for BigInteger is around 2^16 billion, though it has been noted that some functions don't behave correctly after about 2^2 billion.
My guess is that your console or IDE has problems displaying very long lines.

Do you need the whole thing? There is also a BigInteger.modpow(power, modulus) method which raises the integer value to the specified power and returning result % modulus -- commonly used in cryptography. This is also MUCH faster when dealing with very large exponents.