Arbitrary array length in java - java

Is there a way to get an array in java which is longer than the length supported by an integer data type?
I'm looking for something that may be indexable using a big integer in Java because the natively supported array length is nowhere near as big as I need it to be for an algorithm I am implementing.

This library should be useful for you: http://fastutil.dsi.unimi.it/
It says:
"...provides also big (64-bit) arrays..."

Int32 gives you 17 8 gigabytes of storage. Do you have so much memory?
I think you should use sparse arrays, i.e. index elements by hash. For example, with just HashMap<BigInteger,YourValueType> or with some libs like BigMemory and alternatives http://terracotta.org/products/bigmemory

Are you sure that you don't have to change the algorithm? Integer.MAX is equal to 2^31-1, which is 2147483647, each int has 4 bytes what gives us: 8589934588 bytes of memory (8GB!!!).

Related

Is a Java array consisting of bytes/shorts instead of integers more memory efficient?

On many posts on Stackoverflow people said that using bytes or shorts instead of integers did not reduce the memory usage or CPU utilization.
However, if I have an array of bytes or shorts, would that array use less memory and/or be faster to iterate over than a similar array of integers.
To be specific: I'm asking about the primitive type.
http://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html
As mentioned in Java documentation byte/short can be used for saving memory in large arrays.
In case of byte, the variable declared as byte can also serve as a form of a documentation since its value is limited by the range. ex of byte (-128 to 127).
Also take a look at the excellent answers in the following post,
In java, is it more efficient to use byte or short instead of int and float instead of double?

can java.util.BitSet hold more than MAX_INT no. of bits?

As the BitSet.get() function uses an int as an argument, I was thinking whether I could store more than 2^32 bits in a BitSet, and if so how would I retrieve them?
I am doing a Project Euler problem where I need to generate primes till 10^10. The algorithm I'm currently using to generate primes is the Erathonesus' Sieve, storing the boolean values as bits in a BitSet. Any workaround for this?
You could use a list of bitsets as List<BitSet> and when the end of one bitset has been reached you could move to the next one.
However, I think your approach is probably incorrect. Even if you use a single bit for each number you need 10^10 bits which is about 1 GB memory (8 bits in a byte and 1024^3 bytes in a GB). Most Project Euler problems should be solvable without needing that much memory.
No, it's limited by the int indexing in its interface.
So they didn't bother exploiting all its potential, (about 64x downsized) probably because it wasn't feasible to use that much RAM.
I worked on a LongBitSet implementation, published it here.
It can take:
//2,147,483,647 For reference, this is Integer.MAX_VALUE
137,438,953,216 LongBitSet max size (bits)
0b1111111_11111111_11111111_11111100_000000L in binary
Had to address corner cases, in the commit history you can see the 1st commit being a copy paste from java.util.BitSet.
See factory method:
public static LongBitSet getMaxSizeInstance() {
// Integer.MAX_VALUE - 3 << ADDRESS_BITS_PER_WORD
return new LongBitSet( 0b1111111_11111111_11111111_11111100_000000L);
}
Note: -Xmx24G -Xms24G -ea Min GB needed to start JVM with to call getMaxSizeInstance() without java.lang.OutOfMemoryError: Java heap space

Representing a single byte

A single byte takes up four bytes of space inside the Java virtual machine(32 bit processor).
Yes,we can use an array of byte which would occupy only the amount of space it actually needs. But I want to use a single byte not an array of bytes.
So,is there any type in Java to represent an 8 bit datum.
A single byte can be allocated more than a single byte of storage, for memory alignment reasons.
Do not worry about the target processor. An array of 10000 bytes will be stored in approximately 10000 bytes of space.
is there any type in Java to represent an 8 bit datum.
Yes, it is called byte.
How much a single byte actually needs only depends on the Java VM.
It's up to the implementation (JVM) how to deal with the internal types. I guess any JVM on an 8bit machine uses 1 byte for the type byte - on 32bit or 64bit machines this might not always be the case, as you noticed :)
If you use byte then Java will use the most efficient method to store it. Might be 8 bits, might be 64 bits, but whatever it is it's for a good reason. Don't fight the compiler, it knows better than you.
A byte does represent an 8 bit datum. Why do you care how many bytes an implementation of a vm uses to store it?

Array size in Java

What would be the memory size/space occupied in bits/bytes by array as follows.
final String[] objects_user1={"1","10","100","1000","10000"};
ROUGH ESTIMATE: 12 bytes for array header, 4x5 bytes for the pointers (8x5 if you're on a 64 bit jvm), each string has 3 ints (+3x4 bytes), and an array of chars (+12 bytes for header + length of the string x2, because it's char).
Did you try to Google it? Here is the first result of my Google search.
Impossible to say, since its an implementation detail of the JRE you're using.
You can get an approximate answer by querying available heap space before & after the memory allocation. Run it a number of times & compute the average, & it will be pretty close to the right answer. But again, the answer is only valid for the specific JVM it's run on.

How can I convert an int number from decimal to binary

how to convert an int array of length 8 to an int array of length 32 each 4 indexes are the binary representation of an index of the first array is there such a method that do all overhead.
Edit, PST: I updated the title. However, this doesn't quite reflect with the notion of "binary", it's just decomposing integers into bytes. Correct and/or add examples if this is not accurate.
ok programers the main thing that i want is >> How can I convert an int number from decimal to binary
You can use ByteBuffer from java.nio. While the NIO can be sometimes cumbersome to use, its ByteBuffers are very nice and easy to use. Also be careful with endianness, by default it is BigEndian, but you can change that.
EDIT
Disregard this, I misread the question. It says convert int array to another int array, not to a byte array. Sorry.
I'm not sure, but maybe this is what you're looking for?
http://download.oracle.com/javase/1.4.2/docs/api/java/lang/Integer.html#toBinaryString(int)
Probably using all or some of the following (there is no direct way to achieve it):
Integer.toBinaryString to get binary representation of your integer at your index.
Array.newInstance
System.arrayCopy
simple for loop etc.

Categories

Resources