I am attepmting to load a .ply file into a vbo, and draw it in lwjgl + java. The file loads correctly and all the data is parsed correctly, the only problem is thatit is a model with A LOT of indices (faces). The indices are stored in a byte[] array. In java (and probably everywhere else), bytes can only go up to 128, but the indices required to draw the mesh well surpass 128 in value, so I get this error.
Exception in thread "main" java.lang.NumberFormatException: Value out of range. Value:"128" Radix:10
at java.lang.Byte.parseByte(Byte.java:151) at java.lang.Byte.parseByte(Byte.java:151)
at game_3d.ModelLoader.loadGeometry(ModelLoader.java:333)
at game_3d.Game_3D.setupGeometry(Game_3D.java:399)
at game_3d.Game_3D.<init>(Game_3D.java:81)
at game_3d.Game_3D.main(Game_3D.java:70)
When I tried to use an int[] array, the model didn't draw correctly. I don't know how or if you even can increase the max value of a byte in a byte[] array, I already tried experimenting with the radix, but the minimum was 10 (which gets me 128).
Well, a byte is a byte. It can only have 256 different values. There is a slight oddity related to the index range when using Java OpenGL bindings. Since Java does not have unsigned data types, the maximum supported value may seem to be 127. But since the arrays of byte values will be passed through to native code, which will treat them as unsigned values, the maximum number of vertices you can address with byte indices is in fact 255.
For example, say you have a value of 160 as an int, and cast it to a byte value for storage in a byte[] array. If you look at the byte value in Java, it will appear to have changed to -96 (160 - 256). But once native code sees the value, and interprets it as an unsigned byte, it will in fact be 160.
See this question for more details about casting int to byte in Java: How are integers cast to bytes in Java?.
Now, if you need more than 256 vertices, you will have to use an index type that supports a larger range. That will work just fine as long as you're consistent:
Using short as the type of your index array in Java, you can address 65,536 vertices. The matching type argument of the glDrawElements() call is GL_UNSIGNED_SHORT.
Using int as the type of your index array in Java, you can address... a lot of vertices (2 ^ 32, which is more than 4 billion). The matching type argument of the glDrawElements() call is GL_UNSIGNED_INT.
Related
I just found out that there is BitSet in java. There are already arrays and similar data structures. Where can BitSet be used?
As the above answer only explains what a BitSet is, I am providing here an answer of how I use BitSet and why. At first, I did not knew that the BitSet construct exists. I have a QR Code generator in C++ and for flexible reasons I don't want to use a specific Bitmap structures in returning this QR Code back to the caller. The QR Code is just black and white and can be represented as a series of bits. The problem was that in the JNI C++, I have to return the byte array that represents these series of bits and then I have to return the count of bits. Note that the size of the bytes array alone could not tell the count of bits. In effect, I am face with a scenario wherein my JNI C++ has to return two values:
the byte[] array
the count of bits
My first solution, was to return an array of boolean. The content of this array are the QR Code pixels, and the square root of the length of the array is the length of the side. Of course this worked but I felt wasted because it is supposed to be a series of bits. My next attempt was to return Pair<int, byte[]> object which, after lots of hair pulling i am not able to make it work in C++. Here comes the BitSet(145) construct. By returning this BitSet object, I am conveying two types of information i listed above. But there is minor trick. If QR Code pixel has total 144 pixels, because one side is 12, then you have to allocate BitSet(145) and do obj.set(144). That is, we introduce an artificial last bit that we then set, but this last bit is not part of the QR Code pixels. This ensures that, BitSet::length() correctly returns the bit count. So in Kotlin:
var pixels:BitSet = getqrpixels(inputdata)
var pixels_len = pixels.length() - 1
var side = sqrt(pixels_len.toFloat()).toInt()
drawSquareBitmap(pixels, side)
And thus, is my unexpected use case of this mysterious BitSet.
Take a look at this:
https://docs.oracle.com/javase/7/docs/api/java/util/BitSet.html
A BitSet is a vector of bits. Each entry in the list is either true (1) or false (0). The BitSet class comes with methods that resemble the bitwise operators. It is a little bit more flexible then a normal binary type.
BitSet, unlike a boolean[], is actually a dynamically sized bitmask. Essentially, instead of using booleans to store values, it uses longs, where each of the longs 64 bits are used to store a single bit.
Consider this situation. I am having a thousand int variables in my program. We know that an int variable occupies 2 bytes of memory space in Java. So, the total amount of space occupied by my variables would be 2000 bytes. But, the problem is that, the value in every variable occupies only half of the space it has. So, the actual required space used would be 1000 bytes, which means that 1000 bytes goes waste. How can I address this problem? Is there a way to do address this problem in Java?
Your data fits in a byte, so just use byte instead of int (which is 4 bytes).
Checking the space required by values every time they are stored would mean creating an additional useless overhead.
If you know which size your values will be, it is up to you to properly chose their type while developing. If you don't know how much space your value might occupy, you have to choose the largest possible one.
During runtime, if a value is an int for example, this means it can possibly be between -2^31 and (2^31)-1. If you know this value will never be this size, you can use smaller sized primitives such as byte or short.
We could check and allocate the exact amount of memory required for each element in memory but it would take more time. On the contrary, using a fixed amount of memory takes up more space evidently but is faster, this is how Java works. Unfortunately, we can't do without a compromise.
An int in Java is 32 bit (4 bytes) long
The value held in the int does not affect the int size in memory, e.g. if your int has the value 0x55
So, in order to occupy 2000 bytes in memory, you will need 500 ints, if you create an int array, there will be overhead of the array itself, so it will take up more memory.
int data type is a 32-bit signed two's complement integer. And hence occupies 4 bytes of memory. For int:
Minimum value is - 2,147,483,648.(-2^31)
Maximum value is 2,147,483,647(inclusive).(2^31 -1)
If your values are not going to be so large then you can use either byte or short as per your requirement.
Byte data type is an 8-bit signed two's complement integer.
Minimum value is -128 (-2^7)
Maximum value is 127 (inclusive)(2^7 -1)
Short data type is a 16-bit signed two's complement integer.
Minimum value is -32,768 (-2^15)
Maximum value is 32,767 (inclusive) (2^15 -1)
You could also use a linked list if you really are not sure on how many elements are going to be in a large list.
The principal benefit of a linked list over a conventional array is that the list elements can easily be inserted or removed without reallocation or reorganization of the entire structure because the data items need not be stored continuously in memory or on disk, while an array has to be declared in the source code, before compiling and running the program. Linked lists allow insertion and removal of nodes at any point in the list, and can do so with a constant number of operations if the link previous to the link being added or removed is maintained during list traversal.
In general use a linked list for int data type or heavier data types as they actually incur a per-element overhead of at least 1 pointer size, so if your elements are small, you're actually worse off.
See the Oracle docs:
int: By default, the int data type is a 32-bit signed two's complement
integer
So it means that int occupies 4 bytes in memory.
And
byte: The byte data type is an 8-bit signed two's complement integer
So in your case you can change your datatype to byte instead of int.
I am trying to solve competitive programming questions using Java. In questions involving the use of arrays, I will have to write my programs such that the array can hold a large number of values(beyond the range of int) and also each array element is also beyond the int range.
So I am using long for this. But if I try accessing say
a[i++] = b[j++];
where i & j & a& b are all of long type, I get the following error:
error: possible loss of precision
for the index access.
How do I access such elements? I tried typecast of long to int for the index values, and the error goes away, but won't that affect the precision too?
variable declarations are as follows:
long numTest = in.nextLong();
long [] sizes = new long[numTest];
The Java Language Specification says that you can't use longs as array indexes:
Arrays must be indexed by int values... An attempt to access an array component with a long index value results in a compile-time error.
So you could expect that tha maximum size would be Integer.MAX_VALUE, but that depends on your JVM which is discussed here
Unfortunately Java can't handle arrays bigger than 2^31 elements, the maximum size for a signed integer. A so big array wouldn't probably fit in memory anyways. Let's consider this case for example:
Array with 2^32 long elements
Size is 2^32 * 8 bytes = 2^35 bytes = 32 GB (!)
In this example the array size is just slightly bigger than the integer maximum value but we already reached 32 gigabytes of used memory.
I think you should find some alternative solution such as memory-mapped files or dinamically loading parts of the data as needed.
I'd also like to link to this existing answer: https://stackoverflow.com/a/10787175
Java arrays must be indexed by int values. That is a rule stated in the language specification. So if you use a long, as an index, the compiler is helping you obey the rules and is implicitly coercing it to an int. That is why you get the message.
To have an array containing a long number of objects you will need a custom implementation of array.
Java allows you to create an array maximum of integer range and hence the indexes should be within the integer range.
To achieve array of long ranges, i suggest you to use the concept of ArrayLet. I.e create 2 dimension arrays where the top level array can help you to cross the integer range, for example see below,
long[][] array = new long[x][Integer.MAX_VALUE];
where x could be any number that defines the multiples of Integer.MAX_VALUE values that your program requires.
I read this line in the Java tutorial:
byte: The byte data type is an 8-bit signed two's complement integer. It has
a minimum value of -128 and a maximum value of 127 (inclusive). The
byte data type can be useful for saving memory in large arrays, where
the memory savings actually matters. They can also be used in place of
int where their limits help to clarify your code; the fact that a
variable's range is limited can serve as a form of documentation.
I don't clearly understand the bold line. Can somebody explain it for me?
Byte has a (signed) range from -128 to 127, where as int has a (also signed) range of −2,147,483,648 to 2,147,483,647.
What it means is that since the values you're going to use will always be between that range, by using the byte type you're telling anyone reading your code this value will be at most between -128 to 127 always without having to document about it.
Still, proper documentation is always key and you should only use it in the case specified for readability purposes, not as a replacement for documentation.
If you're using a variable which maximum value is 127 you can use byte instead of int so others know without reading any if conditions after, which may check the boundaries, that this variable can only have a value between -128 and 127.
So it's kind of self-documenting code - as mentioned in the text you're citing.
Personally, I do not recommend this kind of "documentation" - only because a variable can only hold a maximum value of 127 doesn't reveal it's really purpose.
Integers in Java are stored in 32 bits; bytes are stored in 8 bits.
Let's say you have an array with one million entries. Yikes! That's huge!
int[] foo = new int[1000000];
Now, for each of these integers in foo, you use 32 bits or 4 bytes of memory. In total, that's 4 million bytes, or 4MB.
Remember that an integer in Java is a whole number between -2,147,483,648 and 2,147,483,647 inclusively. What if your array foo only needs to contain whole numbers between, say, 1 and 100? That's a whole lot of numbers you aren't using, by declaring foo as an int array.
This is when byte becomes helpful. Bytes store whole numbers between -128 and 127 inclusively, which is perfect for what you need! But why choose bytes? Because they use one-fourth of the space of integers. Now your array is wasting less memory:
byte[] foo = new byte[1000000];
Now each entry in foo takes up 8 bits or 1 byte of memory, so in total, foo takes up only 1 million bytes or 1MB of memory.
That's a huge improvement over using int[] - you just saved 3MB of memory.
Clearly, you wouldn't want to use this for arrays that hold numbers that would exceed 127, so another way of reading the bold line you mentioned is, Since bytes are limited in range, this lets developers know that the variable is strictly limited to these bounds. There is no reason for a developer to assume that a number stored as a byte would ever exceed 127 or be less than -128. Using appropriate data types saves space and informs other developers of the limitations imposed on the variable.
I imagine one can use byte for anything dealing with actual bytes.
Also, the parts (red, green and blue) of colors commonly have a range of 0-255 (although byte is technically -128 to 127, but that's the same amount of numbers).
There may also be other uses.
The general opposition I have to using byte (and probably why it isn't seen as often as it can be) is that there's lots of casting needed. For example, whenever you do arithmetic operations on a byte (except X=), it is automatically promoted to int (even byte+byte), so you have to cast it if you want to put it back into a byte.
A very elementary example:
FileInputStream::read returns a byte wrapped in an int (or -1). This can be cast to an byte to make it clearer. I'm not supporting this example as such (because I don't really (at this moment) see the point of doing the below), just saying something similar may make sense.
It could also have returned a byte in the first place (and possibly thrown an exception if end-of-file). This may have been even clearer, but the way it was done does make sense.
FileInputStream file = new FileInputStream("Somefile.txt");
int val;
while ((val = file.read()) != -1)
{
byte b = (byte)val;
// ...
}
If you don't know much about FileInputStream, you may not know what read returns, so you see an int and you may assume the valid range is the entire range of int (-2^31 to 2^31-1), or possibly the range of a char (0-65535) (not a bad assumption for file operations), but then you see the cast to byte and you give that a second thought.
If the return type were to have been byte, you would know the valid range from the start.
Another example:
One of Color's constructors could have been changed from 3 int's to 3 byte's instead, since their range is limited to 0-255.
It means that knowing that a value is explicitly declared as a very small number might help you recall the purpose of it.
Go for real docs when you have to create a documentation for your code, though, relying on datatypes is not documentation.
An int covers the values from 0 to 4294967295 or 2 to the 32nd power. This is a huge range and if you are scoring a test that is out of 100 then you are wasting that extra spacce if all of your numbers are between 0 and 100. It just takes more memory and harddisk space to store ints, and in serious data driven applications this translates to money wasted if you are not using the extra range that ints provide.
byte data types are generally used when you want to handle data in the forms of streams either from file or from network. Reason behind this is because network and files works on the concept of byte.
Example: FileOutStream always takes byte array as input parameter.
I believe conversion exactly to BigInteger[] would be optimal in my case. Anyone had done or found this written in Java and willing to share?
So imagine I have arbitrary size byte[] = {0xff,0x3e,0x12,0x45,0x1d,0x11,0x2a,0x80,0x81,0x45,0x1d,0x11,0x2a,0x80,0x81}
How do I convert it to array of BigInteger's and then be able to recover it back the original byte array safely?
ty in advance.
Use BigInteger.toByteArray() and BigInteger(byte[]).
According to the javadoc, the latter ...
Translates a byte array containing the two's-complement binary representation of a BigInteger into a BigInteger. The input array is assumed to be in big-endian byte-order: the most significant byte is in the zeroth element.
If your byte-wise representation is different, you may need to apply some extra transformations.
EDIT - if you need to preserve leading (i.e. non-significant) zeros, do the following:
When you convert from the byte array to a BigInteger, also make a note of the size of the byte array. This information is not encoded in the BigInteger value.
When you convert from the BigInteger to a byte array, sign-extend the byte array out to the same length as the original byte array.
EDIT 2 - if you want to turn a byte array into an array of BigIntegers with at most N bytes in each one, you need to create a temporary array of size N, repeatedly 1) fill it with bytes from the input byte array (with left padding at the end) and 2) use it to create BigInteger values using the constructor above. Maybe 20 lines of code?
But I'm frankly baffled that you would (apparently) pick a value for N based on memory usage rather than based on the mathematical algorithm you are trying to implement.