Iterating over consecutive elements of an array is generally considered to be more efficient than iterating over consecutive linked list elements because of caching.
This is undoubtedly true as long as elements have elementary data types. But if the elements are objects, my understanding is that only the references to the objects will be stored in the contiguous memory area of the array (which is likely to be cached) while the actual object data will be stored anywhere in main memory and cannot be cached effectively.
As you normally not only iterate over the container but also need to access object data in each iteration, doesn't this more or less kill the performance benefit of the array over the list?
Edit: The comment about different scenarios varying greatly is probably correct. So let's consider a specific one: You search for a specific object in the container. In order to find it you need to compare a given string to another string that is a class variable of the object.
No, for objects ("pointers") there is an indirection in both. A linked list needs for every node to step to the next one. So it still has an overhead.
But yes, in a relative way the gain concerns only a part, very roughly the half of the pure walk through, counting indirection steps.
And ofcourse every indirection makes access more chaotic, slower.
BTW there is the ArrayList too being similar fast as arrays.
Since day one of learning Java I've been told by various websites and many teachers that arrays are consecutive memory locations which can store the specified number of data all of the same type.
Since an array is an object and object references are stored on the stack, and actual objects live in the heap, object references point to actual objects.
But when I came across examples of how arrays are created in memory, they always show something like this:
(In which a reference to an array object is stored on the stack and that reference points to the actual object in the heap, where there are also explicit indexes pointing to specific memory locations)
But recently I came across online notes of Java in which they stated that arrays' explicit indexes are not specified in the memory. The compiler just knows where to go by looking at the provided array index number during runtime.
Just like this:
After reading the notes, I also searched on Google regarding this matter, but the contents on this issue were either quite ambiguous or non-existent.
I need more clarification on this matter. Are array object indexes explicitly shown in memory or not? If not, then how does Java manage the commands to go to a particular location in an array during runtime?
Does an array object explicitly contain the indexes?
Short answer: No.
Longer answer: Typically not, but it theoretically could do.
Full answer:
Neither the Java Language Specification nor the Java Virtual Machine Specification makes any guarantees about how arrays are implemented internally. All it requires is that array elements are accessed by an int index number having a value from 0 to length-1. How an implementation actually fetches or stores the values of those indexed elements is a detail private to the implementation.
A perfectly conformant JVM could use a hash table to implement arrays. In that case, the elements would be non-consecutive, scattered around memory, and it would need to record the indexes of elements, to know what they are. Or it could send messages to a man on the moon who writes the array values down on labeled pieces of paper and stores them in lots of little filing cabinets. I can't see why a JVM would want to do these things, but it could.
What will happen in practice? A typical JVM will allocate the storage for array elements as a flat, contiguous chunk of memory. Locating a particular element is trivial: multiply the fixed memory size of each element by the index of the wanted element and add that to the memory address of the start of the array: (index * elementSize) + startOfArray. This means that the array storage consists of nothing but raw element values, consecutively, ordered by index. There is no purpose to also storing the index value with each element, because the element's address in memory implies its index, and vice-versa. However, I don't think the diagram you show was trying to say that it explicitly stored the indexes. The diagram is simply labeling the elements on the diagram so you know what they are.
The technique of using contiguous storage and calculating the address of an element by formula is simple and extremely quick. It also has very little memory overhead, assuming programs allocate their arrays only as big as they really need. Programs depend on and expect the particular performance characteristics of arrays, so a JVM that did something weird with array storage would probably perform poorly and be unpopular. So practical JVMs will be constrained to implement contiguous storage, or something that performs similarly.
I can think of only a couple of variations on that scheme that would ever be useful:
Stack-allocated or register-allocated arrays: During optimization, a JVM might determine through escape analysis that an array is only used within one method, and if the array is also a smallish fixed size, it would then be an ideal candidate object for being allocated directly on the stack, calculating the address of elements relative to the stack pointer. If the array is extremely small (fixed size of maybe up to 4 elements), a JVM could go even further and store the elements directly in CPU registers, with all element accesses unrolled & hardcoded.
Packed boolean arrays: The smallest directly addressable unit of memory on a computer is typically an 8-bit byte. That means if a JVM uses a byte for each boolean element, then boolean arrays waste 7 out of every 8 bits. It would use only 1 bit per element if booleans were packed together in memory. This packing isn't done typically because extracting individual bits of bytes is slower, and it needs special consideration to be safe with multithreading. However, packed boolean arrays might make perfect sense in some memory-constrained embedded devices.
Still, neither of those variations requires every element to store its own index.
I want to address a few other details you mentioned:
arrays store the specified number of data all of the same type
Correct.
The fact that all an array's elements are the same type is important because it means all the elements are the same size in memory. That's what allows for elements to be located by simply multiplying by their common size.
This is still technically true if the array element type is a reference type. Although in that case, the value of each element is not the object itself (which could be of varying size) but only an address which refers to an object. Also, in that case, the actual runtime type of objects referred to by each element of the array could be any subclass of the element type. E.g.,
Object[] a = new Object[4]; // array whose element type is Object
// element 0 is a reference to a String (which is a subclass of Object)
a[0] = "foo";
// element 1 is a reference to a Double (which is a subclass of Object)
a[1] = 123.45;
// element 2 is the value null (no object! although null is still assignable to Object type)
a[2] = null;
// element 3 is a reference to another array (all arrays classes are subclasses of Object)
a[3] = new int[] { 2, 3, 5, 7, 11 };
arrays are consecutive memory locations
As discussed above, this doesn't have to be true, although it is almost surely true in practice.
To go further, note that although the JVM might allocate a contiguous chunk of memory from the operating system, that doesn't mean it ends up being contiguous in physical RAM. The OS can give programs a virtual address space that behaves as if contiguous, but with individual pages of memory scattered in various places, including physical RAM, swap files on disk, or regenerated as needed if their contents are known to be currently blank. Even to the extent that pages of the virtual memory space are resident in physical RAM, they could be arranged in physical RAM in an arbitrary order, with complex page tables that define the mapping from virtual to physical addresses. And even if the OS thinks it is dealing with "physical RAM", it still could be running in an emulator. There can be layers upon layers upon layers, is my point, and getting to the bottom of them all to find out what's really going on takes a while!
Part of the purpose of programming language specifications is to separate the apparent behavior from the implementation details. When programming you can often program to the specification alone, free from worrying about how it happens internally. The implementation details become relevant however, when you need to deal with the the real-world constraints of limited speed and memory.
Since an array is an object and object references are stored on the stack, and actual objects live in the heap, object references point to actual objects
This is correct, except what you said about the stack. Object references can be stored on the stack (as local variables), but they can also be stored as static fields or instance fields, or as array elements as seen in the example above.
Also, as I mentioned earlier, clever implementations can sometimes allocate objects directly on the stack or in CPU registers as an optimization, although this has zero effect on your program's apparent behavior, only its performance.
The compiler just knows where to go by looking at the provided array index number during runtime.
In Java, it's not the compiler that does this, but the virtual machine. Arrays are a feature of the JVM itself, so the compiler can translate your source code that uses arrays simply to bytecode that uses arrays. Then it's the JVM's job to decide how to implement arrays, and the compiler neither knows nor cares how they work.
In Java, arrays are objects. See the JLS - Chapter 10. Arrays:
In the Java programming language, arrays are objects (§4.3.1), are dynamically created, and may be assigned to variables of type Object (§4.3.2). All methods of class Object may be invoked on an array.
If you look at 10.7. Array Members chapter, you'll see that the index is not part of the array member:
The members of an array type are all of the following:
The public final field length, which contains the number of components
of the array. length may be positive or zero.
The public method clone, which overrides the method of the same name
in class Object and throws no checked exceptions. The return type of
the clone method of an array type T[] is T[].
All the members inherited from class Object; the only method of Object
that is not inherited is its clone method.
Since the size of each type is known, you can easily determine the location of each component of the array, given the first one.
The complexity of accessing an element is O(1) since it only needs to calculate the address offset. It's worth mentioning that this behavior is not assumed for all programming languages.
The array, as you say, will only store objects of the same type. Each type will have a corresponding size, in bytes. For example in an int[] each element will occupy 4 bytes, each byte in a byte[] will occupy 1 byte, each Object in an Object[] will occupy 1 word (because it's really a pointer to the heap), etc.
The important thing is that each type has a size and every array has a type.
Then, we get to the problem of mapping an index to a memory position at runtime. It's actually very easy because you know where the array starts and, given the array's type, you know the size of each element.
If your array starts at some memory position N you can use the the given index I and element size S to compute that the memory you're looking for will be at memory address N + (S * I).
This is how Java finds memory positions for indexes at runtime without storing them.
Your two diagrams are, apart from labels that are strictly for human consumption, equivalent and identical.
That is to say that in the first diagram, the labels arr[0], arr[1], etc., are not part of the array. They are simply there for illustrative purposes, indicating how the array elements are laid out in memory.
What you were told, namely that arrays are stored in contiguous locations in memory (at least insofar as virtual addresses are concerned; on modern hardware architectures, these need not map into contiguous physical addresses) and array elements are located based on their size and index, is correct. (At least in... well, it is definitely correct in C/C++. It is almost certainly correct in most, if not all, Java implementations. But it is likely incorrect in languages that allow for sparse arrays or arrays that can grow or shrink dynamically.)
The fact that the array reference is created in the stack whereas the array data are placed on the heap is an implementation-specific detail. Compilers that compile Java directly to machine code may implement array storage differently, taking into account the specific characteristics of the target hardware platform. In fact, a clever compiler may place, e.g., small arrays in the stack in their entirety, and use the heap only for larger arrays, to minimize the need for garbage collection, which can impact performance.
On your first picture arr[0] to arr[4] are not references to the array elements. They are just illustrative labels for the location.
The critical piece to understand is that memory allocated for an array is contiguous. So given the address of the initial element of an array, i.e., arr[0], this contiguous memory allocation scheme helps the runtime to determine the address of array element given its index.
Say we have declared int[] arr = new int[5], and its initial array element, arr[0], is at address 100. To reach the third element in the array all that the runtime needs to perform is following the math 100 + ((3-1)*32) = 164 (assuming 32 is the size of an integer). So all that the runtime needs is the address of the initial element of that array. It can derive all other addresses of array elements based on the index and the size of the datatype the array stores.
Just an off-topic note: Although the array occupies a contiguous memory location, the addresses are contiguous only in the virtual address space and not in the physical address space. A huge array could span multiple physical pages that may not be contiguous, but the virtual address used by the array will be contiguous. And mapping of a virtual address to a physical address is done by OS page tables.
The reference of an array is not always on the stack. It could also be stored on the heap if it's a member of a class.
The array itself can hold either primitive values or references to an object. In any case, the data of an array are always of the same kind. Then the compiler can deal with their location without explicit pointers, only with the value/reference size and an index.
See:
* The Java Language Specification, Java SE 8 Edition - Arrays
* The Java Virtual Machine Specification, Java SE 8 Edition - Reference Types and Values
The "consecutive memory locations" is an implementation detail and may be wrong. For example, Objective-C mutable arrays do not use consecutive memory locations.
To you, it mostly doesn't matter. All you need to know is that you can access an array element by supplying the array and an index, and some mechanism unknown to you uses the array and the index to produce the array element.
There is obviously no need for the array to store indexes, since for example every array in the world with five array elements has the indexes 0, 1, 2, 3, and 4. We know these are the indexes, no need to store them.
An array is a contigious memory allocation, which means if you know the address of the first element you can go to the next index by stepping to next memory address.
The reference array is not the array address, but the way to reach the address (done internally) like normal objects. So you can say you have the position from where the array starts, and you can move a memory address by changing the indexes. So this is why indexes are not specified in the memory; the compiler just knows where to go.
Out of interest: Recently, I encountered a situation in one of my Java projects where I could store some data either in a two-dimensional array or make a dedicated class for it whose instances I would put into a one-dimensional array. So I wonder whether there exist some canonical design advice on this topic in terms of performance (runtime, memory consumption)?
Without regard of design patterns (extremely simplified situation), let's say I could store data like
class MyContainer {
public double a;
public double b;
...
}
and then
MyContainer[] myArray = new MyContainer[10000];
for(int i = myArray.length; (--i) >= 0;) {
myArray[i] = new MyContainer();
}
...
versus
double[][] myData = new double[10000][2];
...
I somehow think that the array-based approach should be more compact (memory) and faster (access). Then again, maybe it is not, arrays are objects too and array access needs to check indexes while object member access does not.(?) The allocation of the object array would probably(?) take longer, as I need to iteratively create the instances and my code would be bigger due to the additional class.
Thus, I wonder whether the designs of the common JVMs provide advantages for one approach over the other, in terms of access speed and memory consumption?
Many thanks.
Then again, maybe it is not, arrays are objects too
That's right. So I think this approach will not buy you anything.
If you want to go down that route, you could flatten this out into a one-dimensional array (each of your "objects" then takes two slots). That would give you immediate access to all fields in all objects, without having to follow pointers, and the whole thing is just one big memory allocation: since your component type is primitive, there is just one object as far as memory allocation is concerned (the container array itself).
This is one of the motivations for people wanting to have structs and value types in Java, and similar considerations drive the development of specialized high-performance data structure libraries (that get rid of unneccessary object wrappers).
I would not worry about it, until you really have a huge datastructure, though. Only then will the overhead of the object-oriented way matter.
I somehow think that the array-based approach should be more compact (memory) and faster (access)
It won't. You can easily confirm this by using Java Management interfaces:
com.sun.management.ThreadMXBean b = (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();
long selfId = Thread.currentThread().getId();
long memoryBefore = b.getThreadAllocatedBytes(selfId);
// <-- Put measured code here
long memoryAfter = b.getThreadAllocatedBytes(selfId);
System.out.println(memoryAfter - memoryBefore);
Under measured code put new double[0] and new Object() and you will see that those allocations will require exactly the same amount of memory.
It might be that the JVM/JIT treats arrays in a special way which could make them faster to access in one way or another.
JIT do some vectorization of an array operations if for-loops. But it's more about speed of arithmetic operations rather than speed of access. Beside that, can't think about any.
The canonical advice that I've seen in this situation is that premature optimisation is the root of all evil. Following that means that you should stick with the code that is easiest to write / maintain / get past your code quality regime, and then look at optimisation if you have a measurable performance issue.
In your examples the memory consumption is similar because in the object case you have 10,000 references plus two doubles per reference, and in the 2D array case you have 10,000 references (the first dimension) to little arrays containing two doubles each. So both are one base reference plus 10,000 references plus 20,000 doubles.
A more efficient representation would be two arrays, where you'd have two base references plus 20,000 doubles.
double[] a = new double[10000];
double[] b = new double[10000];
When designing java classes, what are the recommendations for achieving CPU cache friendliness?
What I have learned so far is that one should use POD as much as possible (i.e. int instead of integer). Thus, the data will be allocated consecutively when allocating the containing object. E.g.
class Local
{
private int data0;
private int data1;
// ...
};
is more cache friendly than
class NoSoLocal
{
private Integer data0;
private Integer data1;
//...
};
The latter will require two separate allocations for the Integer objects that can be at arbitrary locations in memory, esp. after a GC run. OTOH the first approach might lead to duplication of data in cases where the data can be reused.
Is there a way to have them located close to each other in memory so that the parent object and its' containing elements will be in the CPU cache at once and not distributed arbitrarily over the whole memory plus the GC will keep them together?
You cannot force JVM to place related objects close to each other (though JVM tries to do it automatically). But there are certain tricks to make Java programs more cache-friendly.
Let me show you some examples from the real-life projects.
BEWARE! This is not a recommended way to code in Java!
Do not adopt the following techniques unless you are absolutely sure why you are doing it.
Inheritance over composition. You've definitely heard the contrary principle "Favor composition over inheritance". But with composition you have an extra reference to follow. This is not good for cache locality and also requires more memory. The classic example of inheritance over composition is JDK 8 Adder and Accumulator classes which extend utility Striped64 class.
Transform arrays of structures into a structure of arrays. This again helps to save memory and to speed-up bulk operations on a single field, e.g. key lookups:
class Entry {
long key;
Object value;
}
Entry[] entries;
will be replaced with
long[] keys;
Object[] values;
Flatten data structures by inlining. My favorite example is inlining 160-bit SHA1 hash represented by byte[]. The code before:
class Blob {
long offset;
int length;
byte[] sha1_hash;
}
The code after:
class Blob {
long offset;
int length;
int hash0, hash1, hash2, hash3, hash4;
}
Replace String with char[]. You know, String in Java contains char[] object under the hood. Why pay performance penalty for an extra reference?
Avoid linked lists. Linked lists are very cache-unfriendly. Hardware works best with linear structures. LinkedList can be often replaced with ArrayList. A classic HashMap may be replaced with an open address hash table.
Use primitive collections. Trove is a high-performance library containg specialized lists, maps, sets etc. for primitive types.
Build your own data layouts on top of arrays or ByteBuffers. A byte array is a perfect linear structure. To achieve the best cache locality you can pack an object data manually into a single byte array.
the first approach might lead to duplication of data in cases where the data can be reused.
But not in the case you mention. An int is 4 bytes, a reference is typically 4-bytes so you don't gain anything by using Integer. For a more complex type, it can make a big difference however.
Is there a way to have them located close to each other in memory so that the parent object and its' containing elements will be in the CPU cache at once and not distributed arbitrarily over the whole memory plus the GC will keep them together?
The GC will do this anyway, provided the objects are only used in one place. If the objects are used in multiple places, they will be close to one reference.
Note: this is not guaranteed to be the case, however when allocating objects they will typically be continuous in memory as this is the simplest allocation strategy. When copying retained objects, the HotSpot GC will copy them in reverse order of discovery. i.e. they are still together but in reverse order.
Note 2: Using 4 bytes for an int is still going to be more efficient than using 28 bytes for an Integer (4 bytes for reference, 16 bytes for object header, 4 bytes for value and 4 bytes for padding)
Note 3: Above all, you should favour clarity over performance, unless you have measured you need and have a more performant solution. In this case, an int cannot be a null but an integer can be null. If you want a value which should not be null use int, not for performance but for clarity.
I have to store millions of X/Y double pairs for reference in my Java program. I'd like to keep memory consumption as low as possible as well as the number of object references. So after some thinking I decided holding the two points in a tiny double array might be a good idea, it's setup looks like so:
double[] node = new double[2];
node[0] = x;
node[1] = y;
I figured using the array would prevent the link between the class and my X and Y variables used in a class, as follows:
class Node {
public double x, y;
}
However after reading into the way public fields in classes are stored, it dawned on me that fields may not actually be structured as pointer like structures, perhaps the JVM is simply storing these values in contiguous memory and knows how to find them without an address thus making the class representation of my point smaller than the array.
So the question is, which has a smaller memory footprint? And why?
I'm particularly interested in whether or not class fields use a pointer, and thus have a 32-bit overhead, or not.
The latter has the smaller footprint.
Primitive types are stored inline in the containing class. So your Node requires one object header and two 64-bit slots. The array you specify uses one array header (>= an object header) plust two 64-bit slots.
If you're going to allocate 100 variables this way, then it doesn't matter so much, as it is just the header sizes which are different.
Caveat: all of this is somewhat speculative as you did not specify the JVM - some of these details may vary by JVM.
I don't think your biggest problem is going to be storing the data, I think it's going to be retrieving, indexing, and manipulating it.
However, an array, fundamentally, is the way to go. If you want to save on pointers, use a one dimensional array. (Someone has already said that).
First, it must be stated that the actual space usage depends on the JVM you are using. It is strictly implementation specific. The following is for a typical mainstream JVM.
So the question is, which has a smaller memory footprint? And why?
The 2nd version is smaller. An array has the overhead of the 32 bit field in the object header that holds the array's length. In the case of a non-array object, the size is implicit in the class and does not need to be represented separately.
But note that this is a fixed over head per array object. The larger the array is, the less important the overhead is in practical terms. And the flipside of using a class rather than array is that indexing won't work and your code may be more complicated (and slower) as a result.
A Java 2D array is actually and array of 1D arrays (etcetera), so you can apply the same analysis to arrays with higher dimensionality. The larger the size an array has in any dimension, the less impact the overhead has. The overhead in a 2x10 array will be less than in a 10x2 array. (Think it through ... 1 array of length 2 + 2 of length 10 versus 1 array of length 10 + 10 of length 2. The overhead is proportional to the number of arrays.)
I'm particularly interested in whether or not class fields use a pointer, and thus have a 32-bit overhead, or not.
(You are actually talking about instance fields, not class fields. These fields are not static ...)
Fields whose type is a primitive type are stored directly in the heap node of the object without any references. There is no pointer overhead in this case.
However, if the field types were wrapper types (e.g. Double rather than double) then there could be the overhead of a reference AND the overheads of the object header for the Double object.