This post would likely be a good candidate for frequently asked questions at OpenHFT.
I am playing with ChronicleMap considering it for an idea but having lots of questions. I am sure most junior programmers who are looking into this product have similar considerations.
Would you explain how memory is managed in this API?
ChronicleMap proclaims some remarkable TBs off-heap memory resources available to processing its data and I would like to get a clear vision on that.
Lets get down to a programmer with a laptop of 500GB HD and 4GB RAM. In this case pure math sais - total resource of 'swapped' memory available is 504GB. Let's give the OS and other programs half and we are left with 250GB HD and 2GB RAM. Can you elaborate on the actual available memory ChronicleMap can allocate in numbers relative to available resources?
Next related questions are relative to the implementation of ChronicleMap.
My understanding is that each ChronicleMap allocates chunk of memory it works with and optimal performance/memory usage is achieved when we can accurately predict the amount of data passed through. However, this is a dynamic world.
Lets set an (exaggerated but possible) example:
Suppose a map of K (key) 'cities' and their V (value) - 'description' (of the cities) and allowing users large limits on the description length.
First user enters: K = "Amsterdam", V = "City of bicycles" and this entry is used to declare the map
- it sets the precedent for the pair like this:
ChronicleMap<Integer, PostalCodeRange> cityPostalCodes = ChronicleMap
.of(CharSequence.class, CharSequence.class)
.averageKey("Amsterdam")
.averageValue("City of bicycles")
.entries(5_000)
.createOrRecoverPersistedTo(citiesAndDescriptions);
Now, next user gets carried away and writes an assay about Prague
He passes to: K = "Prague", V = "City of 100 towers is located in the hard of Europe ... blah, blah... million words ..."
Now the programmer had expected max 5_000 entries, but it gets out of his hands and there are many thousands of entries.
Does ChronicleMap allocate memory automatically for such cases? If yes is there some better approach of declaring ChronicleMaps for this dynamic solution? If no, would you recommend an approach (best in code example) how to best handle such scenarios?
How does this work with persistence to file?
Can ChronicleMaps deplete my RAM and/or disk space? Best practice to avoid that?
In other words, please explain how memory is managed in case of under-estimation and over-estimation of the value (and/or key) lengths and number of entries.
Which of these are applicable in ChronicleMap?
If I allocate big chunk (.entries(1_000_000), .averageValueSize(1_000_000) and actual usage is - Entries = 100, and Average Value Size = 100.
What happens?:
1.1. - all works fine, but there will be large wasted chunk - unused?
1.2. - all works fine, the unused memory is available to:
1.2.1 - ChronicleMap
1.2.2 - given thread using ChronicleMap
1.2.3 - given process
1.2.4 - given JVM
1.2.5 - the OS
1.3. - please explain if something else happens with the unused memory
1.4. - what does the over sized declaration do to my persistence file?
Opposite of case 1 - I allocate small chunk (.entries(10), .averageValueSize(10) and the actual usage is 1_000_000s of entries, and Average Value Size = 1_000s of bytes.
What happens?:
Lets get down to a programmer with a laptop of 500GB HD and 4GB RAM. In this case pure math sais - total resource of 'swapped' memory available is 504GB. Let's give the OS and other programs half and we are left with 250GB HD and 2GB RAM. Can you elaborate on the actual available memory ChronicleMap can allocate in numbers relative to available resources?
Under such conditions Chronicle Map will be very slow, with on average 2 random disk reads and writes (4 random disk operations in total) on each operation with Chronicle Map. Traditional disk-based db engines, like RocksDB or LevelDB, should work better when the database size is much bigger than memory.
Now the programmer had expected max 5_000 entries, but it gets out of his hands and there are many thousands of entries.
Does ChronicleMap allocate memory automatically for such cases? If yes is there some better approach of declaring ChronicleMaps for this dynamic solution? If no, would you recommend an approach (best in code example) how to best handle such scenarios?
Chronicle Map will allocate memory until the actual number of entries inserted divided by the number configured through ChronicleMapBuilder.entries() is not higher than the configured ChronicleMapBuilder.maxBloatFactor(). E. g. if you create a map as
ChronicleMap<Integer, PostalCodeRange> cityPostalCodes = ChronicleMap
.of(CharSequence.class, CharSequence.class)
.averageKey("Amsterdam")
.averageValue("City of bicycles")
.entries(5_000)
.maxBloatFactor(5.0)
.createOrRecoverPersistedTo(citiesAndDescriptions);
It will start throwing IllegalStateException on attempts to insert new entries, when the size will be ~ 25 000.
However, Chronicle Map works progressively slower when the actual size grows far beyond the configured size, so the maximum possible maxBloatFactor() is artificially limited to 1000.
The solution right now is to configure the future size of the Chronicle Map via entries() (and averageKey(), and averageValue()) at least approximately correctly.
The requirement to configure plausible Chronicle Map's size in advance is acknowledged to be a usability problem. There is a way to fix this and it's on the project roadmap.
In other words, please explain how memory is managed in case of under-estimation and over-estimation of the value (and/or key) lengths and number of entries.
Key/value size underestimation: space is wasted in hash lookup area, ~ 8 bytes * underestimation factor, per entry. So it could be pretty bad if the actual average entry size (key + value) is small, e. g. 50 bytes, and you have configured it as 20 bytes, you will waste ~ 8 * 50 / 20 = 20 bytes, or 40%. Bigger the average entry size, smaller the waste.
Key/value size overestimation: if you configure just key and value average size, but not actualChunkSize() directly, the actual chunk size is automatically chosen between 1/8th and 1/4th of the average entry size (key + value). The actual chunk size is the allocation unit in Chronicle Map. So if you configured average entry size as ~ 1000 bytes, the actual chunk size will be chosen between 125 and 250 bytes. If the actual average entry size is just 100 bytes, you will lose a lot of space. If the overestimation is small, the expected space losses are limited to about 20% of the data size.
So if you are afraid you may overestimate the average key/value size, configure actualChunkSize() explicitly.
Number of entries underestimation: discussed above. No particular space waste, but Chronicle Map works slower, the worse the underestimation.
Number of entries overestimation: memory is wasted in hash lookups area, ~ 8 bytes * overestimation factor, per entry. See the section key/value size underestimation above on how good or bad it could be, depending on the actual average entry data size.
Related
Heap space have only 80 MB memory available. We need to put 100 elements(of 1MB each) in TreeMap.
What is the behavior in this case when 80th element is put into map?
How will garbage collection work in this scenario?
what is the effective way to handle such scenario?
This is a bit un-clear, since you put Key and Value to a Map, not individual elements. I'll assume your Key + Value combined will give you 1MB.
If they indeed have 1MB each - your memory will blow up a lot faster then even 50 elements that you put into your TreeMap, since each object has 2 headers (12 bytes combined on typical VM's); there is padding involved + each Key + Value is wrapped into an Entry internally that add some overhead too.
GC is not going to be able to help you much, you have limited memory to begin with; your VM is blow up with an OutOfMemory. GC will be triggered when you are getting close to the limits (this heavily depends on the GC used and the input parameters), but since there will be not much to clean - it will basically not do much, if anything at all.
I have the following function:
public void scanText(char[] T){
int q=0;
for(int i=0;i<T.length;i++){
q = transFunc[preCompRow[q]+T[i]];
if(q==pattern.length){
System.out.println("match found at position: "+(i-pattern.length+2));
}
}
}
This function scans a char Array searching for matches of a given pattern, which is stored as a finite automata. The transition function of the automata is stored in the variable called transFunc.
I am testing this function in a text with 8 millions of characters and using 800000 patterns. The thing is the accession of the array preCompRow[q] (which is an int[]) is very slow. The performance is greatly improved if I delete the preCompRow[q] of the code. I think this might be because in every loop the q variable has a different non-sequential value (2, 56, 18, 9 ..).
Is there any better way to access to an array in a non-sequential manner?
Thanks in advance!
One possible explanation is that your code is seeing poor memory performance due to poor locality in its memory access patterns.
The role of the memory caches in a modern computer is to deal with the speed mismatch between processor instruction times (less than 1 ns) and main memory (5 to 10 ns or more). They work best when your code gets a cache hit most time it fetches from memory.
A modern Intel chipset caches memory in blocks of 64 bytes, and loads from main memory in burst mode. (That corresponds to 16 int values.) The L1 cache on (say) an I7 processor is 2MB.
If your application is able to access the data in a large array (roughly) sequentially, then 7 out of 8 accesses will be a cache hits. If the access pattern is non-sequential and the "working set" of is a large multiple of the cache size, then you may end up with a cache miss on each memory access.
If memory access locality is the root of yoiur problems, then your option are limited:
redesign your algorithm so that locality of memory references is better
buy hardware with larger caches
(maybe) redesign your algorithm to use GPUs or some other strategy to reduce the memory traffic
Recoding your existing in C or C++ may give a performance improvement, but the same memory locality problems will bite you there as well.
I am not aware of any tools that can be used to measure cache performance in Java applications.
How many entries theoretically can ChronicleMap contain at maximum? What is the number of maximum entries one can put to ChronicleMap?
The theoretical maximum is limited by the virtual memory you can have. This can be as low as 128 TB depending on your OS. By comparison Chronicle Queue doesn't have this limit, as it swaps in memory mappings, but would be much slower for random access as a result.
In practice, limiting the size of your map to 2x - 40x main memory size seems to be a realistic upper bound.
In short, the smaller the entries, the more you can have.
I have a large amount of data that I'm currently storing in an AtomicReferenceArray<X>, and processing from a large number of threads concurrently.
Each element is quite small and I've just got to the point where I'm going to have more than Integer.MAX_VALUE entries. Unfortunately List and arrays in java are limited to Integer.MAX_VALUE (or just less) values. Now I have enough memory to keep a larger structure in memory - with the machine having about 250GB of memory in a 64b VM.
Is there a replacement for AtomicReferenceArray<X> that is indexed by longs? (Otherwise I'm going to have to create my own wrapper that stores several smaller AtomicReferenceArray and maps long accesses to int accesses in the smaller ones.)
Sounds like it is time to use native memory. Having 4+ billion objects is going to cause some dramatic GC pause times. However if you use native memory you can do this with almost no impact on the heap. You can also use memory mapped files to support faster restarts and sharing the data between JVMs.
Not sure what your specific needs are but there are a number of open source data structures which do this like; HugeArray, Chronicle Queue and Chronicle Map You can create an array which 1 TB but uses almost no heap and has no GC impact.
BTW For each object you create, there is a 8 byte reference and a 16 byte header. By using native memory you can save 24 bytes per object e.g. 4 bn * 24 is 96 GB of memory.
My Java application deals with large binary data files using memory mapped file (MappedByteBuffer, FileChannel and RandomAccessFile). It often needs to grow the binary file - my current approach is to re-map the file with a larger region.
It works, however there are two problems
Grow takes more and more time as the file becomes larger.
If grow is conducted very rapidly (E.G. in a while(true) loop), JVM will hang forever after the re-map operation is done for about 30,000+ times.
What are the alternative approaches, and what is the best way to do this?
Also I cannot figure out why the second problem occurs. Please also suggest your opinion on that problem.
Thank you!
Current code for growing a file, if it helps:
(set! data (.map ^FileChannel data-fc FileChannel$MapMode/READ_WRITE
0 (+ (.limit ^MappedByteBuffer data) (+ DOC-HDR room))))
You probably want to grow your file in larger chunks. Use a doubling each time you remap, like a dynamic array, so that the cost for growing is an amortized constant.
I don't know why the remap hangs after 30,000 times, that seems odd. But you should be able to get away with a lot less than 30,000 remaps if you use the scheme I suggest.
The JVM doesn't clean up memory mappings even if you call the cleaner explicitly. Thank you #EJP for the correction.
If you create 32,000 of these they could be all in existence at once. BTW: I suspect you might be hitting some 15-bit limit.
The only solution for this is; don't create so many mapping. You can map an entire disk 4 TB disk with less than 4K mapping.
I wouldn't create a mapping less than 16 to 128 MB if you know the usage will grow and I would consider up to 1 GB per mapping. The reason you can do this with little cost is that the main memory and disk space will not be allocated until you actually use the pages. i.e. the main memory usage grows 4 KB at a time.
The only reason I wouldn't create a 2 GB mapping is Java doesn't support this due to an Integer.MAX_VALUE size limit :( If you have 2 GB or more you have to create multiple mappings.
Unless you can afford an exponential growth on the file like doubling, or any other constant multiplier, you need to consider whether you really need a MappedByteBuffer at all, considering their limitations (unable to grow the file, no GC, etc). I personally would either be reviewing the problem or else using a RandomAccessFile in "rw" mode, probably with a virtual-array layer over the top of it.