Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I have application with google map, because googlemaps not provide api to create routes on map in Android, i'm using WebApi to get routes.
I store route in ArrayList, and pass then throught Intent into my activity where drow a route. If LatLng objects less then 2000 it works nice, else if it more then 2000 objects then i have Java Binder Exception, maybe Intent have limit size.
To handle situation with more then 2000 objects i save LatLng latitude and longitude to file and pass only path, then in activity i load file. But this files very big ~ 0.5-3 mB.
How i can solve my problem, or how i can compress my arraylist with double values?
A couple of thoughts
Don't use an object to encapsulate lat/long as that has additional overhead. Use a data structure that is more concise such as an array of Doubles or a geohash of the lat/long coordinates
Compress the output as another answer suggested
A more advanced option could be to use a trie to encode the list of geohashs. Given that a route is going to consist of numerous coordinates very close to each other the 'compression' is likely to be quite high.
You could compress your data to an ZipOutputStream and pass this as raw bytes or use the way over a ZipFile. Compression should be quite good, if values ocure more often.
Make sure you don't waste space when writing the data. Don't use text/ASCII/XML data structures/files for storing the data.
Use DataOutputStream and writeDouble(..) method for writing the data (result: 8 byte per double value). Alternatively you could convert the double values to float and write them via writeFloat(..) - that would halve the data to be written. Usually for displaying points on a map a float would still be precise enough.
Reading works of course just the other way round using DataInputStream.
Compression would also be an option but better to minimize the data to be written first as compression can take much CPU resources, meaning it consumes power and time.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 5 years ago.
Improve this question
Currently I am working on transfer image from C++ to Java.
The destination location is allocate by Java,
the source location is the image generated by C++, so.
I have a
uint8_t* pixelPtr
, I want to move the content of this to a
__uint8_t* data
without copy.
I have 1920*1080*3 bytes in total, so I want to move rather than copy to be fast in computation, I am wondering is there any trick way to do so?
Thank you in advance!
Let's recap:
The source is a buffer allocated in C++ by an image generation function.
The destination is a buffer allocated in Java by some other code somewhere.
You want to transfer data between the two buffers.
As long as those two buffers are distinct, there is no "trick" to avoid this. "Moving" in this context would mean swapping the pointers around, but that does nothing to the underlying buffers. You will just have to copy the data.
Explore solutions such as generating the data in the destination buffer in the first place, or making use of appropriate functionality exposed by the C++ image generation function (or the Java code). Unfortunately we can't speculate on the possible existence or form of such solutions, from here.
The standard way is, you should modify your C++ code so it creates the data not wherever it wants, but in the given place. That is, if you have code like this
uint8_t* GenerateImage(...parameters...)
{
uint8_t* output = ... allocate ...
return output;
}
you should change it to receive the destination as a parameter
void GenerateImage(...parameters..., __uint8_t* destination)
{
... fill the destination ...
}
The latter is better C++ design anyway - this way you don't need to make a separate DestroyImage function - the memory is managed entirely by Java.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I have a C++ program and a Java program. On the C++ side, I have a vector having millions of entries. I need to transfer this data to my Java program. So far, I have tried the following:
Created a unix socket, converted the vector to a long string (Serialized) and send it through the unix socket
Created a thrift server/client model to transfer the data
Both approaches worked very well, but the performance I'm getting is quite low. I don't even see it using the full network bandwidth (in the case of thrift).
Also with the unix socket approach, Since I'm serializing it to String and then again converting this string back to a string array (received byte[] to String and split) on the Java side is very expensive operation.
What is the best way to transfer data faster from the C++ world to the Java world with lesser overhead on reconstructing/serializing the object?
If both problems are on the same machine, I would use a shared memory mapped file.
This way both programs can access the data at full memory speed without serialization/deserialization esp if the values are int, long or double values.
You can place the file on a tmpfs, or ram drive to avoid hitting a hard drive.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I have several matrices in Java that I would like to transfer to Python as efficiently as possible, without requiring anything but standard libraries on both the Java and Python sides.
Currently I serialize them to file using the writeDouble function to write the entries out one by one, and writeInt to write the dimensions of the matrices. Now I would like to read these matrices back into Python. I can get the integers using struct.unpack, but Java's serialization of doubles does not correspond to an algorithm that struct.unpack can implement.
How can I decode a Java double in the binary format that writeDouble uses? I have trouble even finding a specification for the encoding that writeDouble uses.
You're overengineering it; DataOutputStream.writeDouble() and related methods are for manually serializing a Java Object, so it can be re-read as a Java Object. If all you need is to transfer data, you can simply write them out as text (or bytes), then read them back in. Common formats are CSV, JSON, XML, and ProtoBuf.
If all you're doing is trying to transfer a list of doubles, you can even just write them out one per line, and read them right back in with Python.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I am working on a project where I will have a binary file. The file is split into multiple sections, each of which represents a list of primitive values. I need a solution where I can have a collection of objects, each of which represents a section of the file. These collections are then all held within a "file" object that represents the file as a whole.
Each collections object will need to provide sequential access to each value in the represented section of the file. What method would provide the fastest data retrieval without loading all the data into memory first?
Also it would be nice if two separate collections of the same "file" object could be accessed by two separate Threads, but this is not as important.
A good approach is to divide the solution into layers, here: one for the file i/o, mapping bytes to Java shorts and ints, another one for the abstraction of the file sections and the entire file.
java.nio's MappedByteBuffer provides a good interface between the "byte array" of a random access file and what you need for getting the Java typed data from that.
As Kayaman has mentioned, FileChannel.map() returns a MappedByteBuffer and you can navigate easily on that with its methods.
The implemention should make use of the OS feature for mapping memory pages to file pages, actually accessing on the file only what you really access in memory. (I've used this recently with Java 8 and Linux, and it performed well on files exceeding even the capacity of a single MappedByteBuffer.)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
Using a general-purpose programming language like Java, what is the most efficient way to search through a ~20 page document to replace a set of 5000+ strings with some predetermined replacement string? The program should not replace any strings that have already been replaced. What data structure would be optimal to store the 5000+ strings and each of their replacements - two arrays, a dictionary, or something else?
Here are some of the options that I have considered so far:
Iterate through the entire .txt document once time per string using string.replace. The problem is that the algorithm must iterate through the entire .txt document an extra time for each string stored.
Iterate through the .txt once while replacing string as necessary while creating a new string by appending replacements. This seems more efficient, but each step would still require checking the entire set of 5000+ strings for any strings to replace.
Is there a more optimized means of solving this problem, or is one of the above attempts already optimal?
Also, would it be possible to run this algorithm more efficiently in a lower-level language like C?
You want to replace some string in 5000 strings and you want to make it optimal ... Now my question to you is: How will you know if you have to replace a string if you dont read the string? It's not possible, you have to read everything. And the shortest way to do that is to go line by line and replace immediatly. And somebody can correct me if i'm wrong, but reading a file is one of the most basic operations there is so using a library for that besides what is available by default in the programming language seems total overkill to me. Furthermore, every language has basic io and if it doesn't then don't use it.
To store strings, it all depends what you want to do with them. Different data structures have different purposes and some are better suited in some situations then others. If you just need to store them then a simple array is fine. However, if you need more advanced functions then you need to consider your options. But again it's all up to what you want to do with them later.
And there is the memory issue, you need to calculate how much memory your 5000+ strings will take, because you might run out of memory. Then you need to think if it's worth it to use all that memory. check this link
Finally your question about C, ofcourse it will be more efficient. Java runs in a virtual machine that adds considerable overhead. So basically your Java program runs in another Java program and if you know that there is a cost for every single operation then you understand that C will be more efficient then Java in terms of performance.
I would use the commons-lang library, which I think has exactly what you are looking for. Basically you create one array with all the strings you want to substitute and another array with the substitutions. See http://commons.apache.org/proper/commons-lang/javadocs/api-release/index.html for details on the StringUtils#replaceEach method.