Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
So I have this app, a Java servlet. It uses a dictionary object that reads words from a file specified as a constructor parameter on instantiation and then serves queries.
I can do basically the same on PHP, but it's my understanding the class will be instantiated on each and every request, and the file will be read again every time. In fact, I did it and it works, but it collapses my humble amazon EC2 micro instance at the ridiculous amount of 11 requests per second or more.
My question is: Shouldn't some kind of compiler/file system optimization be kicking in and making the performance impact insignificant when the file does not change at all?
If the answer is no, I guess my design is quite poor and I should try to improve it. In that case, my second question is: What would be the best approach to improve it?
Building a servlet-like service so the code is properly reused?
Using memcached to keep the words file content in memory?
Using a RDBMS instead of a plain text file and have my dictionary querying it?
(despite the dictionary being only a few KB of static data and despite having to perform some complex queries such as selecting a
(cryptographically safe) random word from those having a length
higher than some per-request user setting and such?)
Something else?
Your best bet is to generate a PHP file which contains the final structure of the dictionary in PHP code. You could then include() that cache file into your code or write a new one when the file changes. You should store it on the filesystem, no databases. You could cache it in memory as well. But I don't think this is really needed at this point.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I'm about to start a program that is constantly getting information from a web API so I want it read the API JSONs as fast as possible in order not to lose any time (I want to check some values that are constantly updating).
As I'm coding using Java, I thought I could use gson to parse those JSON information but I don't know if it's the most efficient way.
Also, is Java a good language in order to read APIs efficiently?
If it's about performance there are some issues to look at / to optimize:
query speed for the data (e.g. the used http or network client)
speed of parsing the result
speed of the output of the data
....
If you want to improve the performance, I would suggest you analyze at first, where the most performance is lost in that chain, for example by logging durations for each part.
When you found the bottleneck, you can improve it and measure it again, to see wether your program became faster.
For example you could perhaps use UDP instead of tcp. Or use http2 instead of old http protocol and so on.
If it's really the parsing part, which makes critical durations, you could try to use the fact, that the data is always in the same structure. For example you could look at "keywords" in your JSON format and extract the text right before or after these keywords. Then your program doesn't have to parse (or "understand") the whole structure and can operate (possibly) faster.
Or you can extract the facts you search for with certain positions (for example the info is always after the sixth curly open brace).
But you should only optimize, if it's a real performance gain (see first part of the answer) because it's quite likely that your code gets less readable, when you optimize it for the sake of performance. That's often the tradeoff, one has to choose.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I have to build a search functionality where GUI will provide a search field to search objects in Oracle database. There are currently 30K objects I have to search on but they will grow in number over time. 300-400 per month approx.
As a part of the requirement, when user types in any text in search Like for example "ABC", then all objects in the DB that contains ABC should appear in a datatable more like system is predicting results based on what user has types in the search field.
Question is how to architect such feature?
Simple way to do is to load everything in the GUI Javascript object and run search on it. Since JS is ridiculously fast, performance wont be an issue.
Another way is to run query in the Database everytime user types in text in search field. This does not seem convenient as it will put unnecessary load on the database.
Is there a better way to architect this feature? Please share thoughts.
premature optimization is seldom useful.
300-400 object growth per month with a 30k base object is nothing at all for any DB to handle.
loading all 30k object at once on the browser is awful and may affect performance while querying result in the DB will not have this problem until you have LOT of and LOT of users accessing the DB.
You should be building the service using the Database and then if/when you reach a bottleneck you can think about optimization trick such as caching frequent queries on the database.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I am working on a project where I will have a binary file. The file is split into multiple sections, each of which represents a list of primitive values. I need a solution where I can have a collection of objects, each of which represents a section of the file. These collections are then all held within a "file" object that represents the file as a whole.
Each collections object will need to provide sequential access to each value in the represented section of the file. What method would provide the fastest data retrieval without loading all the data into memory first?
Also it would be nice if two separate collections of the same "file" object could be accessed by two separate Threads, but this is not as important.
A good approach is to divide the solution into layers, here: one for the file i/o, mapping bytes to Java shorts and ints, another one for the abstraction of the file sections and the entire file.
java.nio's MappedByteBuffer provides a good interface between the "byte array" of a random access file and what you need for getting the Java typed data from that.
As Kayaman has mentioned, FileChannel.map() returns a MappedByteBuffer and you can navigate easily on that with its methods.
The implemention should make use of the OS feature for mapping memory pages to file pages, actually accessing on the file only what you really access in memory. (I've used this recently with Java 8 and Linux, and it performed well on files exceeding even the capacity of a single MappedByteBuffer.)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
Let us say that I have a method that writes to a file or database. What if different parts of application calls this method too many times at the same time or in same interval of time. All those method calls are maintained in some stack/queue in memory and wait for previous requests to be served ?
Writing to the same file is platform dependent, like Unix allows concurrent writes to the same file.
You need to see the synchronization techniques - how you want to manage the read write operations.
If you see from DB perspective the db engine handles it properly - whichever comes first will be served. The next insert would depend on the first insert(in case you already inserted with the same key in the previous operation - then obviously it ll throw an exception)
Also I would say if different parts of your application are appending data to the same file at the same time - there could be design flaw and you need to reconsider the design
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Given a .JSP, would there be any reason to store static text in a .properties file, pull the key/value pair, and display it? Is that the best practice / standardized way? If so, why? Or can you "hard code" the static text into the .JSP? Are there benefits (performance, code readability, etc.) with each way?
My response could differ depending on what you are referring to as static text? Storing the values within a properties file is one way of approaching it, but I would advise that you store these types of key value pairs in a database that you could later retrieve and cache at start-up.
This depends on what the "text" is. If it's user-visible, such as form labels, then storing it in a properties file allows you to make use of the built-in i18n capabilities of JSP and similar technologies, and it also permits easy updates in case of typos or such. Externalizing user-visible text is always a good best practice.
If the text is something that's "static" but likely to change on a reasonably frequent basis, then it belongs in a persistent store such as the database you're using for your other persistence, or a document database like MongoDB.
Or you can store your text in the database. I think this is most preferrable way.