Having a lot of data in an application - java

I am planning on creating an android application sometime in the future in which I'll want it to display a lot of constant data on the screen.
I'm not sure the best way to do this but I see two options:
Storing the data within the code itself such as creating a constants class.
Using an embedded database to hold the data.
I'm guessing option #2 is the best way? But it just seems weird using a database if I'm not going to be doing any updating to the database, I would only be selecting.
The total amount of data that I need the application to display is maybe about 400 lines consisting of a string and two integers...
Is there a different way people use for such a situation that I don't know about?

But it just seems weird using a database if I'm not going to be doing
any updating to the database
I am totally disagree with you. Database is not only for updating. It can be used as a better storage and definitely a best way for searching. So as you want to preserve the data then it is definitely wise to use database.
But if you want to handle data which will not persists , i,e you will use different datas for different run then you can use temporary class or other data structure to store data.
Finally, If you are planning to have portability then File storage is an easier solution.
SO you can see, that it totally depends on what you want.

Related

Java : relational database vs static variable

I have a web application in which I'm maintaining many static Maps to store my relevant information. Since the application is deployed on a server. Each and every hit to the server side java uses these maps to match the key and get appropriate result and send back to the client side. My code contains a rank and retrieval feature so I have to read the entire keySet of each of these Maps.
My question is:
1. Is working with static variables better than storing this data in a local embedded DB like Apache Derby and then using it?
2. The use of this data is very frequent. So if I use database will that be faster approach? Since I read the full keyset the where clause may not come handy in many operations.
3. How does the server's memory gets impacted on holding data in static variables?
My no. of maps are fixed but the size of the Maps keeps increasing? Please suggest the better solution.
If you want the data to be saved regularly an embedded database like H2 makes sense. You then also have snapshots of the data, and development, structural changes are a bit more safe.
A real database also has an incredible power behind it: concurrency, caching and so on. An embedded (when file based) database less so.
The problem with maps is that the data extraction can become several indirections. It is more versatile to have SQL queries with joins on the tables.
So SQL is more abstract (does not prescribe the actual query implementation), and easier to test. SQL for instance releases the developer of programming reports.
So go for a database IMHO, when you are really doing hard work.
What you might want to consider is to store the data searched in map when it's searched.
For instance, if a user searches for something specific, that something is stored in the map so that the next user who searches for that gets the data directly from the map rather than the database.
There are some downsides though, as you need to make sure that if the data is changed on the database, the hashmap/cache should be cleared or updated with the new data, as to prevent feeding outdated data to the user.
As for the impact on the server's memory, it depends on the size of the data you're storing. It's hard to give you a precise answer, but you can however test that on your own:
long memoryBefore = Runtime.getRuntime().freeMemory();
// populate your map
long memoryAfter = Runtime.getRuntime().freeMemory();
System.out.println(memoryBefore - memoryAfter);
That should give you the amount of bytes used (more or less, depending on the operations you run between memoryBefore and memoryAfter, as you may have instantiated other classes/variables unrelated to the hashmap)

Using hashmap or H2 database?

I am developing a web application in which I need to store session, user messages etc. I am thinking of using HashMap or H2 database.
Please let me know which is better approach in terms of performance and memory utilization. The web site has to support 10,000 users.
Thanks.
As usual with these questions, I would worry about performance as/when you know it's an issue.
10000 users is not a lot of data to hold in memory. I would likely start off with a standard Java collection, and look at performance when you predict it's going to cause you grief.
Abstract out the access to this Java collection such that when you substitute it, the refactoring required is localised (and perhaps make it configurable, such that you can easily perform before/after performance tests with your different solutions -H2, Derby, Oracle, etc. etc.)
If your session objects aren't too big (which should be the case), there is no need to persist them in a database.
Using a database for this would add a lot of complexity in a case when you can start with a few lines of code. So don't use a database, simply store them in a ligth memory structure (HashMap for example).
You may need to implement a way to clean your HashMap if you don't want to keep sessions in memory when the user left from a long time. Many solutions are available (the easiest is simply to have a background thread removing from time to time the too old sessions). Note that it's usually easier to clean a hashmap than a database.
Both H2 and Hash Map are gonna keep the data in memory (So from space point of view they are almost the same).
If look ups are simple like KEY VALUE then looking up in the Hash Map will be quicker.
If you have to do comparisons like KEY < 100 etc use H2.
In fact 10K user info is not that high a number.
If you don't need to save user messages - use the collections. But if the message is should be saved, be sure to use a database. Because after restart you lost all data.
The problem with using a HashMap for storing objects is that you would run into issues when your site becomes too big for one server and would need to be clustered in order to scale with demand. Then you would face problems with how to synchronise the HashMap instances on different servers.
A possible alternative would be to use a key-value store like Redis as you won't need the structure of a database or even use the distributed cache abilities of something like EHCache

Android arraylist large number of string objects

I am developing a Android application for a website. It has large number of users around 100000. I have to fetch these users to an Arraylist for a custom user search. Is that a good practice to store this much amount of data in an Arraylist (particularly in Android). If not I am planning to use a Sqlite database any suggestions?
You do not want to use a list of any type.
Databases are optimized to store and search through large amounts of data, if you store these usernames in an ArrayList, you would be responsible for ensuring that you efficiently search.
This seems like a poor idea in the first place. Why would you want to have a local copy of all 1lakh+ usernames? This is terrible waste of bandwidth! It would be better if the application could query the database for the usernames it is searching for. You could then store the results only on the client.
ex: SELECT * FROM `user` WHERE `name` LIKE "david"
Store only the results from the query. Not every user.
Make Data Classes and make it Serializable and use file storage.. Because if your using DataBase getting and putting data is a different task... storing file Object is better for data handling..
It seems that it is not a good idea to store the content in an ArrayList. Depending upon the data or your application, you may get a 'OutOfMemory' error. Try persisting the information to a SQLite database or file.
On the other hand, I do not find the necessity to bulk download the 1 lakh user data and store it locally for search on device. You could make your service to do the search and return only the search results. If this is possible, then storing it in ArrayList is not bad. If the size of your arraylist exceeds the amount tolerated by the DVM, you could override onLowMemory callback and empty the list contents. By this way you could prevent your app from being killed

Architecture of Android application with simple database

I want to build a fairly simple Android application.
The basic data model of my app consist on several basic elements with relationships between them.
I want the data to be persistant so I'm thinking about using an sqlite DB.
I looked at the Android Developer website, and the closest thing that I could find that is relevant to my question is the "NotePad Tutorial" which make use of an sqlite DB to store notes.
I think by now I got the handle on the basics, but I still have some questions though:
In the tutorial they have only one table in the DB. If my application requires a more complicated scheme - should I still use the same method? that is - putting everything inside a subclass of SQLiteOpenHelper? Or is there a more "structured" way to go?
Should I bother creating classes for all the data elements stored in my DB? I mean this is what I learned that I should do, but in the documentation there is no hint about that.
If I Should create classes - How should I use them correctly? I mean, since the result of the query is a Cursor object, and not a collection of rows, Should/Can I parse that into objects?
Thanks!
Define all the tables in the same subclass, this makes easy to see all the table at one place and possibly write SQL for upgade etc
Yes, that would be easier for manipulation in the java side and makes code clean
Read from the cursor and initialize an arraylist of objects one by one.

Should I load mysql data into arrays or just query the database using java

I am making a java desktop application for billing customers that will be using a mysql database (so I can make a php frontend using the same database later). I was wondering if I should make a class that puts all the mysql info into arrays on startup so I can work with the arrays or if I should just query the database when I need to access data.
I was wondering what is the most efficient, fastest etc... Has anyone got an good pointers?
You should query the database when you need the data. That's what databases are for. If you bring all the data into Java arrays, then you will end up building querying methods on those arrays, or limiting yourself to simplistic ways of accessing the data.
If your data is small enough to fit easily into RAM, then MySQL will cache it all anyway, and it will go just as fast as if you had pulled it into arrays first.
Putting data into arrays might make sense if it's static - I'd call that caching.
But billing data seems more dynamic to me, depending on how you define it. In that case, I'd query the database each time.
Query as needed rather than pre-loading all the information. This will use potentially a lot less memory. Some of your data may need to be cached while working, but odds are most of it doesn't. The RDBMS is already designed and optimized to store and retrieve data as needed, so it is best allowed to do its job.

Categories

Resources