I am developing a Android application for a website. It has large number of users around 100000. I have to fetch these users to an Arraylist for a custom user search. Is that a good practice to store this much amount of data in an Arraylist (particularly in Android). If not I am planning to use a Sqlite database any suggestions?
You do not want to use a list of any type.
Databases are optimized to store and search through large amounts of data, if you store these usernames in an ArrayList, you would be responsible for ensuring that you efficiently search.
This seems like a poor idea in the first place. Why would you want to have a local copy of all 1lakh+ usernames? This is terrible waste of bandwidth! It would be better if the application could query the database for the usernames it is searching for. You could then store the results only on the client.
ex: SELECT * FROM `user` WHERE `name` LIKE "david"
Store only the results from the query. Not every user.
Make Data Classes and make it Serializable and use file storage.. Because if your using DataBase getting and putting data is a different task... storing file Object is better for data handling..
It seems that it is not a good idea to store the content in an ArrayList. Depending upon the data or your application, you may get a 'OutOfMemory' error. Try persisting the information to a SQLite database or file.
On the other hand, I do not find the necessity to bulk download the 1 lakh user data and store it locally for search on device. You could make your service to do the search and return only the search results. If this is possible, then storing it in ArrayList is not bad. If the size of your arraylist exceeds the amount tolerated by the DVM, you could override onLowMemory callback and empty the list contents. By this way you could prevent your app from being killed
Related
I'm fairly new to Android-Development and I got a general question about How-To:
My App gets Sensor-Data from Step-Detector (Detected steps gets added up).
Now I need to store those Steps (which will be a lot of Data).
The steps should be stored like this:
If Todays
steps are stored on per Hour basis.
Else
steps are stored on per Day basis
SharedPreferences falls out of this as it only stores KeyValues.
But can SQLite handle this? Or is there any other way?
A future feature could be to sync those data with a Server.
I mean this could end up in thousands of Entries, and the app will also support other large data sets which need to get stored in similar way.
Try using Realm noSql database for it. The point is, you can save entire database on sd card as separate file for each day and process it later. It is native and work very fast with large amount of data. You can process all your readings later on - open database, transform readings (perhaps interpolate values for older to shring data in size) and then upload it to the cloud and delete database file.
But, anyways, a database is just implementation details, consider abstracting out all your operations so you can replace db later on.
As far as I know, sqLite stores all tables in a single file, so you will need column for a date and all records will be stored in single table. Realm is more flexible for this task.
SQL Lite can be used , it will be there as long as your application exist in the device, however if you want you can use Cloud Service, Azure provides simple and easy to use App Service , which have easy tables , in which you can directly call the APIs and internally it takes care of making connection and inserting the data into table.You can use Free Tier of App Service to test the concept.
I need to save permanently a big vocabulary and associate to each word some information (and use it to search words efficiently).
Is it better to store it in a DB (in a simply table and let the DBMS make the work of structuring data based on the key) or is it better to create a
trie data structure and then serialize it to a file and deserialize once the program is started, or maybe instead of serialization use a XML file?
Edit: the vocabulary would be in the order of 5 thousend to 10 thousend words in size, and for each word the metadata are structured in array of 10 Integer. The access to the word is very frequent (this is why I thought to trie data structure that have a search time ~O(1) instead of DB that use B-tree or something like that where the search is ~O(logn)).
p.s. using java.
Thanks!
using DB is better.
many companies are merged to DB, like the erp divalto was using serializations and now merged to DB to get performance
you have many choices between DBMS, if you want to see all data in one file the simple way is to use SQLITE. his advantage it not need any server DBMS running.
Let's say that I have a table with columns TABLE_ID, CUSTOMER_ID, ACCOUNT_NUMBER, PURCHASE_DATE, PRODUCT_CATEGORY, PRODUCT_PRICE.
This table contains all purchases made in some store.
Please don't concentrate on changing the database model (there are obvious improvement possibilities) because this is a made-up example and I can't change the actual database model, which is far from perfect.
The only thing I can change is the code which uses the already existing database model.
Now, I don't want to access the database all the time, so I have to store the data into cache and then read it from there. The problem is, my program has to support all sorts of things:
What is the total value of purchases made by customer X on date Y?
What is the total value of purchases made for products from category X?
Give me a list of total amounts spent grouped by customer_id.
etc.
I have to be able to preserve this hierarchy in my cache.
One possible solution is to have a map inside a map inside a map... etc.
However, that gets messy very quickly, because I need an extra nesting level for every attribute in the table.
Is there a smarter way to do this?
Have you already established that you need a cache? Are you sure the performance of your application requires it? The database itself can optimize queries, have things in memory, etc.
If you're sure you need a cache, you also need to think about cache invalidation: is the data changing from beneath your feet, i.e. is another process changing the data in the database, or is the database data immutable, or is your application the only process modifying your data.
What do you want your cache to do? Just keep track of queries and results that have been requested so the second time a query is run, you can return the result from the cache? Or do you want to aggressively pre calculate some aggregates? Can the cache data fit into your app memory or do you want to use ReferenceMaps for example that shrink when memory gets tight?
For your actual question, why do you need maps inside maps? You probably should design something that's closer to your business model, and store objects that represent the data in a meaningful way. You could have each query (PurchasesByCustomer, PurchasesByCategory) represented as an object and store them in different maps so you get some type safety. Similarly don't use maps for the result but the actual objects you want.
Sorry, your question is quite vague, but hopefully I've given you some food for thoughts.
I am planning on creating an android application sometime in the future in which I'll want it to display a lot of constant data on the screen.
I'm not sure the best way to do this but I see two options:
Storing the data within the code itself such as creating a constants class.
Using an embedded database to hold the data.
I'm guessing option #2 is the best way? But it just seems weird using a database if I'm not going to be doing any updating to the database, I would only be selecting.
The total amount of data that I need the application to display is maybe about 400 lines consisting of a string and two integers...
Is there a different way people use for such a situation that I don't know about?
But it just seems weird using a database if I'm not going to be doing
any updating to the database
I am totally disagree with you. Database is not only for updating. It can be used as a better storage and definitely a best way for searching. So as you want to preserve the data then it is definitely wise to use database.
But if you want to handle data which will not persists , i,e you will use different datas for different run then you can use temporary class or other data structure to store data.
Finally, If you are planning to have portability then File storage is an easier solution.
SO you can see, that it totally depends on what you want.
I am making a java desktop application for billing customers that will be using a mysql database (so I can make a php frontend using the same database later). I was wondering if I should make a class that puts all the mysql info into arrays on startup so I can work with the arrays or if I should just query the database when I need to access data.
I was wondering what is the most efficient, fastest etc... Has anyone got an good pointers?
You should query the database when you need the data. That's what databases are for. If you bring all the data into Java arrays, then you will end up building querying methods on those arrays, or limiting yourself to simplistic ways of accessing the data.
If your data is small enough to fit easily into RAM, then MySQL will cache it all anyway, and it will go just as fast as if you had pulled it into arrays first.
Putting data into arrays might make sense if it's static - I'd call that caching.
But billing data seems more dynamic to me, depending on how you define it. In that case, I'd query the database each time.
Query as needed rather than pre-loading all the information. This will use potentially a lot less memory. Some of your data may need to be cached while working, but odds are most of it doesn't. The RDBMS is already designed and optimized to store and retrieve data as needed, so it is best allowed to do its job.