I am developing a java application which will be graphically depicting and organizing hundreds of objects. Each of these objects is loaded in from a SQL database, for use in this program.
I plan to keep a local copy of all the data, retrived through a JDBC connection, and then sort it locally (Likely with Merge Sort as that is my favorite)
How would you recommend I store the data locally? Ideally, it would be very easy to traverse, sort and compare to the source Database.
Currently, I have objects called Ideas which are populated from the database. Then, to display them, I create the same number of IdeaGraphics objects and add them to the appropriate JPanels. This can be quite slow at times.
You can implement the Comparator interface and use Collections.sort(), a modified mergesort, as shown here.
Maybe you already did that but it's not so clear in your question:
You can get almost as many instances of your data class (Ideas) but it's not the same at all with graphic classes. Limit the number of widgets and graphics to 1.
Build your own custom JPanel class, give it a list of Ideas and override the paintComponent method to draw all items in the list.
If your visible area is large, double buffer it.
I disagree with that and not good idea to holding lots of data in local Memory, with idea to simulate, excelent optimalized preprocesor from Sql Engine (most of todays engines), then basically faster as best EndUser PC
and if you have some WHATEVER permisions to the Database, then there are still exist option(s) to loading data from DbEngine to the Embedded database (on Memory), maybe to H2 or maybe JavaDB
Related
So I have a homework assignment to build a console application in Java while utilizing mySQL databases.
Our teacher wants us to use objects and ArrayLists, as Java is an "object oriented language".
I am not sure why I must parse the mySQL ResultSets into objects, and not print them directly in my methods in a nice printf format.
Similarly, why should I make class objects when inserting new data and not altering the database directly through my connection.
Can someone please explain and point me in the right direction.
I don't need any code. Just trying to understand.
One printf method for a ResultSet is not the correct way to implement! A resultset can have a different amount of different types of data.
The correct way would be to parse them to an object. For each table you might have an own class with constructor. For example:
You have a table 'Person' with attributes Name, age, address. Then you create a constructor
public Person(String name, int age, String address){}
or you could create a static method to parse like this:
public static Person parseFromResultSet(ResultSet r) throws ParseException {}
and then you can even write your own 'PersonParseExcpetion'.
"I am not sure why I must parse the mySQL ResultSets into objects, and
not print them directly in a nice printf format. Similarly, why should I make class objects when inserting new data and not altering the database directly through my connection."
...mostly because in a real-world application you're likely to do a lot more with the records than just print them directly to the console, and at that point it's useful to put them into a solid data structure you can move around and manipulate easily.
It can also make the code easier to maintain - real-world apps get updated and altered lots of times usually, and may contain a lot of different data types and structures, some of which may be complex representations consisting of several other structures put together - and it's easier to build a complex object if you already have a lot of smaller objects to make it from (imagine building furniture from ready-made sections rather than starting by sawing all the planks and making all the screws yourself).
You can think of it as an intermediate layer between what's on screen and what's in storage - which, in more complicated applications, will not always be structured or displayed in the exact same way as each other, hence the need for an intermediary. Sometimes the display and the storage will not even be on the same computer (like in websites such as this one).
So I imagine the purpose of this assignment might be to get you used to using those structures, but without making the overall goal too complicated. The true purpose of the exercise is not always the obvious one.
I want to preserve data during service restart, which uses a arraylist of {arraylist of integers} and some other variables.
Since it is about 40-60 MB, I don't want it be generated each time the service restarts(it takes a lot of time); I want to generate data once, and maybe copy it for next service restart.
How can it be done?
Please consider how will I go about putting a data structure similar to multidimensional array(3d or above) into file, before suggesting writing the data in a file; which when done, will likely take significant time to read too.
You can try writing your data after generation to a file. Then on next service restart, you can simply read that from the file.
If you need persistent data, then put it into database
https://developer.android.com/guide/topics/data/data-storage
or try some object database like http://objectbox.io/
So you're afraid reading from the file would take along time due to its size, the number and size of the rows (the inner arrays).
I think it might be worthy to stop for a minute and ask yourself whether you need all this data at once. Maybe you only need a portion of it at any given time and there are scenarios in which you don't use some (or maybe most) of the data? If this is likely, I would suggest that you'll compute the data on demand, when required, and only keep a memory based cache for future demand in the current session.
Otherwise, if you do need all the data at a given time, you have a trade-off here. Trade-off between size on disk and processing time. You can shrink the data using some algorithm, but it would be at the expense of the processing time. On the hand, you can just serialize your object of data and save it to disk as is. Less time, more disk space.
Another solution for your scenario, could be, to just use a DB and a cursor (room on top sqlite). I don't exactly know what it is that you're trying to do, but your arrays can easily be modeled into a DB. Model a single row as you'd like and add to that model the outer index of the array. Then save the models into the DB, potentially making the outer index field the primary key if the DB.
Regardless of the things I wrote, try to think if you really need this data persistent on your client, maybe you can store it at the server side? If so, there are other storage and access solutions which are not included at the Android client side.
Thank you all for answering this question.
This is what I have finally settled for:
Instead of using the structure as part of the app, I made this into a
tool, which will prepare data to be used with the main app. In doing
so, it also stopped the concern regarding service restart.
This tool will first read all the strings from input file(s).
Then put all of them into the structure one at a time.(This will be
the part which I was having doubts, and asked the question about.
Since all the data is into the structure here, as soon as program
terminates, this structured data is unusable.)
Now, I prepared another structure for putting this data into file,
and put all this data into file so that I do not need to read to all
input file again and again, but only few lines.
Then I thought, why spend time "read"ing files while I can hard code
it into my app. So, as final step of this preprocessing tool, I made
it into a class which has switch(input){case X: return Y}.
Now I will just have to put this class into the app I wanted to make.
I know this all sounds very abstract, even stretching the concept of abstract, if you want to know details, please let me know. I am also including link of my "tool". Please visit and let me know if there would have been some better way.
P.S. There could be errors in this tool yet, which if you find, let me know to fix them.
P.P.S.
link: Kompressor Tool
I am reading this blog post about making animations with Gnuplot and Cairo -terminal which algo's plan is simply
to save png-images to working directory, and
to save latest the video to working directory.
I would like to have something more such that the user can also browse the images real time when the images are being converted:
Data-parallelism model - data structure regularly arranged in an array
to give the user some list in some interface which the user can browse by arrow buttons
in this interface, new images are being added to the end of the list
the user can also remove bad images from the stream in real time
which may work well in Data parallelism model of Parallel programming i.e. a data set regularly structured in an array.
The operations (additions, deletions) can operate on this data, but independently on distinct processes.
Let's assume that there is no need for efficient searches for simplicity in Version 1.
However, if you come with a model which can do that also, I am happy to consider it - let's call it Version 2.
I think a list is not a good data structure here because of the wanted opportunity for deletions and continuous easy addition to the end of the data structure.
The data structure stack is not going to work either because of deletions.
I think some sort of tree data structure can work because of rather cheap deletions and cheap search there.
However, a simple array in the Data-parallelism model can be sufficient.
Languages
I think Java is a good option here because of parallelism.
However, any language and pseudocode are good too.
Frontend
I have an intuition that requirements for such a system in the frontend should be qT as a terminal emulator.
What is a better data structure for cheap deletions and continuous additions to the end?
Java LinkedList seems to be the thing you could use for version 1. you can use its single param add() to append to the list in constant time. if by "real-time" you mean when the image is in user's display and thus pointed to somehow, can delete them in constant time as well.
optimum use of memory and no re-instantiation as you'd have with an Arraylist.
any doubly linked list implemented on objects (as opposed to an array) would do.
your second version isn't clear enough.
i want to use a list which will store objects of some type (lets say for simplicity - books) so i can show them in a listview object.
im kinda new to this, so i ask for the help of more advanced and experienced users about the following debates -
which one to use? linkedlist is something im familiar with. however, how do i make the app maintain the list? should i save the details of each object in a XML? if i do that, isnt it just better to use Arraylist? (please exclude in your answer things related to proccessing time).
if not via xml - how do i 'store' a list for later use even when the app is shut down and later on activated?
Thanks!
ArrayLists are good to use when you want random access via an indexed lookup. They're just as well suited for iterating through as LinkedLists.
OTOH, LinkedList doesn't need to be resized, it only runs out of room when you run out of memory to hold more nodes. If you have lots of data growth, or you're doing lots of sequential add/removes, then LinkedLists will win out in performance.
Sometimes you need both random access and growth, in those cases you need to make a judgment call on which criteria you want to be more performant.
In your current use case, I'd probably choose an ArrayList, you'll likely know how big the list should be, it won't be changing in size that often, and if you want to display this thing in a GUI, you're likely to need to do indexed lookups.
As far as storing the list, XML is as good a means as any, CSV files (or plain line-delimited text files), YAML, JSON and even class serialization are some alternatives, choose what's easiest and most convenient for you.
You must storage your data into SQLite. Android provides a very easy way. Look at this tutorial: http://www.vogella.de/articles/AndroidSQLite/article.html
I would prefer ArrayList over LinkedList because it has methods to manipulate the size of the array that is used internally to store the list
If i am going to use it as a stack, queue, or double-ended queue then I would use a LinkedList
There must be a better way to manage string values than to have a bunch of strings in strings.xml file.
I am looking for database like solution, however I don`t want it to connect to a database on the internet. It is just that I need some advanced sorting and categorising to be done that is all.
I am not very experienced with JAVA so pardon me if I just lack the knowledge.
EDIT: It would be nice that I could synchronize both database on the internet and on user`s smartphone.
Maybe effect of synchronization can be achieved by adding additional databases and sending out already modified data.
You can use a SQLite database for your app. See http://developer.android.com/guide/topics/data/data-storage.html#db
If you don't need to save them you could just use an array of strings or an array list if you have the user entering a bunch of strings or need dynamic sizing
Well, there are simpler alternatives to SQLite - you should at least consider some kind of ORM for Android.
It'll let you to persist Java objects in a few lines of code instead of using SELECT, UPDATE, and marshalling data.
Consider db4o or something like that - it's rather large (~1M), though.
Why would you want to use SQLite for this? Remember that using a database connection for this (and only this) takes up resources that could be used elsewhere. If its strings that are only used as text (which never changes) in your program you are better of using the strings.xml. Not only is it faster, but it is also a Android Standard. Besides, if you decide one day into the future to translate your application to a different language I would guess it is much easier using the strings.xml file.