I am reading this blog post about making animations with Gnuplot and Cairo -terminal which algo's plan is simply
to save png-images to working directory, and
to save latest the video to working directory.
I would like to have something more such that the user can also browse the images real time when the images are being converted:
Data-parallelism model - data structure regularly arranged in an array
to give the user some list in some interface which the user can browse by arrow buttons
in this interface, new images are being added to the end of the list
the user can also remove bad images from the stream in real time
which may work well in Data parallelism model of Parallel programming i.e. a data set regularly structured in an array.
The operations (additions, deletions) can operate on this data, but independently on distinct processes.
Let's assume that there is no need for efficient searches for simplicity in Version 1.
However, if you come with a model which can do that also, I am happy to consider it - let's call it Version 2.
I think a list is not a good data structure here because of the wanted opportunity for deletions and continuous easy addition to the end of the data structure.
The data structure stack is not going to work either because of deletions.
I think some sort of tree data structure can work because of rather cheap deletions and cheap search there.
However, a simple array in the Data-parallelism model can be sufficient.
Languages
I think Java is a good option here because of parallelism.
However, any language and pseudocode are good too.
Frontend
I have an intuition that requirements for such a system in the frontend should be qT as a terminal emulator.
What is a better data structure for cheap deletions and continuous additions to the end?
Java LinkedList seems to be the thing you could use for version 1. you can use its single param add() to append to the list in constant time. if by "real-time" you mean when the image is in user's display and thus pointed to somehow, can delete them in constant time as well.
optimum use of memory and no re-instantiation as you'd have with an Arraylist.
any doubly linked list implemented on objects (as opposed to an array) would do.
your second version isn't clear enough.
Related
I am currently developing my own app (just another remake of the famous "Game of Life") and I want to add a "revert" button. My game basically consists of a two dimensional array: Cell[][]...
So: My idea was to create an ArrayList which that array is being added to every update... (With a limit of 50 entries)
But then I thought, that that would be a lot of Objects in that list... So:
Would it be more performant to have an ArrayList in each cell in the 2-dimensional array, containing the history of itself, or to have a huge ArrayList containing entire game states as a history?
(I don't think you need any of my code to answer this question, but I will post it if you do)
Instead of storing cell state, why not keep the events that transform the cell state? An event would simply encapsulate the previous state info and the next state, or maybe even a coding enum that can be executed in reverse. This would already significantly reduce the data you need to store.
Also, it makes a difference if you're working natively, or on a garbage-collecting VM platform such as Java or C#. The former tends to be faster and is more forgiving in terms of large addressing storage, while the latter can start trashing wildly if you store lots of objects.
But the real proof of the pudding is in the eating / tinstatfc: there is no such thing as the fastest code (c) Michael Abrash -> i.e. benchmark your code and find out! (and come tell us)
In my opinion, an ArrayList per cell would work fine; however, if you only want to store 50 states, you'll have to create a custom ADT or utilize Guava's Collections API.
I recommend using a Stack rather than an ArrayList, as reverting every cell would be a simple operation. In Java, the correct ADT to use is an ArrayDeque, but you'll have to handle capacity yourself.
Background: I have done a bit of looking into Caching in Spring and it seems like a great way to save time for common read operations. My code currently has a loop over a large number of items, where I am performing logic to see if certain other objects are connected in a way through common items. A way to think about this is similar to a shopping website's related items showing up when you view a certain item. The values I use to determine this are complex, but that is the basic idea.
On loading the item page there is a very long load time trying to compute and figure out which other items are related in some way as to display links to them. Instead of computing this list every time an item page loads, I have started "caching" items with a list of their recommended items. Many things in the system can trigger a need to recalculate these relations: adding/removing properties to items, adding/removing items, etc.
Problem: My "cache" is simply a singleton object containing a Map for items and their related objects. The process of iterating through every item in the system when any change to the cache is needed is very time consuming and process intensive. Java Caches don't seem to be the right answer due to constant changes to items. Is there any other design patterns that I am overlooking for this design? Caches seem to be close, but I am not sure if this problem fits into the mold of caching, due to it being a little more complex then a bulk amount of reads to a single item.
Are caches the way to go with this? If caching isn't the right solution, what is?
It seems that caches are not a solution for your problem, but they might help you in reaching a solution.
For example instead of caching the created items another approach is to cache information that rarely changes but is crucial to create the lists.
Spring function based caching (ie #Cachable) might come in handy, either for caching or invalidation.
The next level is to examine different types of caches (ie. redis) and what they offer in terms of algorithms, sorting and Pub/Sub.
I have finished my studying of a basic trading system structure and going to build one now. I am going to put a table of historical data (having columns of day high, day low...etcs of days of many years) into an array or other kind of data structure, then use the date to do analysis, and some analysis would need to put the result into another array (or another data structure).
So basically it would be few "tables" with each having around 6 columns and around 2000-5000 rows. I would do calculations within these tables and then store the result to another similar size table.
Array is good enough? Or I should choose another data structure like linked list?
It really depends on your requirements.
If you are just doing simple analysis on relatively small amounts of data (sounds like this is the case?) then there's no need to be too fancy. Probably a simple ArrayList of rows would work well. Keep it simple.
If performance is a concern then you need to understand the usage pattern a lot more. For example, if you are doing a lot of read-only lookups then you may want to create indexes into the data (e.g. using HashMaps). But that's getting quite advanced.
So for now, I suggest sticking with an ArrayList.
i want to use a list which will store objects of some type (lets say for simplicity - books) so i can show them in a listview object.
im kinda new to this, so i ask for the help of more advanced and experienced users about the following debates -
which one to use? linkedlist is something im familiar with. however, how do i make the app maintain the list? should i save the details of each object in a XML? if i do that, isnt it just better to use Arraylist? (please exclude in your answer things related to proccessing time).
if not via xml - how do i 'store' a list for later use even when the app is shut down and later on activated?
Thanks!
ArrayLists are good to use when you want random access via an indexed lookup. They're just as well suited for iterating through as LinkedLists.
OTOH, LinkedList doesn't need to be resized, it only runs out of room when you run out of memory to hold more nodes. If you have lots of data growth, or you're doing lots of sequential add/removes, then LinkedLists will win out in performance.
Sometimes you need both random access and growth, in those cases you need to make a judgment call on which criteria you want to be more performant.
In your current use case, I'd probably choose an ArrayList, you'll likely know how big the list should be, it won't be changing in size that often, and if you want to display this thing in a GUI, you're likely to need to do indexed lookups.
As far as storing the list, XML is as good a means as any, CSV files (or plain line-delimited text files), YAML, JSON and even class serialization are some alternatives, choose what's easiest and most convenient for you.
You must storage your data into SQLite. Android provides a very easy way. Look at this tutorial: http://www.vogella.de/articles/AndroidSQLite/article.html
I would prefer ArrayList over LinkedList because it has methods to manipulate the size of the array that is used internally to store the list
If i am going to use it as a stack, queue, or double-ended queue then I would use a LinkedList
Not-so-critical data can be stored in memcache. However, how can complicated data be stored there while being updated simultaneously by different user sessions.
Say a graph, tree or linked list? It is OK to miss a node, but it is bad to loose the whole graph/tree/list if a node is evicted. An example here is sending user update notification among online-users.
App engine's appstats use a predetermined 1000 bucket which is good ( I think there is no dependency among them). But I am thinking about more complicated cases...
Any tutorial, example or theory would be considered helpful...
( Memcached tag was added, but I know it is not for app engine )
If you want to store more complex data, you need to serialize it before you place it into shared memory.
When you want to update this data, you will need to deserialize back into your complex structure, update the structure then serialize again to place the structure back into shared memory.
I am curious - why memcache? There are many other shared memory storage systems out there such as MemBase, Redis and Hazelcast, with Hazelcast adding some help to hide some of the complexity of storing some more complex structures (like lists and maps). Hazelcast also adds nice features like cluster wide locks and data listeners which can come in useful (full disclosure: I decided on Hazelcast).
Of course if you want to spend real money for licensing, you always have Terracotta which can completely abstract you from this complexity.
If you're storing a graph, and don't mind nodes going missing, just store each node under its own memcache key.
If all the data going away at once is "bad", though, you probably shouldn't be using memcache in the first place. Complete wipes are rare, but can happen.