I have a table that displays the output from a database, this can often be up to or over 10,000 rows. I'm looking to implement a 'quick find' function that will allow the user to easily scroll to a row with the value for which they search.
The idea is for this to be very similar to Chrome or Firefox's quick find. So:
User hits 'CTRL+F'
A box pops up:
The user can then quickly skim through the table pushing / or /\ to each matching value
My question is, does such a table already exist? I have looked at NatTable and XViewer but they both seem like rather large additions for a relatively minor change.
Another library I've looked at that is close to fitting the billl is Lucene TableSearcher unfortunately this is designed for filtering rather than highlighting. However it may be an avenue worth investigating.
It's not quite the same, but JFace does have support for filtering views. You type in a text box and view contents shrink to what matches the filter. It's basically another way to present search/find functionality to the user.
You can see this in action in Eclipse Preferences (the filter text field about the navigation tree in the dialog). See org.eclipse.ui.dialogs.FilteredTree class. I don't believe there is a FilteredTable or equivalent, but underlying facilities are pretty generic, so it shouldn't take much work to put one together. Take a look at FilteredTree source and go from there.
The solution I chose wasn't as simple as I'd hoped but has a satisfactory performance.
The tables involved are virtual tables that can be very large. So on top of this upon loading the dataset we start a thread to build an index in the form of a list. Because these tables can be very large (but aren't often) we set a 50mb limit to this list and stop indexing it when we hit it. For most cases this is perfectly acceptable but for the larger tables it means that whilst it may be slower, it is considerably more stable than other options.
The 'search' is a simple SWT textbox with ModifyListener that iterates over the list, performing a 'contains' on each entry. Upon a match the row and column are calculated from the index number (0=0,0, 1=0,1 ..).
It would have been nice to have a simple 'drop in' solution for this but all of the tables on offer had too many features.
Use FilteredItemsSelectionDialog dialog box
Related
Background: I have done a bit of looking into Caching in Spring and it seems like a great way to save time for common read operations. My code currently has a loop over a large number of items, where I am performing logic to see if certain other objects are connected in a way through common items. A way to think about this is similar to a shopping website's related items showing up when you view a certain item. The values I use to determine this are complex, but that is the basic idea.
On loading the item page there is a very long load time trying to compute and figure out which other items are related in some way as to display links to them. Instead of computing this list every time an item page loads, I have started "caching" items with a list of their recommended items. Many things in the system can trigger a need to recalculate these relations: adding/removing properties to items, adding/removing items, etc.
Problem: My "cache" is simply a singleton object containing a Map for items and their related objects. The process of iterating through every item in the system when any change to the cache is needed is very time consuming and process intensive. Java Caches don't seem to be the right answer due to constant changes to items. Is there any other design patterns that I am overlooking for this design? Caches seem to be close, but I am not sure if this problem fits into the mold of caching, due to it being a little more complex then a bulk amount of reads to a single item.
Are caches the way to go with this? If caching isn't the right solution, what is?
It seems that caches are not a solution for your problem, but they might help you in reaching a solution.
For example instead of caching the created items another approach is to cache information that rarely changes but is crucial to create the lists.
Spring function based caching (ie #Cachable) might come in handy, either for caching or invalidation.
The next level is to examine different types of caches (ie. redis) and what they offer in terms of algorithms, sorting and Pub/Sub.
Didn't know quite how to title this question, so suggested edits are welcome.
Right now, I have a fairly simple pair of JComboBoxes for searching row & column headers in a heatmap display. It's been augmented with the ability to allow wildcards. Here's what it looks like:
It works well enough, but a new requirement for the next version is to consolidate the pair of search boxes into 1 search box plus a few other enhancements. Basically, what we'd like to do is enter a search term and have the resulting dropdown have 2 sections: 1 for options and another for the matching results. The desired options would be things like this:
Search rows/columns checkboxes
song/artist/genre checkboxes (to use the iTunes analogy) or google/bing (to use the browser analogy)
Search within previous search results: like have a previous search term with an 'x' next to it to clear it. It would act like a filter.
The thing about this is, the desire to consolidate 2 search boxes to 1 is to simplify the interface, but at the same time, we want this additional functionality.
A JComboBox doesn't seem to be able to handle something like this, and my search for alternatives isn't yielding anything. I know this question is somewhat of a broad design question, perhaps not appropriate for stack exchange. I'm still fairly new to Java, so any suggestions are appreciated.
UPDATE: One of the commenters asked for something more specific and pointed out the existence of different types of iTunes searches based on context, which I'd like to disambiguate. Taking a look at how iTunes does it, I decided to try to mock up a graphic of what I was thinking I could do:
So given this specific example, I envision separating the combo box and the search options, perhaps with a JMenu revealed using a button made to look like iTunes' search options feature.
So my evolved questions are (based on the above mock-up):
Can JMenu do this or is there another type of button-initiated menu I should use?
Can menu selections trigger a re-population of the list the JComboBox provides or would that have bad side-effects?
How can I alleviate the confusion of having a drop-down on either side of the search field?
I am working on a small, toy application to expand my knowledge about Java JavaFx and Sql. I have a MySQL server in my local network, which I am able to communicate with and a simple Tableview can be populated, sorted ... etc. The data has only to be shown to the user, no editing. Everything nice and clean.
The Problems:
There are around 170 000 rows with 10 col., all chars, to display, which seems to be rather hard to do in reasonable time. The query is done during startup and it take around 1 1/2 min before I can see the table.
Also the memory footprint is enormous, the application without populated Tableview around 70 mb, with all the data it has 600-700 mb (the xml file which is used to populate the mysql is 70 mb in size ... ) !
Sorting is REALLY slow, I am using Stringproperty which should give a boost according to: JavaFx tableview sort is really slow how to improve sort speed as in java swing (If I understood that correctly) However the custom sort, I did not try so far.
My thoughts:
Similar to the application design for mobile, I think an adapter-pattern can fix these problems. Hence, I create an OberservableList with the correct size of elements, but only populate a limit of rows in the beginning. When I am scrolling done (scroll wheel) the List has to be updated with new elements in advance via sql-queries. This should give me a performance boost for the first Problem. Nice idea but what am I going to do if the user is going to scroll done via the scrollbar(click and drag down), then I would skip certain entries, but I need the information to give the user the feedback where to scroll to.
How could I fix this ?
For the sorting, I would use the sql sorting methods, so each sort will be performed on the sql server and a new OberservableList will be created. As before, only a certain amount of data would be loaded in the first query.
If this approach would also effect the memory footprint, I am not sure.
Your opinion:
Are my ideas reasonable and do-able in Java, JavaFx ?
I would love to hear your ideas about these problems.
Thank you.
I found out that JVx is capable of providing the lazy-loading function. This should do the trick
The problem is how to store (and search) a set of items a user likes and dislikes. Although each user may have 2-100 items in their set, the possible values for the items numbers in the tens of thousands (and is expanding).
Associated with each item is a value say from 10 (like) to 0 (neutral) to -10 (dislike).
So given a user with a particular set, how to find users with similar sets (say a percentage overlap on the intersection)? Ideally the set of matches could be reduced via a filter that includes only items with like/dislike values within a certain percentage.
I don't see how to use key/value or column-store for this, and walking relational table of items for each user would seem to consume too many resources. Making the sets into documents would seem to lose clarity.
The web app is in Java. I've searched ORMS, NoSQL, ElasticSearch and related tools and databases. Any suggestions?
Ok this seems like the actual storage isn’t the problem, but you want to make a suggestion system based on the likes/dislikes.
The point is that you can store things however you want, even in SQL, most SQL RDBMS will be good enough for your data store, but you can of course also use anything else you want. The point, is that no SQL solution (which I know of) will give you good results with this. The thing you are looking for is a suggestion system based on artificial intelligence, and the best one for distributed systems, where they have many libraries implemented, is Apache Mahout.
According to what I’ve learned about it so far, it can do what you need basically out of the box. I know that it’s based on Hadoop and Yarn but I’m not sure if you can import data from anywhere you want, or need to have it in HDFS.
Other option would be to implement a machine learning algorithm on your own, which would run only on one machine, but you just won’t get the results you want with a simple query in any sql system.
The reason you need machine learning algorithms and a query with some numbers won’t be enough in most of the cases, is the diversity of users you are facing… What if you have a user B which liked / disliked everything he has in common with user A the same way - but the coverage is only 15%. On the other hand you have user C which is pretty similar to A (while not at 100%, the directions are pretty much the same) and C has marked over 90% of the things, which A also marked. In this scenario C is much closer to A than B would be, but B has 100% coverage. There are many other scenarios where most simple percentages won’t be enough, and that’s why many companies which have suggestion systems (Amazon, Netflix, Spotify, …) use Apache Mahout and similar systems to get those done.
I have to generate results of searching i Vaadin.
Probably the fastest way to do it is to put results into table, but I need some custom layout and the FormLayout for every row would be the best.
Unfortunately if I put rows formatted by FormLayout into table - the generation of the results takes about 15-20 seconds.
If i put just some labels/buttons without putting it in a layout the generation takes about a second - and that's fine.
I've read the vaadin manual about optimising UI and I still don't know what to do to shorten the time needed to generate the results.
How do you generate results in Vaadin if it can't be presented as a table?
Ok, the solution is... use CssLayout inside a table. Although CssLayout still consists of 3 div's it's much faster than FormLayout.
I think that using a FlowPanel from GWT would be even better but I didn't have enough time to check it.
And don't forget about setPageLength method of a Table - the lazy loading is the main reason to use a Table :).
Offering a few more layout options:
WeeLayout add-on: fast layout, not many expensive layout calculations in JavaScript, and a lighter DOM structure than in CssLayout
DashLayout add-on: even lighter DOM structure (but more JS calculations), and offers more flexible styling options for CSS (arbitrary margin/border/padding values).