I want to display a list of all the users in my site but I only want to display 10 people per age. I don't know how exactly to do this. I know how to do it by just displaying all the users in one page but that's not very good is it?
If I do it with the code I have now, it will only get the first ten users over and over again.
I want to be able to get all the users for a one time query, store it globally and then move through the list retrieving the next 10 and so on for display.
I am developing on appengine using Java and the Spring Framework some of the solutions I have been thinking about,
Store in the session and go through the list (very bad I guess)
hand it to the JSP, specifically to one of the scopes, page, request etc. But I think request will not work.
Look for a Spring controller that can handle this.
Generally speaking, you would use a form variable on your page (via GET or POST) called 'page', which would be a number. When you receive that in the servlet you would calculate a range based on the page number and configured rows per page.
Take a look at Paging through large datasets (yes it's Python but the same principles apply) and Queries and Indexes from the Google App Engine documentation.
Take a look at http://valuelist.sourceforge.net/
If you keep page size at 10 then you can retrieve your 10 users per age group for each page based on page number:
SELECT TOP 10 users FROM myusers
WHERE AGE = function(page_number)
ORDER BY some_ordering
I hope that JPA + appengine support such type of query.
Some database engines provide handy extensions to SQL for just this purpose. Like, in MySQL you can say something like "select ... whatever ... limit 50,10", where "50" is the row to start with and 10 is the number of rows to retrieve. Then on your display page you simply put next and previous buttons that pass the appropriate starting row number back to the server for the next run at the query.
If the SQL engine you're using has no such handy function, then you have to build an query-specific "where" clause based on the sort order.
To take a simple case, suppose in your example you are displaying the records in order by "user_name". You can use Statement.setMaxRows(10) to limit any queries to 10 rows. Then on your first call you execute, say, "select ... whatever ... from user order by user_name". Save the last user_name found. In your next button, you pass this user_name back to the server, and the query for the next call is "select ... whatever ... from user where user_name>'xxx' order by user_name", where 'xxx' is the last user_name from the previous call. Do the setMaxRows again so you are again limited to 10 rows of output. You can then let the user step through the entire output this way.
Letting the user go backwards is a bit of a pain. I've done it by keeping a table in a session variable with the starting key value for each page.
Related
Application i worked on has a user search API which will do a user search based on Name,Address,DateOfBirth,and Pincode. Of these only pincode is the mandatory field. And any 2 of other data (Date of birth, address, name) can be send. Also in one request clients can search for multiple users. So against each search key we have to give response like either found USER object or NOT FOUND response.
This is how our algorithm works for each search keys .
Fetch all the user profile from table with the given pincode.
Iterate users from step 1 and check if any two of date of birth,
name, address matches then that is a valid candidate. We are doing
this using predicate as shown below
stream(profiles.spliterator(), false).filter(new
UserSearchPredicate(key)).
This predicate will check if any 2 of the
values matched and if yes return true.
Check if only one user we got from step 2 then DONE. Otherwise Exception
Return user.
The above algorithm has a problem , it is searching the users table by pincode.
Total 20 million records are there in the table which makes the query slow. AGainst one pincode we may get 16000 or more records , iterating it is another bottle neck area. ALso this has to be done for all search keys. There can be 50 or more search keys in one request, which took almost a minute to respond.
What we have done:
Pincode column indexing is properly DONE in database. We are using oracle DB. Also tried parallel processing as well. Parallel processing has a problem with DB connections in connection pool which is not a solution we concluded.
Adding one more column to the query is not logically possible because if i add name , then where pincode='' and name='' will be the query. But the matching record can be pincode and date of birth . Apart from pincode nothing is mandatory.
Question: Is there a better way to handle this problem.Am particularly looking for a better algorithm. please help.
As the title states, I want to only retrieve a maximum of for example 1000 rows, but if the queries result would be 1001, i would like to know in some way. I have seen examples which would check the amount of rows in result with a a second query, but i would like to have it in the query i use to get the 1000 rows. I am using hibernate and criteria to receive my results from my database. Database is MS SQL
What you want is not posssible in a generic way.
The 2 usual patterns for pagination are :
use 2 queries : a first one that count, the next one that get a page of result
use only one query, where you fetch one result more than what you show on the page
With the first pattern, your pagination have more functionalities because you can display the total number of pages, and allow the user to jump to the page he wants directly, but you get this possibility at the cost of an additional sql query.
With the second pattern you can just say to the user if there is one more page of data or not. The user can then just jump to the next page, (or any previous page he already saw).
You want to have two information that results from two distinct queries :
select (count) from...
select col1, col2, from...
You cannot do it in a single executed Criteria or JPQL query.
But you can do it with a native SQL query (by using a subquery by the way) with a different way according to the DBMS used.
By making it, you would make more complex your code, make it more dependent to a specific DBMS and you would probably not gained really something in terms of performance.
I think that you should use rather a count and a second query to get the rows.
And if later you want to exploit the result of the count to fetch next results, you should favor the use of the pagination mechanisms provided by Hibernate rather doing it in a custom way.
I'm going to build an app. Until now everything runs very well. Now I have a problem. The app gets its content from a mysql database.A column is called item.I have a ratingbar. The user can rate the item there.Every time the user evaluates an item the value is stored on the database in the respective item line.The values are then added. In other words, when a user evaluates 20 times with 5 stars, the value adds up to 100 and so on.
I want to limit this. I will that the user can evaluate each day only once an item. I will it without a registration mask for the user. How can I solve this problem?
I know that i can identifier the WIFI MAC Adreess and other Unique Identifiers, but how can i solve this with them?
I can not use sqlite database, because the items should update with the time from the mysql database.
A registration mask should not be excluded. If this process is quite possible with them, then I supplement it with it.
I am looking forward to every comment
every computer has a machine ID, you will hash that and encrypt that to use as your identifier..most telecomms do not like using MAc addresses as IDs
One option would be to create UUID during every installation and sending this UUID to your server along with every request. In server, you can control very well if user can provide feedback once a day or others based on your requirement. Pls refer this link on how to create UUID.
https://developer.android.com/reference/java/util/UUID.html
I have a database with 20,000 records. Each record has a name. When a user wants to view a record, he can visit a webapp and type the name of the record in an inputfield. While typing, results from the database would be shown/filtered matchin what the user typed. I would like to know the basic architecture/concepts on how to program this
I'm using the following language stack:
frontend: html5/javascript (+ajax to make instant calls while user is typing)
backend: java + jdbc to connect to simple sql database
My initial idea is:
A user types text
Whenever a character is entered or removed in the inputfield, make an ajax request to the backend
The backend does a LIKE %input% query on the name field in the database
All data found by the query is send as a json string to the frontend
The frontend processes the json string and displays whatever results it finds
My two concerns are: the high amount of ajax requests to process, in conjunction with the possibly very heavy LIKE queries. What are ways to optimize this? Only search for every two characters they type/remove? Only query for the first ten results?
Do you know of websites that utilise these optimizations?
NOTE: assume the records are persons and names are like real people names, so some names are more common than others.
You can choose SPA approach - load all 20 000 names/ids to client side and then filter it in memory - it's supposed to be the fastest way with minimal load to the database and back-end
Here are possible solutions:
Restirct search to prefix search - LIKE 'prefix%' can be executed efficiently using BTREE-type index.
Measure performance of naive LIKE '%str%' solution - it you are working on B2B application, database will likely load that table in memory and do queries fast enough.
Look at documentation for your database - there could be special features for that like inverted index
as #Stepan Novikov suggested, load your data in memory and search manually
Use specialized search indexers like SOLR or ElasticSearch (likely overkill for only 20k records)
If you are feeling ninja, implement your own N-gram index.
I use GWT for UI and Hibernate/Spring for buisness-layer.Following GWT widget is used to display the records.(http://collectionofdemos.appspot.com/demo/com.google.gwt.gen2.demo.scrolltable.PagingScrollTableDemo/PagingScrollTableDemo.html).I assume the sorting is done in client side.
I do not retrieve the entire result set since its huge.
I use
principals = getHibernateTemplate().findByCriteria(criteria,
fromIndex, numOfRecords);
to retrive data.Theres no criteria for sorting in Hibernate layer.
This approach does not give the correct behaviour since it only Sorts the current dataset in the client.
What is the best solution for this problem?
NOTE : I can get the primary-Sort-column and other sort Columns using the UI framework.
May be I can sort the result using primary-sort-column in the hibernate layer?
You need to sort on the server.
Then you can either:
send the complete resultset to the client and handle pagination on the client side. The problem is that the resultset may be big to retrieve from db and sent to the client.
handle the pagination on the server side. The client and the server request only one page at a time from the db. The problem then is that you will order the same data again and again to extract page 1, page 2, etc. each time you ask the db for a specific page. This can be a problem with large database.
have a trade-off between both (for large database):
Set a limit, say 300 items
The server asks the db for the first 301 items according to the order by
The server keept the resultset (up to 301 items) in a cache
The client request the server page by page
The server handles the pagination using the cache
If there are 301 items, the client displays "The hit list contains more than 300 items. It has been truncated".
Note 1: Usually, the client doesn't care if he can't go to the last page. You can improve the solution to count for the total number of rows first (no need of order by then) so that you can display message that is better to the user, e.g. "Result contained 2023 elements, only first 300 can be viewed".
Note 2: if you request the data page by page in the database without using any order criterion, most db (at least Oracle) don't guarantee any ordering. So you may have the same item in page 1 and 2 if you make two requests to the database. The same problem happens if multiple items have the same value that is use to order by (e.g. same date). The db doesn't guarantee any ordering between element with the same value. If this is the case, I would then suggest to use the PK as the last order criterion to order by (e.g. ORDER BY date, PK) so that the paging is done in a consistent way.
Note 3: I speak about client and server, but you can adapt the idea to your particular situation.
Always have a sort column. By default it could by "name" or "id"
Use server side paging. I.e. pass the current page index and fetch the appropriate data subset.
In the fetch criteria / query use the sort column. If none is selected by the client, use the default.
Thus you will have your desired behaviour without trade-offs.
It will be confusing to the user if you sort on a partial result in the GUI, and page on the server.
Since the data set is huge, sending the entire data set to the user and do both paging and sorting there is a no-go.
That only leaves both sorting and paging on the server. You can use Criteria.addOrder() to do sorting in hibernate. See this tutorial.