I need the implementation logic, where mine logic is failing. Let me explain the functionality.
a) There are two searches in screen , SearchA and SearchB . When i perform SearchA it gives me some result and i display on the screen.
b) When i perform the SearchB, application will search the data and display on the screen.
Expected Result :
When SearchB is performed both the search results should be show on the application.
Present Result:
When searchB is performed the search result of SearchA is disappearing and only SearchB is displaying. Visa Versa
Please dnt suggest to place the search result in session. Its a huge data (Millions of records) . So please suggest any other apt implementation.
I am assuming that both search, results in page to be reloaded and search result being request scope, the first response gets wiped off on searching the second time (SearchB)
You could paint the result portion of the page using Ajax instead of full page refresh, thereby retaining the search results.
Say you have 2 div SearchAResults and SearchBResults. SearchA will perform ajax request and populate the SearchAResults div once it gets results from server. There is no page load at this time. Similarly, SearchB will just paint the SearchBResults.
Also, if memory(and not design) is the only criteria stopping you from storing result in session, and the search results need to be reused - you could look into some external cache solutions like Memcached whereby there would be dedicated servers just to handle these humongous data.
Related
I have a case in which I shouldn't make requests to get the scroll_id - I have to manage it somehow so I can get the URL for next pages offline (I am making GET requests against a certain site that exposes their Elasticsearch instance)
So basically, I have a certain URL containing Elasticsearch query and it returns me only 20 results out of 40(20 per request is the max size). I want to get an URL for the next pages - so given I had the connection to the Internet, I would just get the scroll_id from the first request and use it to make next ones.
But I want to avoid it and see if I can have a helper class that builds scroll ids by itself.
Is it possible?
Thanks in advance.
The scroll_id ties directly to some internal state (i.e. the context of the initial query) managed by ES and which eventually times out after the a given period of time.
Once the period times out, the search context is cleared and the scroll id is not valid anymore. I'm afraid there's no way you can craft a scroll id by hand.
But if the result set contains 40 results ans you can only retrieve 20 at a time, I suggest you simple set from: 20 in your second query and you'll be fine.
I'm looking for an opportunity to split a big request like:
rest/api/2/search?jql=(project in (project1, project2, project3....project10)) AND issuetype = Bug AND (component not in (projectA, projectB) OR component = EMPTY). The result will containe > 500 Bugs -> It's very very slow. I want to get them with different requests (methode to performe the request will be annotated with #Asynchronous) but the jql needs to be the same. I don't want to search separately for project1, project2...project10. Would be nice if someone has an idea to resolve my problem.
Thank you :)
You need to calculate pagination. First get the metadata.
rest/api/2/search?jql=[complete search query]&fields=*none&maxResults=0
you should get something like this:
{"startAt":0,"maxResults":0,"total":100,"issues":[]}
so completely without fields, just pagination metadata.
Than create search URI like this.
rest/api/2/search?jql=[complete search query]&startAt=0&maxResults=10
rest/api/2/search?jql=[complete search query]&startAt=10&maxResults=10
..etc
Beware data should change so you should be prepared that you won't recieve all the data and also pagination metadata if calculation is expensive (exspecially "total") should not be presented. More Paged API
Can you not break into 2 parts? If you are displaying in a web page ( display what you can without performance hit. If its a report then get all objects gradually and show once completed.
Get the count in total for JQL & just get the minimum information needed for step 2 - assume its 900
Use the pagination feature (maxResults=100) make multiple calls.
Work on each request.
If you don't want to run the two requests at once and need paging of bugs by user request, you can:
Make a request with the 'maxResults' property set to how much you need.
On the next request set the 'maxResults' property and the 'startAt' with the same value.
If you need to fetch more data, make new request with the same 'maxResults' but update 'startAt' to be the count of bugs you fetched in the previous requests.
So far I've seen examples that use the following logic:
Create a table / grid object
Set its data source (Collection such as array list/ set)
The table shows the entries on the client side!
Problem is, we have millions of rows to display, (on a side note I tried to load the container with all the entries, it took tons of time, and the client side performance were lacking)
So that raises the question:
How do you show huge amount of data on the zk tables \ grids? Wishful
thinking points me to think that instead of an array list data source
i could set a DB connection or something instead, and that will manage
the results on demand with paging.
Any ideas?
Why loading data when you are not displaying all the rows at a time.
Retieve only those data that should be displayed and load the other data on demand not while the page is initially loading.
if you try to fetch 1 million rows and try to bind it to a control, it will hugely effect your application performance and increases the time for your page to load.
So, my adivce should be fetch only those rows that needs to be displayed. if the request comes from the user for the next set of pages then load that data and bind.
you can make ajax calls to avoid every time entire page refershing
hope this helps..
ZK give BigListbox to show huge record
I've developed a module with Spring framework and for the view i've used some Spring JSTL tags like <form:hidden>
I have a table on the jsp which i store using an Arraylist.
Now when i do some other action, i have to maintain the Table and since we are not using AJAX(Client doesnt wants it!!) , what i've done is that i've put all the list elements one by one into <form:hidden>.Now everytime i do a select for one of the elements of the list, i have to maintain the list and that is taken care off via the tag.
But when i go on selecting multiple records one by one, i've noticed( System.out.println("Request Size : " + request.getContentLength())), the size increases everytime and when it reaches 3MB, the system crashes. Is there any way i can increase the size of the POST method, in eclipse or websphere? or is there any way i can clear the request so that the size doesnt increase? please help.
instead of using form:hidden to submit all the array values, maybe you could use form:hidden to submit only the index of the elements in the array
You should maintain the state at the server side, possible in HTTPSession. Whenever, the state changes on the page and has to be committed, only the state changes should be POSTed back to the server. Sending 3 MB worth of data in request will not scale.
The scenario I have is this.
User does a search
Handler finds results, stores in session
User see results, decides to click one of them to view
After viewing, user clicks to "Back to Search"
Handler detects its a back to search, skips search and instead retrieves from session
User sees the same results as expected
At #5, if there was a new item created and fits the user's search criteria, thus it should be part of the results. But since in #5 I'm just retrieving from session it will not detect it.
My question is, should I be doing an extra step of checking? If so, how to check effectively without doing an actual retrieve (which would defeat the purpose)? Maybe do select count(*) .... and compare that with count of resultset in session?
Caching something search results in a session is something I strongly advise against. Web apps should strive to have the smallest session state possible. Putting in blanket logic to cache search results (presumably several kb at least) against user session state is really asking for memory problems down the road.
Instead, you should have a singleton search service which manages its own cache. Although this appears similar in strategy to caching inside the session, it has several advantages:
you can re-use common search results among users; depending on the types of searches this could be significant
you can manage cache size in the service layer; something like ehcache is easy to implement and gives you lots of configurability (and protection against out of memory issues)
you can manage cache validity in the service layer; i.e. if the "update item" service has had its save() method triggered, it can tell the search service to invalidate either its entire cache or just the cached results that correspond with the newly updated/created item.
The third point above addresses your main question.
It depends on your business needs. If it's imperative that the user have the latest up to date results then you'll have to repull them.
A count wouldn't be 100% because there could be corresponding deletions.
You might be able to compare timestamps or something but I suspect all the complexity involved would just introduce further issues.
Keep it simple and rerun your search.
In order to see if there are new items, you likely will have to rerun your search - even just to get a count.
You are effectively caching the search results. The normal answer is therefore either to expire the results after a set amount of time (eg. the results are only valid for 1 minute) or have a system that when the data is changed, the cache is invalidated, causing the search to have to run again.
Are there likely to be any new results by the time the user gets back there? You could just put a 'refresh' button on the search results pages to cause the search to be run again.
What kind of refresh rate are you expecting in the DB items? Would the search results change drastically even for short intervals, because I am not aware of such a scenario but you might have a different case.
Assuming that you have a scenario where your DB is populated by a separate thread or threads and you have another independent thread to search for results, keep track of the timestamp of the latest item inserted into the DB in your cache.
Now, when user wants to see search results again compare the timestamps i.e. compare your cache timestamp with that of the last item inserted into the DB. If there is no match then re-query else show from your cache.
If your scenario confirms to my assumption that the DB is not getting updated too frequently (w.r.t. to a specific search term or criteria) then this could save you from querying the DB too often.