I am using a JTable in java for listing the values from database.
I need something like, I need to list few set of values in the JTable. And when we scroll down or scroll up using scroll pane of JTable, next set of values must be loaded from database. so that instead of loading all values, i can list few values and scrolling action will retrieve next range of values.
How can I do this?
Can any one suggest me an idea for this?
You need to implement a TableModel, like subclassing DefaultTableModel. Need to track which rows you already retrieved from database and when table request the table model more rows, grab from database. This will occurs as user scrolls the table. Be careful if the total number of rows is huge, as you should discard rows retrieved and not shown currently in the table to save memory.
This question is not easy and implementations could vary. You can maintain an open connection and read from the ResultSet, you can open a new connection each time... If retrieving the rows takes too much time, the user will experience the scroll "freezes" while retrieving the new rows. Another problem is calculating the total number of rows, as table needs it for several operations (calculate the scrolling available...), implementing getRowCount() could be problematic with very large queries. You should perform a SELECT COUNT before to get the total number of rows and then perform the SELECT to start grabbing they.
Mi advice is to retrieve all rows from database and pass to the table. If the total number of rows is not too much large (<10000), JTable handles without problem such number of rows (some memory is required!) and the scroll will be perfectly smooth and only one database access is performed. If you know user already will not scroll the entire set of rows, try to adopt another technique, like limiting the total number of rows returned from database, setting a filter and getting all the rows (limited for example to 100) and show all in the table in order to avoid memory usage and database access.
In the days of Java 1.4.1 and Pentium IV (2004), we use to populate the JTable with all rows returned from database, limiting they to 10000 (WHERE ROWNUM < 10000 in Oracle) and the application needs more memory but works fine and smooth. The time required to retrieve all the rows was larger, but use a waiting dialog and let the user wait for the data (or he/she can filter the data to retrieve fewer rows).
I am not sure how to do it with the scroll action. You could have next and back buttons instead. You could initially show the first set of values. Clicking on next or back will cause the next set to be retrieved. You might need some reference to the first and last value in the displayed list.
Related
I am having 50000 entries in MySQL DB, which need to be fetches and iterated in java. Is it required to do pagination ?
This table could contain 20 columns which are varchar(100).
If its not possible, Is pagination required if I am fetching only 1 column from each row for these 50000 entries ?
This very much depends on your use case. If you want to display it to an end user you will very likely need pagination as displaying this much data in one go may make your UI sluggish (apart from not being very user-friendly). If you just need to do calculations in a background task, you can probably live without pagination.
So far I've seen examples that use the following logic:
Create a table / grid object
Set its data source (Collection such as array list/ set)
The table shows the entries on the client side!
Problem is, we have millions of rows to display, (on a side note I tried to load the container with all the entries, it took tons of time, and the client side performance were lacking)
So that raises the question:
How do you show huge amount of data on the zk tables \ grids? Wishful
thinking points me to think that instead of an array list data source
i could set a DB connection or something instead, and that will manage
the results on demand with paging.
Any ideas?
Why loading data when you are not displaying all the rows at a time.
Retieve only those data that should be displayed and load the other data on demand not while the page is initially loading.
if you try to fetch 1 million rows and try to bind it to a control, it will hugely effect your application performance and increases the time for your page to load.
So, my adivce should be fetch only those rows that needs to be displayed. if the request comes from the user for the next set of pages then load that data and bind.
you can make ajax calls to avoid every time entire page refershing
hope this helps..
ZK give BigListbox to show huge record
I have a database query returning a large number of rows. JDBC doesn't tell you how many rows come back and there are too many rows to count them before I display it in a JTable.
So the model I'm working with is an Iterable<R>, but unfortunately, TableRowModel has to know the row count up-front.
I can imagine a few strategies which might work:
Load all rows, but do it in the background and add them to the model in batches.
Somehow detect requests to render the lower rows (via the model I guess) and dynamically load more rows if this happens.
Have a button on the screen to explicitly load more rows.
Some kind of non-JTable component which supports rendering rows without knowing the row count in advance (ideal, but I can't find one.)
I am wondering if there is a "normal" way to do this, because it isn't often that I see this sort of thing in a UI. The few JDBC tools I have checked out (I figured these would be the best bet) seem to either eagerly load the results or page them (as a user, I really dislike paging in GUI apps, so I would like to avoid that.)
Operations like "Find" over the table should ideally work in an "understandable" way though... so it seems like this is not going to be too easy.
Use a SwingWorker in your implementation of AbstractTableModel. Let the worker partition the query so that a reasonable number of records are available promptly, while the rest are read in the background. Use a PropertyChangeListener to show progress and implement a cancel button.
You have 2 possibilities:
Write query, which returns the number of rows, and then implement
table model, which tries to load rows from the database, if table
want to show the rows which are not in the model. In this case you need the third query,
which can index based load the rows (these queries are possible in
MS-SQL and Oracle). For example: you loads the first 100 rows, and
the number of results. If table model is asked for rows from
position 100 to 199 you makes new query, which loads these rows
interval (if it's possible for your database) or you tries to
iterate over your result set, and load the next data batch. But if
user presses the "End" key you definitly needs the index based
loading (or you need to iterate over the complete result set).
You write model, which shows, the first batch (for example 100
rows). If user, scrolls to view the last row (model.getValueAt(row,
col) -> col == 99) you simply asks the result set for next batch and
add the new rows to the model.
First variant is better for the user (because the user can see how large is the table and can directly scroll to last row), but the second is easier to implement because it does not need index based loading of rows.
I have a wicket application with several pages, many of which use an AJAX table to display the data. But until now their columns have all been static, only the rows change. The new screen does the following:
The user enters various query params and clicks submit. One parameter is a date unit (hours, days, weeks or months) which alongside a from/to date specifies the number of columns to display.
the data provider gets hold of the params and refreshes the data to display
.. but how to update the columns in the table?
Looking at the DataGridView object in the table code, it looks as if the columns are unchangeable without a major reimplementation of the data table object. Does anyone have any examples of how to update both the rows and columns of a table when the underlying data changes? Is it even possible?
if you hold a reference to the List that you give the data table in the constructor, in theory you should be able to manipulate it to remove/add columns. however, i am not sure if all the code surrounding the data table takes that usecase into account.
the safest way to do this is to simply to replace the data table instance with a new one that has the correct list of columns.
so something like this
addOrReplace(new DataTable("table", ...))
whenever the columns need to change.
I would like to display 100000 records on browser / multiple pages with minimal impact on memory. ie Per page 100 records.
I would like to move page back and forth. My doubts are
1. Can I maintain all the record inside the memory ? Is this good Idea ?
2) Can I make database connection/query for ever page ? If so how do write a query?
Could anyone please help me..
It's generally not a good idea to maintain so much records in memory. If the application is accessed by several users at the same time, the memory impact will be huge.
I don't know what DBMS are you using, but in MySQL and several others, you can rely on the DB for pagination with a query such as:
SELECT * FROM MyTable
LIMIT 0, 100
The first number after limit is the offset (how many records it will skip) and the second is the number of records it will fetch.
Bear in mind that this is SQL does not have the same syntax on every DB (some don't even support it).
I would not hold the data in memory (either in the browser or in the serving application). Instead I'd page through the results using SQL.
How you do this can be database-specific. See here for one example in MySql. Mechanisms will exist for other databases.
1) No, having all the records in memory kind of defeats the point of having a database. Look into having a scrollable result set, that way you can get the functionality you want without having to play with the SQL. You can also adjust how many records are fetched at a time so that you don't load more records than you need.
2) Db connections are expensive to create and destroy but any serious system will pool the connections so the impact on performance won't be that great.
If you want to get a bit more fancy you can do away with pages altogether and just load more records as the user scrolls through the list.
It would not be a good idea, as you are making the browser executable hold all of that.
When I had something like this to do used javascript to render the page, and just made ajax calls to get the next page. There is a slight delay in displaying the next table, as you fetch it, but users are used to that.
If you are showing 100 records/page, use json to pass the data from the server, as javascript can parse it quickly, and then use innerHTML to put the html, as the DOM is much slower in rendering tables.
As mentioned by others here, it is not a good idea to store a large list of results in memory. Query for results for each page is certainly a much better approach. To do that you have two options. One is to use whatever the database specific features your DBMS provides for targeting a specific subsection of results from a query. The other approach is to use the generic methods provided by JDBC to achieve the same effect. This keeps your code from being tied to a specific database:
// get a ResultSet from some query
ResultSet results = ...
if (count > 0) {
results.setFetchSize(count + 1);
results.setFetchDirection(ResultSet.FETCH_FORWARD);
results.absolute(count * beginIndex);
}
for (int rowNumber = 0; results.next(); ++rowNumber) {
if (count > 0 && rowNumber > count) {
break;
}
// process the ResultSet below
...
}
Using a library like Spring JDBC or Hibernate can make this even easier.
In many SQL language, you have a notion of LIMIT (mysql, ...) or OFFSET (mssql).
You can use this kind of thing to limit rows per page
Depends on the data. 100k int's might not be too bad if you are caching that.
T-SQL has SET ##ROWCOUNT = 100 to limit the amount of records returned.
But to do it right and return the total # of pages, you need a more advanced paging SPROC.
It's a pretty hotly dedated topic and there are many ways to do it.
Here's a sample of an old sproc I wrote
CREATE PROCEDURE Objects_GetPaged
(
#sort VARCHAR(255),
#Page INT,
#RecsPerPage INT,
#Total INT OUTPUT
)
AS
SET NOCOUNT ON
--Create a temporary table
CREATE TABLE #TempItems
(
id INT IDENTITY,
memberid int
)
INSERT INTO #TempItems (memberid)
SELECT Objects.id
FROM Objects
ORDER BY CASE #sort WHEN 'Alphabetical' THEN Objects.UserName ELSE NULL END ASC,
CASE #sort WHEN 'Created' THEN Objects.Created ELSE NULL END DESC,
CASE #sort WHEN 'LastLogin' THEN Objects.LastLogin ELSE NULL END DESC
SELECT #Total=COUNT(*) FROM #TempItems
-- Find out the first and last record we want
DECLARE #FirstRec int, #LastRec int
SELECT #FirstRec = (#Page - 1) * #RecsPerPage
SELECT #LastRec = (#Page * #RecsPerPage + 1)
SELECT *
FROM #TempItems
INNER JOIN Objects ON(Objects.id = #TempItems.id)
WHERE #TempItems.ID > #FirstRec AND #TempItems.ID < #LastRec
ORDER BY #TempItems.Id
I would recommend that you choose using CachedRowSet .
A CachedRowSet object is a container for rows of data that caches its rows in memory, which makes it possible to operate without always being connected to its data source.
A CachedRowSet object is a disconnected rowset, which means that it makes use of a connection to its data source only briefly. It connects to its data source while it is reading data to populate itself with rows and again while it is propagating changes back to its underlying data source.
Because a CachedRowSet object stores data in memory, the amount of data that it can contain at any one time is determined by the amount of memory available. To get around this limitation, a CachedRowSet object can retrieve data from a ResultSet object in chunks of data, called pages. To take advantage of this mechanism, an application sets the number of rows to be included in a page using the method setPageSize. In other words, if the page size is set to five, a chunk of five rows of data will be fetched from the data source at one time. An application can also optionally set the maximum number of rows that may be fetched at one time. If the maximum number of rows is set to zero, or no maximum number of rows is set, there is no limit to the number of rows that may be fetched at a time.
After properties have been set, the CachedRowSet object must be populated with data using either the method populate or the method execute. The following lines of code demonstrate using the method populate. Note that this version of the method takes two parameters, a ResultSet handle and the row in the ResultSet object from which to start retrieving rows.
CachedRowSet crs = new CachedRowSetImpl();
crs.setMaxRows(20);
crs.setPageSize(4);
crs.populate(rsHandle, 10);
When this code runs, crs will be populated with four rows from rsHandle starting with the tenth row.
On the similar path, you could build upon a strategy to paginate your data on the JSP and so on and so forth.