I have a table named "preference" which includes more than 100 columns in oracle,I wrote a little bit complicated SQL which need use keyword UNION/INTERSECT/MINUS to do a query.
Take a simple example:
select a.* from preference a where a.id = ? union
select a.* from preference a where a.id = ?
The business care have been changed due to unlimited length string storage on demand. one column need to be re-defined to Clob type. Oracle don't allow union on the clob type, so ideally the a.* cannot be used here.
I changed SQL to like below:
select a.a,a.b,a.c... from preference a where a.id = ? union
select a.a,a.b,a.c... from preference a where a.id = ?
It lists all columns except clob and then I have to do another selection to append the Clob value together. Is that a good idea?
The Another issue brought from above case is that: as I mentioned this table has large columns, list all columns in sql it make SQL much longer. Is there expression I can select all columns but getting rid of specific one?
Oracle when delaing with log does not allow union/minus but allows union all, may be you can rewrite your query using union all and use a select . in the select clause you can issue a select a. or list every column.
After reading your question my main concern is memory usage on Java, are you using an orm to load the data? or are you using the jdbc api?
If you are loading all the clobs into some strings you could end with an OutOfMemoryError. My advice is to load the clob only for rows you need to show to the user (or for the rows where the clob filed has to be processed).
Can you give more insight about your application (the numer fo rows it has to process) and your data (epsecially the clob size)?
Related
I have the following query:
#Select("SELECT* FROM "+MyData.TABLE_NAME+" where data_date = #{refDate}")
public List<MyData> getMyData(#Param("refDate") Date refDate);
This table data is HUGE! Loading so many rows in memory is not the best way!
Is it possible to have this same query return a resultset so that I can just iterate over one item?
edit:
I tried adding:
#ResultType(java.sql.ResultSet.class)
public ResultSet getMyData(#Param("refDate") Date refDate);
but it gives me:
nested exception is org.apache.ibatis.reflection.ReflectionException: Error instantiating interface java.sql.ResultSet with invalid types () or values (). Cause: java.lang.NoSuchMethodException: java.sql.ResultSet.<init>()
I'd suggest you use limit in your query. limit X, Y syntax is good for you. Try it.
If the table is huge, the query will become slower and slower. Then the best way to to iterate will be to filter based on id and use limit.
such as
select * from table where id>0 limit 100 and then
select * from table where id>100 limit 100 etc
There are multiple options you have ...
Use pagination on database side
I will just suppose the database is oracle. However other db vendors would also work. In oracle you have a rownum with which you can limit number of records to return. To return desired number of records you need to prepare a where clause using this rownum. Now, the question is how to supply a dynamic rownum in a query. This is where dynamic sqls of mybatis comes in use. You can pass these rownum values inside a parameter map which there onwards you can use in your query inside a mapper xml using a #{} syntax. With this approach you filter the records on db level itself and only bring or prepare java objects which are needed or in the current page.
Use pagination on mybatis side
Mybatis select method on sqlSession has a Rowbounds attribute. Populate this as per your needs and it will bring you those number of records only. Here, you are limiting number of records on mybatis side whereas in first approach the same was performed on db side which is better performant .
Use a Result handlers
Mybatis will give you control of actual jdbc result set. So, you can do/iterate over the result one by one here itself.
See this blog entry for more details.
I'm working on a feature which allow some users to define their own SQL queries and run them on the database.
Basically a query could look like this:
1. SELECT first_name, last_name FROM user;
2. SELECT first_name, last_name, id, address, email FROM user.
As you can see there may be a different number of columns in the result table.
Is there a way to handle this in Hibernate?
For instance, the basic usage displayed below does not help me in any way because I cannot be sure that each result row has at least 2 columns.
Query query = session.getSession().createSQLQuery(queryStr);
ScrollableResults results = query.scroll(ScrollMode.FORWARD_ONLY);
while (results.next()) {
data.put("firstName", String.valueOf(results.get(0)));
data.put("lastName", String.valueOf(results.get(1)));
}
Furthermore, I don't think I can use the select new map because the users have to run native MySQL queries.
Is there any solution to this?
Thanks in advance for your time and suggestions!
First of all, I do not think that it is a good idea to expose an interface that allows SQL input to users.
Anyway, in your case you could program the SQL result set extraction yourself. There is no need for object relational mapping, if you do not map into objects. You could then go and just check for the existence of the columns and map them right away into your data structure. Nice way to learn plain JDBC. Just for the case you use Hibernate for other issues in your application, you can even mix the approaches.
I need to get the list of columns in a table using mybatis/ibatis in java 1.5.
That's not a typical requirement (99.99% of applications using iBatis or whatever ORM knows the DB schema). iBatis is a SQL mapper, you must write the SQL query yourself. And there is no standard SQL query (AFAIK) that gives you the number of columns in a table.
I can only suggest two approaches:
Make a SQL query selecting from the catalog tables. That's the normal way of knowing about your DB metadata. But that depends on your particular database engine. And it's not related to iBatis.
QUick and dirty: make an ad-hoc query SELECT * FROM MYTABLE LIMIT 1 (replace LIMIT for your DB analog), map that in iBatis through a HashMap, and in your DAO just count the number of keys.
For Mybatis:You need to use resultType instead of resultmap.
resultType must be of returning collection data type, by knowing the size of collection you can get no. of columns and more over if you are going with HashMap you can get column names too in keys.
i am migrating from ms access database to mysql database with java frontend so that my application can be used in linux as well .
in ms access here's what i would do
go to create query . write a select statement . call give the name of the query as query1.
when double clicking on query1 you would get the result of select statement in a tabular form.
next i would write a query2 which is also a select query. this query would be fetching data not from a table but query1 e.g select a,b from query1;
now i am on a mysql database using java
what would be the java statement for select a,b from query1 ?
what i mean to say is that i would connect to mysql using jdbc.
have query1 like this
string query1 = " select * from users " ;
then execute query using executeQuery(query1)
but i dont think i can do something like this.
string query2 = " select a,b from query1 " ;
and then executeQuery(query2)
so what is the way out ?
I ran into the exact same problems when I went from using MS Access to using a lot of SQL queries against a MySQL DB.
There are two ways I would approach this:
VIEWS:
Views are a great way to emulate a lot of the functionality you found in Access. One of the things I really liked about Access was the ability to separate my SQL into smaller queries and then re-use those queries in other queries. Views allow you to do essentially the same thing in that you define a query in a View and then you can write another query or View against that original View.
In my experience, however, Views tend to be really slow, especially when referencing calculated columns. With MySQL, I very rarely use Views (though perhaps others have found for efficient ways of implementing them).
SUBQUERIES (NESTED QUERIES)
As others have mentioned, subqueries are a great way to write multiple queries within one query. With a subquery, instead of putting the query name (as in Access) or View name (as explained above) within the SELECT portion of your code, you simply paste the entire SQL statement of the subquery.
You might write code like this to find only the 2009 sales and salesperson name for customers in a database:
SELECT
customer.Name,
customer.AccountNumber,
customer.SalespersonName,
ch.`2009 Sales`
FROM
customer
Left Join (
SELECT
customerhistory.AccountNumber,
SUM ( CASE WHEN customerhistory.`Year` = 2009
THEN customerhistory.`Sales`
ELSE 0
END
) AS `2009 Sales`
FROM
customerhistory
GROUP BY
customerhistory.AccountNumber
) ch ON customer.AccountNumber = ch.AccountNumber
In my work, I tend to use mostly subqueries since I find they run a lot faster than views, but your experience may vary.
You can do this all in MySQL. The query would look like
SELECT * FROM (
SELECT * FROM users
) query1;
You can either do nested queries (subqueries) like #muffinista suggested.
But i think you are looking for Views: http://dev.mysql.com/doc/refman/5.0/en/create-view.html.
In short, a view is a "pseudo table" that is a result of a query.
you can
create view q1 as
select * from table1 where f1>1
select * from q1 where f2<100
select * from table2 where user_id in (select user_id from users)
I would like to do something in Java (using iBatis, JDBC, etc., really in SQL) like:
SELECT SUM(rowName) FROM myTable WHERE id = [myArrayOfIds]
Where myArrayOfIds can be almost any length. Now I know you can do:
SELECT SUM(rowName) FROM myTable WHERE id IN (x, y, z)
but what happens for longer lists? For example my list could be as little as a few items to hundreds or more items. How can I do this?
I think it depends on your flavour of SQL. For instance, Oracle does not allow more than 1000 values in an IN() list. Other flavours may vary.
one alternative would be to insert those ids to a table, then do a join
SELECT SUM(rowName) FROM myTable ta inner join tempTable tb on ta.id = tb.id
Oracle definitely allows more than 1000 items in the IN clause. It's your persistence tool that is limiting this. iBatis or Hibernate, whatever. Use Oracle Sqlplus and you'll see this is not an Oracle limit.
Suggestion from BlackTigerX would work, or you could call the query multiple times, passing 1000 items at a time and aggregating the results. Either way, you're just working around your persistence tool limitation.