I'm having a weird problem with an Grails application accessing data. Going deeper I've isolated the problem to a plain java8 small application using PreparedStatement.executeQuery vs Statement.executeQuery.
Consider the following snippet of code:
// executes in milliseconds
directSql = "select top(10) * from vdocuments where codcli = 'CCCC' and serial = 'SSSS' ORDER BY otherField DESC;";
stmt = con.createStatement();
rs = stmt.executeQuery(directSql);
// More than 10 minutes
sqlPrepared = "select top(10) * from vdocuments where codCli = ? and serial = ? ORDER BY otherField DESC;";
PreparedStatement pStatement = con.prepareStatement( sqlPrepared );
pStatement.setString(1, "CCCC");
pStatement.setString(2, "SSSS");
rsPrepared = pStatement.executeQuery();
Same query.
Data comes from a view on SqlServer (2008, I think, have no access right now) from a table with more than 15 Million records. There are indexes for all needed fields and the same query (the first one) executed from console runs also quite fast.
If I execute the slow PreparedStatement query without the ORDER clause it also runs fast.
It looks clear to me that for any cause the database it's not using indexes and make a full scan when using preparedStatement, but maybe I'm wrong so I'm open to any idea.
I thought maybe the driver (sqlserver official latest and jtds has been tested) was holding the data waiting for any kind of EOF from connection but I've checked with tcpdump on my side and no data is received.
I can't find why this is happening so any idea will be welcomed.
Thank you in advanced!
I've finally found a solution, at least in for my case. I got it here http://mehmoodbluffs.blogspot.com.es/2015/03/hibernate-queries-are-slow-sql-servers.html . Telling (driver? sqlServer?) not to send parameters as Unicode have resolved the problem.
Current connection string it's now:
String connectionUrl = "jdbc:sqlserver://server:port;databaseName=myDataBase;sendStringParametersAsUnicode=false";
And now both direct queries and preparedStatements runs at millisecond speed.
Thank you #DanGuzman for your suggestions!
Related
I am trying to retrieve data from DB2 using Java prepared statement
String select_statement = "SELECT * FROM schema_name.table_name where NME='xxx002' and LINE =7200 and FILE_NME='720001042021XYZ002' with ur";
try (Connection connection = DataBaseConnection.getGeoCarDBConnection_TESTDATA();
PreparedStatement ps = connection.prepareStatement(select_statement);) {
ResultSet rs = null;
rs = ps.executeQuery();
}
The problem I am facing is that I include the FILE_NME in the where clause of the query, as shown above, 0 rows are returned. But any other string fields can be passed and I get the desired number of rows.
Any integer fields in the where clause works too.
But only the string fields that are large(In this case, FILE_NME field) are not working. In the DB2 table, where I am pulling the data from, the FILE_NME field is of varchar(30).
Things that did not work for me was
String select_statement = "SELECT * FROM schema_name.table_name where NME='xxx002' and LINE =7200 and FILE_NME = ? with ur";
then I set the String value using,
ps.setString(1, "'720001042021XYZ002'")
ps.setString(1, "720001042021XYZ002")
Both did not work.
None of the google links were helpful. Have spent more than a day on it.
This code used to work flawlessly before, Even the java version hasn't changed(as per my knowledge)
I am running it in windows 10.
Java version : 1.8 ((build 1.8.0_221-b11))
I run the same query in the database client and it works.
Someone please help me or point me in the right direction. I don't know what I am missing
Thank in advance
The problem was that there was indeed no data, I was checking the same query in the database client in a different environment. I am closing this.
I have a java servlet application and I'm using a prepared query to update a record in a SQL Server Database table.
Lets say I want to execute UPDATE MyTable SET name = 'test' WHERE id = '10'. (Yes, id is a varchar)
I used the following code to make this happen:
PreparedStatement pstmt = con.prepareStatement("UPDATE MyTable SET name = ? WHERE id = ?");
pstmt.setString(1, getName() );
pstmt.setString(2, getID() );
pstmt.executeUpdate();
I found out that while I was running a JMeter script to simulate 2 users, this statement causes a deadlock in my database.
I wanted to check what my values were in the SQL Profiler so I used the following code, so I could check the values.
String query = String.format("UPDATE MyTable SET name = '%s' WHERE id = '%s' ", getName(), getID() );
PreparedStatement pstmt = con.prepareStatement(query);
pstmt.executeUpdate();
Suddenly my deadlock was gone! It's a shame the last approach is vulnerable to SQL injection.
Is there somebody who can tell me what is going on and/or how to fix it?
Ok I finally found the problem and solution to my problem.
It seemed that the combination of the jTDS JDBC driver with MSSQL was the 'problem'.
This article explained my situation exactly. And with the help of this FAQ I was able to set the datasource to the right configuration.
From what I understand:
If you have statement that uses a String-like index (Like in my situation), the table performs an index SCAN instead of an index SEEK. This causes the whole table to be locked and vulnerable to deadlocks.
I hope this will help other people too.
I am using this code:
double timeBefore = System.currentTimeMillis();
ResultSet rs = st.executeQuery(sql);
double timeAfter = System.currentTimeMillis();
System.out.println(timeAfter - timeBefore);
The return is 9904.0
While when I do the exact same query from WorkBench MySQL client:
SELECT DISTINCT completeAddress FROM DB_M3_Medium.AvailableAddressesV2 where postNr = 2300 ORDER BY completeAddress ASC;
it takes 0.285s
How is that possible?
PS: I tried it with different payload sizes and it's always approx. 10s with Java JDBC
EDIT:
I tried PreparedStatement with the same query as above and it took the same time, approx. 1s less.
I have also tried pinging with following code:
String query = "/* ping */ SELECT 1";
double timeBefore = System.currentTimeMillis();
PreparedStatement preparedStatement = DBConnect.getInstance().con.prepareStatement(query);
ResultSet rs = preparedStatement.executeQuery(query);
double timeAfter = System.currentTimeMillis();
System.out.println(timeAfter - timeBefore);
And the response was: 1306.0 which is not perfect, but better.
But I am still not getting what is wrong with it.
EDIT2:
I have figured out that the time that it takes is related to the amount of data in the DB (not the payload that I am retrieving). It appears to me like indexing didn't work. But why would I then have the issue only when I go with JDBC but not with WorkBench.
while you code in java, you are creating connection,then passing the query. That query is compiled(as you are using Statement) in the sql server and then you will get the result. This whole process needs some time.But when you execute direclty in workbench you are neither creating connection nor compiling,you are simply running the sql.Hence the time taken is less
As #SpringLearner suggested in JDBC every time you execute a query and made a new connection and cost some time. You can use a Data Source to avoid this overhead and better performance.
One thing to bear in mind is that the JDBC driver is pure Java, so you are probably running into some early JIT compilation that would not apply with the MySQL workbench. After the JDBC driver code has been through the JIT, you will probably see comparable performance. The real test for you would be put that code few more times after that and see what happens.
You can also use a PreparedStatement and see if that helps since that should be the API most comparable to what the MySQL workbench is using to avoid unnecessary recompilation of the query.
I have table called mpi which contains 23 columns. I have introduced the search field with button for every column where user can enter the query to fetch the records using query
query="select * from mpi where Genus ='"+genus+"'
Now I want to fetch records by giving keywords using LIKE %% but it is not working and not giving any records but if type type the full name it is working perfectly. Here is the code
String uname=request.getParameter("uname");
String full="%"+uname+"%";
dbconn=new DatabaseConnection();
conn=dbconn.setConnection();
pstmt=conn.prepareStatement("select * from mpi where Genus LIKE ?");
pstmt.setString(1, full);
res=pstmt.executeQuery
Could any one tell me where is the mistake and why I am not getting the records when I use half keyword like %keyword%.
It works (apart from the missing parentheses) and the approach with a prepared statement is entirely correct.
However I have seen a couple of code pieces like that, and always the problem lay with variables mix-up or not closing, or simple oversight. Better declare as close as possible.
try (ResultSet res = pstmt.executeQuery()) {
while (res.next()) {
..
}
} // Automatically closes res.
Also handle the life-cycle of pstmt correctly, with closing.
I'm using a library that delegates to a JDBC driver for PostgreSQL, and some queries are very complex and require more memory. I don't want to set work_mem to something large for all queries, just this subset. The problem is that executing the following code results in an error:
// pseudo-code for what is happening
String sql = "set work_mem = 102400;";
sql += "SELECT * FROM expensive_query";
ResultSet rs = DatabaseDriver.execute(sql);
When I run this I get an error that:
set work_mem = 102400;
returns no results.
This works in pgAdmin because you can execute multiple queries at once. Is there a better way to do this or do I need to execute arbitrary SQL and then extract the result set I want?
I have no idea what DatabaseDriver does, but with "plain" JDBC you just need to do the following:
Statment stmt = connection.createStatement();
stmt.execute("set work_mem = 102400");
ResultSet rs = stmt.executeQuery("select ...");
Try to do that using Batch Processing