I'm having a hard time getting decent performances with Google Cloud SQL, I'm doing some pretty basic CRUD operations, for instance:
public BaseUser getUser(String token) throws SQLException{
Connection conn = DriverManager.getConnection(JDBC_CON_STRING);
PreparedStatement ps = conn.prepareStatement(GET_USER_BY_TOKEN_QUERY);
ps.setString(1, token);
ResultSet rs = ps.executeQuery();
List<BaseUser> users = inflateUser(rs);
if(users.size() == 0){
return null;
}
if(users.size() > 1){
logger.info("DATABASE MAY BE CORRUPTED, THERE'S MORE THAN 1 USER WITH THE SAME TOKEN");
}
ps.close();
rs.close();
conn.close();
return users.get(0);
}
And getting an average of 450ms reponse time for each query. += 150 for openConnection, 150 for operation, 150 for close. See the img. below.
I've read the google documentation, forums and multiple blogs and still can't see what I'm doing wrong (I must be doing something wrong, 450ms/query is wayyy to much...)
UPDATE 1: I'ts definitively a Google Cloud SQL issue, I installed my own local MySQL server and I'm having way better performances (80ms for an "insert or update", then select finally commit.), hope I could get some hints from Google dev. team, I really like the whole Google cloud platform, but it's simply impossible to work with that level of latency =(
UPDATE 2: 2014/05/06 The latency problem is the same with a D0 or a D16. Trying to insert 10000 rows (3 varchar and a ~100bytes blob) takes 32s from a Google ComputeEngine VM because of the latency. The duration is the same with 10000 inserts and a single batch insert. If I use 64 threads, then the duration is down to 3s. I tested with the native mysql jdbc driver.
Any suggestions?
Thanks!
Give it a try with the MySQL Connector/J driver in AppEngine, it may give better performance.
See https://developers.google.com/appengine/docs/java/cloud-sql/#Java_Connect_to_your_database for the classname and URL format to use.
Rob
I had problem with GAE + cloud SQL used with php and the socket.
The problem was my query that used actually 3 queries and read thousands of rows even tough only outputtet couple.
I found it with HeidiSQL and putting EXPLAIN in front of the select query.
Had to make the join in php side so that the load time (inspected in chrome devtools, not mysql run time) of that query-set dropped from 600 ms to 151 ms
Related
I'm having a weird problem with an Grails application accessing data. Going deeper I've isolated the problem to a plain java8 small application using PreparedStatement.executeQuery vs Statement.executeQuery.
Consider the following snippet of code:
// executes in milliseconds
directSql = "select top(10) * from vdocuments where codcli = 'CCCC' and serial = 'SSSS' ORDER BY otherField DESC;";
stmt = con.createStatement();
rs = stmt.executeQuery(directSql);
// More than 10 minutes
sqlPrepared = "select top(10) * from vdocuments where codCli = ? and serial = ? ORDER BY otherField DESC;";
PreparedStatement pStatement = con.prepareStatement( sqlPrepared );
pStatement.setString(1, "CCCC");
pStatement.setString(2, "SSSS");
rsPrepared = pStatement.executeQuery();
Same query.
Data comes from a view on SqlServer (2008, I think, have no access right now) from a table with more than 15 Million records. There are indexes for all needed fields and the same query (the first one) executed from console runs also quite fast.
If I execute the slow PreparedStatement query without the ORDER clause it also runs fast.
It looks clear to me that for any cause the database it's not using indexes and make a full scan when using preparedStatement, but maybe I'm wrong so I'm open to any idea.
I thought maybe the driver (sqlserver official latest and jtds has been tested) was holding the data waiting for any kind of EOF from connection but I've checked with tcpdump on my side and no data is received.
I can't find why this is happening so any idea will be welcomed.
Thank you in advanced!
I've finally found a solution, at least in for my case. I got it here http://mehmoodbluffs.blogspot.com.es/2015/03/hibernate-queries-are-slow-sql-servers.html . Telling (driver? sqlServer?) not to send parameters as Unicode have resolved the problem.
Current connection string it's now:
String connectionUrl = "jdbc:sqlserver://server:port;databaseName=myDataBase;sendStringParametersAsUnicode=false";
And now both direct queries and preparedStatements runs at millisecond speed.
Thank you #DanGuzman for your suggestions!
I am creating a basic java application with Oracle database. I use Oracle SQL Developer to interact with oracle database.
I created tables and inserted data using Oracle SQL Developer and all operations were successful.But when I try to get those records in the java program, it does not get any results.
I debugged the java program and found out the issue is the resultset not getting any records from DB.(Other things are fine)
try {
stmt = dbConnector().createStatement();
rs = stmt.executeQuery(sql);
while (rs.next()) {
System.out.println(rs.getString("AGE"));
}
} catch (SQLException e) {
e.printStackTrace();
Because of that, I tried to verify using SQL command line and it did not show the entries I entered using SQL developer. I cannot figure out what is wrong. Please refer the attached.
A help regarding this is much appreciated.
Please make sure that you have commit the transactions done in Oracle SQL Developer. Most probably that may be the reason for not showing records both in SQL plus & your java program.
You may commit by clicking on the commit button on Oracle SQL Developer toolbar (i.e. 6th button from left)
I am using this code:
double timeBefore = System.currentTimeMillis();
ResultSet rs = st.executeQuery(sql);
double timeAfter = System.currentTimeMillis();
System.out.println(timeAfter - timeBefore);
The return is 9904.0
While when I do the exact same query from WorkBench MySQL client:
SELECT DISTINCT completeAddress FROM DB_M3_Medium.AvailableAddressesV2 where postNr = 2300 ORDER BY completeAddress ASC;
it takes 0.285s
How is that possible?
PS: I tried it with different payload sizes and it's always approx. 10s with Java JDBC
EDIT:
I tried PreparedStatement with the same query as above and it took the same time, approx. 1s less.
I have also tried pinging with following code:
String query = "/* ping */ SELECT 1";
double timeBefore = System.currentTimeMillis();
PreparedStatement preparedStatement = DBConnect.getInstance().con.prepareStatement(query);
ResultSet rs = preparedStatement.executeQuery(query);
double timeAfter = System.currentTimeMillis();
System.out.println(timeAfter - timeBefore);
And the response was: 1306.0 which is not perfect, but better.
But I am still not getting what is wrong with it.
EDIT2:
I have figured out that the time that it takes is related to the amount of data in the DB (not the payload that I am retrieving). It appears to me like indexing didn't work. But why would I then have the issue only when I go with JDBC but not with WorkBench.
while you code in java, you are creating connection,then passing the query. That query is compiled(as you are using Statement) in the sql server and then you will get the result. This whole process needs some time.But when you execute direclty in workbench you are neither creating connection nor compiling,you are simply running the sql.Hence the time taken is less
As #SpringLearner suggested in JDBC every time you execute a query and made a new connection and cost some time. You can use a Data Source to avoid this overhead and better performance.
One thing to bear in mind is that the JDBC driver is pure Java, so you are probably running into some early JIT compilation that would not apply with the MySQL workbench. After the JDBC driver code has been through the JIT, you will probably see comparable performance. The real test for you would be put that code few more times after that and see what happens.
You can also use a PreparedStatement and see if that helps since that should be the API most comparable to what the MySQL workbench is using to avoid unnecessary recompilation of the query.
So I have been playing around with querying databases using the standard
Statement s = connection.createStatement();
ResultSet rs = s.executQuery(queryString);
ResultSetMetadata rsmd = rs.getMetaData();
while(rs.next)){
String code = "";
String value = "";
for(int i = 1; i <= rsmd.getColumnCount(); i++){
Object obj = rs.getObject(i);
if(i == 1){
code = obj.toString():
}
else{
label = obj.toString();
}
}
//Store code and labels in map
}
...go on to close statement and move on.
I am trying to select two columns from a table in each instance.
For the most part this works well. When working with MySql & Microsoft Sql databases I get a result set full of data in the table. However when I try to do this with an Oracle database I get an empty result set.
I have tested my query string in the SQL Developer application and it works fine, returns my data. But the result set doesnt contain anything. The resultSet metadata says that it has two columns though. Is there anything I need to do when interacting with an Oracle Database that is different from the other two? Thanks.
If your query works when you run it against the Oracle database, and you know the code works since you've run it against MySQL, then some other things to try are:
1.) Make sure your JDBC connection URL is correct. Are you sure you are connecting to the database that you intend to? (i.e. - the one that would return the rows you expect?)
2.) Take into account credentials. Make sure you are using the same credentials through JDBC that you are when connecting to Oracle directly.
3.) Make sure both connections are being made from the same machine and with the same environment. Oracle drivers rely on environment variables to find a file (I believe it is called tnsnames.ora, or something like that) that contains the alias & connection info. Getting different versions of that file could point you to different Oracle instances.
4.) Try manually specifying your schema name in the query. So instead of select * from my_table use select * from my_schema.my_table. Sometimes Oracle clients will configure their sessions to have default schemas set up in their preferences.
5.) If your are attempting to select data that you've inserted with your Oracle client, make sure you've committed the transaction in your Oracle client so that the data is visible to other sessions.
One last debugging tool to use is to try connecting via the Squirrel DB client. Squirrel is a 100% pure java SQL client that connects to any DB using JDBC. It would be a good test to make sure your JDBC Driver, Connection URL, etc. are all valid.
The database table has records but the JDBC client can't retrieve the records. Means the JDBC client doesn't have the select privileges. Please run the below query on command line:
grant all on emp to hr;
i have the following query:
String updatequery = "UPDATE tbl_page SET linkCount = ?, pageProcessed = 1 WHERE pageUrl =?";
PreparedStatement updatestmt = kon.prepareStatement(updatequery);
updatestmt.clearParameters();
//updatestmt.setQueryTimeout(10);
updatestmt.setInt(1, linkCount);
updatestmt.setString(2, urlLink);
updatestmt.executeUpdate();
When i set the query timeout for 10 seconds it will catch an exception the query timed out. but when i dont it goes on waiting. Whats wrong with the query? pageUrl column is the Primary Key with varchar(900)
I know something might be wrong with the prepared statement because when i run this query in MS SQl Server Management Studio ('?' replaced with its value) it works fine.
Am i missing something in Java or MSSQL?
Since the code looks just fine, this could be an issue at database side. May be someone else has blocked the row by updating it and not doing a commit/rollback (most possibly from you MS-SQL Server Management studio !). You could look for locks owned by other processes for the same record so that you can be sure that this is not a database issue.
Create an index on pageUrl:
create index tbl_page_pageUrl_index on tbl_page(pageUrl);
That will allow speedy access to the rows you want to update.
Without this index, the database must do a full table scan, and when combined with an update command, if likely to lead to lock contention and possibly even deadlocks, depending on your locking options.