We have a Java EE application using JPA.
User can ask the app to generate some output (big charts, etc) based on results of a large query, so in that case we decided to use directly JDBC and looping over the resultset.
We have implemented our code like this:
#Resource(mappedName = "jdbc/resource")
private DataSource dataSource;
Connection connection = dataSource.getConnection();
[...]
Connection conn = dataSource.getConnection();
stmt = conn.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(Integer.MIN_VALUE);
[...]
ResultSet rs = stmt.executeQuery(sql);
while (rs.hasNext()) {
[....]
}
Anyway it always load all the results in memory, we have tried with different techniques (useCursorFetch true and fetch size = 10, etc) with no luck, we always fail to have a stream resultset.
The "jdbc/resource" is the same resource used by jpa entity manager (so it's working fine), the connection have no "weird" options, we are using glassfish 4.1 for our tests and we are using the connector/J version 5.1.26 and 5.1.32.
It there a way to have a working streaming resultset?
Related
I agree that the database user privileges should be tuned to prevent him for running DML or DDL queries but I'm trying to do a similar thing from the client side. The JDBC side.
I downloaded Oracle driver found here.
While the driver and it's companion JARs are in the classpath, I ran the following piece test code:
Connection connection = DriverManager.getConnection(url, username, password);
connection.setAutoCommit(false);
// connection.setReadOnly(true); // Doesn't help
Statement statement = connection.createStatement();
statement.executeQuery("ALTER TABLE GELBANA_TEST.TABLE1 ADD HQ varchar2(40) DEFAULT 'NY'");
connection.rollBack();
But the column is added to the table !!
I can also drop the column.
Shouldn't this be enough to prevent the user from running DML or DDL queries ? I need to stop him from making any changes to the database. He should be allowed to only select data. I checked the connection URL parameters for something that could reject such queries but I found nothing close.
I'm running some SQL queries in an AWS Lambda, and was hoping to utilize AWS-XRay's tracing capabilities to get some more detailed information on these calls.
This documentation shows examples of configuration with Spring and Tomcat, but neither of which makes sense to use in my obviously serverless and supposed-to-be lightweight Lambda. Here's how I establish my connections currently:
public Connection getDatabaseConnection(String jdbcUrl, String dbUser, String dbPassword) throws SQLException
{
return DriverManager.getConnection(jdbcUrl, dbUser, dbPassword);
}
try (Connection connection = getDatabaseConnection(getJdbcUrl(), getDbUser(), getDbPassword()))
{
try(ResultSet results = connection.createStatement().executeQuery("SELECT stuff FROM whatever LIMIT 1))
{
return (results.getLong(1));
}
}
Is there any way to utilize AWS-XRay SQL tracing in my use case?
I think one approach would be to use a MySQL statement interceptor:
https://github.com/spullara/mysql-connector-java/blob/master/src/main/java/com/mysql/jdbc/StatementInterceptor.java
You can use AWSXRay.beginSubsegment() in preProcess() method and then AWSXRay.endSubsegment() in the postProcess().
Would be a nice addition to AWS X-Ray SDK for Java which is in open source.
in case you get it working.
For reference of spring based implementation for X-Ray: DataSource based intereceptor
You can use the statementinterceptors property as part of the connection URL to intercept the statement as documented here:
https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html
Update
My teammate pointed out, a newer version of StatementInterceptor is available:
https://github.com/spullara/mysql-connector-java/blob/master/src/main/java/com/mysql/jdbc/StatementInterceptorV2.java
You may want to use that.
I am trying to connect to one of my MySql Databases through a System DSN I set up. The DSN is set up correctly with my SSL certs, username, password, port, and the databases populate the DSN database drop down and the "Test" connection passes. I can't seem to get a connection in Java. I have spent 2 days looking through some examples on Stack but they all refer to an Access database and using JDBC-ODBC bridge which is no longer available in Java 8. I tried using UCanAccess with Jackcess but I have gotten no where. The code below is what I have been tinkering with the last few hours. I normally connect to MySql databases with PHP and receive result in JSON or directly with JDBC driver but for this project neither are really an option. Any ideas. I appreciate the help.
//String username = "<username>";
//String password = "<password>";
//String database = "<database_name>";
try {
Class.forName("net.ucanaccess.jdbc.UcanaccessDriver");
//Connect to cllients MySql Database
Connection conn = DriverManager.getConnection("jdbc:ucanaccess:" + database);
//Call VerifyLabel(<MAC>,<MODEL>); Call provided client
CallableStatement cStmt = conn.prepareCall("{CALL verify(?, ?)}");
//MAC
cStmt.setString(1, "mac address");
//model
cStmt.setString(2, "model");
cStmt.execute();
//Getting results from "Status" column
ResultSet rs1 = cStmt.getResultSet();
//Iterate results and print.
while (rs1.next()) {
System.out.println(rs1.getString("Status"));
}
//Close connection conn
rs1.close();
} catch (SQLException ex) {
Logger.getLogger(CambiumStoredTest.class.getName()).log(Level.SEVERE, null, ex);
} catch (ClassNotFoundException ex) {
Logger.getLogger(CambiumStoredTest.class.getName()).log(Level.SEVERE, null, ex);
}
Using MySql Driver:
Class.forName("com.mysql.jdbc.Driver");
Connection conn = DriverManager.getConnection("jdbc:mysql:"+ database);
also tried:
Class.forName("com.mysql.jdbc.Driver");
Connection conn = DriverManager.getConnection("jdbc:mysql://localhost:3306/"+ database);
Error for MySql Driver:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
1) DSN is most commonly assocatiated with ODBC (and often with MS-Access). Hence all the links. ODBC is NOT required for a DSN.
2) Do NOT use Ucanaccess. Use J/Connector for mySQL.
3) Make sure you can communicate with mySQL from the command line. Then focus on getting a small "hello world" JDBC app to connect. Your second and third examples look OK. Be sure to check the mySQL logs for any warnings/errors.
Well, after an entire day of trying to get this to work and sleeping on it for a couple hours I finally got it to work. UCanAccess and mysql-connector did not work. The easiest thing since no other method of connecting to this clients database was acceptable was to push this application in Java 7 rather than 8. This allowed me to Coo=nnect to my DSN with no problems. I understand that this method is not the best solution but it is what is working flawlessly and efficiently. Also, instead of using some rigged up 3rd party libs and jars, I am able to use Connector/J. Thanks everyone for trying to help me. Just incase anyone else runs into this issue, this is how I made it work.
Develope app in Java 7 - not 8.
Set Up System DSN
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
//You do not need to provide username or password if it is setup in DSN
Connection conn = DriverManager.getConnection("jdbc:odbc:"+ database);
I'm trying to activate the statementPool with the db2 jdbc driver, and i m having trouble doing so.
I've tried different techniques mentionned here : http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=%2Fcom.ibm.db2z10.doc.java%2Fsrc%2Ftpc%2Fimjcc_c0052041.htm
Here is the java code :
csUrl += ":maxStatements=1000;";
System.err.println("Connecting to csUrl : " + csUrl);
DB2ConnectionPoolDataSource ds = new DB2ConnectionPoolDataSource();
//DB2SimpleDataSource ds = new DB2SimpleDataSource();
//ds = new DB2ConnectionPoolDataSource();
ds.setServerName("edited");
ds.setPortNumber(edited);
ds.setDatabaseName("edited");
ds.setCurrentSchema("edited");
ds.setMaxStatements(1000);
ds.setDriverType(4);
Connection connection = ds.getPooledConnection(csUser, csPassword).getConnection();
//Connection connection = ds.getConnection(csUser, csPassword);
//Connection connection = DriverManager.getConnection(csUrl);
//Connection connection = m_driver.connect(csUrl, propertiesUserPassword);
boolean pooling = connection.getMetaData().supportsStatementPooling();
if (pooling) {
System.err.println(">>> Pooling is ON!");
} else {
System.err.println(">>> Pooling is OFF! <<<");
}
All techniques fail according to : connection.getMetaData().supportsStatementPooling() and statementPooling is always deactivated.
I'm running DB2 Express /Linux :
Database server = DB2/LINUXX8664 10.1.2
with the matching jdbc driver.
Same code with a different db2 backend (db2 10 zOs) gives the same result, so i guess it s not an Express issue.
Client code is running on windows.
We have a in-house statement cache, but i wanted to play around with DB2's especially since it can apparently share the pool in between logical connections provided thru a PooledDataSource
Thanks :)
I'm trying to follow Java's JDBC tutorials to write a Java program that can connect to SQL Server 2008. I'm getting lost at the point of making a connection.
The following snippet is from the tutorial:
InitialContext ic = new InitialContext();
DataSource ds = ic.lookup("java:comp/env/jdbc/myDB");
Connection con = ds.getConnection();
DataSource ds = (DataSource) org.apache.derby.jdbc.ClientDataSource()
ds.setPort(1527);
ds.setHost("localhost");
ds.setUser("APP")
ds.setPassword("APP");
Connection con = ds.getConnection();
There's no explanation of what comp/env/jdbc/myDB should point to, and I don't know how I should choose a port. Also, the object ds seems to be defined twice.
I'm using the JSQLDataSource driver, for the record. Can anyone point me in the right direction here?
http://java.sun.com/docs/books/tutorial/jdbc/basics/connecting.html
I'm not sure anyone above has really answered the question.
I found this microsoft sample useful.
The key information in there is really that the class you need is SQLServerDataSource
that is basically a configuration object - you use it something like this:
SQLServerDataSource dataSource = new SQLServerDataSource();
dataSource.setUser("aUser");
dataSource.setPassword("password");
dataSource.setServerName("hostname");
dataSource.setDatabaseName("db");
You would then call
dataSource.getConnection();
to get a connection object which is basically the thing you use to talk to the database.
Use
connection.prepareStatement("some sql with ? substitutions");
to make something for firing off sql and:
connection.prepareCall
for calling stored procedures.
Start with the JDBC tutorial or the Microsoft docs.
and this:
String driver = "com.microsoft.jdbc.sqlserver.SQLServerDriver";
Class.forName(driver);
String url = "jdbc:microsoft:sqlserver://host:1433/database";
Connection conn = DriverManager.getConnection(url, "username", "password");
Fill in your values for host, database, username, and password. The default port for SQL server is 1433.
UPDATE: Good point below. JDBC drivers can be had from both Microsoft and jTDS. I prefer the latter.
JNDI lookups have to do with Java EE app servers that support connection pooling. You can ask the app server to create a pool of connections, which can be an expensive thing to do, and loan them out to clients like library books as needed.
If you aren't using a Java EE app server or connection pooling, you have to create the connection on your own. That's where manual processes and DriverManager come in.
EXPLANATION: As for why the Sun tutorial shows DataSource twice, I'd say it's a case of poor editing. If you look above the code sample it says you can get a DataSource "by lookup or manually". The code snippet below shows both together, when it should be one or the other.
You know it's an inadvertent error because there's no way the code as written could compile. You have "ds" declared twice.
So it should read "...lookup", followed by its code snippet, and then "...manually", followed by its code snippet.
I like the jTDS driver for connecting to SQL Server. A URL will look like this:
jdbc:jtds:sqlserver://localhost/Finance;instance=sqlexpress
Check this for jTDS Url Info.
This also has some interesting information to help troubleshoot jtds to sql express sorts of problems.
DataSource ds = new SimpleDriverDataSource(new com.mysql.jdbc.Driver(),
"jdbc:mysql://database:1433;databaseName=name", "username", "password");
JdbcTemplate jdbc = new JdbcTemplate(ds);
This question has already been answered long time ago. The question was asked about JNDI lookup. With lookup you have to see the application server log to see what the connection is bound to. For example in Jboss startup, I can see:
[ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:service=DataSourceBinding,name=myDB' to JNDI name 'java:myDB'
Using that name=myDB you lookup
InitialContext ic = new InitialContext();
DataSource ds = ic.lookup("java:myDB");
Notice how the server log and the code both point to the JNDI name java:myDB.