Java/Derby - Should I manage the application memory myself? - java
I wrote this simple class in NetBeans, in order to learn about connecting to and managing a Derby database through Java code, and am using the sample database provided by the IDE as the connection target. I believe it's a rather straightforward database connection example:
import java.io.IOException;
import java.io.PrintWriter;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.logging.Level;
import java.util.logging.Logger;
public class MainClass {
public static void main(String[] args) {
//Some code that allows me to start the connection only after keyboard input
System.out.println("Program will now connect to database");
try {
System.in.read();
}
catch (IOException ex) {
Logger.getLogger(MainClass.class.getName()).log(Level.SEVERE, null, ex);
}
// Redirecting DriverManager's output to the console's standard output
DriverManager.setLogWriter(new PrintWriter(System.out));
// This is where I connect to the database, execute a query and print a result
// Still barebones, I am learning SQL basics yet
String dbURL ="jdbc:derby:C:\\users\\project2100\\.netbeans-derby\\sample";
try (Connection conn = DriverManager.getConnection(dbURL, "app", "app")) {
System.out.println("Connected");
try (Statement stmt = conn.createStatement()) {
ResultSet output = stmt.executeQuery("SELECT * FROM CUSTOMER");
while (output.next()) {
System.out.println(output.getString("NAME"));
}
}
}
catch (SQLException ex) {
Logger.getLogger(MainClass.class.getName()).log(Level.SEVERE, null, ex);
}
// Recently added to keep the application alive
// This way, I can monitor memory usage on Task Manager
try {
System.in.read();
}
catch (IOException ex) {
Logger.getLogger(MainClass.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
Java's standard output gives me the results I expect, but DriverManager's output tells me something I'm worrying about; here's the full console output from an application run:
run:
Program will now connect to database
DriverManager.getConnection("jdbc:derby:C:\users\project2100\.netbeans-derby\sample")
trying org.apache.derby.jdbc.AutoloadedDriver40
SQLState(08004) vendor code(40000)
java.sql.SQLException: Connection refused : java.lang.OutOfMemoryError
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<clinit>(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.Driver20.connect(Unknown Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at javadbpg.MainClass.main(MainClass.java:35)
SQLState(08004) vendor code(40000)
java.sql.SQLNonTransientConnectionException: Connection refused : java.lang.OutOfMemoryError
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<clinit>(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.Driver20.connect(Unknown Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at javadbpg.MainClass.main(MainClass.java:35)
Caused by: java.sql.SQLException: Connection refused : java.lang.OutOfMemoryError
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
... 11 more
getConnection returning org.apache.derby.jdbc.AutoloadedDriver40
Connected
Jumbo Eagle Corp
New Enterprises
Wren Computers
Small Bill Company
Bob Hosting Corp.
Early CentralComp
John Valley Computers
Big Network Systems
West Valley Inc.
Zed Motor Co
Big Car Parts
Old Media Productions
Yankee Computer Repair Ltd
BUILD SUCCESSFUL (total time: 23 seconds)
NOTE: There is a 1-2 seconds hang between the first two errors.
Just recently, after looking around the Internet for what might be causing these OutOfMemoryErrors, I added that last try block and observed the application's memory footprint. It goes from an initial ~9MB to ~40MB.
Therefore, should I set some memory parameters inside the application, in order to reduce these errors, and eventually gain performance? (If so, could I have some pointers?) Or should i just leave everything as is and let DriverManager work by itself?
2014-08-07 Follow-up:
This time, I ran the application on debug mode, and wrote down the private memory usage step by step. Points of interest are as follows:
Memory amount right before the try (Connection conn = ... statement is 12MB;
Following this, the program hangs for about 5 seconds and fires all the OutOfMemoryErrors described above;
After the 5 seconds have elapsed, the program reserves a total memory of 37MB right before the System.out.println("Connected"); statement;
The executeQuery statement adds up another 7MB, total: 45MB. Execution time is negligible (<1/2sec);
Here's the derby.log file as requested by Bryan Pendleton, its state is after all the previous tests:
----------------------------------------------------------------
Thu Aug 07 10:10:11 CEST 2014:
Booting Derby version The Apache Software Foundation - Apache Derby - 10.10.1.3 - (1557168): instance a816c00e-0147-af84-a381-0000263f2b92
on database directory C:\Users\Project2100\.netbeans-derby\C3Subjects with class loader sun.misc.Launcher$AppClassLoader#58644d46
Loaded from file:/C:/Program%20Files/Java/jdk1.8.0_05/db/lib/derby.jar
java.vendor=Oracle Corporation
java.runtime.version=1.8.0_05-b13
user.dir=C:\Users\Project2100\Documents\NetBeansProjects\JavaDB Proving Grounds
os.name=Windows 7
os.arch=amd64
os.version=6.1
derby.system.home=C:\users\Project2100\.netbeans-derby
Database Class Loader started - derby.database.classpath=''
Yesterday, I kept searching on the Internet about memory management, and attempted to use the CLI arguments -Xms and -Xns to raise the application's starting memory, to no avail; these options manipulate only the application's reserved memory, and leaves the actual memory initially used at the usual 9-10MB. I'm getting the same memory values, errors and hangs anyway.
I guess that, for now, I'm going to chalk this behavior under loading database... and go on with SQL itself, since the system console doesn't list any complaints at all. However, if there is indeed a way to preemptively allocate more memory on database connection, I'd be grateful to get an answer about it.
Many thanks for your help!
There is a static variable in the org.apache.derby.impl.jdbc.EmbedConnection:
/**
* Static exception to be thrown when a Connection request can not
* be fulfilled due to lack of memory. A static exception as the lack
* of memory would most likely cause another OutOfMemoryException and
* if there is not enough memory to create the OOME exception then something
* like the VM dying could occur. Simpler just to throw a static.
*/
public static final SQLException NO_MEM =
newSQLException(SQLState.LOGIN_FAILED, "java.lang.OutOfMemoryError");
When you set up the logwriter for DriverManager, the SQLException will be printed:
public SQLException(String reason, String SQLState, int vendorCode) {
super(reason);
this.SQLState = SQLState;
this.vendorCode = vendorCode;
if (!(this instanceof SQLWarning)) {
if (DriverManager.getLogWriter() != null) {
DriverManager.println("SQLState(" + SQLState +
") vendor code(" + vendorCode + ")");
printStackTrace(DriverManager.getLogWriter());
}
}
}
That's why we have an ugly stack trace.
Related
Java issue within connection to Microsoft SQL Server
How can I connect to Microsoft SQL Server from my java code? Code: public class insert { public static void main(String[] args) { try { Class.forName("com.microsoft.jdbc.sqlserver.SQLServerDriver"); String url = "jdbc:microsoft:sqlserver://LENOVO-PC\\SQLEXPRESS;DatabaseName=dbtest"; Connection connection = DriverManager.getConnection(url , "sa" , "Aa123456"); Statement st = connection.createStatement(); st.executeUpdate("INSERT INTO [dbo].[table] VALUES ('come')"); } catch (ClassNotFoundException e) { e.printStackTrace(); System.exit(1); } catch (SQLException e) { e.printStackTrace(); System.exit(2); } } } Error: java.lang.ClassNotFoundException: com.microsoft.jdbc.sqlserver.SQLServerDriver at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Unknown Source) at insert.main(insert.java:12) Microsoft SQL Server name is: LENOVO-PC\SQLEXPRESS and sqljdbc.jar is already added to the referenced libraries.
The class name com.microsoft.jdbc.sqlserver.SQLServerDriver you're trying to load is from a very old version of the Microsoft SQL Server 2000 version of the driver. Around 2005, Microsoft changed this to com.microsoft.sqlserver.jdbc.SQLServerDriver (notice the switch in order between jdbc and sqlserver). At that time, they also changed the driver URL prefix from jdbc:microsoft:sqlserver: to jdbc:sqlserver:. In other words, you need to: Use Class.forName("com.microsoft.jdbc.sqlserver.SQLServerDriver") (optional, driver will be loaded automatically) Change your URL to jdbc:sqlserver://LENOVO-PC\\SQLEXPRESS;databaseName=dbtest (note the prefix change and DatabaseName -> databaseName. See also Building the Connection URL. With recent JDBC drivers, it is not necessary to explicitly load the driver class in simple Java applications, so instead of step 1, you could also remove the Class.forName line. Also make sure you are using a recent version of the driver, see the mssql-jdbc project on GitHub (latest stable version at time of writing is 7.0.0).
Correct the driver class name: Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
ORMLite exception : java.sql.SQLException: ResultSet already requested
In one of our Java app we have a SQLite DB and multiple threads read and write to the same DB (different tables). These multiple threads however use the same database connection object (ORMLite's DaoManager.createDao()). We are getting this SQLException (Resultset already requested) randomly whenever some insert/update statement is executing. Below is the stack trace of one such instance. java.sql.SQLException: Unable to run insert stmt on object ABCD-164: INSERT INTO `pos_order` (`bill_number` ,`order_status` ,`order_type` ,`payment_mode` ,`reference_bill_number` ,`table_selected` ,`sub_total` ,`discountable_sub_total` ,`total` ,`total_discount` ,`discount_type` ,`discount_value` ,`total_tax` ,`service_charge` ,`service_tax` ,`vat` ,`delivery_charge` ,`packaging_charge` ,`amount_paid` ,`waiter` ,`delivery_boy` ,`order_source` ,`delivery_source` ,`card_type` ,`card_name` ,`client_creation_time` ,`client_updation_time` ,`order_comment` ,`customer_feedback` ,`is_deleted` ,`payment_status` ,`pos_outlet_id` ,`is_sync` ,`delivery_status` ,`shipment_id` ,`response_code` ,`delivery_time` ,`custom_packaging_charge_enable` ,`custom_delivery_charge_enable` ,`receipt_printed` ,`total_person` ,`order_origin` ,`json` ,`oo_system_id` ,`pre_order` ,`pre_order_time` ) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) at com.j256.ormlite.misc.SqlExceptionUtil.create(SqlExceptionUtil.java:22) at com.j256.ormlite.stmt.mapped.MappedCreate.insert(MappedCreate.java:135) at com.j256.ormlite.stmt.StatementExecutor.create(StatementExecutor.java:450) at com.j256.ormlite.dao.BaseDaoImpl.create(BaseDaoImpl.java:310) at com.j256.ormlite.dao.BaseDaoImpl.createOrUpdate(BaseDaoImpl.java:336) at com.limetray.pos.dbmanagers.implementations.PosOrderDaoImpl.insert(PosOrderDaoImpl.java:50) at com.limetray.pos.controllers.BillingSectionController.lambda$10(BillingSectionController.java:808) at com.limetray.pos.utilities.SingleTaskExecutor$1.call(SingleTaskExecutor.java:37) at javafx.concurrent.Task$TaskCallable.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.sql.SQLException: ResultSet already requested at org.sqlite.jdbc3.JDBC3Statement.getResultSet(JDBC3Statement.java:136) at org.sqlite.jdbc3.JDBC3PreparedStatement.executeQuery(JDBC3PreparedStatement.java:69) at org.sqlite.jdbc3.JDBC3DatabaseMetaData.getGeneratedKeys(JDBC3DatabaseMetaData.java:1796) at org.sqlite.jdbc3.JDBC3Statement.getGeneratedKeys(JDBC3Statement.java:346) at com.j256.ormlite.jdbc.JdbcDatabaseConnection.insert(JdbcDatabaseConnection.java:173) at com.j256.ormlite.stmt.mapped.MappedCreate.insert(MappedCreate.java:91) ... 13 more Any help would be much appreciated. Thanks.
In one of our Java app we have a SQLite DB and multiple threads read and write to the same DB (different tables). With the SQLite database, you can only have one connection to it at a time. The database has internal locking that ensures that no two threads are using the same connection at the same time but that doesn't stop the DAOs from allowing multiple threads to cause problems like you are having. Caused by: java.sql.SQLException: ResultSet already requested In your case, because multiple threads are using the same connection, this exception is getting thrown when one thread asks for results on a connection that has already been asked my another thread. Their queries are improperly being interleaved. It's important to realize that this has nothing to do with the underlying database. Sqlite has its own locking that blocks concurrent access by multiple threads. But in this case, we are having problems with multiple threads using the same connection. Right now ORMLite does not have a ConnectionSource that provides a synchronized single connection to the underlying database so you will have to do your own external locking. You could extend the JdbcSingleConnectionSource class and use a ReentrantLock to enforce the controls. Maybe something like: public class LockedJdbcSingleConnectionSource extends JdbcSingleConnectionSource { private final ReentrantLock lock = new ReentrantLock(); // called from JdbcSingleConnectionSource.getReadOnlyConnection() ... public DatabaseConnection getReadWriteConnection() { lock.lock(); return super.getReadWriteConnection(); } public void releaseConnection(DatabaseConnection connection) { super.releaseConnection(connection); lock.unlock(); } }
Oracle JDBC DriverManager.getConnection() hangs
We have several servers that each run an Oracle database 11g Release 11.2.0.1.0 - 64bit. We are connecting via JDBC like this: public Connection createConnection(String drvClass, String connURL, String user, String pass) throws ClassNotFoundException, SQLException { Class.forName(drvClass); Connection conn = DriverManager.getConnection(connURL, user, pass); for (SQLWarning warn = conn.getWarnings(); warn != null; warn = warn.getNextWarning()) { System.out.println("SQL Warning:"); System.out.println("State : " + warn.getSQLState()); System.out.println("Message: " + warn.getMessage()); System.out.println("Error : " + warn.getErrorCode()); } return conn; } drvClass would be oracle.jdbc.OracleDriver. The program which tries to connect runs on each server. The database is reachable from out of other programs with the exact same connection properties. It is also possible to let this program run on another server and let it connect to the problematic database. It can establish the connection. The program does not work if it's running on the server locally. We tried both IP and servername. On one server the code hangs at DriverManager.getConnection() an I cannot find out why. Does anyone have any idea what could cause this? There is no entry about this in the DB logs. Stacktrace of blocking thread: "pool-27-thread-1" - Thread t#1483 java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at oracle.net.ns.Packet.receive(Packet.java:239) at oracle.net.ns.NSProtocol.connect(NSProtocol.java:255) at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:973) at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:291) at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:490) at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:202) at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:33) at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:474) at java.sql.DriverManager.getConnection(Unknown Source) at java.sql.DriverManager.getConnection(Unknown Source) at com.companyname.DBConnectionInternal.DBConnection.createConnection(DBConnection.java:19) at com.companyname.exportadapter.ExportCollector.initDatabase(ExportCollector.java:259) at com.companyname.exportadapter.ExportCollector.run(ExportCollector.java:120) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Locked ownable synchronizers: - locked <50be77> (a java.util.concurrent.ThreadPoolExecutor$Worker) If i set DriverManager.setLoginTimeout(10) then i get Io exception: Socket read timed out.
you may making some unneccessary connections. make Connection class static ,whenever you are creating new connection check older is alive or close then and then you must create new connection other wise return old connection. like if(conn!=null & !conn.isClosed()){ // code for create connection } It also depends on how the database side is configured, so check it with DBA of your system. I would like to suggest using Connection pooling. hope this helps.
You might want to enable JDBC debug logging for the ojdbc driver: http://docs.oracle.com/cd/B28359_01/java.111/b31224/diagnose.htm That might give you some information about what the driver is doing. Have you tried telnet-ing to the database server from the client machine (to assert it's reachable)?
The server was misconfigured. For some reason it had a virtual adapter configured which returned an ip to which nothing could connect. From outside the resolving worked correctly. Don't know why it never timed out with the wrong IP though.
Lotus Notes Agent - Java Security Error
I have a Lotus Notes Agent written in Java which should pop up a message to the user after it is executed by clicking a button in the Notes client. Trying to display the pop-up causes an error which I see in the Java Debug console as : java.security.AccessControlException: Access denied (java.lang.RuntimePermission exitVM.0) at java.security.AccessController.checkPermission(AccessController.java:108) at java.lang.SecurityManager.checkPermission(SecurityManager.java:544) at COM.ibm.JEmpower.applet.AppletSecurity.superDotCheckPermission(AppletSecurity.java:1449) at COM.ibm.JEmpower.applet.AppletSecurity.checkRuntimePermission(AppletSecurity.java:1311) at COM.ibm.JEmpower.applet.AppletSecurity.checkPermission(AppletSecurity.java:1611) at COM.ibm.JEmpower.applet.AppletSecurity.checkPermission(AppletSecurity.java:1464) at java.lang.SecurityManager.checkExit(SecurityManager.java:756) at java.lang.Runtime.exit(Runtime.java:99) at java.lang.System.exit(System.java:279) at ClipboardTest.main(Unknown Source) at JavaAgent.NotesMain(Unknown Source) at lotus.domino.AgentBase.runNotes(Unknown Source) at lotus.domino.NotesThread.run(Unknown Source) My agent code is import lotus.domino.*; import javax.swing.JOptionPane; public class JavaAgent extends AgentBase { public void NotesMain() { try { Session session = getSession(); AgentContext agentContext = session.getAgentContext(); // (Your code goes here) Document cdoc = agentContext.getDocumentContext(); String[] notesURL = new String[] {cdoc.getNotesURL()}; ClipboardTest.main(notesURL); JOptionPane.showMessageDialog(null,"message","title",JOptionPane.WARNING_MESSAGE); } catch(Exception e) { e.printStackTrace(); } } } I have manager access to the Database. What could be causing this error?
First look at what the error actually is: Access denied (java.lang.RuntimePermission exitVM.0) You are telling the VM to die and you do not have the rights to do this. If you had, you would cause anything else running on that JVM instance to die as well, possibly leading to a hang/crash. Secondly your code is failing in the ClipboardTest.main() method, which you haven't posted any code for. In the agent properties there is an option to add debug data. This will give you the exact line number that is causing the issue. My guess is you have a System.exit() call in the ClipboardTest.main() . It shouldn't be there.
Set Oracle 10g database connection timeout in Java
I tried to set a connection timeout with the following code: public class ConnectionTimeout { public static void main(String[] args) throws Exception { String entry = "jdbc:oracle:thin:#xxx:1521:xxx"; Connection con=null; Class.forName("oracle.jdbc.driver.OracleDriver"); DriverManager.setLoginTimeout(1); con=DriverManager.getConnection(entry,"username","password"); Statement s=con.createStatement(); s.execute("select 1 from dual"); s.close(); con.close(); } } The instance xxx is not existing. But I get the following exception: Exception in thread "main" java.sql.SQLException: E/A-Exception: Socket is not connected at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255) at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:387) at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:439) at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:165) at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:35) at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:801) at java.sql.DriverManager.getConnection(Unknown Source) at java.sql.DriverManager.getConnection(Unknown Source) at my.package.connection.timeout.ConnectionTimeout.main(ConnectionTimeout.java:22) How I can implement a timeout to an not existing or available Oracle database instance? Edit: If I set the DriverManager.setLoginTimeout(30); to 30 second, the exception happens so fast as before!
Your DriverManager.setLoginTimeout(1); sets the maximum time in seconds for the driver to wait while connecting to the database. In your case, it's set to 1. To have no limit, set setLoginTimeout(0) where 0 means no limit. I hope this helps. Update if your instance xxx doesn't exists, how would you expect your Oracle Driver to connect to the database? It won't make any difference how long you set your loginTimeout there's not "host" to connect to.
Because, in Java doc, it shows: timeout in seconds, but in the implementation of JDBC Oracle, it is milliseconds. You can try using a measure of milliseconds.