I'm trying to set up a service which listens to a RabbitMQ server and I've set up code using the RabbitMQ Sample code from Github, which includes the following try-with-resources block
try (Connection connection = factory.newConnection();
Channel channel = connection.createChannel()) {
// code here
}
When I use the same code and build and run this service using java -cp myJar.jar MyService, it just starts and ends immediately (and echo $? returns 0)
However, if I replace the block with the following, then it works fine with the same command, and I'm able to start a listener to a RabbitMQ instance
try {
Connection connection = factory.newConnection();
Channel channel = connection.createChannel());
// code here
}
The same happens even when I create database connections
try (Connection connection = dataSource.getConnection()) {
//code here
}
fails but
try {
Connection connection = dataSource.getConnection();
//code here
}
works fine, and allows me to use the connection to make entries into the DB as well.
Why is this happening? I'm using OpenJDK 11.0.2 and this service is standalone, but the rest of the codebase is a JAX-RS driven Rest API if it helps.
The try-with-resources construct closes the resources when the block exits. Your basic try block code doesn't close anything. That's the difference between the two.
When you use try-with-resources, you need to actually DO something with the resource.
If you're writing code where the resource you are opening needs to be 'long lived', then try-with-resources is not the right construct (though, generally, that means the class you're writing should itself be AutoClosable).
When using any concept in programming you have to understand what you are doing!
The whole point of try-with-resource is that the resources are guaranteed to be closed when the try block ends.
But note: just using a try block is not sufficient. Such critical resources, like a database connection require distinct and intentional life cycle management. In other words: you should keep such resources around exactly as long as you need them. And you better make sure that you don't mistakenly forget to close a resource when it is no longer required!
Thus you should not lightheartedly give up on try-with-resource. Rather make sure that code that requires such a resource gets called within the scope of such a try-with-resource block!
Related
I have a bukkit plugin (minecraft) that requires a connection to the database.
Should a database connection stay open all the time, or be opened and closed when needed?
The database connection must be opened only when its needed and closed after doing all the necessary job with it. Code sample:
Prior to Java 7:
Connection con = null;
try {
con = ... //retrieve the database connection
//do your work...
} catch (SQLException e) {
//handle the exception
} finally {
try {
if (con != null) {
con.close();
}
} catch (SQLException shouldNotHandleMe) {
//...
}
}
Java 7:
try (Connection con = ...) {
} catch (SQLException e) {
}
//no need to call Connection#close since now Connection interface extends Autocloseable
But since manually opening a database connection is too expensive, it is highly recommended to use a database connection pool, represented in Java with DataSource interface. This will handle the physical database connections for you and when you close it (i.e. calling Connection#close), the physical database connection will just be in SLEEP mode and still be open.
Related Q/A:
Java Connection Pooling
Some tools to handle database connection pooling:
BoneCP
c3po
Apache Commons DBCP
HikariCP
Depends on what are your needs.
Creating a connection takes some time, so if you need to access database frequently it's better to keep the connection open. Also it's better to create a pool, so that many users can access database simultaneously(if it's needed).
If you need to use this connection only few times you may not keep it open, but you will have delay when you would like to access database. So i suggest you to make a timer that will keep connection open for some time(connection timeout).
You need to close your connections after each query executions.Sometimes you need to execute multiple queries at the same time because the queries are hanging from each other.Such as "first insert task then assign it to the employees".At this time execute your queries on the same transaction and commit it, if some errors occur then rollback.By default autocommit is disabled in JDBC. Example
Use connection pooling.If you are developing a webapplication then use App Server connection pooling.App server will use the same pooling for each of your applications so you can control the connection count from the one point.Highly recommend the Apache Tomcat Connection pooling.Example
As an additional info:
Connection, Statement and ResultSet.
1.If you close connection you don't need close statement or resultset.Both of them will be closed automatically
2.If you close Statement it will close ResultSet also
3.if you use try-with-resources like this:
try (Connection con = ...) {
} catch (SQLException e) {
}
it will close the connection automatically.Because try-with-resources require autoclosable objects and Connection is autocloseable.You can see the details about try-with-resources here
Actually, it's all matter on how you write your application! It's an art, but sadly everyone takes a tutorial for a good practice like Microsoft's tutorials.
If you know what you are coding, then you keep your connection open for the lifetime of the application. It's simple, not because you have to go at work in the morning that everyday we have to build a special route just for you! You take that single route or 2 or 4 like everyone does! You judge for the traffics and you build 2, 4 or 6 routes as needed. If there is traffic with these 4 or 6 routes, you wait!
Happy coding.
The Connection should be opened only when required. If it is open before the actual need, it reduces one active connection from the connection pool..so it ultimately effects the users of the application.
So,it is always a better practice to open connection only when required and closing it after completion of process.
Always try puttting you connection close logic inside the finally block that will ensure that your connection will be closed,even if any exception occurs in the application
finally
{
connection.close()
}
I am currently working on a java project which implements web-scraping and I am facing a weird issue so far.
Here is what I do :
Get an URL Connection with a page of a website
Parse the HTML code to get some content (OpenData)
Add the content in my database
Move onto the next page and go back to Step 1
This is actually very long and it can last for days so I need to let the script running. The problem is that sometimes, it stops for no reason (no errors, no messages, no window close ; It just litterally stops and I need to press one of my button to restart it). I have implemented a short code which restarts the application from where it stopped. I believe it's a connection problem to the database so I would like to know how could I fix it.
I use a static class which creates an instance of this class at the beginning of the application and then I use static methods from this class to run my queries like this for example :
ConnexionBDD.con.prepareStatement(query);
public static Connection loadDriver() {
try {
Class.forName(Driver);
con = DriverManager.getConnection(ConnectionString, user, pwd);
} catch (ClassNotFoundException e) {
System.err.println("Classe not found : Class.forName(...)");
} catch (SQLException e) {
System.err.println(e.getMessage());
}
return con;
}
I am not sure I am doing the right thing to make my connection lasts forever (in theory) and eventually close it when It has finished to iterate over my links.
You're jumping the gun a bit here. There's no evidence that the database connection is actually the problem. Usually if you were having DB connection issues you'd be getting an exception from the connection when you try to perform operations on it, a timeout, etc.
You need to:
Add detailed logging to your application, so you can see what it's doing as it progresses, and what it's trying to do when it stops; and
Run it with -Xdebug and other suitable options for remote debugging, so you can attach a debugger to it when it stops and examine its state to see what it is doing at the time. Use the debugger user interface from NetBeans, Eclipse, or whatever you prefer to attach to the program when the logging indicates that it's stopped progressing.
For logging, you can use java.util.logging. See the javadoc and the logging overview docs.
Here's an example of how to do remote debugging with Eclipse. You'll be able to find similar guides for your chosen IDE. Java also has a command line debugger, but it's pretty painful.
You also need to check to see whether the program might be crashing or exiting, rather than just stopping working. You should capture any standard error output from the program and check the program's error return code from the shell. Also look for hs_error files in the directory the program runs in, in case there's a JVM crash, though that should generate output on stderr as well.
You should also:
Set an application_name when you establish a connection to PostgreSQL, so you can easily see what your client is doing with the database. You can specify application_name as a JDBC connection parameter, or run a SET application_name = 'blah' statement after connecting.
When logging (or however you currently tell that your program is no longer progressing) indicates that the program has stopped working, examine pg_stat_activity in the server, looking at the entry/entries for your application. See if the connection is idle, idle in transaction, or running a statement, and what that statement is. If it's running a statement, query against pg_locks to see if it's blocked on an ungranted lock.
My team has to make some changes and renew an old web application. This application has one main thread and 5 to 15 daemon threads used as workers to retrieve and insert data in a DB.
All those threads have this design (here simplified for convenience):
public MyDaemon implements Runnable {
// initialization and some other stuffs
public void run() {
...
while(isEnabled) {
Engine.doTask1();
Engine.doTask2();
...
Thread.sleep(someTime);
}
}
}
The Engine class provides a series of static methods used to maipulate other methods of DataAccessor classes, some of those methods been static:
public Engine {
public static doTask1() {
ThisDataAccessor.retrieve(DataType data);
// some complicated operations
ThisDataAccessor.insertOrUpdate(DataType data);
}
public static doTask2() {
ThatDataAccessor da = new ThatDataAccessor();
da.retrieve(DataType data);
// etc.
}
...
}
DataAccessor classes usually interact with DB using simple JDBC statements enclosed in synchronized methods (static for some classes). DataSource is configured in the server.
public ThatDataAccessor {
public synchronized void retrieve(DataType data) {
Connection conn = DataSource.getConnection();
// JDBC stuff
conn.close();
}
...
}
The problem is that the main thread needs to connect to DB and when these daemon threads are working we run easily out of available connections from the pool, getting "waiting for connection timeout" exceptions. In addition, sometimes even those daemon threads get the same exception.
We have to get rid of this problem.
We have a connection pool configured with 20 connections, and no more can be added since that "20" is our production environment standard. Some blocks of code need to be synchronized, even if we plan to move the "synchronized" keyword only where really needed. But I don't think that it would make really the difference.
We are not experienced in multithreading programming and we've never faced this connection pooling problem before, that's why I'm asking: is the problem due to the design of those threads? Is there any flaw we haven't noticed?
I have profiled thread classes one by one and as long as they are not running in parallel it seems that there's no bottleneck to justify those "waiting for connection timeout".
The app is running on WebSphere 7, using Oracle 11g.
You are likely missing a finally block somewhere to return the connections back to the pool. With hibernate, I think this is probably done when you call close() or possibly for transactions, when you call rollback(). But I would call close anyway.
For example, I wrote a quick and dirty pool myself to extend an old app to make it multithreaded, and here is some of the handling code (which should be meaningless to you except the finnally block):
try {
connection = pool.getInstance();
connection.beginTransaction();
processFile(connection, ...);
connection.endTransaction();
logger_multiThreaded.info("Done processing file: " + ... );
} catch (IOException e) {
logger_multiThreaded.severe("Failed to process file: " + ... );
e.printStackTrace();
} finally {
if (connection != null) {
pool.releaseInstance(connection);
}
}
It is fairly common for people to fail to use finally blocks properly... For example, look at this hibernate tutorial, and skip to the very bottom example. You will see that in the try{} he uses tx.commit() and in the catch{} he uses tx.rollback(), but he has no session.close(), and no finally. So even if he added a "session.close()" in try and in catch, if his try block threw something other than a RuntimeException, or his catch caused an additional Exception before the try or a non-HibernateException before the rollback(), his connection would not be closed. And without session.close(), I don't think that is actually very good code. But even if the code is seemingly working, a finally gives you assurance that you are protected from this type of problem.
So I would rewrite his methods that use Session to match the idiom shown on this hibernate documentation page. (and also I don't recommend his throwing a RuntimeException, but that is a different topic).
So if you are using Hibernate, I think the above is good enough. But otherwise, you'll need to be more specific if you want specific code help, but otherwise the simple idea that you should use a finally to ensure the connection is closed is enough.
How does async JMS work? I've below sample code:
public class JmsAdapter implements MessageListener, ExceptionListener
{
private ConnectionFactory connFactory = null;
private Connection conn = null;
private Session session = null;
public void receiveMessages()
{
try
{
this.session = this.conn.createSession(true, Session.SESSION_TRANSACTED);
this.conn.setExceptionListener(this);
Destination destination = this.session.createQueue("SOME_QUEUE_NAME");
this.consumer = this.session.createConsumer(destination);
this.consumer.setMessageListener(this);
this.conn.start();
}
catch (JMSException e)
{
//Handle JMS Exceptions Here
}
}
#Override
public void onMessage(Message message)
{
try
{
//Do Message Processing Here
//Message sucessfully processed... Go ahead and commit the transaction.
this.session.commit();
}
catch(SomeApplicationException e)
{
//Message processing failed.
//Do whatever you need to do here for the exception.
//NOTE: You may need to check the redelivery count of this message first
//and just commit it after it fails a predefined number of times (Make sure you
//store it somewhere if you don't want to lose it). This way you're process isn't
//handling the same failed message over and over again.
this.session.rollback()
}
}
}
But I'm new to Java & JMS. I'll probably consume messages in onMessage method. But I don't know how does it work exactly.
Do I need to add main method in JmsAdapter class? After adding main method, do I need to create a jar & then run the jar as "java -jar abc.jar"?
Any help is much appreciated.
UPDATE: What I want to know is that if I add main method, should I simply call receiveMessages() in main? And then after running, will the listener keep on running? And if there are messages, will it retrieve automatically in onMessage method?
Also, if the listener is continuously listening, doesn't it take CPU??? In case of threads, when we create a thread & put it in sleep, the CPU utilization is zero, how doe it work in case of listener?
Note: I've only Tomcat server & I'll not be using any jms server. I'm not sure if listener needs any specific jms server such as JBoss? But in any case, please assume that I'll not be having anything except tomcat.
Thanks!
You need to learn to walk before you start trying to run.
Read / do a tutorial on Java programming. This should explain (among other things) how to compile and run a Java program from the command line.
Read / do a tutorial on JMS.
Read the Oracle material on how to create an executable JAR file.
Figure out what it is you are trying to do ... and design your application.
Looking at what you've shown and told us:
You could add a main method to that class, but to make an executable JAR file, you've got to create your JAR file with a manifest entry that specifies the name of the class with the main method.
There's a lot more that you have to do before that code will work:
add code to (at least) log the exceptions that you are catching
add code to process the messages
add code to initialize the connection factory and connection objects
And like I said above, you probably need some kind of design ... so that you don't end up with everything in a "kitchen sink" class.
if I add main method, should I simply call receiveMessages() in main?
That is one approach. But like I said, you really need to design your application.
And then after running, will the listener keep on running?
It is not entirely clear. It should keep running as long as the main thread is alive, but it is not immediately obvious what happens when your main method returns. (It depends on whether the JMS threads are created as daemon threads, and that's not specified.)
And if there are messages, will it retrieve automatically in onMessage method?
It would appear that each message is retrieved (read from the socket) before your onMessage method is called.
Also, if the listener is continuously listening, doesn't it take CPU???
Not if it is implemented properly.
In case of threads, when we create a thread & put it in sleep, the CPU utilization is zero, how doe it work in case of listener?
At a certain level, a listener thread will make a system call that waits for data to arrive on a network socket. I don't know how it is exactly implemented, but this could be as simple as an read() call on the network socket's InoutStream. No CPU is used by a thread while it waits in a blocking system call.
This link looks like a pretty good place with examples using Oracle AQ. There's an examples section that tells you how to setup the examples and run them. Hopefully this can help.
Link to Oracle Advanced Queueing
Sometimes when I call connect() on a third-party proprietary JDBC driver, it never returns and a stack trace shows that it is stuck waiting for a socket read (usually). Is there a generic way to forcibly cancel this operation from another thread? It's a blocking I/O call so Thread.interrupt() won't work, and I can't directly close the socket because I don't have access to it since it's created inside the proprietary code.
I'm looking for generic solutions because I have a heterogeneous DB environment (Oracle, MySQL, Sybase etc). But driver-specific suggestions are also welcome. Thanks,
There is no standard JDBC interface to set connection or read timeouts, so you are bound to use proprietary extensions, if the JDBC driver supports timeouts at all. For the Oracle JDBC thin driver, you can e.g. set the system properties "oracle.net.CONNECT_TIMEOUT" and or "oracle.jdbc.ReadTimeout" or pass a Properties instance to DriverManager.getConnection with these properties set. Although not particulary well documented, the Oracle specific properties are listed in the API documentation.
For other JDBC drivers, the documentation should contain the relevant references.
Ah ... the joys of using closed-source libraries ...
If interrupt() doesn't work, and you cannot set some kind of timeout, then I think there is no safe way to do it. Calling Thread.kill() might do the job, but the method is deprecated because it is horribly unsafe. And this is the kind of scenario where the unsafe-ness of Thread.kill() could come back and bite you.
I suggest that you simply code your application to abandon the stuck thread. Assuming that your application doesn't repeatedly try to connect to the DB, a stuck thread isn't a huge overhead.
Alternatively use a better JDBC driver. (And on your way out of the door, complain to the supplier about their driver being too inflexible. There is a slight chance that someone might listen to you ...)
At least one JDBC driver (not one of those you listed, though) will cleanly close the connection if the thread this connection attempt is running on is interrupted. I don't know if this will work for all drivers though.
This is a problem with Java, not the JDBC driver. In certain circumstances, the socket connect call ignores the timeout parameters and can take minutes to return. This happens to us when firewall blocks the port. It happens to all TCP connections (HTTP, RMI).
The only solution I find is to open connection in a different thread like this,
private static final ExecutorService THREADPOOL
= Executors.newCachedThreadPool();
private static <T> T call(Callable<T> c, long timeout, TimeUnit timeUnit)
throws InterruptedException, ExecutionException, TimeoutException
{
FutureTask<T> t = new FutureTask<T>(c);
THREADPOOL.execute(t);
return t.get(timeout, timeUnit);
}
try {
Data data = call(new Callable<Data>() {
public Data call() throws Exception
{
// Open connection, get data here
return data;
}, 2, TimeUnit.SECONDS);
} catch (TimeoutException e) {
System.err.println("Data call timed-out");
}