I ran into the exact issue that setNetworkTimeout is supposed to solve according to Oracle. A query got stuck in socket.read() for several minutes.
But I have little idea what the first parameter for this method needs to be. Submitting a null causes an AbstractMethodError exception, so... does the implementation actually need some sort of thread pool just to set a network timeout?
Is there some way to achieve the same effect without running a thread pool just for this one condition?
It seems like the documentation explains this horribly, but without looking at any code behind the class my guess would be that you are expected to pass an Executor instance to the method so that implementations can spawn jobs/threads in order to check on the status of the connection.
Since connection reads will block, in order to implement any sort of timeout logic it's necessary to have another thread besides the reading one which can check on the status of the connection.
It sounds like a design decision was made that instead of the JDBC driver implementing the logic internally, of how/when to spawn threads to handle this, the API wants you as the client to pass in an Executor that will be used to check on the timeouts. This way you as the client can control things like how often the check executes, preventing it from spawning more threads in your container than you like, etc.
If you don't already have an Executor instance around you can just create a default one:
conn.setNetworkTimeout(Executors.newFixedThreadPool(numThreads), yourTimeout);
As far as Postgres JDBC driver is concerned (postgresql-42.2.2.jar), the setNetworkTimeout implementation does not make use of the Executor parameter. It simply sets the specified timeout as the underlying socket's timeout using the Socket.setSoTimeout method.
It looks like the java.sql.Connection interface is trying not to make any assumptions about the implementation and provides for an executor that may be used if the implementation needs it.
Related
I am trying to integrate QFJ into a single-threaded application. At first I was trying to utilize QFJ with my own TCP layer, but I haven't been able to work that out. Now I am just trying to integrate an initiator. Based on my research into QFJ, I would think the overall design should be as follows:
The application will no longer be single-threaded, since the QFJ initiator will create threads, so some synchronization is needed.
Here I am using an SocketInitiator (I only handle a single FIX session), but I would expect a similar setup should I go for the threaded version later on.
There are 2 aspects to the integration of the initiator into my application:
Receiving side (fromApp callback): I believe this is straightforward, I simply push messages to a thread-safe queue consumed by my MainProcessThread.
Sending side: I'm struggling to find documentation on this front. How should I handle synchronization? Is it safe to call Session.sendToTarget() from the MainProcessThread? Or is there some synchronization I need to put in place?
As Michael already said, it is perfectly safe to call Session.sendToTarget() from multiple threads, even concurrently. But as far as I see it you only utilize one thread anyway (MainProcessThread).
The relevant part of the Session class is in method sendRaw():
private boolean sendRaw(Message message, int num) {
// sequence number must be locked until application
// callback returns since it may be effectively rolled
// back if the callback fails.
state.lockSenderMsgSeqNum();
try {
.... some logic here
} finally {
state.unlockSenderMsgSeqNum();
}
Other points:
Here I am using an SocketInitiator (I only handle a single FIX session), but I would expect a similar setup should I go for the threaded version later on.
Will you always use only one Session? If yes, then there is no use in utilizing the ThreadedSocketInitiator since all it does is creating a thread per Session.
The application will no longer be single threaded, since the QFJ initiator will create threads
As already stated here Use own TCP layer implementation with QuickFIX/J you could try passing an ExecutorFactory. But this might not be applicable to your specific use case.
I've been using StackOverFlow for a long time now and always found existing answers, but this time I couldn't find any information about what I'm trying to do.
Using java, I have a process composed of about 10 different tasks that gather distinct data from the database using pure jdbc (no ejb/jpa here). Each task (callable) can actually be run concurrently and is responsible for obtaining a connection, which is what we are doing. However we're randomly experiencing trouble with the connection pool (accessed via jndi), sometimes we're blocked because the connection pool doesn't have any available connection.
To solve this problem, I thought we could change the way we're obtaining the connections, instead of letting each callable opening and closing a connection ( following the number of tasks to execute and the number of threads to use in the ThreadPoolExecutor), I would like to create some kind of local connections pool dedicated to this process, so that we're sure nothing will block later (eventually if we can't acquire all the requested connections, we would then adapt the number of threads to launch with a minimum of 1)
My colleagues approve this idea, but what surprises me is that I can't found any similar approaches or discussion on the web (maybe I'm not using the right keywords).
I would like to know what you think about this idea, whether you already tried something similar or if I'm missing something important.
In advance, thank you.
You have not mentioned which connection pool is used. If it is not HikariCP and you are allowed to switch, having contributed there I recommend it.
HikariCP seems rather interesting finally, i'll have to check this further. But this isn't directly related to the question :)
Just a little return of experience, my idea is working, with one caveat, I couldn't get rid of one downcast from a runnable to my implementation on which I can do .setConnection() during the before() of my ExecutorService. And all tasks must have been given to the executor with the execute() method, otherwise the runnable is autolatically wrapped in a FutureTask without the ability to access the inner runnable. Maybe one of you know of to do this correctly ?
I want to implement mechanism which will be closing connections if there are not used by specific period of time. This time is constant for all of the connections. Opened connections can be used many times, so I need to update usage time and always compute difference between current time and usage time. I also need to close connections which excess my timeout.
My opened connections are in Map. (Map<Id, Connection>) where Id is an Integer.
I thought about resolving my problem with DelayQueue, but there is no possible to update usage (in this case delay) time in this type of Queue.
I also know that this mechanism should work in separate thread.
Please, give me some tip about the best way of implementation or example. What kind of data structure should I use?
I can use Spring also (maybe there is some good mechanism and I don't know about it).
If you're speaking about database connectivity then just use connection poolers such as c3po, hikariCP, BoneCP and so on. Don't reinvent the wheel.
Take a look at the HikariCP code. Specifically, look at:
ConcurrentBag
PoolBagEntry
BaseDataSource.getConnection()
BaseDataSource.releaseConnection()
HouseKeeper inner class
While HikariCP is a database connection pool, you can use ConcurrentBag as is, use the HouseKeeper basically as is, slightly modify PoolBagEntry, and lift the basic gist from getConnection() and releaseConnection(), to create a generic pool.
Hi guys am getting following error am using Websocket and Tomcat8.
java.lang.IllegalStateException: The remote endpoint was in state [TEXT_FULL_WRITING] which is an invalid state for called method
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase$StateMachine.checkState(WsRemoteEndpointImplBase.java:1092)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase$StateMachine.textStart(WsRemoteEndpointImplBase.java:1055)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendString(WsRemoteEndpointImplBase.java:186)
at org.apache.tomcat.websocket.WsRemoteEndpointBasic.sendText(WsRemoteEndpointBasic.java:37)
at com.iri.monitor.webSocket.IRIMonitorSocketServlet.broadcastData(IRIMonitorSocketServlet.java:369)
at com.iri.monitor.webSocket.IRIMonitorSocketServlet.access$0(IRIMonitorSocketServlet.java:356)
at com.iri.monitor.webSocket.IRIMonitorSocketServlet$5.run(IRIMonitorSocketServlet.java:279)
You are trying to write to a websocket that is not in a ready state. The websocket is currently in writing mode and you are trying to write another message to that websocket which raises an error. Using an async write or as not such good practice a sleep can prevent this from happening. This error is also normally raised when a websocket program is not thread safe.
Neither async or sleep can help.
The key problem is the send-method can not be called concurrently.
So it's just about concurrency, you can use locks or some other thing. Here is how I handle it.
In fact, I write a actor to wrap the socketSession. It will produce an event when the send-method is called. Each actor will be registered in an Looper which contains a work thread and an event queue. Meanwhile the work thread keeps sending message.
So, I will use the sync-send method inside, the actor model will make sure about the concurrency.
The key problem now is about the number of Looper. You know, you can't make neither too much or too few threads. But you can still estimate a number by your business cases, and keep adjusting it.
it is actually not a concurrency issue, you will have the same error in a single-threaded environment. It is about asynchronous calls that must not overlap.
You should use session.get**Basic**Remote().sendText instead of session.get**Async**Remote().sendText() to avoid this problem. Should not be an issue as long as the amount of data you are writing stays reasonable small.
I am using java to create an interface to connect to a database. Each time I want to make a call to the database I need to create new connections to the database, which would make calling the database say 10 times slow.
To avoid having to create new connections each time I want to call the database I have a java thread running that holds all of the connection information.
To write/read from the database I want to create a thread that uses the connection information stored in the thread that's already running, use it to execute specified read/write functions, and then exit.
However I am having trouble accessing this information from the thread which is already running. What would be the best way to accomplish this?
This is a terrible idea, because java.sql.Connection is not thread-safe.
A better idea would be to use a connection pool. Let each thread check out a connection, use it, and put it back.
best way is not to re-invent the wheel. there are good open spource implementations of the connection pooling and i suggest you use them.
if you are already running in a container then use DataSource. look into c3p0 (http://sourceforge.net/projects/c3p0/) and commons-dbcp (http://commons.apache.org/dbcp/)
Why do you need a thread running to keep your connection open, just store it somewhere and execute queries as soon as you need it.. should it work?
In any case if you really want a thread you should care about having a synchronized collection (check Collections.asSynchronizedList) that can be accessed and managed from your thread and others too.
To overcome visibility problems just declare it as a static final variable, so you won't have any problems in accessing it from outside the thread you declared it into.
Another easy solution (since connection seems to be not thread-safe) is not to use a thread but use just a monitor: you can easily manage a wait()/notify() mechanism for which a thread that wants to execute a query checks if connection is "free". if it is occupies the monitor and do whatever it wants before notifying all waiting threads.
why are you doing this? There are frameworks, like Spring or equivalent, which will manage your connections for you. Don't reinvent the wheel man....
I would recommend to use a generic object pool instead of building your own solution and suggest to check Commons Pool from Apache Commons (this is an API for generic Object pooling, this isn't DBCP).