Assume the following pseudo code for a simple two thread scenario:
I have two threads, I would like to insert data to different tables to database. On thread1, I would like to insert to some table, at same time, I want to insert other data to thread 2. My question is how/where to place connection.close(), if I place it on thread 1 and it executes while thread2 is still processing, or vice versa, if thread2 has finished and closed the connection, yet thread1 hasn't finished.
Note, the database is just an example, it can be anything like a file,logger..etc.
class Thread1{
DataBaseConnection connection;
main(){
threadPool = Executors.newFixedThreadPool(1);
connection.open();
if(ThisMightTakeSomeTime)
threadPool.submit(new MyRunnable(connection));
InsertDataToDataBase(Table A, Table B));
connection.Close(); //What if thread2 isn't done yet?
}
}
public class MyRunnable implements Runnable {
MyRunnable(connection){}
#override
void Run() { ...}
void TaskThatMayTakeWhile(){
...get data ...
...Connection.InsertToTables(table X, table Y)
}
}
My question is how/where to place connection.close(),
To start, as far as I know, you should not be sharing a single connection with 2 different threads. Each thread should have it's own database connection, possibly utilizing a database connection pool such as Apache's DBCP.
Once you have multiple connections, I would have each thread manage and release its own connection back to the pool. You should make sure this is done in a finally block to make sure that if there is a database exception, the connection is still released.
If you are forced to have multiple threads share the same connection then they will have to use synchronized to make sure they have an exclusive lock to it:
synchronized (connection) {
// use the connection
}
As to when to close it if it is shared, you could have a shared usage counter (maybe an AtomicInteger) and close it when the counter goes to 0. Or as others have recommended you could use a thread-pool and then the thread pool is done free the connection.
Note, the database is just an example, it can be anything like a file,logger..etc.
In terms of a more generic answer I always try to mirror where the thing is created. If a method opens the stream then it should have the finally that closes the stream.
public void someMethod() {
InputStream stream = ...
try {
// process the stream here probably by calling other methods
} finally {
// stream should be closed in the same method for parity
stream.close();
}
}
The exception to this pattern is a thread handler. Then the Thread should close the stream or release connection in a finally block at the end of the run() or call() method.
public void serverLoopMethod() {
while (weAcceptConnections) {
Connection connection = accept(...);
threadPool.submit(new ConnectionHandler(connection);
}
}
...
private static class ConnectionHandler implements Runnable {
private Connection connection;
public ConnectionHandler(Connection connection) {
this.connection = connection;
}
// run (or call) method executed in another thread
public void run() {
try {
// work with the connection probably by calling other methods
} finally {
// connection is closed at the end of the thread run method
connection.close();
}
}
}
If you run your code it's likely that database connection will be closed before insert statement execution and of course insert will be unsuccessful.
Proper solutions
If you have multiple insert tasks:
Use ExecutorService instead of Execuutor
Submit all tasks
Invoke executorService.shutdown() it will wait until all submitted tasks are done.
Close connection
If you have only one task to submit:
You should close the connection after Connection.InsertToTables(table X, table Y) in your task.
Good for both scenarios and recommended:
Each tasks has it own connection.
Example:
class Thread1 {
private static DataSource dataSource; // initialize it
public static void main(String[] args){
ExecutorService threadPool = Executors.newFixedThreadPool(1);
threadPool.submit(new MyRunnable(dataSource));
}
}
class MyRunnable implements Runnable {
private final DataSource dataSource;
MyRunnable(DataSource dataSource) {
this.dataSource = dataSource;
}
public void run() {
Connection connection = dataSource.getConnection();
// do something with connection
connection.close();
}
}
class Thread1{
DataBaseConnection connection;
main(){
threadPool = Executors.newFixedThreadPool(1);
connection.open();
if(ThisMightTakeSomeTime)
Future f = threadPool.submit(new MyRunnable(connection));
InsertDataToDataBase(Table A, Table B));
f.get(); // this will hold the program until the Thread finishes.
connection.Close(); //What if thread2 isn't done yet?
}
}
the Future is the reference resulting from the submit call. if we call Future.get(), that will block the current thread until the submited thread finishes.
Related
This question already has answers here:
What is the difference between Thread.start() and Thread.run()?
(9 answers)
Closed 6 years ago.
I was told to run all of my MySQL connection processes on another thread besides the main thread to avoid the main thread from being stuck on a process that takes a few seconds to process.
Therefore, I established a ConnectionPool in a separate thread, so that my GUI launches independently from the establishment of the connection. However, this is not the case. When I run the program, it waits until the connection is established and then it actually runs launch(args); My concern is why is it not running independently when a new thread is being established?
public static void main(String[] args) {
initiateConnection();
launch(args);
}
private static void initiateConnection() {
new Thread(() -> {
try {
connection = new ConnectionPool("jdbc:mysql://127.0.0.0/comm", "root",
"pass");
} catch (Exception e) {
}
}).run();
}
From Javadocs of Thread.run()
If this thread was constructed using a separate Runnable run object, then that Runnable object's run method is called; otherwise, this method does nothing and returns.
So, only the body of lambda expression (which is actually body of the method run of java.lang.Runnable ) is called. It is equivalent to:
(new Runnable() {
#Override
public void run() {
try {
connection = new ConnectionPool("jdbc:mysql://127.0.0.0/comm", "root",
"pass");
} catch (Exception e) {
}
}
}).run();
You should call .start() instead of .run(). The JVM will call .run on your lambda for you.
Many JDBC calls (querying DB and get results) are executed through ExecutorService. I found that when those calls are executed, JDBC connections gets a long time to get closed the connection even though those connections are closed correctly. Why I say so is, when a load test is run through JMeter, the database shows that many connections are in IDLE in transaction. If the number of thread which run the test is high, the number of connections in Idle in transactions goes up. If the test is run slowly, then connections get closed slowly (1, 2 minutes), that means there are connections in IDLE in transactions, but after few minutes they become IDLE. I use connection pool here too. If I run the JDBC querying functions as a sequence ( one after another), then database doesn't show any connections in IDLE in transactions. Below is how I run my runnable tasks which run JDBC queries. TaskManager class handles whole ExecutorService related functions.
public class TaskManager {
final private ThreadServiceFactory threadFactory;
private int concurrentThreadCount;
private ExecutorService executerSV;
private final CountDownLatch latch;
// I keep a count of proposed tas task as servicecount
public TaskManager(int serviceCount) {
threadFactory = new ThreadServiceFactory();
this.concurrentThreadCount = serviceCount;
latch = new CountDownLatch(serviceCount);
}
public void execute( ThreadService runnableTask) {
Object rv = null;
runnableTask.setCountDownLatch(latch);
if(executerSV == null) {
executerSV = Executors.newFixedThreadPool(this.concurrentThreadCount, getThreadFactory());
}
executerSV.execute(runnableTask);
}
public boolean holdUntilComplete(){
try {
latch.await();
executerSV.shutdown();
return true;
} catch (InterruptedException e) {
e.printStackTrace();
return false;
}
}
private ThreadServiceFactory getThreadFactory(){
threadFactory.setDeamon( Boolean.FALSE);
return threadFactory;
}
}
In my test class ;
public void test(){
TaskManager tm = new TaskManager(3);
tm.execute(queryTask1);
tm.execute(queryTask2);
tm.holdUntilComplete();
}
queryTask1 is a Runnable and it calls JDBC select query.
If I run, queryTask1.run(); queryTask2.run(); then there are no any IDLE in connections in DB.
I use java 7. Please any one can let me know where the problem is.
There is no code in your question that opens any connection to a database. As such, it is difficult to suggest an answer. However, since you state that you are using a connection pool, you should better look at the pool configuration parameters, since they dictate how long an idle connection may be open before being elegible for eviction. For instance, if you are running a connection pool in tomcat, you should look particularly at "minIdle", "maxIdle" and "minEvictableIdleTimeMillis" properties. See https://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html
I am developing an application in Enterprise JavaBeans 3.1 and I receive data from a Socket. This application acts as a Listener and once data is received it is processed. This application was single threaded and due to it processing it slowly, the application is implemented using Threads which is now a multi threaded application. By doing this, the application now runs much faster.
However, there are two threads and both threads access the database to insert and update the database. I face the problem of concurrency where one thread inserts and the other updates causing problems. To deal with concurrency, I added a synchronized block to lock an object making sure the full block is executed. By doing this application is now very slow as it was with a single threaded application. The insert and update is done through JDBC.
Is there anything else that can be done so it is processed and processed very quickly without slowing down the application. The below is sample code:
#Startup
#Singleton
public class Listener {
private ServerSocket serverSocket;
private Socket socket;
private Object object;
private InetAddress server;
#Resource
private ScheduledExecutorService executor;
#PostConstruct
public void init() {
object = new Object();
serverSocket = new ServerSocket("somePortNumber");
Runnable runnable = new Runnable() {
public void run() {
checkDatabase();
if(!isServerActive()) {
// send e-mail
listen();
}
else {
listen();
}
}
};
executor.scheduleAtFixedRate(runnable, 0, 0, TimeUnit.SECONDS);
}
public void listen() {
if(socket == null) {
socket = serverSocket.accept();
}
else if(socket.isClosed()) {
socket = serverSocket.accept();
}
startThread(socket);
}
public void startThread(Socket socket) {
Runnable runnable = new Runnable() {
public void run() {
processMessage(socket);
}
};
new Thread(runnable).start();
}
public void processMessage(Socket socket) {
synchronized(object) {
// build data from Socket
// insert into database message, sentDate
// do other things
// update processDate
}
}
public void checkDatabase() {
synchronized(object) {
// get data and further update
}
}
public boolean isServerActive() {
boolean isActive = true;
if(server == null) {
sever = InetAddress.getByName("serverName");
}
if(!server.isNotReachable(5000)) {
isActive = false;
if(socket != null) {
socket.close();
}
}
return isActive;
}
}
EDIT:
Table name: Audit
Message: VARCHAR NOT NULL
SentDate: DATE NOT NULL
ProcessedDate: DATE
AnotherDate: DATE
Query: INSERT INTO AUDIT (message, sentDate, processedDate, receivedDate) VALUES (?, java.sql.Timestamp, null, null)
Assuming a record is inserted without the synchronized block inserting the message and sentDate. The other thread will execute causing this record to be found and further update. The problem is that after the initial insert and processedDate should be updated and then the other thread should be executed.
The processMessage() sends the data over HTTPS asynchronously.
One of the reasons to use Threads was because only one piece of data came to Java. So by introducing threads the full set of data comes to Java.
Even with single thread you can get much better speed by using JDBC batching and running any transactions around the batch instead of committing every individual insert/update statement.
In a multi threaded environment you can avoid concurrency problems if you ensure no two threads act on the same database row at the same time. You can use row level locks to avoid multiple threads updating the same row.
It is not possible to give you any more inputs with the information you have given. You may get more ideas if you provide information about the data you are processing.
The application behaved as single threaded because the processMessage & checkDatabase methods have synchronised block on the same class object , the threads that are listening currently will hold the lock and other threads will have to wait until the message is processed,which will cause the application to slow down. instead of putting synchronised in two separate blocks create separate threads outside of the class that checks this condition and try to invoke then separately based on a condition or you could try with wait() and notifyAll in your synchronized blocks also.
I have a class which is a listener for incoming messages and should be alive forever (So that it can listen for incoming messages) until i explicitly disconnect the connection for it. I have declared the thread as setDaemon(false) but it terminates with the calling methods termination.
Please tell me how to keep that thread alive and also please throw some light on how to implement the Spring TaskExecutor to achieve same.
Thanks in advance.
it is a listener it gets notified when someone sends message... so how do i keep it running ?
The Listener Class
public class MyListnerImpl implements Listener {
private final connectionImpl con;
public MyListnerImpl(ConnectionImpl con) {
if (con.isAuthenticated() && con.isConnected()) {
if (logger.isInfoEnabled()) {
logger.info("Initializing XmppListner:");
}
this.itsCon = con;
Thread t1 = new Thread(this);
t1.setDaemon(false);
t1.start();
}
}
public final void listnerInterfaceMethod(final Chat chat, final Message message) {
System.out.println("Message" + message);
}
public final void run() {
itsCon.getChatManager().addChatListener(new ChatManagerListener() {
public void chatCreated(final Chat chat, final boolean createdLocally) {
if (!createdLocally) {
chat.addMessageListener(itsFbml);
}
}
});
}
}
Calling class simply creates its object and thread gets started by the Listeners constructor.
I want to keep this thread created run until i interrupt it.
There are a few things you could do that would be better than hanging the initial thread forever:
Use otherThread.join(). This will cause the current thread you are running in to sleep until the other thread has finished executing.
As #nanda suggests, use ExcecutorService.shutdown() to wait until a pool of threads has finished.
Use otherThread.setDaemon(false) and simply let your initial thread exit. This will set your new threads as user threads. Java will not shut down until the only threads running are daemon threads.
synchronized(this) {
while (true) {
this.wait();
}
}
This will make the current thread wait on the monitor of the current class until someone calls notify(), or forever.
copied from How do you hang a thread in Java in one line?
A thread says alive until run() returns (or throw an error/exception) If you want to keep it alive, use a loop, don't return and catch any error/exception.
This is how i solved the problems that time,
So this case was not of multi threading , had just a single thread which needed to run for ever,
So Inserted
public final void run() {
while(true)
{
//Run Method Logic...
}
}
And instantiated it from a spring bean.
I was also looking at more fancy things for this single threaded scenario like awaitTermination(); or something like that.
I wrote an application that runs some threads using ExecutorService and waits until they finish like this:
ExecutorService exService;
exService = Executors.newCachedThreadPool();
exService.execute(T1);
exService.execute(T2);
exService.shutdown();
boolean finshed = exService.awaitTermination(5, TimeUnit.MINUTES);
sometimes I need to cancel the execution of these threads (The entire ExecutorService).
I tried exService.shutdownNow() but it throws java.lang.InterruptedException and doesn't cancel threads.
How can I cancel execution of these threads?
EDIT: T1 class code added as nanda's request
public class TC implements Runnable{
private ExtractedDataBuffer Buffer;
private Scraper scraper;
private String AppPath;
private boolean Succeed=false;
private Map<String,Object> Result=null;
private JLabel StatElement;
public TC(ExtractedDataBuffer Buffer,String AppPath,String XMLfile,JLabel Stat) throws FileNotFoundException {
this.Buffer = Buffer;
this.AppPath=AppPath;
this.StatElement=Stat;
ScraperConfiguration config;
config = new ScraperConfiguration(AppPath + Main.XMLFilesPath +XMLfile);
scraper = new Scraper(config, AppPath);
}
private void extract(){
try{
mainF.SetIconStat("working", this.StatElement);
scraper.execute();
if(scraper.getStatus()==Scraper.STATUS_FINISHED){
this.Succeed=true;
Map<String,Object> tmp=new HashMap<String,Object>();
tmp.put("UpdateTime", ((Variable) scraper.getContext().get("UpdateTime")).toString().trim());
Buffer.setVal(this.Result);
mainF.SetIconStat("done", this.StatElement);
}else{
this.Succeed=false;
this.Result=null;
Buffer.setVal(null);
mainF.SetIconStat("error", this.StatElement);
}
}catch(Exception ex){
this.Succeed=false;
this.Result=null;
Buffer.setVal(null);
mainF.SetIconStat("error", this.StatElement);
}
}
public void run() {
this.extract();
}
}
If you change shutdown() to shutdownNow(), you are doing the correct thing in the code that you wrote. But then, check the documentation of shutdownNow():
There are no guarantees beyond best-effort attempts to stop processing
actively executing tasks. For example, typical implementations will
cancel via Thread.interrupt(), so any task that fails to respond to
interrupts may never terminate.
So probably your T1 and T2 are not coded in a correct way and don't respond to the interrupt well enough. Can you maybe copy the code for them?
--
Based on your code, I guess the code that takes long time is scraper.execute(), right? So inside those method, you have to constantly check something like this:
if (Thread.interrupted()) {
throw new InterruptedException();
}
If the interrupt come, the InterruptedException will be thrown and catched in your catch statement and the thread will stop.
My problem with Future#cancel() is that subsequent calls to get() throw a CancellationException. Sometimes I still want be able to call get() on the Future in order to retrieve a partial result after the shutdown of the executor service. In that case one can implement the termination logic in the callables that you submit to the executor service. Use a field like
private volatile boolean terminate;
public void terminate() {
terminate = true;
}
and check for terminate in the callable as often as required. In addition, you have to remember your callables somewhere so you can call terminate() on all of them.
Use the Executor.submit method, it extends base method Executor.execute(java.lang.Runnable) by creating and returning a Future that can be used to cancel execution and/or wait for completion:
task = Executor.submit( T1 )
...
task.cancel( true )