I am working on a practical scenario related with Java;a socket program. The existing system and the expected system are as follows.
Existing System - The system checks that a certain condition is satisfied. If so It will create some message to be sent and put it into a queue.
The queue processor is a separate thread. It periodically check the queue for existence of items in it. If found any items (messages) it just sends the message to a remote host (hardcoded) and remove the item from queue.
Expected System - This is something like that. The message is created when a certain condition is satisfied but in every case the recipient is not same. So there are many approaches.
putting the message into the same queue but with its receiver ID. In this case the 2nd thread can identify the receiver so the message can be sent to that.
Having multiple threads. In this case when the condition is satisfied and if the receiver in "New" it creates a new queue and put the message into that queue. And a new thread initializes to process that queue. If the next messages are directed to same recipient it should put to the same queue and if not a new queue and the thread should be created.
Now I want to implement the 2nd one, bit stucked. How should I do that? A skeleton would be sufficient and you won't need to worry to put how to create queues etc... :)
Update : I also think that the approach 1 is the best way to do that. I read some articles on threading and came to that decision. But it is really worth to learn how to implement the approach 2 as well.
Consider using Java Message Services (JMS) rather than re-inventing the wheel?
Can I suggest that you look at BlockingQueue ? Your dispatch process can write to this queue (put), and clients can take or peek in a threadsafe manner. So you don't need to write the queue implementation at all.
If you have one queue containing different message types, then you will need to implement some peek-type mechanism for each client (i.e. they will have to check the head of the queue and only take what is theirs). To work effectively then consumers will have to extract data required for them in a timely and robust fashion.
If you have one queue/thread per message/consumer type, then that's going to be easier/more reliable.
Your client implementation will simply have to loop on:
while (!done) {
Object item = queue.take();
// process item
}
Note that the queue can make use of generics, and take() is blocking.
Of course, with multiple consumers taking messages of different types, you may want to consider a space-based architecture. This won't have queue (FIFO) characteristics, but will allow you multiple consumers in a very easy fashion.
You have to weigh up slightly whether you have lots of end machines and occasional messages to each, or a few end machines and frequent messages to each.
If you have lots of end machines, then literally having one thread per end machine sounds a bit over the top unless you're really going to be constantly streaming messages to all of those machines. I would suggest having a pool of threads which will only grow between certain bounds. To do this, you could use a ThreadPoolExecutor. When you need to post a message, you actually submit a runnable to the executor which will send the message:
Executor msgExec = new ThreadPoolExecutor(...);
public void sendMessage(final String machineId, byte[] message) {
msgExec.execute(new Runnable() {
public void run() {
sendMessageNow(machineId, message);
}
});
}
private void sendMessageNow(String machineId, byte[] message) {
// open connection to machine and send message, thinking
// about the case of two simultaneous messages to a machine,
// and whether you want to cache connections.
}
If you just have a few end machines, then you could have a BlockingQueue per machine, and a thread per blocking queue sitting waiting for the next message. In this case, the pattern is more like this (beware untested off-top-of-head Sunday morning code):
ConcurrentHashMap<String,BockingQueue> queuePerMachine;
public void sendMessage(String machineId, byte[] message) {
BockingQueue<Message> q = queuePerMachine.get(machineId);
if (q == null) {
q = new BockingQueue<Message>();
BockingQueue<Message> prev = queuePerMachine.putIfAbsent(machineId, q);
if (prev != null) {
q = prev;
} else {
(new QueueProessor(q)).start();
}
}
q.put(new Message(message));
}
private class QueueProessor extends Thread {
private final BockingQueue<Message> q;
QueueProessor(BockingQueue<Message> q) {
this.q = q;
}
public void run() {
Socket s = null;
for (;;) {
boolean needTimeOut = (s != null);
Message m = needTimeOut ?
q.poll(60000, TimeUnit.MILLISECOND) :
q.take();
if (m == null) {
if (s != null)
// close s and null
} else {
if (s == null) {
// open s
}
// send message down s
}
}
// add appropriate error handling and finally
}
}
In this case, we close the connection if no message for that machine arrives within 60 seconds.
Should you use JMS instead? Well, you have to weigh up whether this sounds complicated to you. My personal feeling is it isn't a complicated enough a task to warrant a special framework. But I'm sure opinions differ.
P.S. In reality, now I look at this, you'd probably put the queue inside the thread object and just map machine ID -> thread object. Anyway, you get the idea.
You might try using SomnifugiJMS, an in-vm JMS implementation using java.util.concurrent as the actual "engine" of sorts.
It will probably be somewhat overkill for your purposes, but may well enable your application to be distributed for little to no additional programming (if applicable), you just plug in a different JMS implementation like ActiveMQ and you're done.
First of all, if you are planning to have a lot of receivers, I would not use the ONE-THREAD-AND-QUEUE-PER-RECEIVER approach. You could end up with a lot of threads not doing anything most of the time and I could hurt you performance wide. An alternative is using a thread pool of worker threads, just picking tasks from a shared queue, each task with its own receiver ID, and perhaps, a shared dictionary with socket connections to each receiver for the working threads to use.
Having said so, if you still want to pursue your approach what you could do is:
1) Create a new class to handle your new thread execution:
public class Worker implements Runnable {
private Queue<String> myQueue = new Queue<String>();
public void run()
{
while (true) {
string messageToProcess = null;
synchronized (myQueue) {
if (!myQueue.empty()) {
// get your data from queue
messageToProcess = myQueue.pop();
}
}
if (messageToProcess != null) {
// do your stuff
}
Thread.sleep(500); // to avoid spinning
}
}
public void queueMessage(String message)
{
synchronized(myQueue) {
myQueue.add(message);
}
}
}
2) On your main thread, create the messages and use a dictionary (hash table) to see if the receiver's threads is already created. If is is, the just queue the new message. If not, create a new thread, put it in the hashtable and queue the new message:
while (true) {
String msg = getNewCreatedMessage(); // you get your messages from here
int id = getNewCreatedMessageId(); // you get your rec's id from here
Worker w = myHash(id);
if (w == null) { // create new Worker thread
w = new Worker();
new Thread(w).start();
}
w.queueMessage(msg);
}
Good luck.
Edit: you can improve this solution by using BlockingQueue Brian mentioned with this approach.
Related
I have a producer-consumer model using a blocking queue where 4 threads read files from a directory puts it to the blocking queue and 4 threads(consumer) reads from blocking queue.
My problem is every time only one consumer reads from the Blockingqueue and the other 3 consumer threads are not reading:
final BlockingQueue<byte[]> queue = new LinkedBlockingQueue<>(QUEUE_SIZE);
CompletableFuture<Void> completableFutureProducer = produceUrls(files, queue, checker);
//not providing code for produceData , it is working file with all 4 //threads writing to Blocking queue. Here is the consumer code.
private CompletableFuture<Validator> consumeData(
final Response checker,
final CompletableFuture<Void> urls
) {
return CompletableFuture.supplyAsync(checker, 4)
.whenComplete((result, err) -> {
if (err != null) {
LOG.error("consuming url worker failed!", err);
urls.cancel(true);
}
});
}
completableFutureProducer.join();
completableFutureConsumer.join();
This is my code. Can someone tell me what I am doing wrong? Or help with correct code.
Why is one consumer reading from the Blocking queue.
Adding code for Response class reading from Blocking queue :
#Slf4j
public final class Response implements Supplier<Check> {
private final BlockingQueue<byte[]> data;
private final AtomicBoolean producersComplete;
private final Calendar calendar = Calendar.getInstance();
public ResponseCode(
final BlockingQueue<byte[]> data
) {
this.data = data;
producersDone = new AtomicBoolean();
}
public void notifyProducersDone() {
producersComplete.set(true);
}
#Override
public Check get() {
try {
Check check = null;
try {
while (!data.isEmpty() || !producersDone.get()) {
final byte[] item = data.poll(1, TimeUnit.SECONDS);
if (item != null) {
LOG.info("{}",new String(item));
// I see only one thread printing result here .
validator = validateData(item);
}
}
} catch (InterruptedException | IOException e) {
Thread.currentThread().interrupt();
throw new WriteException("Exception occurred while data validation", e);
}
return check;
} finally {
LOG.info("Done reading data from BlockingQueue");
}
}
}
It's hard to diagnose from this alone, but it's probably not correct to check for data.isEmpty() because the queue may happen to be temporarily empty (but later get items). So your threads might exit as soon as they encounter a temporarily empty queue.
Instead, you can exit if producers were done AND you got an empty result from the poll. That way the threads only exit when there are truly no more items to process.
It's a bit odd though that you are returning the result of the last item (alone). Are you sure this is what you want?
EDIT: I've done something very similar recently. Here is a class that reads from a file, transforms the lines in a multi-threaded way, then writes to a different file (the order of lines are preserved).
It also uses a BlockingQueue. It's very similar to your code, but it doesn't check for quue.isEmpty() for the aforementioned reason. It works fine for me.
4+4 threads is not that many, so you better do not use asynchronous tools like CompletableFuture. Simple multithreaded program would be simpler and work faster.
Having
BlockingQueue<byte[]> data;
don't use data.poll();
use data.take();
When you have lets say 1 item in the queue, and 4 consumers, one of them will poll the item rendering queue to be empty. Then 3 of the rest of the consumers checks if queue.isEmpty(), and since it is - quits the loop.
Producer-Consumer blog post states that:
"2) Producer doesn't need to know about who is consumer or how many consumers are there. Same is true with Consumer."
My problem is that I have an array of data that I need to get from the Webserver to clients as soon as possible. The clients can appear mid-calculation. Multiple clients at different times can request the array of data. Once the calculation is complete it is cached and then it can simply be read.
Exmaple Use Case: While the calculation is occurring I want to serve each and every datum of the array as soon as possible. I can't use a BlockingQueue because say if a second client starts to request the array while the first one has already used .take() on the first half of the array. Then the second client missed half the data! I need a BlockingQueue where you don't have to take(), but you could instead just read(int index).
Solution? I have a good amount of writes on my array, so I wouldn't want to use CopyOnWriteArrayList? The Vector class should work but would be inefficient?
Is it preferable to use a ThreadSafeList like this and just add a waitForElement() function? I just don't want to reinvent the wheel and I prefer crowd tested solutions for multi-threaded problems...
As far as I understand you need to broadcast data to subscribers/clients.
Here are some ways that I know for approaching it.
Pure Java solution, every client has a BlockingQueue and every time you broadcast a message you put it every queue.
for(BlockingQueue client: clients){
client.put(msg);
}
RxJava provides a reactive approach. Clients will be subscribers and ever time you emit a message, subscribers will be notified and they can choose to cancel their subscription
Observable<String> observable = Observable.create(sub->{
String[] msgs = {"msg1","msg2","msg3"};
for (String msg : msgs) {
if(!sub.isUnsubscribed()){
sub.onNext(msg);
}
}
if (!sub.isUnsubscribed()) { // completes
sub.onCompleted();
}
});
Now multiple subscribers can choose to receive messages.
observable.subscribe(System.out::println);
observable.subscribe(System.out::println);
Observables are a bit functional, they can choose what they need.
observable.filter(msg-> msg.equals("msg2")).map(String::length)
.subscribe(msgLength->{
System.out.println(msgLength); // or do something useful
});
Akka provides broadcast routers
This is not exactly a trivial problem; but not too hard to solve either.
Assuming your producer is an imperative program; it generates data chunk by chunk, adding each chunk to the cache; the process terminates either successfully or with an error.
The cache should have this interface for the produce to push data in it
public class Cache
public void add(byte[] bytes)
public void finish(boolean error)
Each consumer obtains a new view from the cache; the view is a blocking data source
public class Cache
public View newView()
public class View
// return null for EOF
public byte[] read() throws Exception
Here's a straightforward implementation
public class Cache
{
final Object lock = new Object();
int state = INIT;
static final int INIT=0, DONE=1, ERROR=2;
ArrayList<byte[]> list = new ArrayList<>();
public void add(byte[] bytes)
{
synchronized (lock)
{
list.add(bytes);
lock.notifyAll();
}
}
public void finish(boolean error)
{
synchronized (lock)
{
state = error? ERROR : DONE;
lock.notifyAll();
}
}
public View newView()
{
return new View();
}
public class View
{
int index;
// return null for EOF
public byte[] read() throws Exception
{
synchronized (lock)
{
while(state==INIT && index==list.size())
lock.wait();
if(state==ERROR)
throw new Exception();
if(index<list.size())
return list.get(index++);
assert state==DONE && index==list.size();
return null;
}
}
}
}
It can be optimized a little; most importantly, after state=DONE, consumers should not need synchronized; a simple volatile read is enough, which can be achieved by a volatile state
I am building a server that sends data via a single TCP socket for each user every 2 seconds and on a separate thread. There are also special events occasionally sent along side with the regular data. Sometimes, data in multiple packets would mix up so I created a queue to make sure it does not happen. However, the issue is still there, is my approach not correct or is there something wrong with my code?
protected void sendData (byte[] data) {
if (isSendingData) {
dataQueue.add(data);
return;
}
isSendingData = true;
Thread sendThread = new Thread() {
public void run () {
try {
BufferedOutputStream outStream = new BufferedOutputStream(connectionSocket.getOutputStream());
outStream.write(data);
outStream.flush();
// check queue, if there are data, send
byte[] moreData = null;
if (dataQueue.size() > 0) {
moreData = dataQueue.remove(0);
}
isSendingData = false;
if (moreData != null) {
sendData(moreData);
}
}
catch (Exception e) {
System.out.println ("Error sending data to peripheral: " + e);
isSendingData = false;
}
}
};
sendThread.start ();
}
The proper idiom to remove concurrency issues using a queue is to have a long-lived thread run an infinite loop which takes elements from the queue and processes them. Typically you'll use a blocking queue so that on each iteration the thread goes to sleep until there is an item to process.
Your solution deviates from the above in many aspects. For example:
if (isSendingData) {
dataQueue.add(data);
return;
}
isSendingData = true;
—if this method is called concurrently, this will result in a race condition: both threads can read isSendingData as false, then concurrently proceed to sending data over the network. If isSendingData isn't volatile, you've also got a data race on it (entirely separate from the race condition explained above).
if (dataQueue.size() > 0) {
moreData = dataQueue.remove(0);
}
—this is another race condition: after you read size as zero, the other thread can add an item to the queue. Now that item will possibly never be processed. It will linger in the queue until another such thread is started.
The more obvious way your solution is not correct is that the thread you start has no loops and is intended to just process one message, plus possibly one extra message in the queue. This should be reworked so that there are no special cases and sendData always, unconditionally, submits to a queue and never does any sending on its own.
I would do this completely differently. You don't want arbitrarily long queues in your application.
Have your hearbeat thread synchronize on the socket when sending the heartbeat.
Don't have it sending anything else.
Get rid of the queue, isSendingData, etc.
Have your main application synchronize on the socket when it wants to send, and just send whenever it needs to.
Use the same BufferedOutputStream or BufferedWriter for all sending, and flush it after each send.
I have a priority queue implemented with a ConcurrentSkipListMap, using 16 different priorities.
class ConcurrentPriorityQueue {
ConcurrentSkipListMap<Long, Message> queue = new ConcurrentSkipListMap<>();
AtomicLong counter16 = new AtomicLong(Long.MAX_VALUE);
AtomicLong counter15 = new AtomicLong(Long.MAX_VALUE / 8 * 7);
AtomicLong counter14 = new AtomicLong(Long.MAX_VALUE / 4 * 3);
// etc
AtomicLong counter1 = new AtomicLong(Long.MIN_VALUE / 8 * 7);
void addPriority16(Message message) {
queue.put(counter16.getAndDecrement(), message);
}
void addPriority15(Message message) {
queue.put(counter15.getAndDecrement(), message);
}
// and so on
}
This isn't exactly how the class is organized (e.g. I've got the AtomicLongs in an array), but I figured this code would be clearer. There's also a DelayQueue to remove old messages or to raise the priority on old messages (depending on the message type).
My problem is that I've got several consumers that are using pollLastEntry() in order to remove the highest priority message from the queue and then sleeping if the queue is empty, but the thing is that queue activity goes in bursts - it will go an hour without containing more than a few messages, and then in the next hour it will never empty. As such I'd like to use a blocking method to remove messages from the queue so that I don't waste resources on repeatedly sleeping threads (I'd use exponential backoff to make them sleep longer when there's less activity, but this would make them unresponsive when the queue starts up again), but am unclear as to the best way to go about implementing this - I have plenty of experience with using blocking queues, but zero experience with implementing them. My first thought was to implement exponential backoff in the sleeping consumers and then interrupt them when queue activity starts up again, but I first wanted to see if there was a better way to do this.
I would implement a wrapper
class Wrapper implements Comparable<Wrapper> {
long priority;
Message message;
Wrapper(long priority, Message message) {
this.priority = priority;
this.message = message;
}
#Override
public int compareTo(Wrapper w) {
return Long.compare(priority, w.priority);
}
}
and used PriorityBlockingQueue instead of ConcurrentSkipListMap
I'm coding a simple Bank simulator where users would login from different locations at once, using sockets. In the Bank server I keep a bounded buffer to store every incoming request, ex: transfer funds, get account balance etc and there's a Background Thread running at Server end (Buffer Reader) to pull out each request from this Request Queue (assume it works as a Thread Scheduler in OS), in FCFS basis.
I have made buffer's put() and get() methods to have conditional synchronization.
ex:
// put method
while(total_buffer_size == current_total_requests) {
System.out.println("Buffer is full");
wait();
}
So my question is, do we have to synchronize methods like get-balance or transfer-funds to avoid corruption of data? I believe it is not necessary since the Buffer Reader takes each request one-by-one and relevant action. Have I avoided any deadlock situations through this? What do you think? Thanks
EDIT2:
public synchronized boolean put(Messenger msg, Thread t, Socket s) throws InterruptedException {
while(total_buffer_size == current_total_requests) {
System.out.println("Buffer is full");
wait();
}
current_total_requests++;
requests[cur_req_in] = new Request(msg, s); // insert into Queue
cur_req_in = (cur_req_in + 1) % total_buffer_size ;
notifyAll();
return true;
}
// take each incoming message in queue. FIFO rule followed
public synchronized Request get() throws InterruptedException {
while(current_total_requests==0) wait();
Request out = requests[cur_req_out];
requests[cur_req_out] = null;
cur_req_out = (cur_req_out + 1) % total_buffer_size ;
current_total_requests--;
notifyAll(); //wake all waiting threads to continue put()
return out;
}
If there is only one consumer (i.e. one thread that consumes the requests from the "buffer") , then you don't need to use any synchronization on the methods relating to the bank account. However, I don't believe that your current implementation of a "bounded buffer" is valid. To be more specific:
while(total_buffer_size == current_total_requests) {
System.out.println("Buffer is full");
wait();
}
There is absolutely no guarantee how many threads will get past the while loop, perform a context switch just before current_total_requests is incremented and queue more requests than what's allowed the buffer size. Unless your put method is synchronized, this approach will be extremely unreliable and prone to race conditions.
If you want a bounded buffer, then just use one of Java's already existing "bounded buffers" or more specifically: the BlockingQueue. The BlockingQueue blocks on put(...):
Inserts the specified element into this queue, waiting if necessary for space to become available.
It also blocks on take() if there is no data in the queue. I don't know if you can use one of the items in the concurrency library, but if you can't then you have to fix your BoundedBuffer.