I have this really simple JCSP(Java Communicating Sequential Processes) code sample in which I'm trying to write an integer to a One2OneInt channel and then read it.
package jcsp;
import org.jcsp.lang.*;
public class JCSP {
public static void main(String[] args) {
One2OneChannelInt chan = Channel.one2oneInt();
chan.out().write(5);
System.out.println("Written...");
System.out.println(chan.in().read());
}
}
It seems that value never gets written on the channel and program just keeps running. "Written..." is never printed out.
So I learned about BlockingQueue and its implementation SynchronousQueue. As stated here, SynchronousQueue works in similar way in which CSP Channels work. This helped me realize what was wrong with my code. Simply put, you can't write and read from channel in same process. Channel is way for processes to communicate.
Similarly to SynchronousQueue's put() which will wait for other process to call take(), CSP Channel's write() which will wait for corresponding read() to be called. The difference is that CSP Channels have objects ChannelOutput and ChannelInput through which objects are written and red. Conversely, you can call put and take directly on instance of SynchronousQueue. Personally, I find SynchronousQueue much easier to understand, which probably relates to JCSP not being very popular.
Still, if you're interested how I made the above code work in JCSP, here it is:
public static class Process1 implements CSProcess {
private ChannelOutputInt output;
public Process1(ChannelOutputInt out) {
output = out;
}
#Override
public void run() {
for (int i = 0; i < 1; i++) {
System.out.println("Written...");
output.write(5);
}
output.write(-1);
}
}
public static class Process2 implements CSProcess {
private ChannelInputInt input;
public Process2(ChannelInputInt in) {
input = in;
}
#Override
public void run() {
int x = 0;
while ((x = input.read()) > 0) {
System.out.println(x);
}
}
}
public static void main(String[] args) {
One2OneChannelInt chan = Channel.one2oneInt();
Process1 process1 = new Process1(chan.out());
Process2 process2 = new Process2(chan.in());
Parallel parallel = new Parallel();
parallel.addProcess(process1);
parallel.addProcess(process2);
parallel.run();
}
The problem is the channel is unbuffered, so your write() call will block until another process reads from the channel. As soon as another process calls read(), 'Written...' will be printed out.
A BlockingQueue with a capacity of 0 behaves similarly to a JCSP channel
Related
I have producers that takes data A ,produce data B and send it
public interface Producer<T>{
void produce(T data);
void flush();
}
public class DataBaseProducer implements Producer<String>{
List<String> producedData = new ArrayList<>();
// create data
public void produce(String data){
producedData.add(transformData(data));
}
// send created data
public void flush(){
sendDataToDatabase(producedData);
}
}
public class MessageProducer implements Producer<String>{
public void produce(String data){
String line =transformData(data)
sendDataToMessageQueue(line);
}
public void flush(){
}
}
public static void main(String[] args) {
// get producer
Producer producer = getProducer(producerName)
BufferedReader reader;
try {
reader = new BufferedReader(new FileReader(file..));
String line = reader.readLine();
while (line != null) {
producer.produce(line)
line = reader.readLine();
}
reader.close();
producer.flush()
} catch (IOException e) {
e.printStackTrace();
}
}
To demonstrate my question, imagine i have producers like above. One loads all data, and then bulk send it at once after it done, and second sends data right away ( so it does not bulk send data, but whenever new data is createdit sends it right away)
Most of the producers will send the data after every data is loaded, but few of them will send them right away. If the producer sends data right away, than the flush() method remains empty. This however seems like bad pratcise and may violates some OOP principles. What is the correct way to implement this?
Thanks for answers!
In my opinion, and after checking the main() code, I think that MessageProducer is not really a Producer because the method produce actually produces and flushes at the same time.
But since it seems you need to treat what you get from getProducer(producerName) the same way independently, then what you may do is something like this:
public class MessageProducer implements Producer<String> {
private boolean flushNeeded;
private String line;
public void produce(String data){
line =transformData(data)
flushNeeded = true;
flush();
}
public void flush(){
if (flushNeeded) {
sendDataToMessageQueue(line);
flushNeeded = false;
}
}
}
But in my humble opinion, it doesn't really make any significant different with leaving the flush() implementation empty, if not simply to make the code a bit clearer to readers (reading an empty flush() may in fact induce in error thinking your implementation never flushes, while it does it but just into the produce method).
Is it possible in java to create a thread that will always work in the background? The problem is that the application instance sometimes crashes with an OutOfMemoryException. Therefore, several instances are launched in parallel. Each instance does some work: it saves something to the database at the request of the user. And the stream, which should work constantly, will look into the database and somehow process the information from it.
Most likely, the sheduler will not work, since the thread must be running constantly and wait for a signal to start working.
First of all, I suggest you investigate and resolve the OutOfMemoryException because it better to avoid these cases. You can instanziate a thread that wait for a request, execute a request and then return to wait for another request. The implementation is like this for thread:
/** Squares integers. */
public class Squarer {
private final BlockingQueue<Integer> in;
private final BlockingQueue<SquareResult> out;
public Squarer(BlockingQueue<Integer> requests,
BlockingQueue<SquareResult> replies) {
this.in = requests;
this.out = replies;
}
public void start() {
new Thread(new Runnable() {
public void run() {
while (true) {
try {
// block until a request arrives
int x = in.take();
// compute the answer and send it back
int y = x * x;
out.put(new SquareResult(x, y));
} catch (InterruptedException ie) {
ie.printStackTrace();
}
}
}
}).start();
}
}
And for the caller method:
public static void main(String[] args) {
BlockingQueue<Integer> requests = new LinkedBlockingQueue<>();
BlockingQueue<SquareResult> replies = new LinkedBlockingQueue<>();
Squarer squarer = new Squarer(requests, replies);
squarer.start();
try {
// make a request
requests.put(42);
// ... maybe do something concurrently ...
// read the reply
System.out.println(replies.take());
} catch (InterruptedException ie) {
ie.printStackTrace();
}
}
To more information, you can start to read the post that I found here to provide you the example.
You basically need an infinitely running thread with some control.
I found this answer to be the simplest and it does what you need.
https://stackoverflow.com/a/2854890/11226302
I have a problem with a Java multithreaded file crawler that I am making. My issue is that I have a workQueue that is a linkedBlockingQueue that contains the names of files that I would like to crawl over with my threads, each thread will take() from the workQueue and while scanning through the file it may put() another file name into the workQueue (It is a dependency checker program). So I am never really certain when the work is all finished and all threads will eventually enter a waiting state when they try to take() from the (eventually) empty workQueue.
So I guess my question is, is there an efficient way to terminate all of threads once all of the work is finished (when all of threads have entered a waiting state)? Currently I just use sleep() on the main thread and then interrupt() all of the worker threads.
Sorry if the question sounds confused.
I've had this problem before, and the only way I found was to send a special marker object to the BlockingQueue. When the Queue .take() the object, if this is the marker, then the Thread ends itself.
I've tried other solutions, like to wake up the thread and detect the Exception, with no success.
There's a pattern called the Poison Pill that's good for this. Basically, when the producers are done, insert a special value into the queue that tells a consumer to stop. You can either insert one pill for each consumer, or, once a consumer gets a poison pill, return it to the queue for the next consumer. Since it sounds like you're just enqueuing strings, something like
public static final String POISON_PILL = "DONE";
Or in Java 8, use Optional to wrap your values, then have not present be the pill.
BlockingQueue<Optional<...>> queue;
Another option is using an ExecutorService (which is actually backed by a BlockingQueue) and submitting each file as its own task, then using executorService.shutdown() when you're done. The problem with this is that it couples your code more tightly than needed, and it makes it harder to reuse resources like database and HTTP connections.
I'd avoid interrupting your workers to signal them because that can cause blocking IO operations to fail.
You might use the approch below. Add observer pattern if you need to.
Or simply - instead of signalling with a death packet, collect a list of waiting Threads and then interrupt() them.
public class AccessCountingLinkedPrioQueue<T> {
private final LinkedBlockingQueue<T> mWrappingQueue = new LinkedBlockingQueue<>();
private final Object mSyncLockObj = new Object();
private final int mMaxBlockingThreads;
private final T mDeathSignallingObject;
private volatile int mNumberOfThreadsInAccessLoop = 0;
public AccessCountingLinkedPrioQueue(final int pMaxBlockingThreads, final T pDeathSignallingObject) {
mMaxBlockingThreads = pMaxBlockingThreads;
mDeathSignallingObject = pDeathSignallingObject;
}
public T take() throws InterruptedException {
final T retVal;
synchronized (mSyncLockObj) {
++mNumberOfThreadsInAccessLoop;
}
synchronized (mWrappingQueue) {
if (mNumberOfThreadsInAccessLoop >= mMaxBlockingThreads && mWrappingQueue.isEmpty()) signalDeath();
retVal = mWrappingQueue.take();
}
synchronized (mSyncLockObj) {
--mNumberOfThreadsInAccessLoop;
}
return retVal;
}
private void signalDeath() {
for (int i = 0; i < mMaxBlockingThreads; i++) {
mWrappingQueue.add(mDeathSignallingObject);
}
}
public int getNumberOfThreadsInAccessLoop() {
return mNumberOfThreadsInAccessLoop;
}
}
class WorkPacket {
// ... your content here
}
class MultiThreadingBoss {
static public final WorkPacket DEATH_FROM_ABOVE = new WorkPacket();
public MultiThreadingBoss() {
final int THREADS = 7;
final AccessCountingLinkedPrioQueue<WorkPacket> prioQ = new AccessCountingLinkedPrioQueue<>(THREADS, DEATH_FROM_ABOVE);
for (int i = 0; i < THREADS; i++) {
final ThreadedWorker w = new ThreadedWorker(prioQ);
new Thread(w).start();
}
}
}
class ThreadedWorker implements Runnable {
private final AccessCountingLinkedPrioQueue<WorkPacket> mPrioQ;
public ThreadedWorker(final AccessCountingLinkedPrioQueue<WorkPacket> pPrioQ) {
mPrioQ = pPrioQ;
}
#Override public void run() {
while (true) {
try {
final WorkPacket p = mPrioQ.take();
if (p == MultiThreadingBoss.DEATH_FROM_ABOVE) break; // or return
// ... do your normal work here
} catch (final InterruptedException e) {
e.printStackTrace();
}
}
}
}
I have to test that a Flash Player is correctly sending requests over the network. For that, I use BrowserMob Proxy to capture traffic.
My issue is I have to wait the end of the video played to test the final request. So as temp solution, in my test, I put a Thread.sleep(videoDuration). But IMO it's not really nice, and overall, the duration can change.
BrowserMob allow to add interceptor on the requests. So I think about notify and wait function.
Here my first try:
protected static boolean percent100Done;
protected static final Object percent100lock = new Object();
#BeforeClass
public static void addInterceptor() {
ProxyServer server = Server.init();
server.addRequestInterceptor((BrowserMobHttpRequest request, Har har) -> {
String percent = request.getProxyRequest().getParameter("percent");
if("100".equals(percent)) {
percent100Done = true;
synchronized (percent100lock) {
percent100lock.notify();
}
}
});
}
#Test
public void testFinalRequest() throws InterruptedException {
driver.get(myurl);
synchronized (percent100lock) {
while (!percent100Done) {
percent100lock.wait();
}
}
//test continue
}
I test this code, and it works. But it seems a bit dirty, especially the new Object line. What happen if I have to add notification for 25, 50 and 75 percents? I will have to 3 others booleans and lock?
Why not store the percentage and have a generic "progress" event, like how XMLHTTPRequest works?
protected static int percent;
protected static final Object progressLock = new Object();
#BeforeClass
public static void addInterceptor() {
ProxyServer server = Server.init();
server.addRequestInterceptor((BrowserMobHttpRequest request, Har har) -> {
synchronized(progressLock) {
percent = Integer.parseInt(request.getProxyRequest().getParameter("percent"));
progressLock.notify();
}
});
}
#Test
public void testFinalRequest() throws InterruptedException {
driver.get(myurl);
synchronized (progressLock) {
int lastPercent = -1;
while (percent != 100) {
if (lastPercent < 50 && percent >= 50) {
System.out.println("50%");
}
lastPercent = percent;
progressLock.wait();
}
}
//test continue
}
In terms of weirdness, I think your solution is fine. It looks strange, but every Object has a lock, so you might as well use it. If you needed to do something more complicated, you might want to look at Lock and it's subclasses. For example, with a ReadWriteLock, multiple readers can hold the lock at the same time, but only one writer can. It doesn't look like you have multiple readers though (and I wouldn't expect locking to effect the performance of this program much).
I wrote a small program in java to send and receive a data file using two threads. I want the two threads to be in the same class. One thread sends the file and the other thread receives the file. I have wrote the code for it but with few errors. Can you help me figure out the errors in the code. I am a student and a beginner in java, so spare me if there are any silly mistakes.
import java.lang.Thread.*;
import java.io.*;
public class sendques implements Runnable
{
int i=0,c;
static Thread[] t= new Thread[2];
FileInputStream fis=new FileInputStream("ip.jpg");
FileOutputStream fos=new FileOutputStream("output.jpg");
sendques() {
for(i=0;i<2;i++){
t[i]=new Thread(this);
t[i].start();
System.out.println("Threads "+i);
}
}
void run() {
while(true) {
wait();
send();
}
}
void send() {
while((c=fis.read())!=-1) {
t[2].receive(c);
wait();
}
}
void receive(int d) {
while(c!=-1) {
fos.write(d);
t[1].notify();
}
}
public static void main(String arg[]) {
sendques sq=new sendques();
t[1].send();
System.out.println("Quiting..");
}
}
Do not use notify, better use notifyAll, since a liveness failure can occur, named: missed signals. It would make a hard time to correct your code, here is the code for an implementation of producer/consumer with different classes:
BaseBoundedBuffer.java
BoundedBuffer.java
Producer.java
Consumer.java
The Buffer classes are used to store the data shared between producer and consumer. They have their own classes and an example you will find in BoundedBuffer.java. Their is no heavy computing task involved, just passing messages between both.
This is a clean implementation, try to work through it.