Develop Reference

Reader Writer implementation in java

I was trying to implement a reader-writer using notify and wait. But i think I'm stuck.
My sequence goes like this.
RRRRRRRRRRWWWWWWWWW This happens if the main start with reader invoked first.
Or
WWWWWWWRRRRRRRRRRR. This happens if the main start with the writer invoked first.
Looks like reads notify isn't working at all. Writer thread never goes into execution.
If i make while loop in run method to run infinite then it's just
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR......... No chance for the writer to write.
Can you have a look at this?
DATA CLASS
public class Data {
private int q ;
private boolean isAnyOneReading;
public Data() {
}
public void readQ() {
synchronized (this){
isAnyOneReading = true;
System.out.println("Read start "+q);
}
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
synchronized (this){
isAnyOneReading = false;
System.out.println("Read end "+q);
notifyAll();
}
}
public synchronized void writeQ(int q) {
System.out.println(isAnyOneReading);
while (isAnyOneReading){
try{
wait();
} catch (InterruptedException e) {
e.printStackTrace();
System.out.println("Done");
Thread.currentThread().interrupt();
}
}
System.out.println("Write start "+q);
this.q = q;
try{
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
Thread.currentThread().interrupt();
}
System.out.println("Write end "+q);
notifyAll();
}
}
READER CLASS
public class Reader implements Runnable {
private Data data;
private Thread readerThread;
public Reader(Data data) {
this.data = data;
readerThread = new Thread(this, "ReaderThread");
}
void startThread(){
readerThread.start();
}
#Override
public void run() {
int i = 0 ;
while (i != 5){
data.readQ();
i++;
}
}
}
WRITER CLASS
public class Writer implements Runnable{
private Data data;
private Thread writerThread;
public Writer(Data data) {
this.data = data;
writerThread = new Thread(this,"WriterThread," );
}
void startThread(){
writerThread.start();
}
#Override
public void run() {
int i = 0 ;
int j = 0 ;
while (j != 5){
data.writeQ(i++);
// i++;
j++;
}
}
}
MAIN CLASS
public class ReaderWriterDemo {
public static void main(String[] args) {
Data data = new Data();
Reader reader = new Reader(data);
Writer writer = new Writer(data);
reader.startThread();
writer.startThread();
}
}

Try removing the Thread.sleep from Data class.
And add Thread.sleep in run methods like so. (pasting one example):
#Override
public void run() {
int i = 0;
while (i != 5) {
data.readQ();
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
i++;
}
}
}

Read's notifyAll() works, but it seems that read() re-called again and changes isAnyOneReading's value prior to any other action in write(). That's why check fails and write() starts waiting again. As Danny Fried suggested moving Thread.sleep() to the run methods will help.

Looks like a simple case of starvation. Consider your writer's main loop:
while (j != 5){
data.writeQ(i++);
// i++;
j++;
}
data.writeQ() is a synchronized method: The very last thing it does before it returns is to unlock the lock. The very first thing it does on the next call is to re-lock the lock. Not much happens in-between--increment and test a local variable is all.
Java synchronized locks are not fair. (i.e., when a lock becomes available, the system does not guarantee that the winner will be the thread that's been waiting the longest.) In fact, it may be be the opposite of fair: The OS may try to maximize efficient use of the CPU(s) by always choosing the thread that's easiest to wake up.
When the writer comes back to call data.writeQ() on each subsequent iteration, it may be that the OS has not even have started to wake up the reader, and it simply lets the writer enter the synchronized block again.
Same thing happens with your reader. The code is a bit more complicated, but just like in the writer, the very last thing that data.readQ() does before returning is to unlock the lock, and the very first thing that it does on the next call is to lock it again.
Brute force solution: replace the synchronized blocks with a fair ReentrantLock object.
Alternate solution, which is more typical of how many programs actually work: Have the threads do something else (e.g., have them do some I/O) in between calls to the locked function, thereby giving the other threads a chance to get in and use the locked resource.

how to pass reference asynchronously between threads? With Observer?

How do I ensure that:
1.) localThread and remoteThread run independent of each other?
2.) pass messages between localThread and remoteThread?
Specifically, a String object from localThread needs to "percolate" up to Telnet through, I think it's known as, a call-back. However, there's not really anything, per se, for Telnet to observe. It's an anonymous reference to LocalIO, and I don't see that explicitly providing a reference helps.
I've read about java.util.concurrent.Semaphore until my head exploded, all I came away with was that it doesn't seem to apply. For these two threads, they should continue to run regardless of what the other thread is doing. However, there needs to be some mechanism to pass object references between the threads...
public class Telnet {
public Telnet() throws InterruptedException {
startThreads();
}
public static void main(String[] args) throws InterruptedException {
new Telnet();
}
public void startThreads() throws InterruptedException {
Semaphore s = new Semaphore(1, true);
Thread localThread = new Thread(new LocalIO());
Thread remoteThread = new Thread(new RemoteIO());
localThread.start();
remoteThread.start();
}
}
The threads themselves are as follows. LocalIO:
public class LocalIO implements Runnable {
#Override
public void run() {
Scanner scanner;
String line;
while (true) {
scanner = new Scanner(System.in);
line = scanner.nextLine();
out.println("\n\nyou entered\t\"" + line + "\"\n");
}
}
}
RemoteIO:
public class RemoteIO implements Runnable {
private static Logger log = Logger.getLogger(RemoteIO.class.getName());
final String host = "rainmaker.wunderground.com";
final int port = 3000;
#Override
public void run() {
log.fine(host + port);
int byteOfData;
try (Socket socket = new Socket(host, port);
InputStream inputStream = socket.getInputStream();
OutputStream ouputStream = socket.getOutputStream();
PrintWriter printWriter = new PrintWriter(socket.getOutputStream(), true);
final BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(System.in))) {
while ((byteOfData = inputStream.read()) != -1) {
out.print((char) byteOfData);
}
} catch (Exception e) {
out.println(e);
}
}
}
Keeping in mind that RemoteIO never closes its connection and runs indefinitely.

The concurrent package is very helpful for this sort of thing:
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/package-summary.html
For example you can just give each thread a ConcurrentLinkedQueue and they can check the queue to see if there is anything to act on whenever they please. Meanwhile other threads can add new objects to the queue whenever they please.

There is one essential difference in the programming paradigm your code can adopt:
synchronous mode: the receiving end runs an endless loop which explicitly takes items off a concurrent queue, blocking when there are no items ready;
asynchronous mode: the receiving end submits a callback to an item exchange mechanism. This callback is called for every item which arrives from the producer thread.
The Observer pattern may loosely apply to the latter case, but not to the former.
Also note that in the latter case, the "item exchange mechanism" is usually implemented in the synchronous mode.

Not sure what yu are trying to do, but if you want to exchange data between threads, you need a volatile variable to make sure changes are seen by other threads. AtomicReferences are non-blocking and provide some API that might help here.

The solution I found:
public class RemoteConnection extends Observable {
private static Logger log = Logger.getLogger(RemoteConnection.class.getName());
private final Socket socket;
private final BufferedInputStream in;
private final BufferedOutputStream out;
private final static String UTF8 = "UTF-8";
public RemoteConnection(String host, int port) throws UnknownHostException, IOException {
socket = new Socket(host, port);
in = new BufferedInputStream(socket.getInputStream());
out = new BufferedOutputStream(socket.getOutputStream());
}
public void write(Deque<String> commands) throws IOException {
String command;
while (!commands.isEmpty()) {
command = commands.pop();
out.write(command.concat("\r\n").getBytes(Charset.forName(UTF8)));
log.info(command);
}
out.flush();
}
void read() { //probably should use BufferedStream to better effect..?
Thread readRemote = new Thread() {
#Override
public void run() {
StringBuilder sb = new StringBuilder();
char ch;
int i;
while (true) {
try {
i = in.read();
ch = (char) i;
sb.append(ch);
System.out.print(ch);
if (i == 13) {
setChanged();
notifyObservers(sb.toString());
log.fine(sb.toString());
sb = new StringBuilder();
}
} catch (IOException ioe) {
log.fine(ioe.toString());
}
}
}
};
readRemote.start();
}
}
By reorganizing the threading, this approximates a poor-mans telnet, with asynchronous threads for i/o. I think reading from the console was blocking...something...
I really don't know why this works but the other approaches didn't. I would've preferred to have the main class start and handle threads, and pass references between the threads, but that just didn't work despite using the various solutions offered here.
LocalConnection has a similar threading approach.

How to pass parameter to an already running thread in java?

How to pass parameter to an already running thread in java -- not in the constructor, & probably without using wait() (possible ??)
Something similar to a comment in How can I pass a parameter to a Java Thread?
Do you mean passing a parameter to an already running thread ? Because all the current answers are about passing parameters to new threads... – Valentin Rocher May 18 '09 at 10:43
[edited]
yes, I was looking for something like the producer/consumer pattern.
I wanted something like a thread in which has the processing & is ready
for keyboard input. The other thread is just to monitor network and pass
on the received text to the processing thread.

Maybe what you really need is blocking queue.When you create the thread, you pass the blocking queue in and the thread should keep checking if there is any element in the queue. Outside the thread, you can put elements to the queue while the thread is "running". Blocking queue can prevent the thread from quit if their is nothing to do.
public class Test {
public static void main(String... args) {
final BlockingQueue<String> queue = new LinkedBlockingQueue<String>();
Thread running = new Thread(new Runnable() {
#Override
public void run() {
while (true) {
try {
String data = queue.take();
//handle the data
} catch (InterruptedException e) {
System.err.println("Error occurred:" + e);
}
}
}
});
running.start();
// Send data to the running thread
for (int i = 0; i < 10; i++) {
queue.offer("data " + i);
}
}
}

The "other thread" will have its own life, so you can't really communicate with it / pass parameters to it, unless it actively reads what you gives to it.
A thread which you allows you to communicate with it typically reads data from some buffered queue.
Have a look at ArrayBlockingQueue for instance, and read up on the Consumer-Producer pattern.

public class T1 implements Runnable {
//parameter of thread T1
public static AtomicBoolean flag = new AtomicBoolean();
#Override
public void run() {
}
}
public class T2 implements Runnable {
#Override
public void run() {
//parameter to an already running thread
T1.flag.set(true);
}
}

What about such way:
class TestRun implements Runnable
{
private int testInt = -1;
public void setInt(int i)
{
this.testInt = i;
}
#Override
public void run()
{
while (!isFinishing())
{
System.out.println("Working thread, int : " + testInt);
try
{
Thread.sleep(2500);
}
catch (InterruptedException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
.....
TestRun first = new TestRun();
TestRun second = new TestRun();
(new Thread(first)).start();
(new Thread(second)).start();
try
{
Thread.sleep(5000);
}
catch (InterruptedException e)
{
}
first.setInt(101);
second.setInt(102);

synchronized methods in a Runnable

I am writing a Runnable class that's packing messages together for some time or until a given size is reached before sending them over the wire. It is designed to allow other threads to change some internal parameters (e.g. the packet size) while it is running via some setter methods. Is it correct to use an internal object lock to make parts of the run() logic mutually exclusive with the setters?
Something like that:
public class Packer implements Runnable {
private BlockingQueue<byte[]> msgQueue;
private Object lock = new Object();
private Packet packet;
private boolean running = false;
public synchronized void append(byte[] payload) throws InterruptedException {
msgQueue.put(payload);
}
public synchronized void setPacketCapacity(int size) {
synchronized (lock) {
// check to see if we need to flush the current packet first, etc.
packet.setCapacity(size);
}
}
public void run() {
running = true;
while (running) {
try {
byte[] msg = msgQueue.take();
synchronized (lock) {
packet.add(msg);
// check if we need to flush the packet, etc.
}
} catch (InterruptedException ex) {
logger.warn("interrupted");
running = false;
} catch (Exception e) {
logger.error(e);
}
}
logger.warn("stop");
}
}
Relatedly, what's the right way for another thread to tell this runnable to stop (and flush)?
Since the run() method might be waiting on the internal queue msgQueue, it might not be sufficient to simply set running=false and I might have to interrupt the thread. Alternatively, I could send a special "End Of Stream" message to the internal queue, but if the queue is full I might have to wait for a while before it gets accepted.

Since you are locking your setter and the logic inside run() with one locking Object it is correct. I would suggest that you remove synchronized from setter method signature since you already lock all the code inside it with the locking Object
You can remove boolean running and write your run() like this:
public void run() {
while (true) {
try {
byte[] msg = msgQueue.take();
synchronized (lock) {
packet.add(msg);
// check if we need to flush the packet, etc.
}
} catch (InterruptedException ex) {
logger.warn("interrupted");
Thread.currentThread.interrupt();
return;
} catch (Exception e) {
logger.error(e);
}
}
logger.warn("stop");
}
Calling thread.interrupt() will force the code in run() method to go to the InterruptedException catch block where you set the interrupted flag and return from run()

Input and Output Stream Pipe in Java

Does anyone have any good suggestions for creating a Pipe object in Java which is both an InputStream and and OutputStream since Java does not have multiple inheritance and both of the streams are abstract classes instead of interfaces?
The underlying need is to have a single object that can be passed to things which need either an InputStream or an OutputStream to pipe output from one thread to input for another.

It seems the point of this question is being missed. If I understand you correctly, you want an object that functions like an InputStream in one thread, and an OutputStream in another to create a means of communicating between the two threads.
Perhaps one answer is to use composition instead of inheritance (which is recommended practice anyway). Create a Pipe which contains a PipedInputStream and a PipedOutputStream connected to each other, with getInputStream() and getOutputStream() methods.
You can't directly pass the Pipe object to something needing a stream, but you can pass the return value of it's get methods to do it.
Does that work for you?

java.io.PipedOutputStream and java.io.PipedInputStream look to be the classes to use for this scenario. They are designed to be used together to pipe data between threads.
If you really want some single object to pass around it would need to contain one of each of these and expose them via getters.

This is a pretty common thing to do, I think. See this question.
Easy way to write contents of a Java InputStream to an OutputStream

You can't create a class which derives both from InputStream and OutputStream because these aren't interfaces and they have common methods and Java doesn't allow multiple inheritance (the compiler doesn't know whether to call InputStream.close() or OutputStream.close() if you call close() on your new object).
The other problem is the buffer. Java wants to allocate a static buffer for the data (which doesn't change). This means when you use the `java.io.PipedXxxStream', the writing data to it will eventually block unless you use two different threads.
So the answer from Apocalisp is correct: You must write a copy loop.
I suggest that you include Apache's commons-io in your project which contains many helper routines just for tasks like this (copy data between streams, files, strings and all combinations thereof).

See http://ostermiller.org/utils/CircularBuffer.html

I had to implement a filter for slow connections to Servlets so basically I wrapped the servlet output stream into a QueueOutputStream which will add every byte (in small buffers), into a queue, and then output those small buffers to a 2nd output stream, so in a way this acts as input/output stream, IMHO this is better than JDK pipes which won't scale that well, basically there is too much context switching in the standard JDK implementation (per read/write), a blocking queue is just perfect for a single producer/consumer scenario:
import java.io.IOException;
import java.io.OutputStream;
import java.util.concurrent.*;
public class QueueOutputStream extends OutputStream
{
private static final int DEFAULT_BUFFER_SIZE=1024;
private static final byte[] END_SIGNAL=new byte[]{};
private final BlockingQueue<byte[]> queue=new LinkedBlockingDeque<>();
private final byte[] buffer;
private boolean closed=false;
private int count=0;
public QueueOutputStream()
{
this(DEFAULT_BUFFER_SIZE);
}
public QueueOutputStream(final int bufferSize)
{
if(bufferSize<=0){
throw new IllegalArgumentException("Buffer size <= 0");
}
this.buffer=new byte[bufferSize];
}
private synchronized void flushBuffer()
{
if(count>0){
final byte[] copy=new byte[count];
System.arraycopy(buffer,0,copy,0,count);
queue.offer(copy);
count=0;
}
}
#Override
public synchronized void write(final int b) throws IOException
{
if(closed){
throw new IllegalStateException("Stream is closed");
}
if(count>=buffer.length){
flushBuffer();
}
buffer[count++]=(byte)b;
}
#Override
public synchronized void write(final byte[] b, final int off, final int len) throws IOException
{
super.write(b,off,len);
}
#Override
public synchronized void close() throws IOException
{
flushBuffer();
queue.offer(END_SIGNAL);
closed=true;
}
public Future<Void> asyncSendToOutputStream(final ExecutorService executor, final OutputStream outputStream)
{
return executor.submit(
new Callable<Void>()
{
#Override
public Void call() throws Exception
{
try{
byte[] buffer=queue.take();
while(buffer!=END_SIGNAL){
outputStream.write(buffer);
buffer=queue.take();
}
outputStream.flush();
} catch(Exception e){
close();
throw e;
} finally{
outputStream.close();
}
return null;
}
}
);
}

Better to use Pipe or ArrayBlockingQueue, I recommend you not to use PipedInput/OutputStream as they have a bad practice even you can see in the link below that they have asked to be deprecated as it causes many issues.
https://bugs.openjdk.java.net/browse/JDK-8223048
For the BlockingQueue and Pipe here a simple example of that
Pipe:
Pipe pipe = Pipe.open();
Pipe.SinkChannel sinkChannel = pipe.sink();
String newData = "New String to write to file..." + System.currentTimeMillis();
ByteBuffer buf = ByteBuffer.allocate(48);
buf.clear();
buf.put(newData.getBytes());
buf.flip();
while(buf.hasRemaining()) {
sinkChannel.write(buf);
}
Pipe.SourceChannel sourceChannel = pipe.source();
ByteBuffer buf = ByteBuffer.allocate(48);
int bytesRead = inChannel.read(buf);
Reference: http://tutorials.jenkov.com/java-nio/pipe.html
BlockingQueue:
//Shared class used by threads
public class Buffer {
// ArrayBlockingQueue
private BlockingQueue<Integer> blockingQueue = new ArrayBlockingQueue<Integer>(1);
public void get() {
// retrieve from ArrayBlockingQueue
try {
System.out.println("Consumer received - " + blockingQueue.take());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public void put(int data) {
try {
// putting in ArrayBlockingQueue
blockingQueue.put(data);
System.out.println("Producer produced - " + data);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public static void main(String[] args) {
// Starting two threads
ExecutorService executorService = null;
try {
Buffer buffer = new Buffer();
executorService = Executors.newFixedThreadPool(2);
executorService.execute(new Producer(buffer));
executorService.execute(new Consumer(buffer));
} catch (Exception e) {
e.printStackTrace();
}finally {
if(executorService != null) {
executorService.shutdown();
}
}
}
public class Consumer implements Runnable {
private Buffer buffer;
public Consumer(Buffer buffer) {
this.buffer = buffer;
}
#Override
public void run() {
while (true) {
try {
buffer.get();
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
public class Producer implements Runnable {
private Buffer buffer;
public Producer(Buffer buffer) {
this.buffer = buffer;
}
#Override
public void run() {
while (true) {
Random random = new Random();
int data = random.nextInt(1000);
buffer.put(data);
}
}
}
Reference:
https://github.com/kishanjavatrainer/ArrayBlockingQueueDemo/tree/master/ArrayBlockingQueueDemo