I've a small android library which handles a serial port, it has basic functionality like open, read, write and close.
I have made an applications that uses this library to write on the serial port and read the responses, within this application there is a thread that periodically opens the serial port asks for the status get the response and close the serial port.
I want to protect the serial communication in a way that if the main thread opens the communication the secondary thread that only checks the status can not open it and wait for the main thread to finish.
class SerialChannel extends Channel
{
private SerialPortUtility serialPortUtility;
private static final String SERIAL_FILE = "/dev/ttyMT2";
private static final String CONTROL_FILE = "/sys/devices/platform/file";
private static final String UNKNOWN_COMMAND = "UNKNOWN COMMAND";
private FileOutputStream fileOutputStream;
private FileInputStream fileInputStream;
#Override
public void open() throws CommunicationException
{
try
{
if (isSerialOpened() != SerialStatus.Open)
{
toggleSerial(SerialStatus.Open.getStatus());
Thread.sleep(100);
}
serialPortUtility = getSerialPortUtility();
fileInputStream = (FileInputStream) serialPortUtility.getInputStream();
fileOutputStream = (FileOutputStream) serialPortUtility.getOutputStream();
currentProcess = Optional.of(Thread.currentThread().getId());
Thread.sleep(500);
}
catch (IOException | InterruptedException e)
{
throw new CommunicationException(e.getMessage());
}
}
#Override
public void close() throws CommunicationException
{
if (serialPortUtility == null)
{
throw new CommunicationException("SerialPort is null");
}
try
{
toggleSerial(SerialStatus.Close.getStatus());
fileOutputStream.close();
fileInputStream.close();
serialPortUtility.close();
fileInputStream = null;
fileOutputStream = null;
serialPortUtility = null;
}
catch (IOException e)
{
throw new CommunicationException(e.getMessage());
}
}
#Override
public void send(byte[] buffer, int timeout, int length) throws CommunicationException
{
if (fileOutputStream == null)
{
throw new CommunicationException("Problem while sending data!");
}
try
{
fileOutputStream.write(buffer);
fileOutputStream.flush();
}
catch (IOException e)
{
throw new CommunicationException(e.getMessage());
}
}
#Override
public byte[] receive(int length, int timeout) throws CommunicationException
{
StringBuilder stringBuilder = new StringBuilder();
byte[] buffer = new byte[length];
int ret;
int totalSize = 0;
if (fileInputStream == null)
{
throw new CommunicationException("FileInputStream is null!");
}
try
{
long millisStart = Calendar.getInstance().getTimeInMillis();
boolean timeoutReached;
while (true)
{
timeoutReached = (Calendar.getInstance().getTimeInMillis() - millisStart > timeout * 1000);
if (fileInputStream.available() <= 0 && timeoutReached)
{
expectingResult = false;
throw new CommunicationException("Error");
}
else if (fileInputStream.available() > 0)
{
break;
}
}
millisStart = Calendar.getInstance().getTimeInMillis();
while (totalSize != length && (ret = fileInputStream.read(buffer)) != -1)
{
String received = new String(buffer);
stringBuilder.append(received);
if(buffer.length == 15 && received.equals(UNKNOWN_COMMAND))
{
break;
}
totalSize += ret;
}
expectingResult = false;
}
catch (IOException e)
{
throw new CommunicationException(e.getMessage());
}
return stringBuilder.toString().getBytes();
}
private SerialPortUtility getSerialPortUtility() throws IOException
{
if (serialPortUtility == null)
{
File file = new File(SERIAL_FILE);
int baudRate = 115200;
return new SerialPortUtility(file, baudRate, 0);
}
return serialPortUtility;
}
private void toggleSerial(String data) throws IOException
{
FileOutputStream fos = new FileOutputStream(new File(CONTROL_FILE));
fos.write(data.getBytes());
fos.flush();
fos.close();
}
private SerialStatus isSerialOpened() throws IOException
{
byte[] buffer = new byte[1];
FileInputStream fis = new FileInputStream(new File(CONTROL_FILE));
int result = fis.read(buffer);
fis.close();
if (result > -1 && buffer[0] == 1)
{
return SerialStatus.Open;
}
return SerialStatus.Close;
}
}
This class extends custom class Channel that implements an interface with the methods open, close, read, send and implements also AutoCloseable.
Now if I make the open method synchronized any thread that enters here will lock, but will lock until it exits the open method, and when the thread moves to the another method let's say read and stay there until it gets a response, the checker thread will come and enters the open method. Using AutoCloseable the close method will execute and close the serial port communication. If I synchronize an object, there still is a window when the object is not synchronized.
How can I tell the checker thread that the communication is already opened and make him wait until the main thread finish.
Checker looks like this, it is within an timer:
try(Channel ch = CommunicationFactory.getInstance().selectChannel(CommunicationType.SERIAL))
{
ch.open();
//do stuff
}
catch (CommunicationException ex)
{
ex.printStackTrace();
}
The "main" thread looks the same only that it is in an AysncTask.
If additional informations are required please let me know!
Thank you in advance for your effort and time!
How can I tell the checker thread that the communication is already opened and make him wait until the main thread finish.
I don't fully understand your code but the critical thing with threads and locking is to make sure that all threads are calling code that is synchronized on the same object instance.
If I synchronize an object, there still is a window when the object is not synchronized.
Not if you use the same instance of the object. Making each of the public methods in SerialChannel synchronized will make sure that only 1 thread can be using the object at once.
I suspect that your real problem is not about protecting the SerialChannel object but more about race-conditions between the threads. They need to make multiple calls to the methods and they can block each other or interleave in an improper manner.
You can get around this with a couple of changes. You can make the send(...) and receive(...) methods auto-opening. Threads would just call send() or receive() which in turn would internally call open() if the fileInputStream or fileOutputStream was null. The thread would be inside of a synchronized so this would not be interrupted by another thread.
Another completely different model to consider would be to have one thread reading from the serial port and another writing to it that are dedicated to that task -- they would be built into the SerialChannel object. They would share data with the external threads using a read BlockingQueue and a write BlockingQueue. Then the serial port is opened early in your application which starts the IO threads and the external threads never worry about the IO. They just put() and take() from the queues. I typically do this (for example) when reading and writing to the console.
Hope something here helps.
Related
I am working on a simple project: Server executes a slave (ReaderF Class) inside a thread that reads a file and then extracts its content and saves it inside the Server himself. The server then needs to execute a pool to send the content to a client when he connects. I started by writing the ReaderF to extract the content. Problem: it never edits the String variable in which he is supposed to stock the content. Here is what I did:
public class Serveur {
private ServerSocket serverSocket;
private Socket socket;
public String res=null; //This is what my thread is supposed to be editing
ExecutorService pool = null;
public static final int port = 33333;
Serveur(int port, int size){
try {
pool = Executors.newFixedThreadPool(5);
serverSocket = new ServerSocket(port, size);
} catch (IOException ex) {
Logger.getLogger(Serveur.class.getName()).log(Level.SEVERE, null, ex);
}
}
void manage () throws IOException {
while(true) {
ReaderF S = null;
try {
S = new ReaderF(serverSocket.accept(), this);
} catch (IOException e) {
e.printStackTrace();
}
Thread t=new Thread(S);
t.start();
}
}
And this is the slave that reads and edit the res variable.
public class ReaderF implements Runnable {
private final Socket socket;
private Serveur serverMaitre;
ReaderF(Socket socket, Serveur serverMaitre) {
this.socket = socket;
this.serverMaitre = serverMaitre;
}
public void run() {
String fileName = "/home/abdou/1.txt";
FileReader fileReader = null;
Writer output =null;
try {
fileReader = new FileReader(fileName);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try (BufferedReader bufferedReader = new BufferedReader(fileReader)) {
output = new OutputStreamWriter(socket.getOutputStream(), "8859_1");
String line;
String res="";
while((line = bufferedReader.readLine()) != null) {
res+=line+"\n";
}
serverMaitre.res=res; // This is where i tell him to edit res inside the server
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
When I execute this main I see nothing on the screen. I know that it is due to the fact that the Server is stuck in the while loop.
public class Main{
public static void main(String[] args) throws IOException {
Serveur serveur = new Serveur(Serveur.port, 1);
serveur.manage();
System.out.println("The res variable content is"+serveur.res);
}}
How would I get out of it please. And Is my thread really changing the res variable ? Thank you
When multiple threads work with the same variables, then they are allowed to work on a local copy of the variable (in their stack) instead of the original variable (in the heap). This means that one thread may not see changes to a variable that another thread did (he is not 'refreshing' its local data), or the other thread is not 'publishing' (technical term: flushing) the updated value. So, when your ReaderF assign a value to 'res', the main thread (which is checking it) might not notice this. Solution: Make the variable volatile:
public volatile String res=null;
This forces the ReaderF to flush the updated value after assigning, and the main thread to refresh it when reading it.
But you might have another problem here, because you have multiple threads that may be all writing the 'res' variable (Depends on how active your socket is). You can get 'lost updates' or other 'race conditions' here. So maybe you even need a synchronized somewhere. But it would be too much to explain Multi-Threading here. Please google about it. Multi-Threading is not trivial unfortunately, you cannot just make new threads as you wish and expect everything to work. And adding a volatile is not the solution most of the time. It might just be enough for what you wanna do here.
Consider this (simplified) piece of code:
public class Test {
// assigned elsewhere
InetSocketAddress socketAddress;
String socketHost;
int socketPort;
Socket socket;
int COMMAND = 10;
int CONNECTION_TIMEOUT = 10 * 1000;
int SOCKET_TIMEOUT = 30 * 1000;
DataOutputStream dos;
DataInputStream dis;
protected void connect() throws IOException, InterruptedException {
socket.connect(socketAddress != null ? socketAddress : new InetSocketAddress(socketHost, socketPort), CONNECTION_TIMEOUT);
socket.setSoTimeout(SOCKET_TIMEOUT);
socket.setTcpNoDelay(true);
}
void initializeDataStreams() throws IOException {
dos = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream(), socket.getSendBufferSize()));
dis = new DataInputStream( new BufferedInputStream( socket.getInputStream(), socket.getReceiveBufferSize()));
}
void run() {
try {
connect();
initializeDataStreams();
sendCommand(COMMAND, true);
sendIdAndUsername(true);
sendSyncPreference(true);
sendBlockedIds(true);
sendHeaders();
// reading from 'dis' here
// ...
} catch (InterruptedException | IOException e){
/* ... */
}
}
void sendCommand(int command, boolean buffered) throws IOException {
dos.write(command);
if (!buffered) {
dos.flush();
}
}
void sendIdAndUsername(boolean buffered) throws IOException {
sendId(true); // always buffered
String username = "user name";
dos.writeBoolean(username != null);
if (username != null) {
dos.writeUTF(username);
}
if (!buffered) {
dos.flush();
}
}
void sendId(boolean buffered) throws IOException {
dos.writeUTF("user id");
if (!buffered) {
dos.flush();
}
}
void sendSyncPreference(boolean buffered) throws IOException {
boolean fullSync = true;
dos.writeBoolean(fullSync);
if (!buffered) {
dos.flush();
}
}
void sendBlockedIds(boolean buffered) throws IOException {
Set<String> blockedCrocoIds = new HashSet<>();
ObjectOutputStream oos = new ObjectOutputStream(dos);
oos.writeObject(blockedCrocoIds);
if (!buffered) {
oos.flush();
}
}
private void sendHeaders() throws IOException {
dos.writeUTF("some string");
dos.writeInt(123);
// some other writes...
// this should flush everything, right?
dos.flush();
}
}
I left it intentionally with all the methods, just in case I've made some terribly obvious mistake there. When I execute Test.run(), sometimes (really hard to predict when exactly) it seems like the flush() in sendHeaders() doesn't work at all.
Server side doesn't receive anything on its ServerSocket.accept() for next 22 seconds (don't ask me where this number comes from, part of the mystery).
The idea was that I wont call flush() on every transmission but call it only once, to save the bandwidth.
So what's wrong with this code? How to ensure writes to my stream are reliable / immediate so the server can read it ASAP?
I also accept answer "there's nothing wrong", in that case it must be something which is being done in parallel and affecting the network stack on Android.
EDIT: Server code is really nothing special:
ListeningThread listeningThread = new ListeningThread();
listeningThread.start();
listeningThread.join();
and then:
public class ListeningThread extends Thread {
private ServerSocket serverSocket;
public ListeningThread() {
try {
// unbound server socket
serverSocket = new ServerSocket();
serverSocket.setReuseAddress(true);
serverSocket.bind(new InetSocketAddress(NetworkUtil.APP_SERVER_PORT));
} catch (IOException e) {
log(e);
}
}
#Override
public void run() {
log("run");
while (serverSocket.isBound() && !isInterrupted()) {
try {
Socket socket = serverSocket.accept();
new CommandThread(socket).start();
} catch (IOException e) {
log(e);
}
}
try {
serverSocket.close();
} catch (IOException e) {
log(e);
}
}
}
and finally:
public class CommandThread extends Thread {
private final Socket socket;
public CommandThread(Socket socket) {
log("CommandThread");
this.socket = socket;
}
#Override
public void run() {
log("run");
try {
socket.setSoTimeout(NetworkUtil.SOCKET_TIMEOUT);
socket.setTcpNoDelay(true);
InputStream is = socket.getInputStream();
int cmd = is.read(); // <========= so actually this is failing
switch (cmd) {
// handling of the command
case COMMAND:
new DownloadMessagesThread(socket).start();
break;
}
} catch (IOException | SQLException e) {
log(e);
}
}
}
As mentioned in the comments, I'd open to agree on anything wrong with the object streams & co but the trouble is that I'm unable to reach (again, it's just sometimes, it's very random...) CommandThread's run(). So unless I'm missing something else, there's no way Object Streams could cause this kind of failure.
EDIT 2: Correction: it's not accept() I cannot reach, it's the first read operation:
03-07 11:22:42.965 00010 CommandThread: CommandThread
03-07 11:22:42.966 00108 CommandThread: run
[... nothing happening ...]
03-07 11:23:04.549 00111 DownloadMessagesThread: run
Could this be caused by mixing the object stream and data stream after all?
You should verify that the ObjectOutputStream creation in sendBlockedIds is not the culprit.
I've already had some protocol "deadlocks" while mixing DataStreams and ObjectStreams, since the creation of the Writer/Reader pair of ObjectStreams implies a kind of handshake that may fail while mixing those streams.
EDIT: While reading again your question, I realized that I had not answered it. So yes, it is reliable. And +1 for EJP answer.
To answer the question in your title, it is 100% reliable, as it doesn't do anything. Only the flush() methods of streams that are buffered actually do anything, and that only includes ObjectOutputStream and BufferedOutputStream, and PrintStream depending on how you construct it. Not DataOutputStream, and not the output stream of the socket itself.
So in this case the only flush method that does anything is the buffered output stream's, and you can certainly rely on that, as it is just code, and has been working for twenty years.
If this is affecting the speed of accept(), there must be something odd about your accept loop that you haven't shown us: typically, doing I/O in the accept loop instead of in the started thread.
And you should certainly not create an ObjectOutputStream in the middle of the connection. Create it at the start and use it for everything, and an ObjectInputStream at the other end.
NB setting the buffer sizes to the socket buffer sizes respectively is really fairly pointless. The defaults are adequate.
I want my app to be able to use a global su instance. I have code that does that, but I have encountered a race condition, I believe.
I am storing some variables for su like so:
public static List<Object> rootObjects = Collections.synchronizedList(new ArrayList<>());
protected void onCreate(Bundle savedInstanceState) {
...
if(PreferenceManager.getDefaultSharedPreferences(
getApplicationContext()).getBoolean("use_su", false) && rootObjects.isEmpty())
{
try {
Process process = Runtime.getRuntime().exec("su");
rootObjects.add(process);
InputStream inputStream = new DataInputStream(process.getInputStream());
rootObjects.add(inputStream);
OutputStream outputStream = new DataOutputStream(process.getOutputStream());
rootObjects.add(outputStream);
} catch (IOException e) {
Log.d(MainActivity.mainActivity.getPackageName(), e.getLocalizedMessage());
}
finally {
synchronized (rootObjects) {
rootObjects.notifyAll();
}
}
}
}
and using them like so:
byte[] getPrivateKeyAsSuperUser() {
byte[] data = null;
DataInputStream inputStream = null;
DataOutputStream outputStream = null;
if(MainActivity.rootObjects.size() != 3)
synchronized (MainActivity.rootObjects)
{
try {
MainActivity.rootObjects.wait();
} catch (InterruptedException e) {
Log.d(MainActivity.mainActivity.getPackageName(), e.getLocalizedMessage());
}
}
for(Object rootObj : MainActivity.rootObjects)
{
if(rootObj instanceof DataInputStream)
inputStream = (DataInputStream) rootObj;
else if(rootObj instanceof DataOutputStream)
outputStream = (DataOutputStream) rootObj;
}
try {
outputStream.writeBytes(String.format("cat \"%s\"\n", sshPrivateKey.getAbsolutePath()));
outputStream.flush();
data = readStream(inputStream);
} catch (IOException e) {
Log.d(MainActivity.mainActivity.getPackageName(), e.getLocalizedMessage());
}
return data;
}
private byte[] readStream(InputStream stream) {
byte[] data = null;
try {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte buff[] = new byte[1024];
int count = 0;
while (stream.available() != 0 && (count = stream.read(buff)) != -1) {
bos.write(buff, 0, count);
}
data = bos.toByteArray();
//System.out.println(new String(data));
} catch (IOException e) {
Log.d(MainActivity.mainActivity.getPackageName(), e.getLocalizedMessage());
}
return data;
}
But it does not wait like I expect, and I instantly receive a Toast that the returned private key is not valid with my sanity check (It's probably null).
The code works if I let Process finish initializing, but I'd like the program to do that for me.
I've tried some other synchronization techniques such as locks, but apparently as soon as you know if an object has a lock your info is stale.
What is the best thread safe approach to have the caller of getPrivateKeyAsSuperUser() wait if Process is not initialized properly?
EDIT:
I would like to add that through some debugging, I have found that I do not want be waiting for Process to initialize (because what I have DOES that), but rather, that the shell spawned by su is valid to accept further commands. I suppose I could have a thread pipe something like echo DONE and loop until I get DONE back, but that seems like that would waste CPU horsepower. If someone could lend some knowledge on the subject, I would be extremely grateful.
You're attempting the singleton pattern here. I'm not sure why you want to store these objects in a list. The most sensible way to store them is in an object that you guarantee to create one instance of. There are a few ways you could do this. I think in your case the following would work
public class SuProcessHolder {
// store the state of the process here - this would be your Process and streams as above
// these should be non-static members of the class
// this would be the singleton instance you'll use - it will be constructed once
// on first use
private static SuProcessHolder singletonInstance = new SuProcessHolder();
public SuProcessHolder() {
// put your construction code in here to create an SU process
}
// this will access your SU process
public static SuProcessHolder getInstance() { return singletonInstance; }
}
Then, wherever you need your SU process, just call
SuProcessHolder.getInstance()
and it will be there like a Michael Jackson song.
I have solved it. I did end up having to echo and check for done, but I have done it without a loop, or sleeping in my thread, so it will fire as soon as it can, without hogging the CPU. The concurrent class I was looking for was CountDownLatch as well.
The assignment look like this:
process = Runtime.getRuntime().exec("su");
outputStream = new DataOutputStream(process.getOutputStream());
outputStream.writeBytes("echo DONE\n");
outputStream.flush();
inputStream = new DataInputStream(process.getInputStream());
byte[] buff = new byte[4];
inputStream.read(buff);
if(new String(buff).equals("DONE"));
MainActivity.rootLatch.countDown();
and getPrivateKeyAsSuperUser() became:
byte[] getPrivateKeyAsSuperUser() {
byte[] data = null;
try {
MainActivity.rootLatch.await();
} catch (InterruptedException e) {
Log.d(MainActivity.mainActivity.getPackageName(), e.getLocalizedMessage());
}
Su su = Su.getStaticInstance();
try {
su.outputStream.writeBytes(String.format("cat \"%s\"\n", sshPrivateKey.getAbsolutePath()));
su.outputStream.flush();
data = readStream(su.inputStream);
} catch (IOException e) {
Log.d(MainActivity.mainActivity.getPackageName(), e.getLocalizedMessage());
}
return data;
}
Although, this feels slightly sloppy, I may end up posting this on Code Review.
I've written a simple application in Java where there are two nodes, each with a ServerSocket open to a port listening for incoming connections. The nodes run two threads each, sending 1000 messages to the other node through a persistent TCP socket created when sending the first message. However, the nodes do not receive all 1000 messages. One may receive 850 while the other only receives 650. This number tends to stay constant over multiple runs.
The sending code is as follows:
public void SendMsg(String dest, Message myMsg) {
Socket sendsock = null;
PrintWriter printwr = null;
try {
if(printwr == null) {
sendsock = new Socket(dest, Main.rcvport);
printwr = new PrintWriter(sendsock.getOutputStream(), true);
}
String msgtosend = myMsg.msgtype.toString() + "=" + Main.myaddy + "=" + myMsg.content + "\n";
printwr.print(msgtosend);
} catch (UnknownHostException ex) {
System.out.println(ex);
//DO: Terminate or restart
} catch (IOException ex) {
System.out.println(ex);
//DO: Terminate or restart
}
}
Performance seems to improve if I use
buffwr = new BufferedWriter(printwr)
as well and use buffwr.write(...) instead of printwr.print(...), though it doesn't seem to be a complete solution for the data loss. There are no exceptions to show that packets weren't delivered, so according to the sender, they were all sent successfully.
On the receiving end, the accepted connection is treated as follows:
BufferedReader inbuff = new BufferedReader(new InputStreamReader(incoming.getInputStream()));
while(running) {
String rcvedln = inbuff.readLine();
if(rcvedln != null) {
count++;
System.out.println(count);
}
}
Is there an problem with how the readers and writers have been used that could be causing the problem? Thanks.
SendMsg() is creating a new socket every call, so you aren't using a persistent TCP connection. The method isn't closing the socket, either, so you have a lot of open collections. You may be reaching a limit to the number of connections the process can make (the sockets may not be closed when the objects are garbage collected).
Finally, as kd304 pointed out, the Javadoc for PrintWriter states this about the autoFlush parameter of the PrintWriter constructor: "if true, the println, printf, or format methods will flush the output buffer". Your code wasn't calling a method that did a flush.
Try this:
public class MessageSender implements Closeable {
private final Socket socket;
private final PrintWriter writer;
public MessageSender(String dest, int port) {
socket = new Socket(dest, port);
writer = new PrintWriter(socket.getOutputStream(), true);
}
public void sendMessage(Message message) {
try {
writer.println(message.toString());
} catch (UnknownHostException ex) {
System.out.println(ex);
//DO: Terminate or restart
} catch (IOException ex) {
System.out.println(ex);
//DO: Terminate or restart
}
}
#Override
public void close() throws IOException {
writer.close();
socket.close();
}
Note I modified the code so that sendMessage() calls Message.toString() to get the formatted message. It doesn't seem right for sendMessage() to reference fields in Message in order to format the message. Instead of using toString() you could create a method in Message specifically for this purpose.
Here's the server side code:
public class Server implements Runnable {
private final ServerSocket serverSocket;
private final ExecutorService executor;
private volatile boolean running = true;
public Server(int port, ExecutorService executor) throws IOException {
serverSocket = new ServerSocket(port);
this.executor = executor;
}
#Override
public void run() throws IOExeption {
while (running) {
Socket socket = serverSocket.accept();
executor.execute(new ConnectionHandler(socket));
}
}
public boolean stop(long timeout, TimeUnit unit) {
running = false;
executor.shutdown();
return executor.awaitTermination(timeout, unit);
}
}
You can use Executors to create an ExecutorService to run the tasks. Note that ConnectionHandler needs to close the socket it is given.
Are you closing out the PrintWriter to flush the stream?
} finally {
printwr.close();
sendsock.close();
}
Ah, sorry. I accidentally removed the commenting from the code. It's actually like this:
public void SendMsg(String dest, Message myMsg) {
Socket sendsock = null;
try {
if(printwr == null) {
sendsock = new Socket(dest, Main.rcvport);
printwr = new PrintWriter(sendsock.getOutputStream(), true);
}
String msgtosend = myMsg.msgtype.toString() + "=" + Main.myaddy + "=" + myMsg.content + "\n";
printwr.print(msgtosend);
} catch (UnknownHostException ex) {
System.out.println(ex);
//DO: Terminate or restart
} catch (IOException ex) {
System.out.println(ex);
//DO: Terminate or restart
}
}
printrw is declared and stored outside the function, so once it's set up, there is no need for sendsock or for reinitializing printrw. In the actual application, I'm storing the PrintWriter for every connection in a HashMap and retrieving it at the start of the SendMsg(...) function.
Since the connections are persistent, every time one is accepted, a new thread is lunch that runs a while loop to check it continuously for data. These threads and connections are only closed once the application is terminated. In addition to my previous question, is there a more efficient way of doing this?
Earlier, I'd implemented this code without the "\n" and using println(...) instead and I still had the issue of some messages not being received, so I'm not sure what is causing the problem. The messages are sent like so:
public class SendPortal2 implements Runnable {
String dest = null;
SendPortal2 (String dest) {
this.dest = dest;
}
public void run() {
for(int i=1; i<1000; i+=2) {
Message myMsg = new Message("Message", Main.myaddy + " " + String.valueOf(i));
Main.myCommMgr.SendMsg(dest, myMsg);
}
}
}
There are two such threads running. When I ran the code again just now, one side got 999 packets whereas the other one only got 500, leading me to believe sometimes the data from an entire thread could be blocked out. Is that likely?
Thanks for the replies!
If I put a Thread.sleep(2) inside the for loop where the SendMsg function is called, more messages are received properly, but it's not always 1000. Could it be possible that the system's resources are being hogged by two threads running while loops continuously?
Does anyone have any good suggestions for creating a Pipe object in Java which is both an InputStream and and OutputStream since Java does not have multiple inheritance and both of the streams are abstract classes instead of interfaces?
The underlying need is to have a single object that can be passed to things which need either an InputStream or an OutputStream to pipe output from one thread to input for another.
It seems the point of this question is being missed. If I understand you correctly, you want an object that functions like an InputStream in one thread, and an OutputStream in another to create a means of communicating between the two threads.
Perhaps one answer is to use composition instead of inheritance (which is recommended practice anyway). Create a Pipe which contains a PipedInputStream and a PipedOutputStream connected to each other, with getInputStream() and getOutputStream() methods.
You can't directly pass the Pipe object to something needing a stream, but you can pass the return value of it's get methods to do it.
Does that work for you?
java.io.PipedOutputStream and java.io.PipedInputStream look to be the classes to use for this scenario. They are designed to be used together to pipe data between threads.
If you really want some single object to pass around it would need to contain one of each of these and expose them via getters.
This is a pretty common thing to do, I think. See this question.
Easy way to write contents of a Java InputStream to an OutputStream
You can't create a class which derives both from InputStream and OutputStream because these aren't interfaces and they have common methods and Java doesn't allow multiple inheritance (the compiler doesn't know whether to call InputStream.close() or OutputStream.close() if you call close() on your new object).
The other problem is the buffer. Java wants to allocate a static buffer for the data (which doesn't change). This means when you use the `java.io.PipedXxxStream', the writing data to it will eventually block unless you use two different threads.
So the answer from Apocalisp is correct: You must write a copy loop.
I suggest that you include Apache's commons-io in your project which contains many helper routines just for tasks like this (copy data between streams, files, strings and all combinations thereof).
See http://ostermiller.org/utils/CircularBuffer.html
I had to implement a filter for slow connections to Servlets so basically I wrapped the servlet output stream into a QueueOutputStream which will add every byte (in small buffers), into a queue, and then output those small buffers to a 2nd output stream, so in a way this acts as input/output stream, IMHO this is better than JDK pipes which won't scale that well, basically there is too much context switching in the standard JDK implementation (per read/write), a blocking queue is just perfect for a single producer/consumer scenario:
import java.io.IOException;
import java.io.OutputStream;
import java.util.concurrent.*;
public class QueueOutputStream extends OutputStream
{
private static final int DEFAULT_BUFFER_SIZE=1024;
private static final byte[] END_SIGNAL=new byte[]{};
private final BlockingQueue<byte[]> queue=new LinkedBlockingDeque<>();
private final byte[] buffer;
private boolean closed=false;
private int count=0;
public QueueOutputStream()
{
this(DEFAULT_BUFFER_SIZE);
}
public QueueOutputStream(final int bufferSize)
{
if(bufferSize<=0){
throw new IllegalArgumentException("Buffer size <= 0");
}
this.buffer=new byte[bufferSize];
}
private synchronized void flushBuffer()
{
if(count>0){
final byte[] copy=new byte[count];
System.arraycopy(buffer,0,copy,0,count);
queue.offer(copy);
count=0;
}
}
#Override
public synchronized void write(final int b) throws IOException
{
if(closed){
throw new IllegalStateException("Stream is closed");
}
if(count>=buffer.length){
flushBuffer();
}
buffer[count++]=(byte)b;
}
#Override
public synchronized void write(final byte[] b, final int off, final int len) throws IOException
{
super.write(b,off,len);
}
#Override
public synchronized void close() throws IOException
{
flushBuffer();
queue.offer(END_SIGNAL);
closed=true;
}
public Future<Void> asyncSendToOutputStream(final ExecutorService executor, final OutputStream outputStream)
{
return executor.submit(
new Callable<Void>()
{
#Override
public Void call() throws Exception
{
try{
byte[] buffer=queue.take();
while(buffer!=END_SIGNAL){
outputStream.write(buffer);
buffer=queue.take();
}
outputStream.flush();
} catch(Exception e){
close();
throw e;
} finally{
outputStream.close();
}
return null;
}
}
);
}
Better to use Pipe or ArrayBlockingQueue, I recommend you not to use PipedInput/OutputStream as they have a bad practice even you can see in the link below that they have asked to be deprecated as it causes many issues.
https://bugs.openjdk.java.net/browse/JDK-8223048
For the BlockingQueue and Pipe here a simple example of that
Pipe:
Pipe pipe = Pipe.open();
Pipe.SinkChannel sinkChannel = pipe.sink();
String newData = "New String to write to file..." + System.currentTimeMillis();
ByteBuffer buf = ByteBuffer.allocate(48);
buf.clear();
buf.put(newData.getBytes());
buf.flip();
while(buf.hasRemaining()) {
sinkChannel.write(buf);
}
Pipe.SourceChannel sourceChannel = pipe.source();
ByteBuffer buf = ByteBuffer.allocate(48);
int bytesRead = inChannel.read(buf);
Reference: http://tutorials.jenkov.com/java-nio/pipe.html
BlockingQueue:
//Shared class used by threads
public class Buffer {
// ArrayBlockingQueue
private BlockingQueue<Integer> blockingQueue = new ArrayBlockingQueue<Integer>(1);
public void get() {
// retrieve from ArrayBlockingQueue
try {
System.out.println("Consumer received - " + blockingQueue.take());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public void put(int data) {
try {
// putting in ArrayBlockingQueue
blockingQueue.put(data);
System.out.println("Producer produced - " + data);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public static void main(String[] args) {
// Starting two threads
ExecutorService executorService = null;
try {
Buffer buffer = new Buffer();
executorService = Executors.newFixedThreadPool(2);
executorService.execute(new Producer(buffer));
executorService.execute(new Consumer(buffer));
} catch (Exception e) {
e.printStackTrace();
}finally {
if(executorService != null) {
executorService.shutdown();
}
}
}
public class Consumer implements Runnable {
private Buffer buffer;
public Consumer(Buffer buffer) {
this.buffer = buffer;
}
#Override
public void run() {
while (true) {
try {
buffer.get();
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
public class Producer implements Runnable {
private Buffer buffer;
public Producer(Buffer buffer) {
this.buffer = buffer;
}
#Override
public void run() {
while (true) {
Random random = new Random();
int data = random.nextInt(1000);
buffer.put(data);
}
}
}
Reference:
https://github.com/kishanjavatrainer/ArrayBlockingQueueDemo/tree/master/ArrayBlockingQueueDemo