Develop Reference

Java - SMTP Transporter requires SocketOutputStream to be open indefinitely

During a refactoring job, I discovered that if I close my outputstream using "try-with-resources" - I always get a MessagingException from java's SMTPTransport. It always complains that the socket was closed.
The code which I identified to have the problems is this:
try (LineOutputStream los = new LineOutputStream(os);) {
los.writeln(signatureHeaderLine);
Enumeration hdrLines = getNonMatchingHeaderLines(ignoreList);
while (hdrLines.hasMoreElements()) {
String notIgnoredLine = (String) hdrLines.nextElement();
los.writeln(notIgnoredLine);
}
los.writeln();
// Send signed mail to waiting DATA command
os.write(osBody.toByteArray());
os.flush();
} catch (MessagingException me) {
// Deal with it
} catch (Exception e) {
// Deal with it
}
The code above is part of an override of MimeMessage.writeTo(OutputStream, String[]) And the issue comes when `issueSendCommand' and 'sendCommand' is eventually called from SMTPTransport.
So does this mean my sockets should remain open all the time? I know from non-technical viewpoint, it doesn't feel right to close the socket since I will be writing messages through it. But I was trying to understand whether this would cause memory leak by any chance in the future.
Regards,

I believe the problem comes because you are using the OutputStream os outside the try-with-resource statement.
The try-with-resource statement ensures that all initialized AutoClosable resources will be closed after execution of the try block. At the moment com.sun.mail.util.LineOutputStream is closed, also the OutputStream os (passed to its constructor) will be closed. Any access to os after the try-with-resource statement acts on a already closed OutputStream.
edit There is an exception, when the OutputStream has a close() method without an effect. Which is the case for example for the ByteArrayOutputStream.
Closing a ByteArrayOutputStream has no effect. The methods in this class can be called after the stream has been closed without generating an IOException.
a snippet to demonstrate
private static void demoMethod(OutputStream os) throws IOException {
try (LineOutputStream los = new LineOutputStream(os)) {
los.writeln("signatureHeaderLine");
los.writeln();
os.write("foo".getBytes());
System.out.println("within try-block");
} catch (Exception e) {
System.out.println(e);
}
os.write("bar".getBytes());
System.out.println("after try-block");
}
calling the method demoMethod with a ByteArrayOutputStream
ByteArrayOutputStream os = new ByteArrayOutputStream();
demoMethod(os);
gives the output
within try-block
after try-block
So the ByteArrayOutputStream can be used even after calling close() on it (which is implicitly called by the LineOutputStream.close() invoked by try-with-resource code).
Doing the same with a FileOutputStream
FileOutputStream os = new FileOutputStream("/tmp/dummy.out");
demoMethod(os);
throws an exception, because the FileOutputStream has been close at the end of the try-with-resource statement.
within try-block
Exception in thread "main" java.io.IOException: Stream Closed
at java.base/java.io.FileOutputStream.writeBytes(Native Method)
at java.base/java.io.FileOutputStream.write(FileOutputStream.java:342)
at Main.demoMethod(Main.java:24)
at Main.main(Main.java:12)

Error while reading data through socket communication

Following scenario that explains my problem.
I've a PLC that acts as a server socket program. I've written a Client Java program to communicate through socket communication with the PLC.
Steps that take place in this process are:
1) For each second my Client program happen to communicate with the PLC, read the data in stream, store the data temporarily in a ByteArrayOutputStream and closing both input stream and socket. Following snippet gives the idea
try {
socket = new Socket(host, port);
is = socket.getInputStream();
outputBuffer = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int read;
if((read = is.read(buffer)) != -1) {
outputBuffer.write(buffer, 0, read);
}
} catch (UnknownHostException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
System.out.println("Before closing the socket");
try {
is.close();
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
System.out.println("After closing the socket");
} catch (Exception e) {
e.printStackTrace();
}
}
2) Processing stored data according to my requirement is what I'm trying to do. So for every 1 second, client program connects to Server, read the data(if data is present), store the data, close socket and process it. And it has to happen for a very long run, probably till the Server program is on. And that may happen till for every few weeks.
3) Problem what I'm facing is, I'm able to run the above show for 1-2 hours, but from then, Client Program unable to fetch the data from the Server Program(PLC in this case), though both are connected through socket. I.e 128 bytes of data present, but Client program isn't able to read that data. And this started happening after program run successfully for almost 2hours
4) Please find the brief code which may help for you to look into.
public class LoggingApplication {
public static void main(String[] args) throws NumberFormatException {
if (args.length > 0 && args.length == 2) {
String ipAddress = mappingService.getIpAddress();
int portNo = (int) mappingService.getPortNo();
ScheduledExecutorService execService = Executors.newScheduledThreadPool(1);
execService.schedule(new MyTask(execService, ipAddress, portNo, mappingService), 1000, TimeUnit.MILLISECONDS);
} else {
throw new IllegalArgumentException("Please pass IPAddress and port no as arguments");
}
}
}
Runnable Code:
public class MyTask implements Runnable {
public ScheduledExecutorService execService;
private String ipAddress;
private int portNo;
private ConfigurationMappingService mappingService;
private MySocketSocketUtil mySocketSocketUtil;
public MyTask(ScheduledExecutorService execService, String ipAddress, int portNo, ConfigurationMappingService mappingService) {
this.execService = execService;
this.ipAddress = ipAddress;
this.portNo = portNo;
this.mappingService = mappingService;
}
public void run() {
MySocketSocketUtil mySocketSocketUtil = new MySocketSocketUtil(ipAddress, portNo);
execService.schedule(new MyTask(execService, ipAddress, portNo, mappingService), 1000, TimeUnit.MILLISECONDS);
mySocketSocketUtil.getData(); //It's able to fetch the data for almost 2 hours but from then, it's just getting empty data and it's keep on giving empty data from then. and so on.
/*
*
*Some code
*/
}
}
Here's where, I'm having the problem
mySocketSocketUtil.getData(); is able to fetch the data for almost 2 hours but from then, it's just getting empty data and it's keep on giving empty data from then. and so on. It's a big question I know, And I want to understand what might have gone wrong.
Edit: I'm ignoring the condition to check end of the stream and closing a socket based on it is because, I knew I'm going to read first 1024 bytes of data only always. And So, I'm closing the socket in finally block

socket = new Socket(host, port);
if(socket != null && socket.isConnected())
It is impossible for socket to be null or socket.isConnected() to be false at this point. Don't write pointless code.
if((read = is.read(buffer)) != -1) {
outputBuffer.write(buffer, 0, read);
};
Here you are ignoring a possible end of stream. If read() returns -1 you must close the socket. It will never not return -1 again. This completely explains your 'empty data':
from then, it's just getting empty data and it's keep on giving empty data from then, and so on
And you should not create a new Socket unless you have received -1 or an exception on the previous socket.
} else {
System.err.println("Socket couldn't be connected");
}
Unreachable: see above. Don't write pointless code.

You should never disconnect from the established connection. Connect once in the LoggingApplication. Once the socket is connected keep it open. Reuse the socket on the next read.

I think there are couple of points you need to fix before getting to the solution to your problem. Please try to follow the following suggestions first:
As #EJP said this code block is not needed.
if(socket != null && socket.isConnected()) {
also you are using a byte array of length 1024 and not using while or for loop to read the data stream. Are you expecting only a block of data which will never exceed 1024 bytes?
byte[] buffer = new byte[1024];
int read;
if((read = is.read(buffer)) != -1) {
This is also not needed as it is unreachable.
} else {
System.err.println("Socket couldn't be connected");
}
Can you explain the data stream behavior you are expecting?
Last but not the least is.read(buffer) is a blocking call so if there is no data to read yet, it will hold the thread execution at that point.
Please try to answer the questions I have asked.
#KishoreKumarKorada from your description in the comment section, it seems like you are monitoring the data change on server side. Socket stream works in a read-once fashion. So,
First thing is, you need to request from server every time and the server needs to RESEND the data on every request.
Second, the way you presented is more like you are operating on byte level, which is not very good way to do that unless you have any legitimate reason to do so. The good way is to wrap the data in JSON or XML format and send it over the stream. But to reduce bandwidth consumption, you may need to operate on byte stream sometimes. You need to decide on that.
Third, for monitoring the data change, the better way is to use some timestamp to compare when the data has changed on the server side and what is the timestamp stored on the client side, if they match, data has not changed. Otherwise fetch the data from the server side and update the client side.
Fourth, when there is data available that you are not able to read, can you debug the ins.read(...) statement to see if its getting executed and the execution goes inside the if block or if statement is evaluated to false? if true then examine the read value and let me know what you have found?
Thanks.

StreamCorruptedException when using ObjectInputStream in socket server side programming [duplicate]

I want to use a single ObjectInputStream to read from a byte array, but I keep getting a StreamCorruptedException every time I start the program.
public void run(){
byte[] receiveBuffer = new byte[65535];
bIn = new ByteArrayInputStream(receiveBuffer);
try {
in = new ObjectInputStream(bIn);
} catch (IOException e1) {
e1.printStackTrace();
}
while(true){
try {
packetIn = new DatagramPacket(receiveBuffer, receiveBuffer.length);
sock.receive(packetIn);
Object o = in.readObject();
//do things with o
}
}
}
I'm just trying to initialize the ObjectInputStream to read from the byte array eventually, but it's throwing that exception even if I remove the while loop.
What am I doing wrong here?

If you take a look at the javadocs for the ObjectInputStream(InputStream) constructor, you'll see:
Creates an ObjectInputStream that reads from the specified InputStream. A serialization stream header is read from the stream and verified. This constructor will block until the corresponding ObjectOutputStream has written and flushed the header.
...
throws
StreamCorruptedException - if the stream header is incorrect
(emphasis added)
In other words, the constructor doesn't just record the InputStream reference you give it, it also reads from that object. In this case, that's a stream of all 0s.
You should defer creating the ObjectInputStream until you have the serialized data (or at least enough of it to read the header).
(In the interest of "teach a person to fish," I'll also note that any time a method/constructor throws an exception you don't expect, that method's javadocs are a good place to start for understanding its behavior. The javadocs for the JDK classes are usually pretty good.)

Ok, this is how object streams work and the solution that works everywhere.
Object stream data is preceded by a 4 byte 'magical' sequence AC ED 00 05. An ObjectInputStream will peek for this data at construction time rather than before the first read. And that's logical: one wants to be sure it is a proper stream before being too far in an application. The sequence is buffered by the ObjectOutputStream at construction time so that it is pushed on the stream at the first write. This method often leads to complexities in buffered situations or transferring via pipes or sockets. Fortunately, there is a just as simple as an effective solution to all these problems:
Flush the ObjectOutputStream immediately after construction!
ObjectOutputStream myStream = new ObjectOutputStream ( anotherStream );
myStream.flush();
In your case, you will have to use a ObjectOutputStream if you want to read from an ObjectInputStream

The stream protocol includes a stream header, which ObjectInputStream reads in the constructor. You have to defer creating the stream until you have received something:
sock.receive(packetIn);
try {
in = new ObjectInputStream(bIn);
Object o = in.readObject();
} catch (IOException e1) {
e1.printStackTrace();
}

Why a new InputStream will still read what is left over from an old InputStream?

Well please see this question and Jon Skeet's answer first.
This time I have this server:
public class SimpleServer {
public static void main(String[] args) throws Exception {
ServerSocket serverSocket = new ServerSocket(8888);
System.out.println("Server Socket created, waiting for client...");
Socket accept = serverSocket.accept();
InputStreamReader inputStreamReader = new InputStreamReader(accept.getInputStream());
char[] chars = new char[5];
System.out.println("Client connected, waiting for input");
while (true) {
inputStreamReader.read(chars,0,chars.length);
for (int i=0;i<5;i++) {
if(chars[i]!='\u0000') {
System.out.print(chars[i]);
}
}
inputStreamReader = new InputStreamReader(accept.getInputStream());
chars = new char[5];
}
}
}
And when I send the characters "123456789" from the client, this is what I exactly see in the Servers terminal, but should not I be seeing only 12345 ?
Why the difference in the behaviour?

Your client has been set up to only send 5 characters at a time, and then flush - so even though the InputStreamReader probably asked for more data than that, it received less, and then found that it could satisfy your request for 5 characters with what it had got.
Try changing the code on the server to only read 3 characters at a time instead of 5 (but leave the client sending 5) and you may well see a difference in behaviour. You may not, mind you - it will depend on a lot of different things around the timing of how the data is moving around.
Basically the lesson should be that you don't want to be constructing multiple readers over the same stream - it becomes hard to predict what will happen, due to buffering.

Java Process "The pipe has been ended" problem

I am using Java Process API to write a class that receives binary input from the network (say via TCP port A), processes it and writes binary output to the network (say via TCP port B). I am using Windows XP. The code looks like this. There are two functions called run() and receive(): run is called once at the start, while receive is called whenever there is a new input received via the network. Run and receive are called from different threads.
The run process starts an exe and receives the input and output stream of the exe. Run also starts a new thread to write output from the exe on to the port B.
public void run() {
try {
Process prc = // some exe is `start`ed using ProcessBuilder
OutputStream procStdIn = new BufferedOutputStream(prc.getOutputStream());
InputStream procStdOut = new BufferedInputStream(prc.getInputStream());
Thread t = new Thread(new ProcStdOutputToPort(procStdOut));
t.start();
prc.waitFor();
t.join();
procStdIn.close();
procStdOut.close();
} catch (Exception e) {
e.printStackTrace();
printError("Error : " + e.getMessage());
}
}
The receive forwards the received input from the port A to the exe.
public void receive(byte[] b) throws Exception {
procStdIn.write(b);
}
class ProcStdOutputToPort implements Runnable {
private BufferedInputStream bis;
public ProcStdOutputToPort(BufferedInputStream bis) {
this.bis = bis;
}
public void run() {
try {
int bytesRead;
int bufLen = 1024;
byte[] buffer = new byte[bufLen];
while ((bytesRead = bis.read(buffer)) != -1) {
// write output to the network
}
} catch (IOException ex) {
Logger.getLogger().log(Level.SEVERE, null, ex);
}
}
}
The problem is that I am getting the following stack inside receive() and the prc.waitfor() returns immediately afterwards. The line number shows that the stack is while writing to the exe.
The pipe has been ended
java.io.IOException: The pipe has been ended
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:260)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
at java.io.FilterOutputStream.write(FilterOutputStream.java:80)
at xxx.receive(xxx.java:86)
Any advice about this will be appreciated.

This means you are writing to the pipe after the other end has already closed it.
That indicates a major error in your application protocol.

I have had the same problem recently and I have found a solution.
First of all, "The pipe has been ended" error is not a Java error - it comes from Windows system. According to MSDN:
The using process has closed the pipe or, if you are trying to write
to the pipe, there are no available readers.
Not very informative. However, if process has closed the pipe itself, it may mean that some errors occurred in process.
To check this, redirect errors coming from process, for instance, to a file:
File f = new File("errors.txt");
pb.redirectError(f);
In my case (I've been trying to execute SrcML parser) file contained this:
.\libs\srcML-Win\src2srcml.exe: unrecognised option `--language Java'
Try 'src2srcml --help' for more information.
Fixing this solved the problem.