Develop Reference

Code after readObject() does not run

I am working on a UI which reads serialized object from zookeeper, deserializes it and then converts it into JSON. For some reason I am not able to deseerialize the MQTopic object. But I am able to do the same with other objects.
This is the part which converts the byte[] into the MQTopic object.
if (tester != null && tester.contains("com.ibm.mq.jms.MQTopic")) {
System.out.println(getValue());
ByteArrayInputStream in = new ByteArrayInputStream(this.value);
ObjectInputStream is = new ObjectInputStream(in);
System.out.println("after deserializing..");
topic = (MQTopic) is.readObject();
System.out.println("after typecasting..");
System.out.println(topic.getTopicName());
System.out.println(topic.toString());
is.close();
in.close();
}
Here value is a byte array of the object after serialization.
Nothing runs after topic = (MQTopic) is.readObject(); . Not even the print statements. The program neither terminates nor an exception is thrown or caught.
EDIT : Whole Method
public String getStrValue() {
FtpConnectionInfo ftp = null;
MQTopic topic = null;
try {
String tester = new String(this.value, "UTF-8");
if (tester != null && tester.contains("FtpConnectionInfo")) {
ByteArrayInputStream in = new ByteArrayInputStream(this.value);
ObjectInputStream is = new ObjectInputStream(in);
ftp = (FtpConnectionInfo) is.readObject();
in.close();
is.close();
Gson gson = new Gson();
return gson.toJson(ftp);
} else if (tester != null
&& tester.contains("com.ibm.mq.jms.MQTopic")) {
ByteArrayInputStream in = new ByteArrayInputStream(this.value);
ObjectInputStream is = new ObjectInputStream(in);
System.out.println("after deserializing..");
topic = (MQTopic) is.readObject();
System.out.println("after typecasting..");
System.out.println(topic.getTopicName());
System.out.println(topic.toString());
is.close();
in.close();
} else {
return new String(this.value, "UTF-8");
}
} catch (UnsupportedEncodingException ex) {
System.out.println("unsupported error ");
ex.printStackTrace();
//logger.error(Arrays.toString(ex.getStackTrace()));
} catch (Exception e) {
System.out.println("Exception in new logic.");
e.printStackTrace();
}
System.out.println("im out of try");
return null;
}
The FTP if loop works fine, but the Topic loop does not work beyond typecasting.
EDIT 2: This how the other team stores the object into Zookeeper
public static byte[] serialize(Object obj) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
ObjectOutputStream os = new ObjectOutputStream(out);
os.writeObject(obj);
return out.toByteArray();
}
The byte[] is stored in Zookeeper and that is what I am retrieving in my UI.
EDIT 3: I did a debug of the process and at the point where is is called, these are the values. Can anyone tell me if the object is right ?

You're doing this wrong. You should deserialize the object first, and then use instanceof to see what type it is. Converting binary data to String is poor practice at the best of times.
Your actual symptom is not credible. An exception must be thrown, or else you are blocking earlier than stated.

ObjectInputStream's readObject is a blocking method. First check using the available method if there is something to read without blocking.
available would most probably return 0 in this case.
This might be only half the solution you are looking for, but I think this would let you know if you have anything to read or not.

Deserializing multiple objects from file without using while(true)

I have a block of code, that deserializes multiple objects from file. How can i avoid using a while(true)?
ObjectInputStream in = new ObjectInputStream(new FileInputStream(
filename));
while (true) {
try {
MyObject o = (MyObject) in.readObject();
// Do something with the object
} catch (EOFException e) {
break;
}
}
in.close();

You should write either a collection (with a size), or a put a marker before each object:
try {
for (;in.readBoolean();) {
MyObject o = (MyObject) in.readObject();
}
} catch (EOFException e) {
// ...
}
When you write your object, write a boolean just before (it will however take 1 byte if I do remember well that part):
for (MyObject o : iterable) {
out.writeBoolean(true);
out.writeObject(o);
}
out.writeBoolean(false);
If iterable is a collection or map, you can use default serialization:
out.writeObject(iterable); // default collection serialization
Beside, don't catch an exception for each item, catch it globally (especially EOFException!): it is better for performance reasons.
I don't know if you work with Java 7, but your code + my for loop can be written like this:
try (ObjectInputStream in = new ObjectInputStream(new FileInputStream( filename))) {
for (;in.readBoolean();) {
MyObject o = (MyObject) in.readObject();
}
} catch (EOFException e) {
// ...
}
// no need to close, the try-with-resources do the job for you.

How can i avoid using a while(true)?
You can't.
More to the point, why do you think you want to?
This is a classic example of the tail wagging the dog. EOFException is thrown to indicate end of stream. Ergo you have to catch it, and ergo you have to loop until it is thrown, ergo you have to use while (true) or one of its cognates.
The exception thought police would have you prepend an object count, taking the curious position that external data structures should be designed to suit the coder's phobias, and overlooking that you may not know it in advance, or may need to change your mind, or may need to exit prematurely; or would have you write a null as an end-of-stream marker, overlooking that it prevents the use of null for any other purpose; and in both cases overlooking the fact that the API is already designed to throw EOFException, and already works the way it already works, so you already have to code accordingly.

The code that I'm proposing let you to serialize and deserialize multiple objects really easily without having any problems and avoiding the awful (in my opinion) while true:
public class EntityClass implements Serializable{
private int intVal;
private String stringVal;
public EntityClass(int intVal, String stringVal) {
this.intVal = intVal;
this.stringVal = stringVal;
}
#Override
public String toString() {
return "EntityClass{" +
"intVal=" + intVal +
", stringVal='" + stringVal + '\'' +
'}';
}
public static void main(String[] args) throws IOException, ClassNotFoundException {
EntityClass a = new EntityClass(1, "1");
EntityClass b = new EntityClass(2, "2");
EntityClass c = new EntityClass(3, "3");
ObjectOutputStream stream = new ObjectOutputStream(new FileOutputStream("out"));
stream.writeObject(a);
stream.writeObject(b);
stream.writeObject(c);
stream.close();
ObjectInputStream streamRead = new ObjectInputStream(new FileInputStream("out"));
EntityClass[] entities = new EntityClass[3];
int cont = 0;
try {
while (streamRead.available() >= 0) {
entities[cont] = (EntityClass) streamRead.readObject();
System.out.println(entities[cont]);
cont++;
}
} catch (EOFException exp) {
} finally {
streamRead.close();
}
}
}

How to write a byte array to OutputStream of process builder (Java)

byte[] bytes = value.getBytes();
Process q = new ProcessBuilder("process","arg1", "arg2").start();
q.getOutputStream().write(bytes);
q.getOutputStream().flush();
System.out.println(q.getInputStream().available());
I'm trying to stream file contents to an executable and capture the output but the output(InputStream) is always empty. I can capture the output if i specify the the file location but not with streamed input.
How might I overcome this?

Try wrapping your streams with BufferedInputStream() and BufferedOutputStream():
http://download.oracle.com/javase/6/docs/api/java/lang/Process.html#getOutputStream%28%29
Implementation note: It is a good idea for the output stream to be buffered.
Implementation note: It is a good idea for the input stream to be buffered.
Even with buffered streams, it is still possible for the buffer to fill if you're dealing with large amounts of data, you can deal with this by starting a separate thread to read from q.getInputStream(), so you can still be reading from the process while writing to the process.

Perhaps the program you execute only starts its work when it detects the end of its input data. This is normally done by waiting for an EOF (end-of-file) symbol. You can send this by closing the output stream to the process:
q.getOutputStream().write(bytes);
q.getOutputStream().close();
Try this together with waiting for the process.

I dont know if something else may also be wrong here, but the other process ("process") does not even have time to respond, you are not waiting for it (the method available() does not block). To try this out you can first insert a sleep(2000) after the flush(), and if that works you should switch to query'ing q.getInputStream().available() multiple times with short pauses in between.

I think, you have to wait, until the process finished.
I implemented something like this this way:
public class ProcessReader {
private static final int PROCESS_LOOP_SLEEP_MILLIS = 100;
private String result;
public ProcessReader(Process process) {
BufferedReader resultReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
StringBuilder resultOutput = new StringBuilder();
try {
while (!checkProcessTerminated(process, resultReader, resultOutput)) {
}
} catch (Exception ex1) {
throw new RuntimeException(ex1);
}
result = resultOutput.toString();
}
public String getResult(){
return result;
}
private boolean checkProcessTerminated(Process process, BufferedReader resultReader, StringBuilder resultOutput) throws Exception {
try {
int exit = process.exitValue();
return true;
} catch (IllegalThreadStateException ex) {
Thread.sleep(PROCESS_LOOP_SLEEP_MILLIS);
} finally {
while (resultReader.ready()) {
String out = resultReader.readLine();
resultOutput.append(out).append("\n");
}
}
return false;
}
}
I just removed now some specific code, that you dont need, but it should work, try it.
Regards

handling EOFException in java

I have created a method in my java assignment to write into a file from a LinkedList (I used serialization) , then I have created another method to read the file into the inkedList. The following is my method's body:
try {
FileInputStream fin = new FileInputStream("c:\\Info.ser");
ObjectInputStream ois = new ObjectInputStream(fin);
Employee e = (Employee) ois.readObject();
linkP.add(e);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
but it doesnt work right. I think this part:
Employee e = (Employee) ois.readObject();
linkP.add(e);
reads only the first object of the file into the linkedlist and ignores other objects. I surrounded it for loop and while loop several times but it causes EOFException. How can I change my method to read all of the file's objects into the LinkedList?

If you used LinkedList for serialization you should expect a LinkedList to deserialize:
linkP= (LinkedList) ois.readObject();
instead of
Employee e = (Employee) ois.readObject();
linkP.add(e);

The easiest way is to include the size of the list as the first thing written to the file. When you read the file, the first thing you retrieve is the size. Then you can read the expected number of objects.

Are you sure that the serialized file contains all of the elements? It looks to me like you might only be serializing one.

Note: Please also add the code where you create the info.ser file, since you may have corrupted the ObjectOutputStream by closing/reopening it for each object.
But to answer your question, the proper way of doing it (without catching exceptions) would be:
#Test
public void testSerializingListByEntries() throws Exception {
List<Serializable> list = new ArrayList<Serializable>();
list.add(new Date());
list.add(new Date());
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeInt(list.size()); // Magic
for(Serializable o : list) {
oos.writeObject(o);
}
oos.close();
ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
ObjectInputStream ois = new ObjectInputStream(bais);
int count = ois.readInt();
List<Object> newList = new ArrayList<Object>();
for(int i = 0; i < count;i++) {
newList.add(ois.readObject());
}
ois.close();
assertEquals(list,newList);
}
Yes, you need to close the streams yourself of course. Omitted for readability.

Would probably need to see how you're writing in the first place but generally:
ObjectInputStream is = null;
try
{
is = new ObjectInputStream(new FileInputStream("c:/Info.ser"));
Object object = null;
while ((object = is.readObject()) != null)
{
linkP.add(object);
}
}
catch(Exception e)
{
//Whatever you need to do
}
finally
{
//Never forget to close your streams or you'll run into memory leaks
try
{
if (is != null)
{
is.close();
}
}
catch (Exception ex)
{
ex.printStackTrace();
}
}
Also, its probably better practice for you to handle the exceptions individually but I can't really tell what the streams throw so replace the (Exception e) with everything else.

surround Employee e = (Employee) ois.readObject();
linkP.add(e);
with a for loop as you suggested and surround the .readObject call with a try/catc(EOFException)

Just catch EOFException separately inside your reading loop and process it accordingly, i.e. break out of the loop.

Capturing large amounts of output from Apache Commons-Exec

I am writing a video application in Java by executing ffmpeg and capturing its output to standard output. I decided to use Apache Commons-Exec instead of Java's Runtime, because it seems better. However, I am have a difficult time capturing all of the output.
I thought using pipes would be the way to go, because it is a standard way of inter-process communication. However, my setup using PipedInputStream and PipedOutputStream is wrong. It seems to work, but only for the first 1042 bytes of the stream, which curiously happens to be the value of PipedInputStream.PIPE_SIZE.
I have no love affair with using pipes, but I want to avoid use disk I/O (if possible), because of speed and volume of data (a 1m 20s video at 512x384 resolution produces 690M of piped data).
Thoughts on the best solution to handle large amounts of data coming from a pipe? My code for my two classes are below. (yes, sleep is bad. Thoughts on that? wait() and notifyAll() ?)
WriteFrames.java
public class WriteFrames {
public static void main(String[] args) {
String commandName = "ffmpeg";
CommandLine commandLine = new CommandLine(commandName);
File filename = new File(args[0]);
String[] options = new String[] {
"-i",
filename.getAbsolutePath(),
"-an",
"-f",
"yuv4mpegpipe",
"-"};
for (String s : options) {
commandLine.addArgument(s);
}
PipedOutputStream output = new PipedOutputStream();
PumpStreamHandler streamHandler = new PumpStreamHandler(output, System.err);
DefaultExecutor executor = new DefaultExecutor();
try {
DataInputStream is = new DataInputStream(new PipedInputStream(output));
YUV4MPEGPipeParser p = new YUV4MPEGPipeParser(is);
p.start();
executor.setStreamHandler(streamHandler);
executor.execute(commandLine);
} catch (IOException e) {
e.printStackTrace();
}
}
}
YUV4MPEGPipeParser.java
public class YUV4MPEGPipeParser extends Thread {
private InputStream is;
int width, height;
public YUV4MPEGPipeParser(InputStream is) {
this.is = is;
}
public void run() {
try {
while (is.available() == 0) {
Thread.sleep(100);
}
while (is.available() != 0) {
// do stuff.... like write out YUV frames
}
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

The problem is in the run method of YUV4MPEGPipeParser class. There are two successive loops. The second loop terminates immediately if there are no data currently available on the stream (e.g. all input so far was processed by parser, and ffmpeg or stream pump were not fast enough to serve some new data for it -> available() == 0 -> loop is terminated -> pump thread finishes).
Just get rid of these two loops and sleep and just perform a simple blocking read() instead of checking if any data are available for processing. There is also probably no need for wait()/notify() or even sleep() because the parser code is started on a separate thread.
You can rewrite the code of run() method like this:
public class YUV4MPEGPipeParser extends Thread {
...
// optimal size of buffer for reading from pipe stream :-)
private static final int BUFSIZE = PipedInputStream.PIPE_SIZE;
public void run() {
try {
byte buffer[] = new byte[BUFSIZE];
int len = 0;
while ((len = is.read(buffer, 0, BUFSIZE) != -1) {
// we have valid data available
// in first 'len' bytes of 'buffer' array.
// do stuff.... like write out YUV frames
}
} catch ...
}
}