Develop Reference

Writing and Reading Objects over Java NIO non blocking SocketChannels

I am new to Socket programming, and I am trying to write a simple chat application.My server utilizes a HashSet of Strings to store user name information and I want to pass the HashSet from the server to the client. I was able to do this before with ObjectOutputStream and ObjectInputStream over regular Sockets. However, now I am using SocketChannel and it does not allow this I get the following exception:
java.nio.channels.IllegalBlockingModeException
So i searched the web on how to do this and everyone says something different, and most go way over my head. Is there a simple way to do this without completely overriding the writeObject and readObject methods myself?
Thank You in advance.

I recommend you either don't use non-blocking mode or don't use serialization.
If you must, you need to serialize to a ByteArrayOutputStream, get the length, send the length, send the bytes, receive the length, keep reading, accumulating data until you have the correct length of data, put it into a ByteArrayInputStream, and deserialize from that. Not easy, especially the hand-waving around 'keep reading'. And it wastes both time and space.

Sending and receiving data using Sockets

i'm trying to learn more about networking so i decided to do a small project using sockets. I have already been successful is creating an echo server which i know is pretty basic.
The next step would be to send all kinds of files (text files, images maybe even videos?) from the server to the client and vice versa. I have been trying to do so with FIleOutputStream, ObjectOutputStream (and input obviously) and i even tried Byte Buffer but by reading the Java docs i'm starting to understand that i really dont know much about Data....
so my questions are:
Can I convert all type of data (text, images, videos) into bytes, send it and then convert it all into the data that it used to be, or do i need a specific function for each type of data?
Can I convert a text file or an image into an Object and then send it with ObjectOutputFile? If so then how can i decompress it in the other side? I've tried ObjectOutputFile.readObject but i dont really know what to do after that.
Here's what I've been trying to do (I won't post all the code because, only this particular part is important i believe:
Server: (out is a DataOutputStream)
ObjectOutputStream oos = new ObjectOutputStream(out);
Path path = Paths.get("stuff/Folder.jpg");
byte[] data = Files.readAllBytes(path);
oos.write(data);
client (in is a DataInputStream):
ObjectInputStream ois = new ObjectInputStream(in);
ois.readObject();
Thanks for all the help in advance, if you think that you know an article or a tutorial that would help me learn more about this it will also help!

Can I convert all type of data (text, images, videos) into bytes, send it and then convert it all into the data that it used to be?
Sure.
Or do I need a specific function for each type of data?
That's not an "or" - if you want to convert several types of data into bytes yourself, then you probably need a separate function for each type. Note that if you are sending files, you don't need any conversion because the content of a file is already just bytes.
Can I convert a text file or an image into an Object and then send it with ObjectOutputFile?
You could send it with ObjectOutputStream.
If so then how can i decompress it in the other side?
With ObjectInputStream.
I've tried ObjectOutputFile.readObject but i dont really know what to do after that.`
There is no such method in Java as ObjectOutputFile.readObject.

it is pretty certain that you can convert any data into bytes. However, the problem is how you could convert it back? Typically, we use Base64 encoder to encode image and when clients get that we can convert it back by Base64 decoder. There is a lot of way to do this. Please Google it.
Yes, you can use Object data to send. You may need to define a new class for the object and then call data back by getter. In client side, you need to declare the Object value as ois.readObject(). Like:
Object object = ois.readObject();
Just add a little bit advice, I feel you are not so familiar with socket right now. So, a text book like Computer Networking(Kurose Ross) will be helpful. Also, if you plan to transmit data between C and Java, then you cannot use Object because C does not support it natively.

Avoid obtaining same InputStream more than once

I can see there are a number of posts regarding reuse InputStream. I understand InputStream is a one-time thing and cannot be reused.
However, I have a use case like this:
I have downloaded the file from DropBox by obtaining the DropBoxInputStream using the DropBox's Java SDK. I then need to upload the file to another system by passing the InputStream. However, as part of the download, I have to provide the MD5 of the file. So I have to read the file from the stream before uploading the file. Because the DropBoxInputStream I received can only be used once, I have to get another DropBoxInputStream after I have calculated the MD5 and before uploading the file. The procedure is like:
Get first DropBoxInputStream
Read from the DropBoxInputStream and calculate MD5
Get the second DropBoxInputStream
Upload file using the MD5 and the second DropBoxInputStream.
I am thinking that, if there are many way for me to "cache" or "backup" the InputStream before I calculate the MD5 so that I can save step 3 of obtaining the same DropBoxInputStream again?
Many thanks
EDIT:
Sorry I missed some information.
What I am currently doing is that I use a MD5DigestOutputStream to calculate MD5. I stream data across the MD5DigestOutputStream and save them locally as a temp file. Once the data goes through the MD5DigestOutputStream, it will calculate the MD5.
I then call a third party library to upload the file using the calculated md5 and a FileInputStream which reads from the temp file.
However, this requires huge disk space sometime and I want to remove the needs to use temp file. The library I use only accepts a MD5 and InputStream. This means I have to calculate the MD5 on my end. My plan is to use my MD5DigestOutputStream to write data to /dev/null (not keeping the file) so that I can calculate theMD5, and get the InputStream from DropBox again and pass that to the library I use. I assume the library will be able to get the file directly from DropBox without the need for me to cache the file either in the memory of at the disk. Will it work?

Input streams aren't really designed for creating copies or re-using, they're specifically for situations where you don't want to read off into a byte array and use array operations on that (this is especially useful when the whole array isn't available, as in, for e.g. socket comunication). You could buffer up into a byte array, which is the process of reading sections from the stream into a byte array buffer until you have enough information.
But that's unnecessary for calculating an md5. Notice that InputStream is abstract, so it needs be implemented in an extended class. It has many implementations- GZIPInputStream, fileinputstream etc. These are, in design pattern speak, decorators of the IO stream: they add extra functionality to the abstract base IO classes. For example, GZIPInputStream gzips up the stream.
So, what you need is a stream to do this for md5. There is, joyfully, a well documented similar thing: see this answer. So you should just be able to pass your dropbox input stream (as it will be itself an input stream) to create a new DigestInputStream, and then you can both take the md5 and continue to read as before.
Worried about type casting? The idea with decorators in Java is that, since the InputStream base class interfaces all the methods and 'beef' you need to do your IO, there's no harm in passing instances of objects inheriting from InputStream in the constructor of each stream implementation, and you can still do the same core IO.
Finally, I should probably answer your actual question- say you still want to "cache" or "backup" the stream anyway? Well, you could just write it to a byte array. This is well documented, but can become faff when your streams get more complicated. Alternatively, try looking at a PushbackInputStream. Here, you can easily write a function to read off n bytes, perform and operation on them, and then restore them to the stream. Generally good to avoid these implementations of streams in Java, as it's bad for memory use, but no worse than buffering everything up which you'd otherwise have to do.
Or, of course, I would have a go with DigestInputStream.
Hope this helps,
Best.

You don't need to open a new InputStream from DropBox.
Once you have read the file from DropBox, you have it locally. So it is either in memory (in a byte array) or you stored it in a local file. Now you can create an InputStream that reads the data from memory (ByteArrayInputStream) or disk (FileInputStream) in order to upload the file.
So instead of caching the InputStream (which you can't) you cache the contents (which you can).

Stream chaining in Java

Is it bad style to keep the references to streams "further down" a filter chain, and use those lower level streams again, or even to swap one type of stream for another? For example:
OutputStream os = new FileOutputStream("file");
PrintWriter pw = new PrintWriter(os);
pw.print("print writer stream");
pw.flush();
pw = null;
DataOutputStream dos = new DataOutputStream(os);
dos.writeBytes("dos writer stream");
dos.flush();
dos = null;
os.close();
If so, what are the alternatives if I need to use the functionality of both streams, e.g. if I want to write a few lines of text to a stream, followed by binary data, or vice versa?

This can be done in some cases, but it's error-prone. You need to be careful about buffers and stuff like the stream headers of ObjectOutputStream.
if I want to write a few lines of text to a stream, followed by binary
data, or vice versa?
For this, all you need to know is that you can convert text to binary data and back but always need to specify an encoding. However, it is also error-prone because people tend to use the API methods that use the platform default encoding, and of course you're basically implementing a parser for a custom binary file format - lots of things can go wrong there.
All in all, if you're creating a file format, especially when mixing text and binary data, it's best to use an existing framework like Google protocol buffers

If you have to do it, then you have to do it. So if you're dealing with an external dependency that you don't have control over, you just have to do it.
I think the bad style is the fact that you would need to do it. If you had to send binary data across sometimes, and text across at others, it would probably be best to have some kind of message object and send the object itself over the wire with Serialization. The data overhead isn't too much if structured properly.

I don't see why not. I mean, the implementations of the various stream classes should protect you from writing invalid data. So long as you're reading it back the same way, and your code is otherwise understandable, I don't see why that would be a problem.
Style doesn't always mean you have to do it the way you've seen others do it. So long as it's logical, and someone reading the code would see what (and why) you're doing it without you needing to write a bunch of comments, then I don't see what the issue is.

Since you're flushing between, it's probably fine. But it might be cleaner to use one OutputStream and just use os.write(string.getBytes()); to write the strings.

How can I send data in binary form over a Java socket?

I've seen lots of examples of sending serialized data over sockets in Java, but all I want is to send some simple integers and a string. And, the problem is I'm trying to communicate these to a binary written in C.
So, bottom line: how can I just send some bytes over a socket in Java?

You can use the simple OutputStream given by the Socket.
From there you can write bytes.
If you want you can also encapsulate this stream in a BufferedOutputStream to have a buffer.

I would really recommend not using the Java Sockets library directly. I've found Netty (from JBoss) to be really easy to implement and really powerful. The Netty ChannelBuffer class comes with a whole host of options for writing different data types and of course to can write your own encoders and decoders to write POJOs down the stream if you wish.
This page is a really good starter - I was able to make a fairly sophisticated client/server with custom encoders and decoders in under 30 minutes reading this: http://docs.jboss.org/netty/3.2/guide/html/start.html.
If you really want to use Java sockets. The socket output stream can be wrapped in a DataOutputStream which allows you to write many different data types as well, for example:
new DataOutputStream(socket.getOutputStream()).writeInt(5);
I hope that's useful.

I would recommend looking into Protocol Buffers for the serialization and ZeroMQ for the data transfer.