Develop Reference

Chat client-server with file transfer and hybrid cryptography

I'm trying to realize a chat client-server (in local) that permits to exchange text and files. I utilize java.security and java.crypto to implement hybrid cryptography (and the tool Swing).
I exchange text in a serialized way (using ObjectInputStream and ObjectOutputStream), inserting it (after I crypted it with an apposite function) in byte[] format in Message (that is an object that i created and that is effectively exchanged between sockets):
import java.io.Serializable;
public class Message implements Serializable{
private static final long serialVersionUID = 1L;
byte[] data;
Message(byte[] data){
this.data = data;
}
byte[] getData(){
return data;
}
}
Chat works fine until I exchange only Message. Now I'm trying to implement file transfer (first I tried to implement file transfer from server to client, without cryptography), therefor I took a file "selectedFile" with FileChoser and I sent it to the client thanks ObjectOutputStream's method writeObject(selectedFile). On the client side I recognized if the object that is arrived is File or Message with:
class ListenFromServer extends Thread{
public void run(){
while(true){
try{
if((Message.class.isInstance(sInput.readObject()))==true){ //I verify if the recived object belongs to Message class
m=(Message) sInput.readObject();//sInput is an instance of ObjectInputStream class, connected to client socket's InputeStream
decryptMessage(m.getData()); //function that decrypts the content of m and inserts the result in a String that after I append in a text area
}
else
{
File recivedFile= (File) sInput.readObject();
File saveFile=new File(path+"/"+ recivedFile.getName());
save(recivedFile,saveFile);//function that insert the recived file in a specific folder
System.out.println("file ricevuto");
}
} catch (ClassNotFoundException ex) {
System.out.println("INFO: Classe non trovata durante la lettura dell'oggetto Messaggio: " + ex.getMessage() + "\n");
} catch(IOException e){
System.out.println("Connection closed");
break;
}
catch(NullPointerException ne){
ne.printStackTrace();
ne.getCause();
System.out.println("Errore"+ ne.getMessage());
}
The problem is that file is recived by the client only clicking twice server's "sendFile" button , moreover now this problem regards also the action of sending text to client, because client receives Message object only when I send it twice (I use two different methods to send a Message object and a File object).
This problem doesn't occur when I eliminate instruction:
if((Message.class.isInstance(sInput.readObject()))==true){
...
}
I ask you how to overcome this problem, or if there is a better way to distinguish File and Message objects in reception.

You're actually reading two objects in sequence, not one.
sInput.readObject()
This is an order to read an object. You give this twice in sequence, so that's two requests to reads different objects.
To fix this, just read the object once, test the type of the object, and cast it when appropriate:
Object inputObject = sInput.readObject(); // Read the object once
if (inputObject instanceof Message) { // If the object is a message
Message m = (Message) inputObject; // cast as a Message
... // use the Message m
} else if (inputObject instanceof File) { // else if it's a file
File f = (File) inputObject; // cast as a File
... // use the File f
}

Too much data on Node.js socket?

I'm currently developing a system that gets data from a battery pack of an electric vehicle, stores it in a database and display it on a screen.
So I have a Java - Application that reads the data from a hardware interface, interprets the values and sends it via Socket to a Node.js-Server. (Java App and Webserver are running on the same computer, so Url = localhost)
JAVA APP:
s = new Socket();
s.connect(new InetSocketAddress(URL, PORT));
out = new PrintWriter( s.getOutputStream(), true);
for (DataEntry e : entries){
out.printf(e.toJson());
}
NODE:
sock.on('data', function(data) {
try{
var data = JSON.parse(data);
db.serialize(function(){
db.run("INSERT INTO DataEntry(value, MessageField, time) values(" + data.value + "," + data.messageFieldID + ", STRFTIME('%Y-%m-%d %H:%M:%f'))");
});
} catch(e){}
});
I get about 20 Messages per second from the hardware interface which are converted into 100 Json - Strings. So the webserver has to process one message in 10 ms, which I thought, is manageable.
But here is the problem: If my entries - List (foreach loop) has more than 2 elements, the webserver gets 2 or more of the Json's in one message.
So the first message was divided into 2 parts (ID 41,42) and was processed correctly. But the second message was divided into 5 parts (ID 43-47), and the first 4 of them weren't sent alone, so only the last one was saved correctly.
How can I ensure, that every Json is sent one another?
Isn't there something like a buffer so that the socket.on method is called correctly for every message I send?
I hope somebody of you can help me
Thank you!
Benedikt :)

TCP sockets are just streams and you shouldn't make any assumptions about how much of a "message" is contained in a single packet.
A simple solution to this is to terminate each message with a newline character since JSON cannot contain such a character. From there it's a simple matter of buffering data until you see a newline character. Then call JSON.parse() on your buffer. For example:
var buf = '';
sock.on('data', function(data) {
buf += data;
var p;
// Use a while loop as it may be possible to have multiple
// messages buffered depending on chunk contents
while (~(p = buf.indexOf('\n'))) {
try {
var msg = JSON.parse(buf.slice(0, p));
} catch (ex) {
console.log('Bad JSON message: ' + ex);
}
buf = buf.slice(p + 1);
}
});
You will also need to change printf() to println() on the Java-side so that a newline character will be appended to each message.

make mail mark as unread after polling - Apache Camel

Am using apache camel, With Polling consumer, when poll my mail is mark as read.
options : delete=false&peek=false&unseen=true
After polling , when i am processing the attachment, if any error occurs , i want to make the mail as "unread". So that i can pool again later.
public void process(Exchange exchange) throws Exception {
Map<String, DataHandler> attachments = exchange.getIn().getAttachments();
Message messageCopy = exchange.getIn().copy();
if (messageCopy.getAttachments().size() > 0) {
for (Map.Entry<String, DataHandler> entry : messageCopy.getAttachments().entrySet()) {
DataHandler dHandler = entry.getValue();
// get the file name
String filename = dHandler.getName();
// get the content and convert it to byte[]
byte[] data =
exchange.getContext().getTypeConverter().convertTo(byte[].class, dHandler.getInputStream());
log.info("Downloading attachment, file name : " + filename);
InputStream fileInputStream = new ByteArrayInputStream(data);
try {
// Processing attachments
// if any error occurs here, i want to make the mail mark as unread
} catch (Exception e) {
log.info(e.getMessage());
}
}
}
}
I noticed the option peek, by setting it to true, It will not make the mail mark as read during polling, in that case is there any option to make it mark as read after processing.

To get the result that you want you should have options
peek=true&unseen=true
The peek=true option is supposed to ensure that messages remain the exact state on the mail server as they where before polling even if there is an exception. However, currently it won't work. This is actually a bug in Camel Mail component. I've submitted a patch to https://issues.apache.org/jira/browse/CAMEL-9106 and this will probably be fixed in a future release.
As a workaround you can set mapMailMessages=false but then you will have to work with the email message content yourself. In Camel 2.15 onward you also have postProcessAction option and with that you could probably remove the SEEN flags from messages with processing errors. Still, I would recommend waiting for the fix though.

We can set the mail unread flag with the following code
public void process(Exchange exchange) throws Exception {
final Message mailMessage = exchange.getIn(MailMessage.class).getMessage();
mailMessage.setFlag(Flag.SEEN, false);
}

BufferedReader inconsistently hanging on my socket's inputstream

I am writing a Java HTTP server. I thought the entire server was working and it is using threading. However, I'm realizing that the piece of code that reads the request into a BufferedReader is not working consistently.
Here is the code that reads an incoming request:
private String receive(WebSocket webSocket) throws IOException {
int chr;
System.out.println("Receiving!");
StringBuffer buffer = new StringBuffer();
while ( (chr = webSocket.in().read() ) != -1) {
buffer.append((char) chr);
if ( !webSocket.in().ready())
break;
}
return buffer.toString();
}
My Websocket class just wraps the Socket and provides an in and an out. I did this so that I could mock out the socket and test my server.
The Websocket class looks like this:
package http.server.socket;
import java.io.*;
import java.net.Socket;
public class SystemSocket implements WebSocket {
private Socket theConnection;
private BufferedReader in;
private OutputStream out;
public SystemSocket(Socket theConnection) throws IOException {
this.theConnection = theConnection;
in = new BufferedReader(new InputStreamReader(theConnection.getInputStream()));
out = new BufferedOutputStream(theConnection.getOutputStream());
}
public BufferedReader in() throws IOException {
return in;
}
public OutputStream out() throws IOException {
return out;
}
public void close() throws IOException {
in.close();
out.close();
theConnection.close();
}
}
The problem is that with each url the user enters in a browser, two requests are made - one for the page requested and one for the favicon. Sometimes - it seems - the favicon request is not coming in and the thread hangs.
Here's some debugging information I have printing to the console when things go right:
Receiving!
Receiving!
REQUEST STRING = GET /color_picker.html HT
[20130821 20:29:23] REQUEST: http://localhost:5000/color_picker.html
[20130821 20:29:23] PAGE RENDERED
REQUEST STRING = GET /favicon.ico HTTP/1.1
[20130821 20:29:23] REQUEST: http://localhost:5000/favicon.ico
[20130821 20:29:23] PAGE RENDERED
The "Receiving" message is getting printed whenever the request is getting read. So, in this case, the "Receiving" message got printed twice, two requests came in and two things were rendered. But then, the same page (but at a different time) will do this (after about 10 seconds):
Receiving!
Receiving!
REQUEST STRING = GET /color_picker.html HTTP/1.1
[20130821 20:41:25] REQUEST: http://localhost:5000/color_picker.html
[20130821 20:41:25] PAGE RENDERED
REQUEST STRING =
Exception in thread "ServerThread" java.lang.ArrayIndexOutOfBoundsException: 1
at http.request.Parser.setRequestLineData(Parser.java:42)
at http.request.Parser.setRequestHash(Parser.java:27)
at http.request.Parser.parse(Parser.java:13)
at http.request.Request.get(Request.java:18)
at http.server.ServerThread.run(ServerThread.java:39)
All the subsequent errors are because the request string is null. But I can't figure out why the Request string is null. I can't even figure out how to debug.
Can anyone help??
Also important to note that if the second request string doesn't come in right away, the user can request a new url and it will cause the second hung process to complete (so then the fourth request url will be what hangs). So, it's only when the user stops requesting things, on the last request after about 10 seconds, I will get the error. Sometimes I can request 20 different pages and it's only after I stop requesting pages and wait a few seconds, that I will see an error. I think this is what is happening??
UPDATE:
Per the request, here is the setRequestLineData() method:
private void setRequestLineData() {
requestHash = new HashMap<String, String>();
if (requestLineParts.length == 3) {
requestHash.put("httpMethod", requestLineParts[0]);
requestHash.put("url", requestLineParts[1]); //line 42
requestHash.put("httpProtocol", requestLineParts[2]);
}
else {
requestHash.put("httpMethod", requestLineParts[0]);
requestHash.put("url", requestLineParts[1]);
requestHash.put("queryString", requestLineParts[2]);
requestHash.put("httpProtocol", requestLineParts[3]);
}
}
UPDATE:
I think I figured out more about what is going on here with my mentor's help. His thought is that once a request is received, the browser starts another request right away to reduce load time for the next request. This sound plausible to me since I can load page after page after page, but it's only about 10 seconds after the last page is requested that I get an error. Currently, I'm handling this with a custom exception, but am working on a better solution. Thanks for all the help guys!

ready() isn't a valid test for end of message. It only tells you whether there is data available to be read without blocking. TCP isn't a message-oriented protocol, it is a byte-stream protocol. If you want messages you must implement them yourself, e.g. as lines, length-value tuples, type-length-value tuples, serialized objects, XML documents, ...
There are few if any correct uses of ready() (or available()), and this isn't one of them.

Java data object for bidirectional I/O

I am developing an interface that takes as input an encrypted byte stream -- probably a very large one -- that generates output of more or less the same format.
The input format is this:
{N byte envelope}
- encryption key IDs &c.
{X byte encrypted body}
The output format is the same.
Here's the usual use case (heavily pseudocoded, of course):
Message incomingMessage = new Message (inputStream);
ProcessingResults results = process (incomingMessage);
MessageEnvelope messageEnvelope = new MessageEnvelope ();
// set message encryption options &c. ...
Message outgoingMessage = new Message ();
outgoingMessage.setEnvelope (messageEnvelope);
writeProcessingResults (results, message);
message.writeToOutput (outputStream);
To me, it seems to make sense to use the same object to encapsulate this behaviour, but I'm at a bit of a loss as to how I should go about this. It isn't practical to load all of the encrypted body in at a time; I need to be able to stream it (so, I'll be using some kind of input stream filter to decrypt it) but at the same time I need to be able to write out new instances of this object. What's a good approach to making this work? What should Message look like internally?

I won't create one class to handle in- and output - one class, one responsibility. I would like two filter streams, one for input/decryption and one for output/encryption:
InputStream decrypted = new DecryptingStream(inputStream, decryptionParameters);
...
OutputStream encrypted = new EncryptingStream(outputSream, encryptionOptions);
They may have something like a lazy init mechanism reading the envelope before first read() call / writing the envelope before first write() call. You also use classes like Message or MessageEnvelope in the filter implementations, but they may stay package protected non API classes.
The processing will know nothing about de-/encryption just working on a stream. You may also use both streams for input and output at the same time during processing streaming the processing input and output.

Can you split the body at arbitrary locations?
If so, I would have two threads, input thread and output thread and have a concurrent queue of strings that the output thread monitors. Something like:
ConcurrentLinkedQueue<String> outputQueue = new ConcurrentLinkedQueue<String>();
...
private void readInput(Stream stream) {
String str;
while ((str = stream.readLine()) != null) {
outputQueue.put(processStream(str));
}
}
private String processStream(String input) {
// do something
return output;
}
private void writeOutput(Stream out) {
while (true) {
while (outputQueue.peek() == null) {
sleep(100);
}
String msg = outputQueue.poll();
out.write(msg);
}
}
Note: This will definitely not work as-is. Just a suggestion of a design. Someone is welcome to edit this.

If you need to read and write same time you either have to use threads (different threads reading and writing) or asynchronous I/O (the java.nio package). Using input and output streams from different threads is not a problem.
If you want to make a streaming API in java, you should usually provide InputStream for reading and OutputStream for writing. This way those can then be passed for other APIs so that you can chain things and so get the streams go all the way as streams.
Input example:
Message message = new Message(inputStream);
results = process(message.getInputStream());
Output example:
Message message = new Message(outputStream);
writeContent(message.getOutputStream());
The message needs to wrap the given streams with a classes that do the needed encryption and decryption.
Note that reading multiple messages at same time or writing multiple messages at same time would need support from the protocol too. You need to get the synchronization correct.

You should check Wikipedia article on different block cipher modes supporting encryption of streams. The different encryption algorithms may support a subset of these.
Buffered streams will allow you to read, encrypt/decrypt and write in a loop.
Examples demonstrating ZipInputStream and ZipOutputStream could provide some guidance on how you may solve this. See example.

What you need is using Cipher Streams (CipherInputStream). Here is an example of how to use it.

I agree with Arne, the data processor shouldn't know about encryption, it just needs to read the decrypted body of the message, and write out the results, and stream filters should take care of encryption. However, since this is logically operating on the same piece of information (a Message), I think they should be packaged inside one class which handles the message format, although the encryption/decryption streams are indeed independent from this.
Here's my idea for the structure, flipping the architecture around somewhat, and moving the Message class outside the encryption streams:
class Message {
InputStream input;
Envelope envelope;
public Message(InputStream input) {
assert input != null;
this.input = input;
}
public Message(Envelope envelope) {
assert envelope != null;
this.envelope = envelope;
}
public Envelope getEnvelope() {
if (envelope == null && input != null) {
// Read envelope from beginning of stream
envelope = new Envelope(input);
}
return envelope
}
public InputStream read() {
assert input != null
// Initialise the decryption stream
return new DecryptingStream(input, getEnvelope().getEncryptionParameters());
}
public OutputStream write(OutputStream output) {
// Write envelope header to output stream
getEnvelope().write(output);
// Initialise the encryption
return new EncryptingStream(output, getEnvelope().getEncryptionParameters());
}
}
Now you can use it by creating a new message for the input, and one for the output:
OutputStream output; // This is the stream for sending the message
Message inputMessage = new Message(input);
Message outputMessage = new Message(inputMessage.getEnvelope());
process(inputMessage.read(), outputMessage.write(output));
Now the process method just needs to read chunks of data as required from the input, and write results to the output:
public void process(InputStream input, OutputStream output) {
byte[] buffer = new byte[1024];
int read;
while ((read = input.read(buffer) > 0) {
// Process buffer, writing to output as you go.
}
}
This all now works in lockstep, and you don't need any extra threads. You can also abort early without having to process the whole message (if the output stream is closed for example).