Develop Reference

Spring-remoting SimpleHttpInvokerRequestExecutor retries request when it shouldn't

I am investigating a quite strange problem. The project I'm working on uses Spring-remoting to invoke methods over http. From what I have gathered so far the following happens:
My client code executes a request to the server
The server starts handling the request, but is slow
25-30 seconds later, a new request comes in to the server
The second request finishes, the client continues its processing
A while later, the first request get completed, but the client no longer cares
Since my client code executes only one request to the Spring remoting client, and the client continuous on after the second invocation it receives is completed, I can only conclude that this occurs somewhere in the Spring remoting client.
The client uses AbstractHttpInvokerRequestExecutor to make the actual http-invocation, and this in turn uses SimpleHttpInvokerRequestExecutor to make the request. But, from what I can read, this has no mechanism to retry the requests. So now I'm quite stuck.
Can anyone think of what might cause this behaviour? (I have tried to keep the question clean, but I have more details if needed.)

Just an idea to give you some direction, not necessarily a solution. Use a third party Http client (not one from Spring) to see if it changes a behavior. That might help you to see if it is SimpleHttpInvokerRequestExecutor that is "guilty" of re-try or something else. Here is a very simple 3d party HttpClient: Provided in MgntUtils Open source library (written by me). Very simple in use. Take a look at Javadoc. Library itself provided as Maven artifacts and on Git (including source code and Javadoc). All in all your code may look like this:
private static void testHttpClient() {
HttpClient client = new HttpClient();
client.setContentType("application/json");
String content = null;
try {
content = client.sendHttpRequest("http://www.google.com/", HttpMethod.GET);
//content holds the response. Do your logic here
} catch (IOException e) {
//Error Handling is here
content = TextUtils.getStacktrace(e, false);
}
}

How to read secure web requests and process web socket requests with a single Java ServerSocket?

Intro
My code can process all bytes that are sent to a server and decide whether or not to let them go through and eventually send responses. I would like to use this to use the server as web server, web socket server and tcp server in one.
Although my code is written for Minecraft, I am not asking this on a Minecraft forum because answering this question doesn't require any prior knowledge about Minecraft or its codebase.
All you need to know about Minecraft
Minecraft is a Java game that can be played online. When it is being played online, there is a server that opens a ServerSocket and all players have their own client that opens a Socket that will communicate with the ServerSocket of the server.
Anyone can create a Minecraft server and install server-side modifications on their server (for those who know Minecraft, these are usually called plug-ins). My application is such a server-side modification. Most Minecraft servers are being hosted by Minecraft host companies. The owners of the servers have some kind of access to the part of the host computer that manages the server files.
Goal
The goal of my modification is to let the Minecraft server serve more clients than just the Minecraft clients. I would like the same server to also work as web server (for http and https requests) as well as (secure) web socket server and tcp server.
Why no multiple server sockets
The most common solution would be to just create a ServerSocket for the other server types and assign a different port to all of them. However, that is not an option in my case. Most hosts forbid you to open other ports or ask extra money for it. So I need to do it all with just the Minecraft ServerSocket.
What I achieved so far
So far, I have managed to let all bytes that are sent to the minecraft server first go through my code. My code can choose whether or not to let the bytes continue to the Minecraft server code. It can also send responses on its own without the need to inform the Minecraft server code at all.
In principle, what I have managed to do is sufficient to accomplish my goal, but I would like some help with how to continue. I will explain below what I have and have not accomplished so far.
The first byte that is sent by a Minecraft client to the server is always the same, namely 16. This is great because it allows me to easily distinguish Minecraft clients from web browsers and tcp clients.
HTTP requests and websocket connections always start with the same byte, namely 71. HTTPS and secure websockets always start with the byte 22. The TCP connections I was talking about will be sent by my own applications, so I can choose exactly what bytes they will send and I can simply program my modification to respond to that.
I managed to distinguish http requests and websocket connections by their connection property. Http requests always send 'Connection: keep-alive' while websocket connections always send 'Connection: upgrade'. (Although some browsers do the k, a and u in uppercase and others do not.)
Handling normal http requests wasn't very hard to do. Handling TCP connections won't be hard either because I will control everything. But I have issues with the remaining connection types:
Problems I need help with
The web socket protocol is quite large and I would prefer not to handle it completely with only my code. (I tried this before, but I kept having problems with the parts that were rarely used and thus not tested.) So I would like to use some library that allows me to only worry about the payload rather than the entire protocol. Unfortunately, web socket libraries generally want to create the ServerSocket, which is not possible in my case. So does anyone have advice on what to do here?
I haven't found any info on how to read https requests properly. Could someone tell me where to look for the details of this protocol or provide a nice link?
For secure web sockets, I will face the same problems as with the 'normal' web socket connections after I find out how to read the requests.
Code
All my code so far can be found at https://github.com/knokko/Multi-Purpose-Server. The most interesting part is probably the part where my code gets the opportunity to process all bytes before they arrive at the Minecraft code, that code is shown below.
Short question(s)
For those who didn't understand exactly what my question (you could see it as 2 questions that are strongly related) is:
-How I should read https requests and secure web socket handshakes?
-Does anyone know a library that can handle web socket input that doesn't require to create the ServerSocket itself?
// This channel handler will be registered for every connection client that will
// inspect
// any message before it reaches the Minecraft code.
pipeline.addFirst("multipurpose_handler_inspector", new ChannelInboundHandlerAdapter() {
private boolean deactivated;
private ChannelListener listener;
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
//super.channelRead will send the content to the minecraft code
try {
if (!deactivated) {
ByteBuf message = (ByteBuf) msg;
if (listener != null) {
listener.read(ctx, message);
} else {
byte firstByte = message.getByte(0);
// All Minecraft connections start with the byte 16
if (firstByte == 16) {
deactivated = true;
super.channelRead(ctx, msg);
}
// All insecure web connections start with the byte 71
else if (firstByte == 71) {
byte[] data = new byte[message.readableBytes()];
message.getBytes(0, data);
WebHandler.Type type = WebHandler.determineConnectionType(data);
if (type == WebHandler.Type.HTTP) {
listener = new HTTPListener();
listener.readInitial(ctx, message);
} else if (type == WebHandler.Type.WEBSOCKET) {
// TODO Find a nice way to handle web socket connections
listener = new WebSocketListener();
listener.readInitial(ctx, message);
} else {
deactivated = true;
super.channelRead(ctx, msg);
}
}
// All secure web connections start with the byte 22
else if (firstByte == 22) {
// TODO implement the secure web protocols and find a way to read this stuff
// and find the difference
System.out.println(
"We are dealing with a secure websocket or https connection");
byte[] data = new byte[message.readableBytes()];
message.getBytes(0, data);
System.out.println(new String(data));
}
// My applications
else if (firstByte == 31) {
listener = new TCPListener();
listener.readInitial(ctx, message);
} else {
System.out.println("Unknown connection type");
deactivated = true;
super.channelRead(ctx, msg);
}
}
} else {
super.channelRead(ctx, msg);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
});

If you can ALWAYS identify the Minecraft traffic, your best bet might be to run an apache/httpd and/or tomcat server on the same box and forward all non-Minecraft traffic to it. If you do this, the HTTPS stuff might just be a matter of correctly configuring the http server for https traffic.
You may have to configure your code as an http proxy--in fact (Just thought of this) you might want to go out and look for an open source http proxy and just tweak it with your code to extract Minecraft traffic and forward it before doing the rest of the proxy stuff.
I wouldn't do the HTTPs stuff from scratch, it's not terribly difficult but I'd call it non-trivial.
Oh and if your problem is "differentiating Minecraft HTTPS traffic from other HTTPS connections on the same port" I can't help except to say that this might be a good subject for your question :)

About socket programming and REST design

We need to design a server that will serve a webpage to several clients but also query a remote database for these clients. One of the requirement for this project is that the whole system must be compliant with the REST architecture style. We need use Java as programming language but many questions arised while we were designing it.
We want to have a main thread that will get connections, as shown in this example:
// System.out.println("Starting a new web server using port " + port)
try {
ServerSocket reciever = new ServerSocket(port);
while (true) {
try {
Socket s = reciever.accept();
Client c = new Client(s);
} catch (IOException e) {
System.err.println("New item creation failed.");
IOUtil.close(reciever);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
} catch (IOException e) {
System.err.println("ServerSocket problem.");
}
Then each connection will be created as a new thread (the Client object in the code) that will take care of reading ONE request. If the request is a GET, then the thread will serve the resource to the client. If it is a POST, then it will add the request to a buffer and let another thread handle the query to the database and also the answer back to the client. After handling this only request, the thread closes the socket and terminates.
Is the use of sockets violating the REST principle? In order to respect the REST architecture, do we need to destroy every Client object (thread & socket) after each HTTP message? Is there another way of client-server communication that does not use sockets?

Ok, I think you are confusing a whole bag of junk together.
Firstly, different between low level IP sockets that allow data to go from A to B and 'websockets' that use HTTP to bootstrap a connection from a client to a server that can be kept open for TWO WAY communication.
Based on your requirements, you just want a 'standard' JEE container. Using something like JAX-RS you can apply some basic annotations to functions such as #PATH('/MyResource/Cars/') and have that function be called for that path.
Using a container will free you from all that boring boilerplate rubbish. No need to manual setup threads to listen, and spawn other threads to handle requests.
Using IP sockets is (indirectly) a mandate of REST; REST has to (according to Fielding, but strictly speaking it is protocol agnostic) be over HTTP, thus over TCP/IP sockets (though obviously you could do HTTP over any other transport protocol). Websockets however are using HTTP to form a persistent stateful connection between client and server, which is fundamentally opposed to REST. Basic HTTP would (and you would do this via the container doing it for you) fully open and close the connection for each isolated request, in practice however HTTP (and thus REST) will allow for the low level connection (the TCP connection that is slow to start) to be maintained for a series of request. This functionality is intended for the scope of loading a HTML page, and all resources in one TCP connection, but over many HTTP requests.

Sockets move bytes over TCP/IP. That's a lower level protocol, you don't want to worry about that. You care about the higher up protocol (which in this case is HTTP).
Sockets are closed in HTTP after every request, so what you're thinking sounds reasonable. Although I'm not sure why you would create a separate thread for a POST request. I'm assuming that your Client implementation already runs in its own thread (if it doesn't, then your server isn't very efficient).

Designing java client

I'm currently developing a client/server application in Java and i can't make up my mind on how to design the client.
At the moment i have a server that can accept multiple connections and every connection has a loop listening for commands and respond to them. The server looks like this:
try{
mIn = new DataInputStream(mSocket.getInputStream());
mOut = new DataOutputStream(mSocket.getOutputStream());
while(true){
byte tPackType = mIn.readByte();
switch(tPackType){
case PackType.LOGIN:
login();
break;
case PackType.REGISTER:
register();
break;
default:
}
}
}catch (IOException e){
mLog.logp(Level.WARNING, this.getClass().getName(), "run()", "IOException in run()", e);
}finally{
try{
mOut.close();
mIn.close();
mSocket.close();
}catch (IOException e){
e.printStackTrace();
}
}
Now, the client is basically just a number of methods sending a request to the server and returns the response. I want to be able to receive updates from the server.
I would appreciate any suggestions on reading that can help me find a solution.
Is there any patterns i could look into?
Thank you in advance, veLr.

I want to be able to receive updates from the server
I think that you have two options to do this.
Option 1 is to make your client poll the server for updates. I would not recommend this since this will eventually increase your load on your server. You will most likely have performance issue the moment you attempt to scale your system up.
Option 2 is to create a small server on your client. So basically, whenever there is an update, the server would connect to the client and send it the updates. I would recommend this procedure since it allows you to establish connections between your server and clients only when needed. The problem, when opposed to the previous method, is that you will need to track of the ports and IP's of your clients. You could implement a HashTable with Client ID's as keys and their connection properties as their keys. Once you have an update, you will look up the client in the HashTable, obtain the connection information, connect and send the information.
The server on the client side will then get the data and do whatever you want the client to do once an update is received.

How to send objects from android to servlet?

I am looking at sending objects over http from an android client to my server that is running java servlets. The object can hold a bitmap image, and I am just wondering if you could show me an example of sending an object from the client to the server.
I read on the forms that people say to use JSON , but it seems to me JSON works with only textual data. If it does not could someone show me how to use it with objects that contain images

To send binary data between a Java client and a Java server which is connected by HTTP, you have basically 2 options.
Serialize it, i.e. let object implement Serializable, have an exact copy of the .class file on both sides and send it by ObjectInputStream and read it by ObjectInputStream. Advantage: ridiculously easy. Disadvantage: poor backwards compatibility (when you change the object to add a new field, you've to write extremely a lot of extra code and checks to ensure backwards compatibitility) and bad reusability (not reusable on other clients/servers than Java ones).
Use HTTP multipart/form-data. Advandage: very compatible (a web standard) and very good reusability (server is reusable on other clients and client is reusable on other servers). Disadvantage: harder to implement (fortunately there are APIs and libraries for this). In Android you can use builtin HttpClient API to send it. In Servlet you can use Apache Commons FileUpload to parse it.

I recommend you use XStream
XStream for your servlet side:
http://x-stream.github.io/tutorial.html
XStream code optimized for Android:
http://jars.de/java/android-xml-serialization-with-xstream
If you are sending images and such, wrap them into a 'envelope' class that contains a byte array like the one here: Serializing and De-Serializing android.graphics.Bitmap in Java
Then use HttpClient in your android app to send the data to your servlet ^^ Also make sure that both the app and the servlet have the same classes ^^

Socket Api is also well.
Creating socket in both side will allow to send raw data to be transmitted from client android application to server.

Here is the code for hitting a servlet and send data to the server.
boolean hitServlet(final Context context, final String data1,final String data2) {
String serverUrl = SERVER_URL + "/YourSevletName";
Map<String, String> params = new HashMap<String, String>();
params.put("data1", data1);
params.put("data2" data2)
long backoff = BACKOFF_MILLI_SECONDS + random.nextInt(1000);
// As the server might be down, we will retry it a couple
// times.
for (int i = 1; i <= MAX_ATTEMPTS; i++) {
try {
post(serverUrl, params);
return true;
} catch (IOException e) {
// Here we are simplifying and retrying on any error; in a real
// application, it should retry only on unrecoverable errors
// (like HTTP error code 503).
Log.e(TAG, "Failed " + i, e);
if (i == MAX_ATTEMPTS) {
break;
}
try {
Log.d(TAG, "Sleeping for " + backoff + " ms before retry");
Thread.sleep(backoff);
} catch (InterruptedException e1) {
// Activity finished before we complete - exit.
Log.d(TAG, "Thread interrupted: abort remaining retries!");
Thread.currentThread().interrupt();
return false;
}
// increase backoff exponentially
backoff *= 2;
}
}
return false;
}