We need to design a server that will serve a webpage to several clients but also query a remote database for these clients. One of the requirement for this project is that the whole system must be compliant with the REST architecture style. We need use Java as programming language but many questions arised while we were designing it.
We want to have a main thread that will get connections, as shown in this example:
// System.out.println("Starting a new web server using port " + port)
try {
ServerSocket reciever = new ServerSocket(port);
while (true) {
try {
Socket s = reciever.accept();
Client c = new Client(s);
} catch (IOException e) {
System.err.println("New item creation failed.");
IOUtil.close(reciever);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
} catch (IOException e) {
System.err.println("ServerSocket problem.");
}
Then each connection will be created as a new thread (the Client object in the code) that will take care of reading ONE request. If the request is a GET, then the thread will serve the resource to the client. If it is a POST, then it will add the request to a buffer and let another thread handle the query to the database and also the answer back to the client. After handling this only request, the thread closes the socket and terminates.
Is the use of sockets violating the REST principle? In order to respect the REST architecture, do we need to destroy every Client object (thread & socket) after each HTTP message? Is there another way of client-server communication that does not use sockets?
Ok, I think you are confusing a whole bag of junk together.
Firstly, different between low level IP sockets that allow data to go from A to B and 'websockets' that use HTTP to bootstrap a connection from a client to a server that can be kept open for TWO WAY communication.
Based on your requirements, you just want a 'standard' JEE container. Using something like JAX-RS you can apply some basic annotations to functions such as #PATH('/MyResource/Cars/') and have that function be called for that path.
Using a container will free you from all that boring boilerplate rubbish. No need to manual setup threads to listen, and spawn other threads to handle requests.
Using IP sockets is (indirectly) a mandate of REST; REST has to (according to Fielding, but strictly speaking it is protocol agnostic) be over HTTP, thus over TCP/IP sockets (though obviously you could do HTTP over any other transport protocol). Websockets however are using HTTP to form a persistent stateful connection between client and server, which is fundamentally opposed to REST. Basic HTTP would (and you would do this via the container doing it for you) fully open and close the connection for each isolated request, in practice however HTTP (and thus REST) will allow for the low level connection (the TCP connection that is slow to start) to be maintained for a series of request. This functionality is intended for the scope of loading a HTML page, and all resources in one TCP connection, but over many HTTP requests.
Sockets move bytes over TCP/IP. That's a lower level protocol, you don't want to worry about that. You care about the higher up protocol (which in this case is HTTP).
Sockets are closed in HTTP after every request, so what you're thinking sounds reasonable. Although I'm not sure why you would create a separate thread for a POST request. I'm assuming that your Client implementation already runs in its own thread (if it doesn't, then your server isn't very efficient).
Related
This question already has answers here:
Java socket API: How to tell if a connection has been closed?
(9 answers)
Closed 2 years ago.
So I'm in the making of a very simple server/client complex using java. So far I have managed to figure out what happens if the client quits, because then the server receives null while listening from any input from the client.
BUT - what happens if the client is connected and the server quits for any reason... the server is supposed to wait for input from the client, but how can the client know that the server is not listening anymore? For me the clients call to the server just goes into the void... nothing happens...
Can I do something to find out when the server goes down? Time-out, ping/pong or something?
As You surely can see I'm quite new at this, I'm just curious. This was a puzzle for me ever since I attended computer science at the university.
Thanks in advance. dr_xemacs.
(I am assuming you are working with blocking server socket and socket and not with non blocking ones)
Similarly to the server, reading from streams of a closed connection will return null.
However if you instead do not want to rely on this or a scared that the connection to the server could somehow persist, you can also use time outs (check this out! ) which will throw SocketTimeoutException when the time is up and, to keep track of whether the server is up or not, create a ping/packet to assure server is still up and running.
Edit: I did a quick search and this could be useful to you! Take a look!
How can the client know that the server is not listening anymore?
If the client doesn't attempt to interact at some level with the service, it won't know.
Assuming that the client has sent a request, a few different scenarios.
If the service is no longer listening on the designated port, the client will typically get a "Connection Refused" exception.
If the service is still running (in a sense) but it is not working properly, then connection attempts from the client are likely to time out.
If the service's host is down, the client liable get a timeout.
If there are network connectivity or firewall issues, the client could get a timeout or some other exception.
Can I do something to find out when the server goes down? Time-out, ping/pong or something?
You attempt to connect and send a request. If it fails or times out, that means the service is down. If you are designing and implementing the service yourself, you could include a special "healthcheck" request for clients to "ping" on. But the flip-side is that network and server resources will be consumed in receiving and responding to these requests. It can affect your ability to scale up the number of clients, for example, if each client pings the service every N seconds.
But a client typically doesn't need to know whether the service is up or down. It typically only cares that service responds when it it sends a real request. And the simplest way to handle that is to just send the request and deal with the outcome. (The client code has to deal with all possible outcomes anyway when doing a real request. The service can go down, etc between the last healthcheck ping and when the client sends a real request.)
Bottom line: Don't bother with checking availability in the client unless the application (i.e. the end user) really needs to know.
Your Server probably may be running on a certain port and so you can add a health check at the client side and update a global flag with status to let client know about its availibity :-
Socket socket = null;
try
{
socket = new Socket(host, port);
return true;
}
catch (Exception e)
{
return false;
}
finally
{
if(socket != null)
try
{
socket.close();
}
catch(Exception e){}
}
Intro
My code can process all bytes that are sent to a server and decide whether or not to let them go through and eventually send responses. I would like to use this to use the server as web server, web socket server and tcp server in one.
Although my code is written for Minecraft, I am not asking this on a Minecraft forum because answering this question doesn't require any prior knowledge about Minecraft or its codebase.
All you need to know about Minecraft
Minecraft is a Java game that can be played online. When it is being played online, there is a server that opens a ServerSocket and all players have their own client that opens a Socket that will communicate with the ServerSocket of the server.
Anyone can create a Minecraft server and install server-side modifications on their server (for those who know Minecraft, these are usually called plug-ins). My application is such a server-side modification. Most Minecraft servers are being hosted by Minecraft host companies. The owners of the servers have some kind of access to the part of the host computer that manages the server files.
Goal
The goal of my modification is to let the Minecraft server serve more clients than just the Minecraft clients. I would like the same server to also work as web server (for http and https requests) as well as (secure) web socket server and tcp server.
Why no multiple server sockets
The most common solution would be to just create a ServerSocket for the other server types and assign a different port to all of them. However, that is not an option in my case. Most hosts forbid you to open other ports or ask extra money for it. So I need to do it all with just the Minecraft ServerSocket.
What I achieved so far
So far, I have managed to let all bytes that are sent to the minecraft server first go through my code. My code can choose whether or not to let the bytes continue to the Minecraft server code. It can also send responses on its own without the need to inform the Minecraft server code at all.
In principle, what I have managed to do is sufficient to accomplish my goal, but I would like some help with how to continue. I will explain below what I have and have not accomplished so far.
The first byte that is sent by a Minecraft client to the server is always the same, namely 16. This is great because it allows me to easily distinguish Minecraft clients from web browsers and tcp clients.
HTTP requests and websocket connections always start with the same byte, namely 71. HTTPS and secure websockets always start with the byte 22. The TCP connections I was talking about will be sent by my own applications, so I can choose exactly what bytes they will send and I can simply program my modification to respond to that.
I managed to distinguish http requests and websocket connections by their connection property. Http requests always send 'Connection: keep-alive' while websocket connections always send 'Connection: upgrade'. (Although some browsers do the k, a and u in uppercase and others do not.)
Handling normal http requests wasn't very hard to do. Handling TCP connections won't be hard either because I will control everything. But I have issues with the remaining connection types:
Problems I need help with
The web socket protocol is quite large and I would prefer not to handle it completely with only my code. (I tried this before, but I kept having problems with the parts that were rarely used and thus not tested.) So I would like to use some library that allows me to only worry about the payload rather than the entire protocol. Unfortunately, web socket libraries generally want to create the ServerSocket, which is not possible in my case. So does anyone have advice on what to do here?
I haven't found any info on how to read https requests properly. Could someone tell me where to look for the details of this protocol or provide a nice link?
For secure web sockets, I will face the same problems as with the 'normal' web socket connections after I find out how to read the requests.
Code
All my code so far can be found at https://github.com/knokko/Multi-Purpose-Server. The most interesting part is probably the part where my code gets the opportunity to process all bytes before they arrive at the Minecraft code, that code is shown below.
Short question(s)
For those who didn't understand exactly what my question (you could see it as 2 questions that are strongly related) is:
-How I should read https requests and secure web socket handshakes?
-Does anyone know a library that can handle web socket input that doesn't require to create the ServerSocket itself?
// This channel handler will be registered for every connection client that will
// inspect
// any message before it reaches the Minecraft code.
pipeline.addFirst("multipurpose_handler_inspector", new ChannelInboundHandlerAdapter() {
private boolean deactivated;
private ChannelListener listener;
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
//super.channelRead will send the content to the minecraft code
try {
if (!deactivated) {
ByteBuf message = (ByteBuf) msg;
if (listener != null) {
listener.read(ctx, message);
} else {
byte firstByte = message.getByte(0);
// All Minecraft connections start with the byte 16
if (firstByte == 16) {
deactivated = true;
super.channelRead(ctx, msg);
}
// All insecure web connections start with the byte 71
else if (firstByte == 71) {
byte[] data = new byte[message.readableBytes()];
message.getBytes(0, data);
WebHandler.Type type = WebHandler.determineConnectionType(data);
if (type == WebHandler.Type.HTTP) {
listener = new HTTPListener();
listener.readInitial(ctx, message);
} else if (type == WebHandler.Type.WEBSOCKET) {
// TODO Find a nice way to handle web socket connections
listener = new WebSocketListener();
listener.readInitial(ctx, message);
} else {
deactivated = true;
super.channelRead(ctx, msg);
}
}
// All secure web connections start with the byte 22
else if (firstByte == 22) {
// TODO implement the secure web protocols and find a way to read this stuff
// and find the difference
System.out.println(
"We are dealing with a secure websocket or https connection");
byte[] data = new byte[message.readableBytes()];
message.getBytes(0, data);
System.out.println(new String(data));
}
// My applications
else if (firstByte == 31) {
listener = new TCPListener();
listener.readInitial(ctx, message);
} else {
System.out.println("Unknown connection type");
deactivated = true;
super.channelRead(ctx, msg);
}
}
} else {
super.channelRead(ctx, msg);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
});
If you can ALWAYS identify the Minecraft traffic, your best bet might be to run an apache/httpd and/or tomcat server on the same box and forward all non-Minecraft traffic to it. If you do this, the HTTPS stuff might just be a matter of correctly configuring the http server for https traffic.
You may have to configure your code as an http proxy--in fact (Just thought of this) you might want to go out and look for an open source http proxy and just tweak it with your code to extract Minecraft traffic and forward it before doing the rest of the proxy stuff.
I wouldn't do the HTTPs stuff from scratch, it's not terribly difficult but I'd call it non-trivial.
Oh and if your problem is "differentiating Minecraft HTTPS traffic from other HTTPS connections on the same port" I can't help except to say that this might be a good subject for your question :)
I have a REST endpoint and I want to access it using UDP for example Java Datagram. I know its not a best practice to try even but my recent project I have some hardware limitations. Hardware can make UDP calls only and my existing services are over REST i.e. HTTP/HTTPS.
I am looking for any way I can reuse my existing services. I have tried following code but received UnknownHostException.
public class UDPClinet {
public static void main(String[] args) {
String hostname = "https://jsonplaceholder.typicode.com/posts/1";
int port = 80;
try {
InetAddress address = InetAddress.getByName(hostname);
DatagramSocket socket = new DatagramSocket();
while (true) {
DatagramPacket request = new DatagramPacket(new byte[1], 1, address, port);
socket.send(request);
byte[] buffer = new byte[512];
DatagramPacket response = new DatagramPacket(buffer, buffer.length);
socket.receive(response);
String quote = new String(buffer, 0, response.getLength());
System.out.println(quote);
System.out.println();
Thread.sleep(10000);
}
} catch (SocketTimeoutException ex) {
System.out.println("Timeout error: " + ex.getMessage());
ex.printStackTrace();
} catch (IOException ex) {
System.out.println("Client error: " + ex.getMessage());
ex.printStackTrace();
} catch (InterruptedException ex) {
ex.printStackTrace();
}
}
}
Is it possible to implement a RESTful service that can be called via UDP.
Yes. (See below)
Can you call your existing RESTful service via UDP?
Probably no. And certainly not without a lot of work.
Typical RESTful services are in fact implemented using HTTP or HTTPS over TCP/IP connections. It is not possible to talk directly to an TCP-based service using UDP. The IP-level packets will have the wrong protocol family and the service's OS won't route them to the service.
However, it is possible (technically speaking) to implement RESTful services over any transport that is capable of sending messages. REST principles are agnostic of the transport protocol.
The problem will be finding a service framework that support RESTful UDP and (conventional) RESTful HTTP at the same time.
There are a couple of other practical problems:
UDP is unreliable, and this is exacerbated if you send datagrams that won't fit into a packet with the default MTU (1500 bytes). So if you want to implement a RESTful service over UDP, you will need to play close attention to the size of request and response payloads.
HTTPS uses TLS so that the client is able to validate the server's authenticity and then send data encrypted. TLS over UDP is possible (it is called DTLS) and supported by JCSE, but using it in a typical RESTful / HTTP framework may be challenging.
If you want to pursue this, look for a RESTful framework that implements CoAP (Constrained Application Protocol - RFC 7252) and DTLS.
You may have a look at available CoAP implementations, see coap.technology for an overview. Though I'm a committer of Eclipse/Californium, a CoAP/DTLS implementation in java, I would recommend to try that out.
CoAP also defines Cross Proxies to HTTP. There is one example available in the google cloud tutorials. It's using a older Californium version as base. Californium itself has an update proxy exmaple (but not out of the box usable for the google cloud), see Californium - Proxy2.
Yes, there's a internet standard now for REST over UDP, it's goes by the name the Constrained Application Protocoal - CoAP. CoApp is defined in a series of internet standards starting with RFC 7252.
CoAP addresses concerns raised in earlier answers to this question, including using retransmissions over UDP to make it reliable, and providing for security using DTLS.
CoAP has been designed to work on microcontrollers with as low as 10 KiB of RAM and 100 KiB of code space (RFC 7228). Furthermore it adds very little overhead to the UDP packets, using a binary fixed header of 4 bytes, and variable length options (like optional HTTP headers)
When creating a standalone server in Java (not using a container like tomcat/jetty), what are the various techniques to keep the service running and not ending?
I have seen where people use a ServerSocket (since you will be communicating with the service presumably), and they use ServerSocket.accept() which blocks until it receives a message. And this is usually done in a while loop:
while(...) {
serverSocket.accept();
}
(http://docs.oracle.com/javase/6/docs/api/java/net/ServerSocket.html#accept())
Is this the only way? If not, what other ways are there and any pros/cons with it?
Are there any libraries that help with building your own service, or its pretty much roll your own.
There are various libraries that help you roll your own Windows/Unix service in Java:
Apache Commons Daemon
Akuma
Java Service Wrapper
How you keep the application running depends on the actual needs of your application. If your application is a server, you would normally want to listen for incoming connections which usually involves some sort of blocking/polling. How you do that again depends on the type of server you want to build. Of the generic solutions there's the ServerSocket class and its method accept() that you already mentioned. Another possibility is to use java.nio and implement a reactor which provides a single-threaded server that can handle multiple connections at once (see http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf for details), but may be a bit hard to grasp and debug.
What you probably want is a multi-threaded server.
Each time the server accepts a connection, the server creates a thread to handle sending/reciving to that client. If you do not use threads in your server, it will only be able to handle one connection at a time.
So, as you meantioned, the server loops infinitly and listens for incomming connections:
while(true){
serverSocket.accept();
ClientHandler c = new ClientHandler(serverSocket);
A instance of the class ClientHandler will be created each time a connection is accepted. This class implements Runnable, and loops for incomming messages using getInputStream and getOutputStream on that socket:
public class ClientHandler implements Runnable{
DataInputStream in;
DataOutputStream out;
//ClientHandler constructor
public ClientHandler(Socket s) throws IOException{
in= new DataInputStream(socket.getInputStream());
out=new DataOutputStream(socket.getOutputStream());
thread.start();
}
The run method:
public void run() {
while(true){
String temp="";
while ((temp = (String) in.readUTF()) != null){ // Read from the input stream each iteration. When temp is not null a message is recived
System.out.println(temp);
Please that the above code does not take into account different exceptions that might occur and is very basic. But it should give you a basic idea on how a server using Sockets can be implemented.
For a quick solution (in a testing environment only!) you can go for something often dubbed as "Enterprise Loop" (because it is too often found in production systems):
while (true)
try {
// do something
} catch (Throwable t) {
// maybe log
}
However, this is not good style in the production code.
(see [1] for a parody of that idiom)
To create a service, you want one of the libraries from this answer.
If you "just need multithreading", have a look into the Java concurrency framework. I stronly suggest reading Java Concurrency in Practice, as multi-threading is much more that just starting another thread and errors are hard to debug.
[1] http://blog.antiblau.de/2016/01/26/java-enterprise-loop/
I am trying to figure out how to create a java program that can listen to multiple ports and perform different actions depending on which port the client speaks to.
I've seen and understand the basic client-server program:
http://systembash.com/content/a-simple-java-tcp-server-and-tcp-client/
Just to reiterate, I want to create this same relationship, but instead of the server only listening on one port and performing one action when it receives input, I want it to listen on multiple ports and depending which port the client connects and sends data to, perform a different action.
I'm hoping to make each port accept a GET and PUT command in the future, but for now I'm just trying to figure out how to set up the basic structure of the server which will be able to listen to multiple ports. I've tried googling, but I can't seem to find much, so any help is appreciated.
Thanks in advance.
-Anthony
The tutorial you've mentioned is very basic. You cannot write any reasonable server without using threads. In order to have two server sockets, you must spawn a new thread for each port, like this (pseudocode):
new Thread() {
public void run() {
ServerSocket server = new ServerSocket(6788);
while(true) {
Socket client1 = server.accept();
//handle client1
}
}.start();
and (notice the different port):
new Thread() {
public void run() {
ServerSocket server = new ServerSocket(6789);
while(true) {
Socket client1 = server.accept();
//handle client2
}
}.start();
Having client1 and client2 sockets you can handle them separately. Also, handling client connection should be done in a different thread so that you can serve multiple clients. Of course this code introduces a lot of duplication, but consider this as a starting point.
To wrap things up - if your goal is to implement HTTP GET and PUT, use servlet and get away from all this hustle.
A socket can only be open to a particular port, so you need multiple server sockets (e.g. 1 socket per port). I think you also need one thread per socket so that network activity on one socket doesn't interfere with activity on the others.
Are you implementing a server as an academic exercise? If not, I'd really really really really strongly recommend using a preexisting server, like Tomcat.
You could just open multiple server sockets of course.
You might also want to look at jboss netty which can help you implement protocols.