I have a REST endpoint and I want to access it using UDP for example Java Datagram. I know its not a best practice to try even but my recent project I have some hardware limitations. Hardware can make UDP calls only and my existing services are over REST i.e. HTTP/HTTPS.
I am looking for any way I can reuse my existing services. I have tried following code but received UnknownHostException.
public class UDPClinet {
public static void main(String[] args) {
String hostname = "https://jsonplaceholder.typicode.com/posts/1";
int port = 80;
try {
InetAddress address = InetAddress.getByName(hostname);
DatagramSocket socket = new DatagramSocket();
while (true) {
DatagramPacket request = new DatagramPacket(new byte[1], 1, address, port);
socket.send(request);
byte[] buffer = new byte[512];
DatagramPacket response = new DatagramPacket(buffer, buffer.length);
socket.receive(response);
String quote = new String(buffer, 0, response.getLength());
System.out.println(quote);
System.out.println();
Thread.sleep(10000);
}
} catch (SocketTimeoutException ex) {
System.out.println("Timeout error: " + ex.getMessage());
ex.printStackTrace();
} catch (IOException ex) {
System.out.println("Client error: " + ex.getMessage());
ex.printStackTrace();
} catch (InterruptedException ex) {
ex.printStackTrace();
}
}
}
Is it possible to implement a RESTful service that can be called via UDP.
Yes. (See below)
Can you call your existing RESTful service via UDP?
Probably no. And certainly not without a lot of work.
Typical RESTful services are in fact implemented using HTTP or HTTPS over TCP/IP connections. It is not possible to talk directly to an TCP-based service using UDP. The IP-level packets will have the wrong protocol family and the service's OS won't route them to the service.
However, it is possible (technically speaking) to implement RESTful services over any transport that is capable of sending messages. REST principles are agnostic of the transport protocol.
The problem will be finding a service framework that support RESTful UDP and (conventional) RESTful HTTP at the same time.
There are a couple of other practical problems:
UDP is unreliable, and this is exacerbated if you send datagrams that won't fit into a packet with the default MTU (1500 bytes). So if you want to implement a RESTful service over UDP, you will need to play close attention to the size of request and response payloads.
HTTPS uses TLS so that the client is able to validate the server's authenticity and then send data encrypted. TLS over UDP is possible (it is called DTLS) and supported by JCSE, but using it in a typical RESTful / HTTP framework may be challenging.
If you want to pursue this, look for a RESTful framework that implements CoAP (Constrained Application Protocol - RFC 7252) and DTLS.
You may have a look at available CoAP implementations, see coap.technology for an overview. Though I'm a committer of Eclipse/Californium, a CoAP/DTLS implementation in java, I would recommend to try that out.
CoAP also defines Cross Proxies to HTTP. There is one example available in the google cloud tutorials. It's using a older Californium version as base. Californium itself has an update proxy exmaple (but not out of the box usable for the google cloud), see Californium - Proxy2.
Yes, there's a internet standard now for REST over UDP, it's goes by the name the Constrained Application Protocoal - CoAP. CoApp is defined in a series of internet standards starting with RFC 7252.
CoAP addresses concerns raised in earlier answers to this question, including using retransmissions over UDP to make it reliable, and providing for security using DTLS.
CoAP has been designed to work on microcontrollers with as low as 10 KiB of RAM and 100 KiB of code space (RFC 7228). Furthermore it adds very little overhead to the UDP packets, using a binary fixed header of 4 bytes, and variable length options (like optional HTTP headers)
Related
Intro
My code can process all bytes that are sent to a server and decide whether or not to let them go through and eventually send responses. I would like to use this to use the server as web server, web socket server and tcp server in one.
Although my code is written for Minecraft, I am not asking this on a Minecraft forum because answering this question doesn't require any prior knowledge about Minecraft or its codebase.
All you need to know about Minecraft
Minecraft is a Java game that can be played online. When it is being played online, there is a server that opens a ServerSocket and all players have their own client that opens a Socket that will communicate with the ServerSocket of the server.
Anyone can create a Minecraft server and install server-side modifications on their server (for those who know Minecraft, these are usually called plug-ins). My application is such a server-side modification. Most Minecraft servers are being hosted by Minecraft host companies. The owners of the servers have some kind of access to the part of the host computer that manages the server files.
Goal
The goal of my modification is to let the Minecraft server serve more clients than just the Minecraft clients. I would like the same server to also work as web server (for http and https requests) as well as (secure) web socket server and tcp server.
Why no multiple server sockets
The most common solution would be to just create a ServerSocket for the other server types and assign a different port to all of them. However, that is not an option in my case. Most hosts forbid you to open other ports or ask extra money for it. So I need to do it all with just the Minecraft ServerSocket.
What I achieved so far
So far, I have managed to let all bytes that are sent to the minecraft server first go through my code. My code can choose whether or not to let the bytes continue to the Minecraft server code. It can also send responses on its own without the need to inform the Minecraft server code at all.
In principle, what I have managed to do is sufficient to accomplish my goal, but I would like some help with how to continue. I will explain below what I have and have not accomplished so far.
The first byte that is sent by a Minecraft client to the server is always the same, namely 16. This is great because it allows me to easily distinguish Minecraft clients from web browsers and tcp clients.
HTTP requests and websocket connections always start with the same byte, namely 71. HTTPS and secure websockets always start with the byte 22. The TCP connections I was talking about will be sent by my own applications, so I can choose exactly what bytes they will send and I can simply program my modification to respond to that.
I managed to distinguish http requests and websocket connections by their connection property. Http requests always send 'Connection: keep-alive' while websocket connections always send 'Connection: upgrade'. (Although some browsers do the k, a and u in uppercase and others do not.)
Handling normal http requests wasn't very hard to do. Handling TCP connections won't be hard either because I will control everything. But I have issues with the remaining connection types:
Problems I need help with
The web socket protocol is quite large and I would prefer not to handle it completely with only my code. (I tried this before, but I kept having problems with the parts that were rarely used and thus not tested.) So I would like to use some library that allows me to only worry about the payload rather than the entire protocol. Unfortunately, web socket libraries generally want to create the ServerSocket, which is not possible in my case. So does anyone have advice on what to do here?
I haven't found any info on how to read https requests properly. Could someone tell me where to look for the details of this protocol or provide a nice link?
For secure web sockets, I will face the same problems as with the 'normal' web socket connections after I find out how to read the requests.
Code
All my code so far can be found at https://github.com/knokko/Multi-Purpose-Server. The most interesting part is probably the part where my code gets the opportunity to process all bytes before they arrive at the Minecraft code, that code is shown below.
Short question(s)
For those who didn't understand exactly what my question (you could see it as 2 questions that are strongly related) is:
-How I should read https requests and secure web socket handshakes?
-Does anyone know a library that can handle web socket input that doesn't require to create the ServerSocket itself?
// This channel handler will be registered for every connection client that will
// inspect
// any message before it reaches the Minecraft code.
pipeline.addFirst("multipurpose_handler_inspector", new ChannelInboundHandlerAdapter() {
private boolean deactivated;
private ChannelListener listener;
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
//super.channelRead will send the content to the minecraft code
try {
if (!deactivated) {
ByteBuf message = (ByteBuf) msg;
if (listener != null) {
listener.read(ctx, message);
} else {
byte firstByte = message.getByte(0);
// All Minecraft connections start with the byte 16
if (firstByte == 16) {
deactivated = true;
super.channelRead(ctx, msg);
}
// All insecure web connections start with the byte 71
else if (firstByte == 71) {
byte[] data = new byte[message.readableBytes()];
message.getBytes(0, data);
WebHandler.Type type = WebHandler.determineConnectionType(data);
if (type == WebHandler.Type.HTTP) {
listener = new HTTPListener();
listener.readInitial(ctx, message);
} else if (type == WebHandler.Type.WEBSOCKET) {
// TODO Find a nice way to handle web socket connections
listener = new WebSocketListener();
listener.readInitial(ctx, message);
} else {
deactivated = true;
super.channelRead(ctx, msg);
}
}
// All secure web connections start with the byte 22
else if (firstByte == 22) {
// TODO implement the secure web protocols and find a way to read this stuff
// and find the difference
System.out.println(
"We are dealing with a secure websocket or https connection");
byte[] data = new byte[message.readableBytes()];
message.getBytes(0, data);
System.out.println(new String(data));
}
// My applications
else if (firstByte == 31) {
listener = new TCPListener();
listener.readInitial(ctx, message);
} else {
System.out.println("Unknown connection type");
deactivated = true;
super.channelRead(ctx, msg);
}
}
} else {
super.channelRead(ctx, msg);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
});
If you can ALWAYS identify the Minecraft traffic, your best bet might be to run an apache/httpd and/or tomcat server on the same box and forward all non-Minecraft traffic to it. If you do this, the HTTPS stuff might just be a matter of correctly configuring the http server for https traffic.
You may have to configure your code as an http proxy--in fact (Just thought of this) you might want to go out and look for an open source http proxy and just tweak it with your code to extract Minecraft traffic and forward it before doing the rest of the proxy stuff.
I wouldn't do the HTTPs stuff from scratch, it's not terribly difficult but I'd call it non-trivial.
Oh and if your problem is "differentiating Minecraft HTTPS traffic from other HTTPS connections on the same port" I can't help except to say that this might be a good subject for your question :)
I am trying to use the VpnService API to capture packets (on Android 4.4). I've been following an example at:
http://www.thegeekstuff.com/2014/06/android-vpn-service/
I establish the vpn connection using a builder, which is configured using the following code (which is borrowed from the ToyVPN sample, as I'm sure many of you will recognize):
// Configure a builder while parsing the parameters.
Builder builder = new Builder();
builder.setMtu(1500);
builder.addAddress("192.168.0.1", 24);
builder.addRoute("0.0.0.0", 0);
try {
mInterface.close();
} catch (Exception e) {
// ignore
}
mInterface = builder.establish();
The general concept is that with VpnService, I am able to get outbound packets and log them. Then I open a Socket to the original destination and forward the packets. When the response comes, it will be via the Socket that was opened.
So my question is, what happens when an IP address sends me a packet before I've opened a Socket to that address? I'm guessing that either it bypasses the VPN and arrives as normal, or is dropped entirely (probably the former?). Any insight is appreciated.
When you addRoute("0.0.0.0",0);
All packets will be routed to this Tun interface.
Now its upto you whether you want to forward them to original destination or not.
So answer to your question is you still get packets from the application which is trying to connect to its server, and now its up to you whether you allow its are not.
We need to design a server that will serve a webpage to several clients but also query a remote database for these clients. One of the requirement for this project is that the whole system must be compliant with the REST architecture style. We need use Java as programming language but many questions arised while we were designing it.
We want to have a main thread that will get connections, as shown in this example:
// System.out.println("Starting a new web server using port " + port)
try {
ServerSocket reciever = new ServerSocket(port);
while (true) {
try {
Socket s = reciever.accept();
Client c = new Client(s);
} catch (IOException e) {
System.err.println("New item creation failed.");
IOUtil.close(reciever);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
} catch (IOException e) {
System.err.println("ServerSocket problem.");
}
Then each connection will be created as a new thread (the Client object in the code) that will take care of reading ONE request. If the request is a GET, then the thread will serve the resource to the client. If it is a POST, then it will add the request to a buffer and let another thread handle the query to the database and also the answer back to the client. After handling this only request, the thread closes the socket and terminates.
Is the use of sockets violating the REST principle? In order to respect the REST architecture, do we need to destroy every Client object (thread & socket) after each HTTP message? Is there another way of client-server communication that does not use sockets?
Ok, I think you are confusing a whole bag of junk together.
Firstly, different between low level IP sockets that allow data to go from A to B and 'websockets' that use HTTP to bootstrap a connection from a client to a server that can be kept open for TWO WAY communication.
Based on your requirements, you just want a 'standard' JEE container. Using something like JAX-RS you can apply some basic annotations to functions such as #PATH('/MyResource/Cars/') and have that function be called for that path.
Using a container will free you from all that boring boilerplate rubbish. No need to manual setup threads to listen, and spawn other threads to handle requests.
Using IP sockets is (indirectly) a mandate of REST; REST has to (according to Fielding, but strictly speaking it is protocol agnostic) be over HTTP, thus over TCP/IP sockets (though obviously you could do HTTP over any other transport protocol). Websockets however are using HTTP to form a persistent stateful connection between client and server, which is fundamentally opposed to REST. Basic HTTP would (and you would do this via the container doing it for you) fully open and close the connection for each isolated request, in practice however HTTP (and thus REST) will allow for the low level connection (the TCP connection that is slow to start) to be maintained for a series of request. This functionality is intended for the scope of loading a HTML page, and all resources in one TCP connection, but over many HTTP requests.
Sockets move bytes over TCP/IP. That's a lower level protocol, you don't want to worry about that. You care about the higher up protocol (which in this case is HTTP).
Sockets are closed in HTTP after every request, so what you're thinking sounds reasonable. Although I'm not sure why you would create a separate thread for a POST request. I'm assuming that your Client implementation already runs in its own thread (if it doesn't, then your server isn't very efficient).
I'm trying some multiplayer game ideas out at the moment and am trying to create a Java application to serve a web browser based multiplayer game.
My development environment is Eclipse on the main machine, and notepad + Google Chrome on this laptop.
I'm creating the websocket using javascript at the client end, and using the java.net.Socket at the server end.
I've managed to get a connection acknowledged at both ends, but can't seem to send or recieve any data between them without the client closing the connection (doesn't even error; just seems to freak out at something and call socket.close).
Does anyone have any ideas?
Here's some code:
Client:
<script type="text/javascript">
var socket;
function init() {
socket = new WebSocket("ws://192.168.0.3:10000");
socket.onopen = function() { alert('OPEN: ' + socket.readyState); }
socket.onmessage = function (msg) { alert('DATA: ' + msg.data); }
socket.onerror = function (msg) { alert('DATA: ' + msg.data); }
socket.onclose = function () { alert('CLOSED: ' + socket.readyState); }
}
function onClick() {
socket.send("YAY!");
}
</script>
Server:
public static void main(String args[])
{
System.out.printLn("Websocket server test");
ServerSocket connectSocket = null;
try
{
Socket clientSocket;
connectSocket = new ServerSocket(10000);
System.out.printLn("Waiting for connection...");
clientSocket = connectSocket.accept();
System.out.printLn("Got one!");
PrintWriter out = new PrintWriter(clientSocket.getOutputStream(), true);
BufferedReader in = new BufferedReader(
new InputStreamReader(clientSocket.getInputStream()));
for(int i=0;i<100;i++) //Shit but easy
{
String data = in.readLine();
System.out.printLn("Got data: " + data);
out.printLn("YAY!");
}
}
catch (IOException e)
{
System.out.printLn("You fail: " + e.getMessage());
}
System.out.printLn("Finished!");
}
Rather than going the painful way of implementing the spec in Java, I'd suggest that you use an existing solution like jWebSocket.
Also if you don't mind leaving Java land, I'd also suggest that you take a look at Node.js for your Server.
Doing both Server and Client in JavaScript will save you lots of time and lots of Code, especially since JSON just doesn't fit that well into static land. Also creating multiplayer servers in Node.js is trivial, since the event based, single threaded model fits the whole thing pretty well.
More information on WebSocket can be found in the FAQ. In case you want to get started with Node.js take a look at the TagWiki.
shameless plug follows
For two multiplayer games that were written using Node.js take a look at my GitHub page.
Try this lib - https://github.com/mrniko/netty-socketio
Based on high performance socket lib Netty. It supports latest protocol of Socket.IO server. Several transports including websocket.
On web side use Socket.IO client javascript lib:
<script type="text/javascript">
var socket = io.connect('http://localhost:81', {
'transports' : [ 'websocket' ],
'reconnection delay' : 2000,
'force new connection' : true
});
socket.on('message', function(data) {
// here is your handler on messages from server
});
// send object to server
var obj = ...
socket.json.send(obj);
</script>
I would suggest our high level solution: Bristleback Server. It contains both server and client, you can choose from several existing low level WebSocket engines (like Jetty, Netty or Tomcat), developing with Bristleback is extremally fast and easy. However, it is still Beta and we are working hard to release a final 1.0.0 version. If you use Maven, we have provided an archetype with ready to use web application.
I am one of the co-creators of Bristleback Server.
As no one yet really answered your question: the reason it does not work, is because you are not implementing the websocket specification. It takes of lot more work to setup a proper websocket connection than just opening a socket, as the websocket connection setup starts with a HTTP upgrade request. Your client is closing the connection, because it does not receive a positive answer on the upgrade request to start with.
I can't help you with sockets, but can i suggest you to use RMI technology? I'm trying to make a multiplayer rpg in java, and i'm using remote method invocation between server and client (it is possible also call-back the client from the server). It's really easy use it, but it uses TCP instead of UDP. In LAN experience there is no lag, on internet I have not tried yet. However, if your game tolerates just a bit retard between request and response, there is no problem.
This is the link of my project, Client and Server classes may be useful for you.
I am trying to build a WCF client for a Java Socket server which talks on a custom XML messages. I have created my own custom binding with MessageEncoder and netTCP transport.
Now what I have seen happen is
on the first call to the server, server accepts the connection. However the server then waits for a specific XML packet. This is built into the method interface from client. WCF does not send the XML packet.
Later WCF reports a timeout exception and the channel is faulted.
Will WCF netTCP transport work only with a WCF TCP Server?
Code:
[System.CodeDom.Compiler.GeneratedCodeAttribute("System.ServiceModel", "3.0.0.0")]
[System.ServiceModel.ServiceContractAttribute(ConfigurationName = "IUserManager", SessionMode=System.ServiceModel.SessionMode.Required)]
public interface IUserManager
{
[System.ServiceModel.OperationContract]
bool SendMessage(string strMessage);
[System.ServiceModel.OperationContract(IsInitiating = true, IsOneWay=true)]
void SendMessageOneWay(string strMessage);
}
I have created a SendMessageOneWay just to initiate the Socket connection. This did not work as well. As I need to reuse the socket, I have set Session.Required.
Please help
The WCF netTCP transport uses a custom TCP-based protocol; it is not a general purpose raw-TCP socket adapter.
Sounds like you will need to create your own custom transport channel. I created a list of resources on writing WCF channels a while ago (might be a bit out of date, but it should still be useful for getting started)
Yes wcf using .net framing protocol which is not interoperable.
http://blogs.msdn.com/b/drnick/archive/2009/01/19/message-framing-part-1.aspx