I'm writing a game server in Java, and I'm using Netty 3.6.2. The game servers should accept no HTTP requests, as they simply handle game client data (which is purely bytes over TCP). When I load http://server-ip:game-servers-port in Chrome, I download a file with the game's handshake packet (which should not happen).
I bind to the game server's port like so:
ChannelFactory factory = new NioServerSocketChannelFactory(Executors.newCachedThreadPool(), Executors.newCachedThreadPool());
clientAcceptor = new ServerBootstrap(factory);
clientAcceptor.setOption("child.tcpNoDelay", true);
clientAcceptor.setOption("child.keepAlive", false);
clientAcceptor.setPipelineFactory(() -> Channels.pipeline(new PacketDecoder(), new ClientHandler()));
clientAcceptor.bind(new InetSocketAddress(Configurations.CHANNEL_GAME_PORT));
And I process requests in a SimpleChannelHandler, like so
#Override
public void channelConnected(ChannelHandlerContext ctx, ChannelStateEvent e) {
// Encryption key code.
Packets.sendHello(ctx.getChannel(), ivRecv, ivSend);
ctx.getChannel().setAttachment(client);
}
How can I go about deciphering if an incoming request is using the HTTP protocol?
Edit: I should also say, it should also block any FTP, WebSocket, etc. protocol (essentially anything that isn't the game's protocol) on channel connect.
You can't expect to be able to disable HTTP requests if the first thing you do with an accepted connection is to send something.
Have the client do the first send in the handshake. Have the server start by doing a receive, and if it isn't the correct initial handshake packet, close the connection.
Related
Intro
My code can process all bytes that are sent to a server and decide whether or not to let them go through and eventually send responses. I would like to use this to use the server as web server, web socket server and tcp server in one.
Although my code is written for Minecraft, I am not asking this on a Minecraft forum because answering this question doesn't require any prior knowledge about Minecraft or its codebase.
All you need to know about Minecraft
Minecraft is a Java game that can be played online. When it is being played online, there is a server that opens a ServerSocket and all players have their own client that opens a Socket that will communicate with the ServerSocket of the server.
Anyone can create a Minecraft server and install server-side modifications on their server (for those who know Minecraft, these are usually called plug-ins). My application is such a server-side modification. Most Minecraft servers are being hosted by Minecraft host companies. The owners of the servers have some kind of access to the part of the host computer that manages the server files.
Goal
The goal of my modification is to let the Minecraft server serve more clients than just the Minecraft clients. I would like the same server to also work as web server (for http and https requests) as well as (secure) web socket server and tcp server.
Why no multiple server sockets
The most common solution would be to just create a ServerSocket for the other server types and assign a different port to all of them. However, that is not an option in my case. Most hosts forbid you to open other ports or ask extra money for it. So I need to do it all with just the Minecraft ServerSocket.
What I achieved so far
So far, I have managed to let all bytes that are sent to the minecraft server first go through my code. My code can choose whether or not to let the bytes continue to the Minecraft server code. It can also send responses on its own without the need to inform the Minecraft server code at all.
In principle, what I have managed to do is sufficient to accomplish my goal, but I would like some help with how to continue. I will explain below what I have and have not accomplished so far.
The first byte that is sent by a Minecraft client to the server is always the same, namely 16. This is great because it allows me to easily distinguish Minecraft clients from web browsers and tcp clients.
HTTP requests and websocket connections always start with the same byte, namely 71. HTTPS and secure websockets always start with the byte 22. The TCP connections I was talking about will be sent by my own applications, so I can choose exactly what bytes they will send and I can simply program my modification to respond to that.
I managed to distinguish http requests and websocket connections by their connection property. Http requests always send 'Connection: keep-alive' while websocket connections always send 'Connection: upgrade'. (Although some browsers do the k, a and u in uppercase and others do not.)
Handling normal http requests wasn't very hard to do. Handling TCP connections won't be hard either because I will control everything. But I have issues with the remaining connection types:
Problems I need help with
The web socket protocol is quite large and I would prefer not to handle it completely with only my code. (I tried this before, but I kept having problems with the parts that were rarely used and thus not tested.) So I would like to use some library that allows me to only worry about the payload rather than the entire protocol. Unfortunately, web socket libraries generally want to create the ServerSocket, which is not possible in my case. So does anyone have advice on what to do here?
I haven't found any info on how to read https requests properly. Could someone tell me where to look for the details of this protocol or provide a nice link?
For secure web sockets, I will face the same problems as with the 'normal' web socket connections after I find out how to read the requests.
Code
All my code so far can be found at https://github.com/knokko/Multi-Purpose-Server. The most interesting part is probably the part where my code gets the opportunity to process all bytes before they arrive at the Minecraft code, that code is shown below.
Short question(s)
For those who didn't understand exactly what my question (you could see it as 2 questions that are strongly related) is:
-How I should read https requests and secure web socket handshakes?
-Does anyone know a library that can handle web socket input that doesn't require to create the ServerSocket itself?
// This channel handler will be registered for every connection client that will
// inspect
// any message before it reaches the Minecraft code.
pipeline.addFirst("multipurpose_handler_inspector", new ChannelInboundHandlerAdapter() {
private boolean deactivated;
private ChannelListener listener;
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
//super.channelRead will send the content to the minecraft code
try {
if (!deactivated) {
ByteBuf message = (ByteBuf) msg;
if (listener != null) {
listener.read(ctx, message);
} else {
byte firstByte = message.getByte(0);
// All Minecraft connections start with the byte 16
if (firstByte == 16) {
deactivated = true;
super.channelRead(ctx, msg);
}
// All insecure web connections start with the byte 71
else if (firstByte == 71) {
byte[] data = new byte[message.readableBytes()];
message.getBytes(0, data);
WebHandler.Type type = WebHandler.determineConnectionType(data);
if (type == WebHandler.Type.HTTP) {
listener = new HTTPListener();
listener.readInitial(ctx, message);
} else if (type == WebHandler.Type.WEBSOCKET) {
// TODO Find a nice way to handle web socket connections
listener = new WebSocketListener();
listener.readInitial(ctx, message);
} else {
deactivated = true;
super.channelRead(ctx, msg);
}
}
// All secure web connections start with the byte 22
else if (firstByte == 22) {
// TODO implement the secure web protocols and find a way to read this stuff
// and find the difference
System.out.println(
"We are dealing with a secure websocket or https connection");
byte[] data = new byte[message.readableBytes()];
message.getBytes(0, data);
System.out.println(new String(data));
}
// My applications
else if (firstByte == 31) {
listener = new TCPListener();
listener.readInitial(ctx, message);
} else {
System.out.println("Unknown connection type");
deactivated = true;
super.channelRead(ctx, msg);
}
}
} else {
super.channelRead(ctx, msg);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
});
If you can ALWAYS identify the Minecraft traffic, your best bet might be to run an apache/httpd and/or tomcat server on the same box and forward all non-Minecraft traffic to it. If you do this, the HTTPS stuff might just be a matter of correctly configuring the http server for https traffic.
You may have to configure your code as an http proxy--in fact (Just thought of this) you might want to go out and look for an open source http proxy and just tweak it with your code to extract Minecraft traffic and forward it before doing the rest of the proxy stuff.
I wouldn't do the HTTPs stuff from scratch, it's not terribly difficult but I'd call it non-trivial.
Oh and if your problem is "differentiating Minecraft HTTPS traffic from other HTTPS connections on the same port" I can't help except to say that this might be a good subject for your question :)
I am trying to use the VpnService API to capture packets (on Android 4.4). I've been following an example at:
http://www.thegeekstuff.com/2014/06/android-vpn-service/
I establish the vpn connection using a builder, which is configured using the following code (which is borrowed from the ToyVPN sample, as I'm sure many of you will recognize):
// Configure a builder while parsing the parameters.
Builder builder = new Builder();
builder.setMtu(1500);
builder.addAddress("192.168.0.1", 24);
builder.addRoute("0.0.0.0", 0);
try {
mInterface.close();
} catch (Exception e) {
// ignore
}
mInterface = builder.establish();
The general concept is that with VpnService, I am able to get outbound packets and log them. Then I open a Socket to the original destination and forward the packets. When the response comes, it will be via the Socket that was opened.
So my question is, what happens when an IP address sends me a packet before I've opened a Socket to that address? I'm guessing that either it bypasses the VPN and arrives as normal, or is dropped entirely (probably the former?). Any insight is appreciated.
When you addRoute("0.0.0.0",0);
All packets will be routed to this Tun interface.
Now its upto you whether you want to forward them to original destination or not.
So answer to your question is you still get packets from the application which is trying to connect to its server, and now its up to you whether you allow its are not.
I'm on the dev team for a socket server which uses Netty. When a client sends a request, and the server sends a single response, the round trip time is quite fast. (GOOD) We recently noticed that if the request from the client triggers two messages from the server, even though the server writes both messages to the client at about the same time, there is a delay of more than 200ms between the first and second message arriving on the remote client. When using a local client the two messages arrive at the same time. If the remote client sends another request before the second message from the server arrives, that second message is sent immediately, but then the two messages from the new request are both sent with the delay of over 200ms.
Since it was noticed while using Netty 3.3.1, I tried upgrading to Netty 3.6.5 but I still see the same behavior. We are using NIO, not OIO, because we need to be able to support large numbers of concurrent clients.
Is there a setting that we need to configure that will reduce that 200+ ms delay?
editing to add a code snippet. I hope these are the most relevant parts.
#Override
public boolean openListener(final Protocol protocol,
InetSocketAddress inetSocketAddress) throws Exception {
ChannelFactory factory = new NioServerSocketChannelFactory(
Executors.newCachedThreadPool(),
Executors.newCachedThreadPool(),
threadingConfiguration.getProcessorThreadCount());
ServerBootstrap bootstrap = new ServerBootstrap(factory);
final ChannelGroup channelGroup = new DefaultChannelGroup();
bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
.... lots of pipeline setup snipped ......
});
Channel channel = bootstrap.bind(inetSocketAddress);
channelGroup.add(channel);
channelGroups.add(channelGroup);
bootstraps.add(bootstrap);
return true;
}
The writer factory uses ChannelBuffers.dynamicBuffer(defaultMessageSize) for the buffer, and when we write a message it's Channels.write(channel, msg).
What else would be useful? The developer who migrated the code to Netty is not currently available, and I'm trying to fill in.
200ms strikes me as the magic number of the Nagle's algorithm. Try setting the TcpNoDelay to true on both sides.
This is how you set the option for the server side.
serverBootstrap.setOption("child.tcpNoDelay", true);
This is for the client side.
clientBootStrap.setOption("tcpNoDelay", true);
Further reading: http://www.stuartcheshire.org/papers/NagleDelayedAck/
I am trying to open a basic connection to an HL7 server where I send a request and get the ACK response. This will be done continuously.
If this is being done continuously, when do I close the socket? Am I implementing this correctly, in this case?
If I close the socket, how do I open it again? The javadocs for ConnectionHub indicates the following:
attach(java.lang.String host, int port, Parser parser,
java.lang.Class<? extends LowerLayerProtocol> llpClass)
Returns a Connection to the given address, opening this Connection if necessary.
However, in real life, it will not open a new connection if it was already closed.
Patient patient = appt.getPatient();
Parser parser = new GenericParser();
Message hl7msg = parser.parse(wlp.getORMString(appt));
//Connect to listening servers
ConnectionHub connectionHub = ConnectionHub.getInstance();
// A connection object represents a socket attached to an HL7 server
Connection connection = connectionHub.attach(serverIP, serverPort,
new PipeParser(), MinLowerLayerProtocol.class);
if (!connection.isOpen()) {
System.out.println("CONNNECTION is CLOSED");
connection = connectionHub.attach(serverIP, serverPort, new PipeParser(),
MinLowerLayerProtocol.class);
if (!connection.isOpen()) {
System.out.println("CONNNECTION is still CLOSED");
}
}
Initiator initiator = connection.getInitiator();
Message response = initiator.sendAndReceive(hl7msg);
String responseString = parser.encode(response);
System.out.println("Received response:\n" + responseString);
connection.close();
Result:
The first pass goes through perfectly, with request sent and ACK received. Any subsequent call to this method results in java.net.SocketException: Socket closed" on the client side.
If I remove the connection.close() call, then it will run fine for a certain amount of time then the socket will close itself.
If you are communicating via HL7 2.X, the expected behavior on the socket is to never disconnect -- you allocate the connection and keep the socket active. Said another way, an HL7 application does not act like a web browser wherein it connects as needed and disconnects when done. Rather, both ends work to keep the socket continuously connected. Most applications will be annoyed if you disconnect. Further, most integration engines have alerts that will fire if you are disconnected for too long.
Once the socket is connected, you need to use the HL7 Minimum Lower Layer Protocol (MLLP or MLP) to communicate the HL7 2.X content. If you are sending data, you should wait for an HL7 Acknowledgment before you send the next message. If you are receiving data, you should generate the HL7 Ack.
References:
MLP - http://www.hl7standards.com/blog/2007/05/02/hl7-mlp-minimum-layer-protocol-defined
Acks - http://www.corepointhealth.com/resource-center/hl7-resources/hl7-acknowledgement
I am trying to build a WCF client for a Java Socket server which talks on a custom XML messages. I have created my own custom binding with MessageEncoder and netTCP transport.
Now what I have seen happen is
on the first call to the server, server accepts the connection. However the server then waits for a specific XML packet. This is built into the method interface from client. WCF does not send the XML packet.
Later WCF reports a timeout exception and the channel is faulted.
Will WCF netTCP transport work only with a WCF TCP Server?
Code:
[System.CodeDom.Compiler.GeneratedCodeAttribute("System.ServiceModel", "3.0.0.0")]
[System.ServiceModel.ServiceContractAttribute(ConfigurationName = "IUserManager", SessionMode=System.ServiceModel.SessionMode.Required)]
public interface IUserManager
{
[System.ServiceModel.OperationContract]
bool SendMessage(string strMessage);
[System.ServiceModel.OperationContract(IsInitiating = true, IsOneWay=true)]
void SendMessageOneWay(string strMessage);
}
I have created a SendMessageOneWay just to initiate the Socket connection. This did not work as well. As I need to reuse the socket, I have set Session.Required.
Please help
The WCF netTCP transport uses a custom TCP-based protocol; it is not a general purpose raw-TCP socket adapter.
Sounds like you will need to create your own custom transport channel. I created a list of resources on writing WCF channels a while ago (might be a bit out of date, but it should still be useful for getting started)
Yes wcf using .net framing protocol which is not interoperable.
http://blogs.msdn.com/b/drnick/archive/2009/01/19/message-framing-part-1.aspx