Develop Reference

In java, how will a client know that the server has quit? [duplicate]

This question already has answers here:
Java socket API: How to tell if a connection has been closed?
(9 answers)
Closed 2 years ago.
So I'm in the making of a very simple server/client complex using java. So far I have managed to figure out what happens if the client quits, because then the server receives null while listening from any input from the client.
BUT - what happens if the client is connected and the server quits for any reason... the server is supposed to wait for input from the client, but how can the client know that the server is not listening anymore? For me the clients call to the server just goes into the void... nothing happens...
Can I do something to find out when the server goes down? Time-out, ping/pong or something?
As You surely can see I'm quite new at this, I'm just curious. This was a puzzle for me ever since I attended computer science at the university.
Thanks in advance. dr_xemacs.

(I am assuming you are working with blocking server socket and socket and not with non blocking ones)
Similarly to the server, reading from streams of a closed connection will return null.
However if you instead do not want to rely on this or a scared that the connection to the server could somehow persist, you can also use time outs (check this out! ) which will throw SocketTimeoutException when the time is up and, to keep track of whether the server is up or not, create a ping/packet to assure server is still up and running.
Edit: I did a quick search and this could be useful to you! Take a look!

How can the client know that the server is not listening anymore?
If the client doesn't attempt to interact at some level with the service, it won't know.
Assuming that the client has sent a request, a few different scenarios.
If the service is no longer listening on the designated port, the client will typically get a "Connection Refused" exception.
If the service is still running (in a sense) but it is not working properly, then connection attempts from the client are likely to time out.
If the service's host is down, the client liable get a timeout.
If there are network connectivity or firewall issues, the client could get a timeout or some other exception.
Can I do something to find out when the server goes down? Time-out, ping/pong or something?
You attempt to connect and send a request. If it fails or times out, that means the service is down. If you are designing and implementing the service yourself, you could include a special "healthcheck" request for clients to "ping" on. But the flip-side is that network and server resources will be consumed in receiving and responding to these requests. It can affect your ability to scale up the number of clients, for example, if each client pings the service every N seconds.
But a client typically doesn't need to know whether the service is up or down. It typically only cares that service responds when it it sends a real request. And the simplest way to handle that is to just send the request and deal with the outcome. (The client code has to deal with all possible outcomes anyway when doing a real request. The service can go down, etc between the last healthcheck ping and when the client sends a real request.)
Bottom line: Don't bother with checking availability in the client unless the application (i.e. the end user) really needs to know.

Your Server probably may be running on a certain port and so you can add a health check at the client side and update a global flag with status to let client know about its availibity :-
Socket socket = null;
try
{
socket = new Socket(host, port);
return true;
}
catch (Exception e)
{
return false;
}
finally
{
if(socket != null)
try
{
socket.close();
}
catch(Exception e){}
}

How to read secure web requests and process web socket requests with a single Java ServerSocket?

Intro
My code can process all bytes that are sent to a server and decide whether or not to let them go through and eventually send responses. I would like to use this to use the server as web server, web socket server and tcp server in one.
Although my code is written for Minecraft, I am not asking this on a Minecraft forum because answering this question doesn't require any prior knowledge about Minecraft or its codebase.
All you need to know about Minecraft
Minecraft is a Java game that can be played online. When it is being played online, there is a server that opens a ServerSocket and all players have their own client that opens a Socket that will communicate with the ServerSocket of the server.
Anyone can create a Minecraft server and install server-side modifications on their server (for those who know Minecraft, these are usually called plug-ins). My application is such a server-side modification. Most Minecraft servers are being hosted by Minecraft host companies. The owners of the servers have some kind of access to the part of the host computer that manages the server files.
Goal
The goal of my modification is to let the Minecraft server serve more clients than just the Minecraft clients. I would like the same server to also work as web server (for http and https requests) as well as (secure) web socket server and tcp server.
Why no multiple server sockets
The most common solution would be to just create a ServerSocket for the other server types and assign a different port to all of them. However, that is not an option in my case. Most hosts forbid you to open other ports or ask extra money for it. So I need to do it all with just the Minecraft ServerSocket.
What I achieved so far
So far, I have managed to let all bytes that are sent to the minecraft server first go through my code. My code can choose whether or not to let the bytes continue to the Minecraft server code. It can also send responses on its own without the need to inform the Minecraft server code at all.
In principle, what I have managed to do is sufficient to accomplish my goal, but I would like some help with how to continue. I will explain below what I have and have not accomplished so far.
The first byte that is sent by a Minecraft client to the server is always the same, namely 16. This is great because it allows me to easily distinguish Minecraft clients from web browsers and tcp clients.
HTTP requests and websocket connections always start with the same byte, namely 71. HTTPS and secure websockets always start with the byte 22. The TCP connections I was talking about will be sent by my own applications, so I can choose exactly what bytes they will send and I can simply program my modification to respond to that.
I managed to distinguish http requests and websocket connections by their connection property. Http requests always send 'Connection: keep-alive' while websocket connections always send 'Connection: upgrade'. (Although some browsers do the k, a and u in uppercase and others do not.)
Handling normal http requests wasn't very hard to do. Handling TCP connections won't be hard either because I will control everything. But I have issues with the remaining connection types:
Problems I need help with
The web socket protocol is quite large and I would prefer not to handle it completely with only my code. (I tried this before, but I kept having problems with the parts that were rarely used and thus not tested.) So I would like to use some library that allows me to only worry about the payload rather than the entire protocol. Unfortunately, web socket libraries generally want to create the ServerSocket, which is not possible in my case. So does anyone have advice on what to do here?
I haven't found any info on how to read https requests properly. Could someone tell me where to look for the details of this protocol or provide a nice link?
For secure web sockets, I will face the same problems as with the 'normal' web socket connections after I find out how to read the requests.
Code
All my code so far can be found at https://github.com/knokko/Multi-Purpose-Server. The most interesting part is probably the part where my code gets the opportunity to process all bytes before they arrive at the Minecraft code, that code is shown below.
Short question(s)
For those who didn't understand exactly what my question (you could see it as 2 questions that are strongly related) is:
-How I should read https requests and secure web socket handshakes?
-Does anyone know a library that can handle web socket input that doesn't require to create the ServerSocket itself?
// This channel handler will be registered for every connection client that will
// inspect
// any message before it reaches the Minecraft code.
pipeline.addFirst("multipurpose_handler_inspector", new ChannelInboundHandlerAdapter() {
private boolean deactivated;
private ChannelListener listener;
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
//super.channelRead will send the content to the minecraft code
try {
if (!deactivated) {
ByteBuf message = (ByteBuf) msg;
if (listener != null) {
listener.read(ctx, message);
} else {
byte firstByte = message.getByte(0);
// All Minecraft connections start with the byte 16
if (firstByte == 16) {
deactivated = true;
super.channelRead(ctx, msg);
}
// All insecure web connections start with the byte 71
else if (firstByte == 71) {
byte[] data = new byte[message.readableBytes()];
message.getBytes(0, data);
WebHandler.Type type = WebHandler.determineConnectionType(data);
if (type == WebHandler.Type.HTTP) {
listener = new HTTPListener();
listener.readInitial(ctx, message);
} else if (type == WebHandler.Type.WEBSOCKET) {
// TODO Find a nice way to handle web socket connections
listener = new WebSocketListener();
listener.readInitial(ctx, message);
} else {
deactivated = true;
super.channelRead(ctx, msg);
}
}
// All secure web connections start with the byte 22
else if (firstByte == 22) {
// TODO implement the secure web protocols and find a way to read this stuff
// and find the difference
System.out.println(
"We are dealing with a secure websocket or https connection");
byte[] data = new byte[message.readableBytes()];
message.getBytes(0, data);
System.out.println(new String(data));
}
// My applications
else if (firstByte == 31) {
listener = new TCPListener();
listener.readInitial(ctx, message);
} else {
System.out.println("Unknown connection type");
deactivated = true;
super.channelRead(ctx, msg);
}
}
} else {
super.channelRead(ctx, msg);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
});

If you can ALWAYS identify the Minecraft traffic, your best bet might be to run an apache/httpd and/or tomcat server on the same box and forward all non-Minecraft traffic to it. If you do this, the HTTPS stuff might just be a matter of correctly configuring the http server for https traffic.
You may have to configure your code as an http proxy--in fact (Just thought of this) you might want to go out and look for an open source http proxy and just tweak it with your code to extract Minecraft traffic and forward it before doing the rest of the proxy stuff.
I wouldn't do the HTTPs stuff from scratch, it's not terribly difficult but I'd call it non-trivial.
Oh and if your problem is "differentiating Minecraft HTTPS traffic from other HTTPS connections on the same port" I can't help except to say that this might be a good subject for your question :)

Sending messages on same socket connection while a file is being sent

My app can transfer files and messages between server and client. Server is multithreaded and clients simply connects to it. While file is being transferred, if sender sends a message, it will be consumed as bytes of file.
I don't want to open more ports,
Can I establish a new connection to the server for file transfer? Or I
should open a separate port for files.
I don't want to block communication while a file is being transferred.
The question was marked as a duplicate but its not, i am trying to send messages and files simultaneously not one by one. I can already receive files one by one. Read again.
Also, as server is multithreaded, I cannot call server socket.accept() again to receive files in new connection because main thread listening for incoming will try to handle it instead. Is there a way around?

Seems to me like trying to multiplex files and messages onto the same socket stream is an XYProblem.
I am not an expert on this, but it sounds like you should do some reading on "ports vs sockets". My understanding is that ip:port is the address of the listening service. Once a client connects, the server will open a socket to actually do the communication.
The trick is that every time a client connects, spawn a new thread (on a new socket) to handle the request. This instantly frees up the main thread to go back to listening for new connections. Your file transfer and your messages can come into the same port, but each new request will get its own socket and its own server thread --> no collision!
See this question for a java implementation:
Multithreading Socket communication Client/Server

you could use some system of all the lines of a file start with a string like this (file:linenum) and then on the other side it puts that in a file then to send text you could do the same thing but with a tag like (text)
Server:
Scanner in = new Scanner(s.getInputStream());
while(true) {
String message = in.nextLine();
if(message.length > 14 && message.substring(0,6).equalsIgnoreCase("(file:") {
int line = Integer.valueOf(message.substring(6).replaceall(")", ""));
saveToFile(message.substring(6).replaceAll(")","").replaceAll("<1-9>",""));
} else {
System.out.println(message);
}
}
I think that code works but I haven't checked it so it might need some slight modifications

You could introduce a handshake protocol where clients can state who they are (probably happening already) and what they want from the given connection. The first connection they make could be about control, and perhaps the messages, and remain in use all the time. File transfer could happen via secondary connections, which may come and go during a session. Having several parallel connections between a client and a server is completely normal, that is what #MikeOunsworth was explaining too.
A shortcut you can take is issuing short-living, one-time tokens which clients can present when opening the secondary connection and then the server will immediately know which file it should start sending. Note that this approach easily can raise various security (if token encodes actual request data) and/or scalability issues (if token is something completely random and has to be looked up in some table).

DatagramSocket Broadcast Behavior (Windows vs. Linux)

Backstory:
I have a wireless device which creates it's own SSID, assigns itself an IP address using auto-ip, and begins broadcasting discovery information to 255.255.255.255. (unfortunately, it does not easily support multicast)
What I'm trying to do:
I need to be able to receive the discovery information, then send configuration information to the device. The problem is, with auto-ip, the "IP negotiation" process can take minutes on Windows, etc (during which time I can see the broadcasts and can even send broadcast information back to the device).
So I enumerate all connected network interfaces (can't directly tell which will be used to talk to the device), create a DatagramSocket for each of their addresses, then start listening. If I receive the discovery information via a particular socket, I know I can use that same socket to send data back to the device. This works on Windows.
The problem:
On Linux and OSX, the following code does not receive broadcast packets:
byte[] addr = {(byte)169, (byte)254, (byte)6, (byte)215};
DatagramSocket foo = new DatagramSocket(new InetSocketAddress(InetAddress.getByAddress(addr), PORT_NUM));
while (true)
{
byte[] buf = new byte[256];
DatagramPacket pct = new DatagramPacket(buf, buf.length);
foo.receive(pct);
System.out.println( IoBuffer.wrap(buf).getHexDump() );
}
In order to receive broadcast packets (on Linux/OSX), I need to create my DatagramSocket using:
DatagramSocket foo = new DatagramSocket(PORT_NUM);
However, when I then use this socket to send data back to the device, the packet is routed by the OS (I'm assuming) and since the interface of interest may be in the middle of auto-ip negotiation, fails.
Thoughts on the following?
How to get the "working" Windows behavior to happen on Linux/OSX
A better way to handle this process
Thanks in advance!

I do not think this is the problem with the code. Have you checked if OSX/Linux has correctly allowed those address / port number through their firewalls? I had this simple problem too in the past =P..
FYI, there is a nice technology called Zero-configuration which was built to solve this problem. It is very easy to learn so I recommend you to having a look at that as well.
Good luck.

Java Sockets and Dropped Connections

What's the most appropriate way to detect if a socket has been dropped or not? Or whether a packet did actually get sent?
I have a library for sending Apple Push Notifications to iPhones through the Apple gatways (available on GitHub). Clients need to open a socket and send a binary representation of each message; but unfortunately Apple doesn't return any acknowledgement whatsoever. The connection can be reused to send multiple messages as well. I'm using the simple Java Socket connections. The relevant code is:
Socket socket = socket(); // returns an reused open socket, or a new one
socket.getOutputStream().write(m.marshall());
socket.getOutputStream().flush();
logger.debug("Message \"{}\" sent", m);
In some cases, if a connection is dropped while a message is sent or right before; Socket.getOutputStream().write() finishes successfully though. I expect it's due to the TCP window isn't exhausted yet.
Is there a way that I can tell for sure whether a packet actually got in the network or not? I experimented with the following two solutions:
Insert an additional socket.getInputStream().read() operation with a 250ms timeout. This forces a read operation that fails when the connection was dropped, but hangs otherwise for 250ms.
set the TCP sending buffer size (e.g. Socket.setSendBufferSize()) to the message binary size.
Both of the methods work, but they significantly degrade the quality of the service; throughput goes from a 100 messages/second to about 10 messages/second at most.
Any suggestions?
UPDATE:
Challenged by multiple answers questioning the possibility of the described. I constructed "unit" tests of the behavior I'm describing. Check out the unit cases at Gist 273786.
Both unit tests have two threads, a server and a client. The server closes while the client is sending data without an IOException thrown anyway. Here is the main method:
public static void main(String[] args) throws Throwable {
final int PORT = 8005;
final int FIRST_BUF_SIZE = 5;
final Throwable[] errors = new Throwable[1];
final Semaphore serverClosing = new Semaphore(0);
final Semaphore messageFlushed = new Semaphore(0);
class ServerThread extends Thread {
public void run() {
try {
ServerSocket ssocket = new ServerSocket(PORT);
Socket socket = ssocket.accept();
InputStream s = socket.getInputStream();
s.read(new byte[FIRST_BUF_SIZE]);
messageFlushed.acquire();
socket.close();
ssocket.close();
System.out.println("Closed socket");
serverClosing.release();
} catch (Throwable e) {
errors[0] = e;
}
}
}
class ClientThread extends Thread {
public void run() {
try {
Socket socket = new Socket("localhost", PORT);
OutputStream st = socket.getOutputStream();
st.write(new byte[FIRST_BUF_SIZE]);
st.flush();
messageFlushed.release();
serverClosing.acquire(1);
System.out.println("writing new packets");
// sending more packets while server already
// closed connection
st.write(32);
st.flush();
st.close();
System.out.println("Sent");
} catch (Throwable e) {
errors[0] = e;
}
}
}
Thread thread1 = new ServerThread();
Thread thread2 = new ClientThread();
thread1.start();
thread2.start();
thread1.join();
thread2.join();
if (errors[0] != null)
throw errors[0];
System.out.println("Run without any errors");
}
[Incidentally, I also have a concurrency testing library, that makes the setup a bit better and clearer. Checkout the sample at gist as well].
When run I get the following output:
Closed socket
writing new packets
Finished writing
Run without any errors

This not be of much help to you, but technically both of your proposed solutions are incorrect. OutputStream.flush() and whatever else API calls you can think of are not going to do what you need.
The only portable and reliable way to determine if a packet has been received by the peer is to wait for a confirmation from the peer. This confirmation can either be an actual response, or a graceful socket shutdown. End of story - there really is no other way, and this not Java specific - it is fundamental network programming.
If this is not a persistent connection - that is, if you just send something and then close the connection - the way you do it is you catch all IOExceptions (any of them indicate an error) and you perform a graceful socket shutdown:
1. socket.shutdownOutput();
2. wait for inputStream.read() to return -1, indicating the peer has also shutdown its socket

After much trouble with dropped connections, I moved my code to use the enhanced format, which pretty much means you change your package to look like this:
This way Apple will not drop a connection if an error happens, but will write a feedback code to the socket.

If you're sending information using the TCP/IP protocol to apple you have to be receiving acknowledgements. However you stated:
Apple doesn't return any
acknowledgement whatsoever
What do you mean by this? TCP/IP guarantees delivery therefore receiver MUST acknowledge receipt. It does not guarantee when the delivery will take place, however.
If you send notification to Apple and you break your connection before receiving the ACK there is no way to tell whether you were successful or not so you simply must send it again. If pushing the same information twice is a problem or not handled properly by the device then there is a problem. The solution is to fix the device handling of the duplicate push notification: there's nothing you can do on the pushing side.
#Comment Clarification/Question
Ok. The first part of what you understand is your answer to the second part. Only the packets that have received ACKS have been sent and received properly. I'm sure we could think of some very complicated scheme of keeping track of each individual packet ourselves, but TCP is suppose to abstract this layer away and handle it for you. On your end you simply have to deal with the multitude of failures that could occur (in Java if any of these occur an exception is raised). If there is no exception the data you just tried to send is sent guaranteed by the TCP/IP protocol.
Is there a situation where data is seemingly "sent" but not guaranteed to be received where no exception is raised? The answer should be no.
#Examples
Nice examples, this clarifies things quite a bit. I would have thought an error would be thrown. In the example posted an error is thrown on the second write, but not the first. This is interesting behavior... and I wasn't able to find much information explaining why it behaves like this. It does however explain why we must develop our own application level protocols to verify delivery.
Looks like you are correct that without a protocol for confirmation their is no guarantee the Apple device will receive the notification. Apple also only queue's the last message. Looking a little bit at the service I was able to determine this service is more for convenience for the customer, but cannot be used to guarantee service and must be combined with other methods. I read this from the following source.
http://blog.boxedice.com/2009/07/10/how-to-build-an-apple-push-notification-provider-server-tutorial/
Seems like the answer is no on whether or not you can tell for sure. You may be able to use a packet sniffer like Wireshark to tell if it was sent, but this still won't guarantee it was received and sent to the device due to the nature of the service.