Develop Reference

How does Android VpnService handle incoming packets before Socket is opened?

I am trying to use the VpnService API to capture packets (on Android 4.4). I've been following an example at:
http://www.thegeekstuff.com/2014/06/android-vpn-service/
I establish the vpn connection using a builder, which is configured using the following code (which is borrowed from the ToyVPN sample, as I'm sure many of you will recognize):
// Configure a builder while parsing the parameters.
Builder builder = new Builder();
builder.setMtu(1500);
builder.addAddress("192.168.0.1", 24);
builder.addRoute("0.0.0.0", 0);
try {
mInterface.close();
} catch (Exception e) {
// ignore
}
mInterface = builder.establish();
The general concept is that with VpnService, I am able to get outbound packets and log them. Then I open a Socket to the original destination and forward the packets. When the response comes, it will be via the Socket that was opened.
So my question is, what happens when an IP address sends me a packet before I've opened a Socket to that address? I'm guessing that either it bypasses the VPN and arrives as normal, or is dropped entirely (probably the former?). Any insight is appreciated.

When you addRoute("0.0.0.0",0);
All packets will be routed to this Tun interface.
Now its upto you whether you want to forward them to original destination or not.
So answer to your question is you still get packets from the application which is trying to connect to its server, and now its up to you whether you allow its are not.

IOException "Socket is closed: Unknown socket_descriptor" using java.net.DatagramSocket with Google App Engine

I am using java.net.DatagramSocket to send UDP packets to a statsd server from a Google App Engine servlet. This generally works; however, we periodically see the following exception:
IOException - Socket is closed: Unknown socket_descriptor..
When these IOExceptions occur, calling DatagramSocket.isClosed() returns false.
This issue happens frequently enough that it is concerning, and although I've put in place some workarounds (allocate a new socket and use a DeferredTask queue to retry), it would be good to understand the underlaying reason for these errors.
The Google docs mention, "Sockets may be reclaimed after 2 minutes of inactivity; any socket operation keeps the socket alive for a further 2 minutes." It is unclear to me how this would play into UDP datagrams; however, one suspicion I have is that this is related to GAE instance lifecycle in some way.
My code (sanitized and extracted) looks like:
DatagramSocket _socket;
void init() {
_socket = new DatagramSocket();
}
void send() {
DatagramPacket packet = new DatagramPacket(<BYTES>, <LENGTH>, <HOST>, <PORT>);
_socket.send(packet);
}
Appreciate any feedback on this!

The approach taken to workaround this issue was simply to manage a single static DatagramSocket instance with a couple of helper methods, getSocket() and releaseSocket() to release sockets throwing IOExceptions through the release method, and then allocate upon next access through the get method. Not shown in this code is retry logic to retry the failed socket.send(). Under load testing, this seems to work reliably.
try {
DatagramPacket packet = new DatagramPacket(<BYTES>, <LENGTH>, <HOST>, <PORT>);
getSocket().send(packet);
} catch (IOException ioe) {
releaseSocket();
}

Sending an image over the network in Java

I am having a problem sending a .PNG image over a network. I have multiple clients connected to a single server. A client chooses the user to whom she would like to send the image to. Here is the code snippet on client side.
BufferedImage img = ImageIO.read(new File(source));
ImageIO.write(img, "png", spGUI.sdtGUI.output);
where spGUI.sdtGUI.output is an output stream wrapped around the output stream of the socket.
I am getting an exception that it cannot write the PNG file.
PS:I don't know how to include the stack trace in the post.

Designing java client

I'm currently developing a client/server application in Java and i can't make up my mind on how to design the client.
At the moment i have a server that can accept multiple connections and every connection has a loop listening for commands and respond to them. The server looks like this:
try{
mIn = new DataInputStream(mSocket.getInputStream());
mOut = new DataOutputStream(mSocket.getOutputStream());
while(true){
byte tPackType = mIn.readByte();
switch(tPackType){
case PackType.LOGIN:
login();
break;
case PackType.REGISTER:
register();
break;
default:
}
}
}catch (IOException e){
mLog.logp(Level.WARNING, this.getClass().getName(), "run()", "IOException in run()", e);
}finally{
try{
mOut.close();
mIn.close();
mSocket.close();
}catch (IOException e){
e.printStackTrace();
}
}
Now, the client is basically just a number of methods sending a request to the server and returns the response. I want to be able to receive updates from the server.
I would appreciate any suggestions on reading that can help me find a solution.
Is there any patterns i could look into?
Thank you in advance, veLr.

I want to be able to receive updates from the server
I think that you have two options to do this.
Option 1 is to make your client poll the server for updates. I would not recommend this since this will eventually increase your load on your server. You will most likely have performance issue the moment you attempt to scale your system up.
Option 2 is to create a small server on your client. So basically, whenever there is an update, the server would connect to the client and send it the updates. I would recommend this procedure since it allows you to establish connections between your server and clients only when needed. The problem, when opposed to the previous method, is that you will need to track of the ports and IP's of your clients. You could implement a HashTable with Client ID's as keys and their connection properties as their keys. Once you have an update, you will look up the client in the HashTable, obtain the connection information, connect and send the information.
The server on the client side will then get the data and do whatever you want the client to do once an update is received.

Java Sockets and Dropped Connections

What's the most appropriate way to detect if a socket has been dropped or not? Or whether a packet did actually get sent?
I have a library for sending Apple Push Notifications to iPhones through the Apple gatways (available on GitHub). Clients need to open a socket and send a binary representation of each message; but unfortunately Apple doesn't return any acknowledgement whatsoever. The connection can be reused to send multiple messages as well. I'm using the simple Java Socket connections. The relevant code is:
Socket socket = socket(); // returns an reused open socket, or a new one
socket.getOutputStream().write(m.marshall());
socket.getOutputStream().flush();
logger.debug("Message \"{}\" sent", m);
In some cases, if a connection is dropped while a message is sent or right before; Socket.getOutputStream().write() finishes successfully though. I expect it's due to the TCP window isn't exhausted yet.
Is there a way that I can tell for sure whether a packet actually got in the network or not? I experimented with the following two solutions:
Insert an additional socket.getInputStream().read() operation with a 250ms timeout. This forces a read operation that fails when the connection was dropped, but hangs otherwise for 250ms.
set the TCP sending buffer size (e.g. Socket.setSendBufferSize()) to the message binary size.
Both of the methods work, but they significantly degrade the quality of the service; throughput goes from a 100 messages/second to about 10 messages/second at most.
Any suggestions?
UPDATE:
Challenged by multiple answers questioning the possibility of the described. I constructed "unit" tests of the behavior I'm describing. Check out the unit cases at Gist 273786.
Both unit tests have two threads, a server and a client. The server closes while the client is sending data without an IOException thrown anyway. Here is the main method:
public static void main(String[] args) throws Throwable {
final int PORT = 8005;
final int FIRST_BUF_SIZE = 5;
final Throwable[] errors = new Throwable[1];
final Semaphore serverClosing = new Semaphore(0);
final Semaphore messageFlushed = new Semaphore(0);
class ServerThread extends Thread {
public void run() {
try {
ServerSocket ssocket = new ServerSocket(PORT);
Socket socket = ssocket.accept();
InputStream s = socket.getInputStream();
s.read(new byte[FIRST_BUF_SIZE]);
messageFlushed.acquire();
socket.close();
ssocket.close();
System.out.println("Closed socket");
serverClosing.release();
} catch (Throwable e) {
errors[0] = e;
}
}
}
class ClientThread extends Thread {
public void run() {
try {
Socket socket = new Socket("localhost", PORT);
OutputStream st = socket.getOutputStream();
st.write(new byte[FIRST_BUF_SIZE]);
st.flush();
messageFlushed.release();
serverClosing.acquire(1);
System.out.println("writing new packets");
// sending more packets while server already
// closed connection
st.write(32);
st.flush();
st.close();
System.out.println("Sent");
} catch (Throwable e) {
errors[0] = e;
}
}
}
Thread thread1 = new ServerThread();
Thread thread2 = new ClientThread();
thread1.start();
thread2.start();
thread1.join();
thread2.join();
if (errors[0] != null)
throw errors[0];
System.out.println("Run without any errors");
}
[Incidentally, I also have a concurrency testing library, that makes the setup a bit better and clearer. Checkout the sample at gist as well].
When run I get the following output:
Closed socket
writing new packets
Finished writing
Run without any errors

This not be of much help to you, but technically both of your proposed solutions are incorrect. OutputStream.flush() and whatever else API calls you can think of are not going to do what you need.
The only portable and reliable way to determine if a packet has been received by the peer is to wait for a confirmation from the peer. This confirmation can either be an actual response, or a graceful socket shutdown. End of story - there really is no other way, and this not Java specific - it is fundamental network programming.
If this is not a persistent connection - that is, if you just send something and then close the connection - the way you do it is you catch all IOExceptions (any of them indicate an error) and you perform a graceful socket shutdown:
1. socket.shutdownOutput();
2. wait for inputStream.read() to return -1, indicating the peer has also shutdown its socket

After much trouble with dropped connections, I moved my code to use the enhanced format, which pretty much means you change your package to look like this:
This way Apple will not drop a connection if an error happens, but will write a feedback code to the socket.

If you're sending information using the TCP/IP protocol to apple you have to be receiving acknowledgements. However you stated:
Apple doesn't return any
acknowledgement whatsoever
What do you mean by this? TCP/IP guarantees delivery therefore receiver MUST acknowledge receipt. It does not guarantee when the delivery will take place, however.
If you send notification to Apple and you break your connection before receiving the ACK there is no way to tell whether you were successful or not so you simply must send it again. If pushing the same information twice is a problem or not handled properly by the device then there is a problem. The solution is to fix the device handling of the duplicate push notification: there's nothing you can do on the pushing side.
#Comment Clarification/Question
Ok. The first part of what you understand is your answer to the second part. Only the packets that have received ACKS have been sent and received properly. I'm sure we could think of some very complicated scheme of keeping track of each individual packet ourselves, but TCP is suppose to abstract this layer away and handle it for you. On your end you simply have to deal with the multitude of failures that could occur (in Java if any of these occur an exception is raised). If there is no exception the data you just tried to send is sent guaranteed by the TCP/IP protocol.
Is there a situation where data is seemingly "sent" but not guaranteed to be received where no exception is raised? The answer should be no.
#Examples
Nice examples, this clarifies things quite a bit. I would have thought an error would be thrown. In the example posted an error is thrown on the second write, but not the first. This is interesting behavior... and I wasn't able to find much information explaining why it behaves like this. It does however explain why we must develop our own application level protocols to verify delivery.
Looks like you are correct that without a protocol for confirmation their is no guarantee the Apple device will receive the notification. Apple also only queue's the last message. Looking a little bit at the service I was able to determine this service is more for convenience for the customer, but cannot be used to guarantee service and must be combined with other methods. I read this from the following source.
http://blog.boxedice.com/2009/07/10/how-to-build-an-apple-push-notification-provider-server-tutorial/
Seems like the answer is no on whether or not you can tell for sure. You may be able to use a packet sniffer like Wireshark to tell if it was sent, but this still won't guarantee it was received and sent to the device due to the nature of the service.