Develop Reference

Java UDP hole punching example - connecting through firewall - java

Lets say I have two computers.
They know each others public and private IPs via ice4j.
One client listening and the other one sending some string.
I'd like to see this happen via UPD hole punching:
Let A be the client requesting the connection
Let B be the client that is responding to the request
Let S be the ice4j STUN server that they contact to initiate the connection
--
A sends a connection request to S
S responds with B's IP and port info, and sends A's IP and port info to B
A sends a UDP packet to B, which B's router firewall drops but it still
punches a hole in A's own firewall where B can connect
B sends a UDP packet to A, that both punches a hole in their own firewall,
and reaches A through the hole that they punched in their own firewall
A and B can now communicate through their established connection without
the help of S
Could any one post pseudo examples of how to go about doing hole punching through symmetric NAT? Assuming there will be server S that will help to guess the port numbers and establish connection between the client A and B.
It would be nice if you accounted for double NAT as well.
NOTE:
You can use STUN to discover the IP and Port but you have to write your own code that would send the IP:Port to your server via keepalive technique.
Once one client identifies the other via unique ID on the server it will be provided with the other's client IP:port info to UDP hole punch the data it needs to send and receive.
Little update:
There is library that is showing up on the horizon for java check it out:
https://github.com/htwg/UCE#readme

This example is in C#, not in Java, but the concepts of NAT traversal are language-agnostic.
See Michael Lidgren's network library which has NAT traversal built in.
Link: http://code.google.com/p/lidgren-network-gen3/
Specific C# File Dealing with NAT Traversal: http://code.google.com/p/lidgren-network-gen3/source/browse/trunk/Lidgren.Network/NetNatIntroduction.cs
The process you've posted is correct. It will work, for only 3 out of 4 general types of NAT devices (I say general because NAT behavior isn't really standardized): Full-Cone NATs, Restricted-Cone NATs, and Port-Restricted-Cone NATs. NAT traversal will not work with Symmetric NATs, which are found mostly in corporate networks for enhanced security. If one party uses a Symmetric NAT and the other party doesn't, it's still possible to traverse the NAT but it requires more guesswork. A Symmetric NAT to Symmetric NAT traversal is extremely difficult - you can read a paper about it here.
But really, the process you've described works exactly. I've implemented it for my own remote screen sharing program (also in C#, unfortunately). Just make sure you've disabled Windows firewall (if you're using Windows) and third-party firewalls. But yes, I can happily confirm that it will work.
Clarifying the Process of NAT Traversal
I'm writing this update to clarify the process of NAT traversal for you and future readers. Hopefully, this can be a clear summary of the history and the process.
Some Reference Sources: http://think-like-a-computer.com/2011/09/16/types-of-nat/, and http://en.wikipedia.org/wiki/Network_address_translation, http://en.wikipedia.org/wiki/IPv4, http://en.wikipedia.org/wiki/IPv4_address_exhaustion.
IPv4 addresses, with the capacity to uniquely name approximately 4.3 billion computers, have run out. Smart people foresaw this problem, and, among other reasons, invented routers to combat IPv4 address exhaustion, by assigning a network of computers connected to itself 1 shared IP address.
There are LAN IPs. And then there are WAN IPs. LAN IPs are Local Area Network IPs which uniquely identify computers in a local network, say the desktops, laptops, printers, and smartphones connected to a home router. WAN IPs uniquely identify computers outside of the local area network in a wide area network - commonly taken to mean The Internet. So these routers assign a group of computers 1 WAN IP. Each computer still has its own LAN IP. LAN IPs are what you see when you type ipconfig in your Command Prompt and get IPv4 Address . . . . . . . . 192.168.1.101. WAN IPs are what you see when you connect to cmyip.com and get 128.120.196.204.
Just as the radio spectrum is bought out, so entire IP ranges are bought out and reserved as well by agencies and organizations, as well as port numbers. The short message is, again, that we don't have any more IPv4 addresses to spare.
What does this have to do with NAT traversal? Well, since routers were invented, direct connections (end-to-end connectivity) have been somewhat ... impossible, without a few hacks. If you have a network of 2 computers (Computer A and Computer B) both sharing the WAN IP of 128.120.196.204, to which computer does a connection go? I'm talking about an external computer (say google.com) initiating a connection to 128.120.196.204. The answer is: nobody knows, and neither does the router, which is why the router drops the connection. If Computer A initiates a connection to, say, google.com, then that's a different story. The router then remembers that Computer A with LAN IP 192.168.1.101 intiated a connection to 74.125.227.64 (google.com). As Computer A's request packet leaves the router, the router actually re-writes LAN IP 192.168.1.101 to the router's WAN IP of 128.120.196.204. So, when google.com receives Computer A's request packet, it sees the sender IP that the router re-wrote, not the LAN IP of Computer A (google.com sees 128.120.196.204 as the IP to reply to). When google.com finally replies, the packet reaches the router, the router remembers (it has a state table) that it was expecting a reply from google.com, and it appropriately forwards the packet to Computer A.
In other words, your router has no problem when you initiate the connection - your router will remember to forward the replying packet back to your computer (through that whole process described above). But, when an external server initiates a connection to you, the router can't know which computer the connection was meant for, since Computer A and Computer B both share the WAN IP of 128.120.196.204 ... unless, there's a clear rule that instructs the router to forward all packets originally going to destination port X, now to go to Computer A, destination port Y. This is known as port-forwarding. Unfortunately, if you're thinking of using port-forwarding for your networking applications, it's not practical, as your users may not understand how to enable it, and may be reluctant to enable it if they think it's a security risk. UPnP simply refers to the technology that allows you to programatically enable port-forwarding. Unfortunately, if you're thinking of using UPnP to port-forward your networking applications, it's not practical either, as UPnP is not always available, and when it is, it may not turned on by default.
So what's the solution then? The solution is to either proxy your entire traffic over your own computer (which you have carefully pre-configured to be globally reachable), or to come up with a way to beat the system. The first solution is (I believe) called TURN, and magically solves all connectivity issues at the price of providing a farm of servers with the available bandwidth. The second solution is called NAT traversal, and it's what we'll be exploring next.
Earlier, I described the process of an external server (say google.com) initiating a connection to 128.120.196.204. I said that, without the router having specific rules to understand which computer to forward google's connection request to, the router would simply drop the connection. This was a generalized scenario, and is not accurate because there are different types of NATs. (Note: A router is the actual physical device that you can drop on the floor. NAT (Network Address Translation) is a software process programmed into the router which helps save IPv4 addresses like trees). So, depending on which NAT the router employs, connection scenarios vary. A router may even combine NAT processes.
There are four types of NATs with standardized behavior: Full-Cone NATs, Restricted-Cone NATs, Port-Restricted-Cone NATs, and Symmetric NATs. Aside from these types, there can be other types of NATs with non-standardized behavior, but it's rarer.
Note: I'm not really too familiar with NATs...it seems like there are many ways of looking at routers, and information on the internet is very spread out on this topic. Classifying NATs by full, restricted, and port-restricted cones has been somewhat deprecated, says Wikipedia? There's something called static and dynamic NATs...just a bunch of various concepts that I can't reconcile together. Nevertheless, the following model worked for my own application. You can find out more about NATs by reading the links below and above and throughout this post. I can't post more about them because I don't really understand much about them.
Hoping for some network gurus to correct/add input, so that we can all learn more about this mysterious process.
To answer your question about gathering the external IP and Port of each client:
The headers of all UDP packets are structured the same with one source IP and one source port. UDP packet headers do not contain an "internal" source IP and an "external" source IP. UDP packet headers only contain one source IP. If you want to get an "internal" and "external" source IP, you need to actually send the internal source IP as part of your payload. But it doesn't sound like you need an internal source IP and port. It sounds like you only need an external IP and port, as your question stated. Which means that your solution it to simply read the source IP and port off the packet like the fields they are.
Two scenarios below (they don't really explain anything else):
LAN Communication
Computer A has a LAN IP of 192.168.1.101. Computer B has a LAN IP of 192.168.1.102. Computer A sends a packet, from port 3000, to Computer B at port 6000. The source IP on the UDP packet will be 192.168.1.101. And that will be the only IP. "External" has no context here, because the network is purely a local area network. In this example, a wide area network (like the Internet) doesn't exist. About ports though, because I'm unsure about NATs, I'm not sure if the port inscribed on the packet will be 3000. The NAT device may re-write the packet's port from 3000 to something random like 49826. Either way, you should use whatever port inscribed on the packet to reply - it's what you're supposed to use to reply. So in this example of LAN communication, you need send only one IP - the LAN IP, because that's all that matters. You don't have to worry about the port - the router takes care of that for you. When you receive the packet, you gather the only IP and port simply by reading it off the packet.
WAN Communication
Computer A has a LAN IP, again, of 192.168.1.101. Computer B has a LAN IP, again, of 192.168.1.102. Both Computer A and Computer B will share a WAN IP of 128.120.196.204. Server S is a server, a globally reachable computer on, let's say, an Amazon EC2 server, with a WAN IP of 1.1.1.1. Server S may have a LAN IP, but it's irrelevant. Computer B is irrelevant too.
Computer A sends a packet, from port 3000, to Server S. On the way out the router, the packet's source LAN IP from Computer A gets re-written to the WAN IP of the router. The router also re-writes the source port of 300 to 32981. What does Server S see, in terms of the external IP and port? Server S sees 128.120.196.204 as the IP, not 192.168.1.101, and Server S sees 32981 as the port, not 3000. Although these aren't the original IP and ports Computer A used to send the packet, these are the correct IPs and ports to reply to. When you receive the packet, you can only know the WAN IP and rewritten port. If that's what you want (you were asking for just the external IP and port), then you're set. Otherwise, if you also wanted the internal IP of the sender, you would need to have transmitted that as normal data separate from your header.
Code:
As stated above (below To answer your question about gathering the external IP), to gather the External IP and Port of each client, you simply read them off the packet. Each datagram sent always has the source IP and source port of the sender; you don't even need a fancy custom protocol because these two fields are always included - every single UDP packet must, by definition, have these two fields.
// Java language
// Buffer for receiving incoming data
byte[] inboundDatagramBuffer = new byte[1024];
DatagramPacket inboundDatagram = new DatagramPacket(inboundDatagramBuffer, inboundDatagramBuffer.length);
// Source IP address
InetAddress sourceAddress = inboundDatagram.getAddress();
// Source port
int sourcePort = inboundDatagram.getPort();
// Actually receive the datagram
socket.receive(inboundDatagram);
Because getAddress() and getPort() can return either the destination or source port, depending on what you set it to be, on the client (sending) machine, call setAddress() and setPort() to the server (receiving) machine, and on the server (receiving) machine, call setAddress() and setPort() back to the client (sending) machine. There must be a way to do this in receive(). Please elaborate if this (getAddress() and getPort() don't return the source IP and port you expect) is your actual roadblock. This is assuming the server to be a "standard" UDP server (it's not a STUN server).
Further Update:
I read your update about "how to use STUN to take the IP and port from one client and give it to the other"? A STUN server isn't designed to exchange endpoints or perform NAT traversal. A STUN server is designed to tell you your public IP, public port, and type of NAT device (whether it's a Full-Cone NAT, Restricted-Cone NAT, or Port-Restricted Cone NAT). I'd call the middleman server responsible for exchanging endpoints and performing the actual NAT traversal the "introducer". In my personal project, I don't actually need to use STUN to perform NAT traversing. My "introducer" (the middleman server that introduces clients A and B) is a standard server listening for UDP datagrams. As both clients A and B register themselves with the introducer, the introducer reads off their public IP and port and private IP (in case they're on a LAN). The public IP is read off the datagram header, like for all standard UDP datagrams. The private IP is written as part of the datagram payload, and the introducer just reads it as part of the payload. So, about STUN's usefulness, you don't need to rely on STUN to get the public IP and public port of each of your clients - any connected socket can tell you this. I'd say STUN is useful only for determining what type of NAT device your client is under so that you know whether to perform NAT traversal (if the NAT device type is Full-Cone, Restricted, or Port-Restricted), or to perform all-out TURN traffic proxying (if the NAT device type is Symmetric).
Please elaborate on your roadblock: if you want advice on best practices for designing an application messaging protocol, and advice on reading the fields off received messages in an orderly and systematic fashion (based on the comment you posted below), could you share your current method?

Your question is really broad - I can't offer an example but the following links might help (specs, libraries, samples etc.):
http://java.net/projects/stun
http://code.google.com/p/ice4j/
http://jstun.javawi.de/ and http://www.t-king.de/blog/index.php?category=JSTUN (BTW: JSTUN is no longer maintained from what I gather)
https://www.rfc-editor.org/rfc/rfc5766
http://www.codeproject.com/Articles/23481/Stunner-A-STUN-Library-and-Client (is in C++ but might illustrate how to make it work)

STUN basically works as follows: your client behind the firewall connects to a STUN server outside the firewall. The STUN server inspects the packet received from the client and sends the client a response back containing the clients IP and port as they appear to the STUN server.
This is how the client behind the firewall discovers its own external IP and port. As far as I know a STUN server does not normally pass address information from one client to another.
Typically STUN is used to set up media streams through firewalls, when the firewall is already open to signalling traffic - e.g. in VoIP: client contacts a STUN server to discover its own external IP and port for UDP traffic, it then sends its signalling request (SIP INVITE or whatever) to the other client on a well-known open port - including its external UDP address info in the payload (SDP or whatever). So generally one client needs to be reachable over an open port for signalling for peer-to-peer communication.

Your issue is not a Java related. If you know how to open a UDP connection, that is enough. Read the content of the following link. Don't be scared by the title, it also covers UDP. The rest is just Java coding.
P.S.: In your scenario, there is a missing step. Both A and B must have an open connection to S, because S needs to tell B that A is trying to reach it. If B does not have an open connection to S, there is no way A and B can start communicating together.
UPDATE
The answer made by Jason contains errors and wild speculation about NAT traversal. One should read the work done by Saikat Guha (mpi-sws.org/~francis/imc05-tcpnat.pdf) to really understand this matter. The cone classification of Wikipedia is completely obsolete and misleading.

Related

In accordance to example in this link:
http://www.javatpoint.com/socket-programming
As i understand port no :6666 is an imaginary or raw port used to illustrate socket programming. I want to know how the PC knows that it has server with port 6666 after running both myClient.java and myServer.java.
Also I want to know while doing Real socket programming the myServer.java needs to be placed in real server location if not then where ? just want to understand where to initialize a serverSocket class object! In server side or in client side ?
Also how Operating system or PC(in general ) search for available ports ?

Your linked document is broken, nevertheless, let me explain a bit how the network sockets work.
A computer has multiple network interfaces. If you're running window you can check them by running ipconfig /all, on linux/osx with ifconfig. You'll see that you have a loopback interface with IP address 127.0.0.1. Also, by convenience it was decided to add a "name" to this loopback interface, and it'd be localhost. You can verify this in /etc/hosts file where a mapping between 127.0.0.1 and localhost exists.
Saying that, a computer can find a route to localhost on himself using the system kernel. This loopback interface is virtual, implemented in the operating system so no packets will go through your Ethernet interface or wifi card.
TCP and UDP are protocols used on top of IP to send data. TCP establishes a connection via the 3-way handshake and packet reception is acknowledged by the server. UDP is non-connection oriented, so a client will send packets to the ports and no acknowledges are sent. That's just a huge summary.
When you want to listen on a port, your application needs to actually tell it to the operating system and when the networking component of the OS receives some packets with the TCP.dst value equal to 6666 (in your case) it will send the payload to your application. The OS is responsible for acknowledging the packets and all the underlaying communication which is transparent from you.
As you might guess, the operating system can only bind the same port port to a single application. That's why if you start twice a web server, the second execution will fail.
You can check which ports are listening with netstat -l on a linux machine.

For a small java game I made, I would like to be able to play it with two computers in the same (home) network. I think I will use RMI and am now trying with computer on ipaddress 192.168.2.3.
I know I can search for a registry on this ipaddress on my other computer at 192.168.2.6, but I would like to show a list of all ipaddresses in the network my computer is connected with. Prefferably only if they actually host a game.
Now I tried some questions here on stackoverflow:
How to enumerate IP addresses of all enabled NIC cards from Java?
How can I find all locally bound IP addresses in Java?
How to get a list of IP connected in same network (subnet) using Java
Why does InetAddress.isReachable return false, when I can ping the IP address?
,but I don't think I need all my computer network interfaces and InetAddress.isReachable() always seem to result to false (even though I can ping via cmd and I have firewall turned off) and calling a commandline
"ping -n 1 192.168.2.i" for all i, where 0<=i<=255,
always exits normal, so it results always in true.
What is the best way to get a list of ipaddresses of computers in the same network as the computer the JVM runs on?

With the linked answers, you should be able to filter the available interfaces down to a few possible options (i.e. interfaces that are up, no loopback, have an IPv4 address, etc.).
To discover game hosts, you can do something like the following.
Let the game hosts listen for UDP broadcasts on a specific port.
Let the clients send out a UDP broadcast to the broadcast address of each of the remaining interfaces from above. The broadcast address can be determined by getBroadcast() in class InterfaceAddress.
The host replies, to let the client know it is waiting. When using UDP, the hosts IP is in the received DatagramPacket. When using TCP, the hosts IP can be determined from the Socket.
Then the client can use the address of the host to establish a direct connection and/or set up RMI.
Edit: I found this blog post, which includes code that does more or less what I described.

I am trying to create a java chat application for my networking class. As of right now I am stuck trying to connect to someone behind a different router. The way I have my project right now is I have a client program and a server program. The client programs first logs into the server program which logs their IP and port in a database and then the server gives them back the list of their friends with their IPs and ports. Then the client closes down the connection to the server and tries to connect to another client using the information the server sent back. So far my program only works connecting to the server and getting the friends IP and port but when I use those values to connect to the other client I cant connect.
socket = new Socket();
socket.setReuseAddress(true);
socket.setKeepAlive(true);
socket.setSoLinger(true, 10);
socket.bind(new InetSocketAddress(Port));
socket.connect(new InetSocketAddress(host, SERVER_PORT));
reusePort = socket.getLocalPort();
Above is a snippet of java code used to connect to the server then below is what i do on the client side.
ss = new ServerSocket(reusePort);
So now technically I am listening on the same port I used to connect to the server with which is logged in and is retrievable to another client and is in the NAT table with my ip and port. I am not sure what I am missing or if there is some protocol or something that I have to do. I have looked at TCP and UDP hole punching but I am not sure how that is actually accomplished or how to implement it.
Any suggestions would be appreciated.

If you want to send a message you'll need to set up port forwarding on any device that acts as a server (any device which creates a socket server). Port forwarding is done on the Router. The reason you cannot connect to the other client is because they are hidden behind their routers firewall. Their address to the rest of the world is actually the address of the router, not of their physical computer. On their local network they have a different address then what the rest of the world sees, and the router figures out what messages from the outside world need to be sent to the client based on an address translation table.
Given your architecture, this would mean that all clients need to have their routers doing port forwarding, which is of course unfeasible (imagine gtalk or aim requiring users to do port forwarding).
The more common architecture is to have the Server do the work of rebroadcasting messages to the connected clients and maintain tables to lookup whose talking with who. This way there is a single server which will need a static ip (or be port forwarded), and all users are simply clients which connect to the server socket and read messages from it.
For actual code describing the second architecture please see http://pirate.shu.edu/~wachsmut/Teaching/CSAS2214/Virtual/Lectures/chat-client-server.html. Then the machine which is running the server code either needs a static ip or if it is behind a router needs traffic from the port it is listening on to be forwarded.
So on the server code you will bind to the ip assigned from your router (something like 192.168.1.2 at some port say 5000). Then go to your routers configuration page (it may be 192.168.1.1 see http://www.wikihow.com/Port-Forward/Open-Ports-on-a-Linksys-Router), and forward port 5000 to the address 192.168.1.2.

The Interactive Connectivity Establishment (ICE) protocol combines various NAT traversal utilities such as the STUN and TURN protocols in order to offer a powerful mechanism that allows Offer/Answer based protocols such as SIP and XMPP to traverse NATs.
This project provides a Java implementation of the ICE protocol that would be usable by both SIP and XMPP applications. The project also provides features such as socket sharing and support for Pseudo TCP.
ice4j is maintained by the Jitsi community.
ice4j

I have spent some time learning about socket programming in Java and have managed to make a couple of simple apps that utilize sockets. (instant messenger, tic-tac-toe, basic things) For my programs I used a client-server relationship with ServerSocket and Socket classes. So far I have been testing all my games on the same machine, meaning the client and the server both run on the same machine and the socket ip I am using is 127.0.0.1. Now I want to make a LAN game using the same logic. One computer will be the server and another will be the client.
The thing I wanted to ask, and pardon me if this is a stupid question, I am not really educated about networks and whatnot, but under what conditions can I establish a socket connection between two machines. What I mean is, I run my socket server on one computer and I want the socket on another computer to connect using the first computer's ip. Say, for example my ip is "192.1.1.4" I want to be able to connect to that computer. Is it possible to establish a connection like this between just any two computers in the world? I know "lan" stands for "local area network" but I am quite ignorant on it beyond that. Sorry it it is a dumb question and I can clarify it if someone needs me to.
Basically, what criteria must be met on two machines for me to be able to establish a socket connection between them using a Java program?

You can make a TCP/IP connection between:
Two machines in the same LAN (private IP)
Two machines with public IP (internet)
A machine in a LAN and a machine with public IP provided that the connection is openned from the LAN to the public IP
You can't open a direct TCP/IP connection to a machine inside a LAN from outside the LAN, unless the gateway is configured to redirect the connections to a specific port to that machine.

On an internal network you do just what you said, client connects to server using server's ip address or hostname on the given port.
over the internet can be tricky because of firewalls and NAT. For example, your computer's ip address on the home network is probably somewhere along the lines of "192.168.0.xxx" - but if you go to: http://www.whatismyipaddress.com you'll see that your internet facing ip address is completely different. What you'll see is basically your router's IP address on the internet (WAN).
So basically, the server will have to setup port forwarding on their router for your game's port to his computer. Then he will have to provide the clients with his internet facing ip address for connection.

The main criterion for establishing a connection - ignoring a multitude of possible factors such as firewall configuration, etc. - is that the two machines are simply on the same network. You may be aware that an IP address starting with 192.168.. always refers to a computer on a local network, which is the situation you are asking about, so if you have two computers connected on a local network (e.g. via a router), and you know the IP address of each machine, then it really is as simple as that - you connect in the same manner you have been using up until now. In fact, the same applies on the internet - even if you have two machines set up on different sides of the world and you know their IP addresses (again, ignoring potentially more significant firewall issues), the process is exactly the same. This is precisely the reason that the internet has proved so scalable, as the process of locating a machine with a particular IP address is handled by the lower layers of the network stack.
tl;dr: It's the same as what you are doing already, just with the appropriate IP addresses.

you can use a public IP address and you don't have to change anything about your own address, this is handled by your router that connects to the internet. What you do have to consider is your firewall settings. The ports you are using in your socket should be added to your 'allowed' list. Normally "established" traffic is allowed by default. This means that you need to allow your incoming ports # serverside to go to your server.

I am working on a project for androids/computers have p2p talking, and we are experimenting with hole-punching in order to get through the wifi's firewall. However the wifi needs to UDP packet needs to look like it is coming from the same machine that the initial Packet was sent to. Using netcat (and choosing which port to send from) from the same machine we can send information back into the computer. However if we are coming from a different device we need to appear as though we have the same IP address, and port. I was wondering if you can easily specify the Sending IP and Port of the packet? I was thinking of using DatagramPacket, but didn't find any methods that would work.

You can't do that from within Java. You can do it with Java plus one of the several Jpcaps in existence, but be aware it's a JNI library with all the risks that entails.