I am trying to put together a basic Java proof-of-concept to have a server (which will most likely just be a Windows or Linux laptop) that can send messages to a number of WIFI-connected android devices on the same subnet (initially 20-30, but it could be quite a number in the future - 2 or 300? - not sure, but I need to allow for the possibility). The app will send a message to all devices (the same message initially - but I may allow for special cases somewhere down the track). Each device can then reply, and I will want to check which ones replied and store the replies.
I tried some test code using UDP (with some help from a tutorial on nakov.com), but it was very unreliable and the phone often locked up while trying to receive a packet - although it worked perfectly sometimes. My client code for the android device is shown below for reference (note, this code just sends a request for a message, and then receives and displays it - I hadn't got around to writing it the way I wanted it since it didn't seem reliable enough to use anyway).
So, I think I probably need to use TCP for reliability (although I am open to suggestions). Would 2-300 individual TCP concurrent connections be ok? And its likely that all the replies would come in to the server at once (or over a 10-15 second period), would this flood it in a way it couldn't handle? Is it likely to be ok for only 20-30 devices?
I know this is a kind of out-there question, but I'm hoping someone with more understanding of TCP than I have can shed some light and give me some pointers.
Cheers
Steve
UDP Client App:
package nz.co.et.quoteclient;
import java.io.IOException;
import java.net.DatagramSocket;
import java.net.DatagramPacket;
import java.net.InetAddress;
import java.net.SocketException;
import java.net.UnknownHostException;
import android.app.Activity;
import android.content.Context;
import android.net.wifi.WifiManager;
import android.os.Bundle;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.Button;
import android.widget.TextView;
import android.widget.Toast;
public class QuoteClient extends Activity {
/** Called when the activity is first created. */
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
Button button = (Button) findViewById(R.id.Button01);
button.setOnClickListener(new OnClickListener() {
public void onClick(View v) {
WifiManager wifiManager = (WifiManager) getSystemService(Context.WIFI_SERVICE);
if (wifiManager.isWifiEnabled()) {
TextView errorText = (TextView) findViewById(R.id.TextView01);
int serverPort = 4445;
InetAddress address = null;
try {
address = InetAddress.getByName("192.168.1.4");
} catch (UnknownHostException e) {
errorText.setText("Failed at setting server IP Address");
e.printStackTrace();
}
byte[] buf = new byte[256];
DatagramPacket packet = new DatagramPacket(buf, buf.length, address, serverPort);
DatagramSocket socket = null;
try {
socket = new DatagramSocket();
} catch (SocketException e) {
errorText.setText("Failed at creating socket");
e.printStackTrace();
}
try {
socket.send(packet);
} catch (IOException e) {
errorText.setText("Failed at sending request packet");
e.printStackTrace();
}
// get response
packet = new DatagramPacket(buf, buf.length);
try {
socket.receive(packet);
} catch (IOException e) {
errorText.setText("Failed at receiving quote packet from server");
e.printStackTrace();
}
// display response
String message = new String(packet.getData());
String data = "";
for (int i = 0; i < message.length(); i++) {
int tmp = (int) message.charAt(i);
data = (tmp <= 0 || tmp >= 256) ? data += "" : data + message.charAt(i);
}
Toast.makeText(getApplicationContext(), data, Toast.LENGTH_LONG).show();
// errorText.setText(data);
}
else {
Toast.makeText(getApplicationContext(), "WIFI not enabled", Toast.LENGTH_LONG).show();
}
}
});
}
}
TCP is certainly going to be more reliable than UDP, since UDP doesn't guarantee packet delivery (which is probably why you application is hanging on the receive).
2-300 connections should be managable, but it's going to depend on things like your hardware capabilities and the amount of data that's being processed / sent back and forth.
You're also going to have to make a few decisions around things like:
What language are you writing your server in (java?)?
Do you maintain a persistent connection from the client to the server, or recreate it each time?
Who is responsible for establishing the connection (the client or the server)?
How is the server going to identify which device has received the message, is it tied to the ip address, or is there some kind of handshaking going on to identify the clients.
If a client becomes connected do you need to send all previously missed information?
Does the information need to get to all of the clients at the same time, or can it be staggered?
How does your client app behave if it loses connection, does it retry etc?
And the list goes on...
Depending upon how your program your server, you ought to be able to handle a few thousand concurrent TCP sessions.
If you pick threading, you might not be able to scale beyond 100-or so clients; each thread brings along enough per-thread storage and scheduling overhead that I really wouldn't want to count on it beyond 50 clients.
The new Java NIO framework provides some of the functionality that can be used to scale into hundreds or thousands of simultaneous connections. If the framework uses select(2) under the hood, it ought to be able to do upwards of 1000 simultaneous connections, but if it uses epoll(2) or kqueue or similar constructs, then even beyond 1000 simultaneous connections should be feasible.
What you do with those sessions might dictate far smaller numbers of simultaneous connections: streaming video will be more intensive than sending a nice polite "welcome to our network" page.
Related
My client-server app works with Apache MINA at both, client and server sides. Sending data via UDP works OK, but after a minute server closes the connection (or MINA's way - "session") and stops answering.
The strange part is that the connection is active the whole time. Client is sending data every 1000ms and server answers to it with the same data. I've found a MINA's mechanism to destroying inactive sessions ExpiringMap, it's got a default value for session's time-to-live public static final int DEFAULT_TIME_TO_LIVE = 60; but I haven't found a way how to change it or better, update time-to-live for sessions.
Imho the time-to-live should update automatically with every incoming packet but I couldn't find a thing why isn't it my server doing. Should I say explicitly that I don't want it to destroy the session yet or what?
My code is quite similar to MINA's tutorials:
SERVER
IoAcceptor acceptor = new NioDatagramAcceptor();
try {
acceptor.setHandler( new UDPHandler() );
acceptor.bind( new InetSocketAddress(RelayConfig.getInstance().getUdpPort()) );
acceptor.getSessionConfig().setReadBufferSize( 2048 );
acceptor.getSessionConfig().setIdleTime( IdleStatus.BOTH_IDLE, IDLE_PERIOD );
System.out.println("RELAY ["+RelayConfig.getInstance().getId()+"]: initialized!");
} catch (IOException e) {
System.out.println("RELAY ["+RelayConfig.getInstance().getId()+"]: failed: "+e.getLocalizedMessage());
//e.printStackTrace();
}
CLIENT
NioDatagramConnector connector = new NioDatagramConnector();
connector.getSessionConfig().setUseReadOperation(true);
handler = new UDPHandler();
connector.setHandler(handler);
connector.getSessionConfig().setReadBufferSize(2048);
// try to connect to server!
try {
System.out.println("Connecting to " + relayIP + ":" + port);
ConnectFuture future = connector.connect(new InetSocketAddress(relayIP, port));
future.addListener(new IoFutureListener<IoFuture>() {
public void operationComplete(IoFuture future) {
ConnectFuture connFuture = (ConnectFuture)future;
if( connFuture.isConnected() ){
UDPClient.setSession(future.getSession());
Timer timer = new Timer("MyTimerTask", true);
timer.scheduleAtFixedRate(new MyTimerTask(), 1000, 1000); // My message is written here every 1000ms
} else {
log.error("Not connected...exiting");
}
}
});
future.awaitUninterruptibly();
} catch (RuntimeIoException e) {
System.err.println("Failed to connect.");
e.printStackTrace();
System.exit(1);
} catch (IllegalArgumentException e) {
System.err.println("Failed to connect. Illegal Argument! Terminating program!");
e.printStackTrace();
System.exit(1);
}
For any additional info please write in comments.
EDIT: Unfortunately I don't have access to that server any more, but problem was not solved back then. If there's anybody else who has the same problem and solved it, let us know.
I did some research and found the link below. You may need to explicitly set the disconnect option to false, but there is also another option to reset the timeout option. A timeout of 30000 is 30 seconds, 60000 is 60 seconds, etc... These solutions are from MINA2. It was not clear if you were using that or an older version. From this you should be able to add the call that implements a specific set of options when you open the UDP port.
MINA2 Documentation
My work is developing software for network capable cameras for retail enviroments. One of the peices of software my team is developing is a webserver that retrieves various reports generated in HTML by the camera itself (which has its own embedded webserver) and stored on the camera. Our software will then GET these reports from the camera and store it on a central webserver.
While we are fine plugging in the IPs of the cameras into our software, I am developing a simple Java class that will query the network and locate all cameras on the network.
The problem though is that while it runs just fine on my PC, and my coworker's PC, when we attempt to run it on the actual webserver PC that will host our software... it runs, but says every IP in the subnet is offline / unreachable EXCEPT for the gateway IP.
For example, if I run it from my PC or my coworkers PC when plugged into the closed LAN, I get the following active IPs found along with a flag telling me if its a camera or not.
(gateway is 192.168.0.1, subnet mask is 255.255.255.0, which means full range of 256 devices to be looked for)
IP:/192.168.0.1 Active:true Camera:false
IP:/192.168.0.100 Active:true Camera:true <- this is camera 1
IP:/192.168.0.101 Active:true Camera:true <- this is camera 2
IP:/192.168.0.103 Active:true Camera:false <- my PC
IP:/192.168.0.104 Active:true Camera:false <- this is our webserver
But for some reason, when running the same program from the webserver PC, using the same JRE, I only get the following found
IP:/192.168.0.1 Active:true Camera:false
Now my code, instead of enumerating through each IP in order on the main Thread, instead creates a seperate Thread for each IP to be checked and runs them concurrently (else it would take little over 21 minutes to enumerate through the entire IP range at a timeout of 5000ms / IP). The main Thread then re-runs these IP scan threads every 15 seconds over and over.
I have checked that all the threads are running to completion on all the PCs, no exceptions are being thrown. Even verified that none of the threads are getting stuck. Each Thread takes about 5001 to 5050ms from start to complete, and those Threads that have an active IP finish sooner (>5000ms), so I know that its correctly waiting the full 5000ms in the ipAddr.isReachable(5000) method.
Me and my coworker are stumped at this point while it seems to reach those active IPs fine when run on our PCs, yet getting no response from the webserver PC???
We have ruled out firewall issues, admin access issues, etc.. The only difference is that our webserver is Embedded Win XP, and our PCs are Windows 7.
This has us stumped. Any ideas why?
Below is the code that is running each IP Thread:
public void CheckIP() {
new Thread() {
#Override
public void run() {
try {
isActive = ipAddr.isReachable(5000);
if (isActive) {
if (!isCamera) {
isCamera = new IpHttpManager().GetResponse(ipAddr.toString());
}
} else {
isCamera = false;
}
} catch (Exception e) {
e.printStackTrace();
}
}
}.start();
}
EDIT: Here is the code that builds each IP to check after determining the range based on gateway and subnet...
for(int i=subMin; i<=subMax; i++) {
byte[] ip = new byte[] {(byte)oct[0],(byte)oct[1],(byte)oct[2],(byte)i};
try {
scanners[subCount] = new IpScan(InetAddress.getByAddress(ip));
subCount++;
} catch (UnknownHostException e) {
e.printStackTrace();
}}
Thanks everyone, but I never did figure out or pinpoint why this oddity was happening. Everything I checked for was not the cause, so this question can be closed.
In any case, I ended up working around it completely. Instead of using InetAddress, I just went native and built my own ICMP ping class instead, via JNA, invoking Windows libraries IPHLPAPI.DLL and WSOCK32.DLL. Here is what I used...
public interface InetAddr extends StdCallLibrary {
InetAddr INSTANCE = (InetAddr)
Native.loadLibrary("wsock32.dll", InetAddr.class);
ULONG inet_addr(String cp); //in_addr creator. Creates the in_addr C struct used below
}
public interface IcmpEcho extends StdCallLibrary {
IcmpEcho INSTANCE = (IcmpEcho)
Native.loadLibrary("iphlpapi.dll", IcmpEcho.class);
int IcmpSendEcho(
HANDLE IcmpHandle, //Handle to the ICMP
ULONG DestinationAddress, //Destination address, in the form of an in_addr C Struct defaulted to ULONG
Pointer RequestData, //Pointer to the buffer where my Message to be sent is
short RequestSize, //size of the above buffer. sizeof(Message)
byte[] RequestOptions, //OPTIONAL!! Can set this to NULL
Pointer ReplyBuffer, //Pointer to the buffer where the replied echo is written to
int ReplySize, //size of the above buffer. Normally its set to the sizeof(ICMP_ECHO_REPLY), but arbitrarily set it to 256 bytes
int Timeout); //time, as int, for timeout
HANDLE IcmpCreateFile(); //win32 ICMP Handle creator
boolean IcmpCloseHandle(HANDLE IcmpHandle); //win32 ICMP Handle destroyer
}
And then using those to create the following method...
public void SendReply(String ipAddress) {
final IcmpEcho icmpecho = IcmpEcho.INSTANCE;
final InetAddr inetAddr = InetAddr.INSTANCE;
HANDLE icmpHandle = icmpecho.IcmpCreateFile();
byte[] message = new String("thisIsMyMessage!".toCharArray()).getBytes();
Memory messageData = new Memory(32); //In C/C++ this would be: void *messageData = (void*) malloc(message.length);
messageData.write(0, message, 0, message.length); //but ignored the length and set it to 32 bytes instead for now
Pointer requestData = messageData;
Pointer replyBuffer = new Memory(256);
replyBuffer.clear(256);
// HERE IS THE NATIVE CALL!!
reply = icmpecho.IcmpSendEcho(icmpHandle,
inetAddr.inet_addr(ipAddress),
requestData,
(short) 32,
null,
replyBuffer,
256,
timeout);
// NATIVE CALL DONE, CHECK REPLY!!
icmpecho.IcmpCloseHandle(icmpHandle);
}
public boolean IsReachable () {
return (reply > 0);
}
My guess is that your iteration logic to determine the different ip address is based upon different configuration hence your pc's fetches all addresses but your webserver doesn't.
Try adding debug in the logic where you build up the list of ip adresses to check.
Ok so the program is designed to take in connections, validate them, and resend that validation code. Before anyone get's angry it's just a simple little project and is not designed to be overly complex haha. However for some very strange reason the function is hanging on send.setAddess(packet.getAddress); I know this because I have commented out each individual line of code that deals with the Datagram packet "send" and have found that it "hangs" (or never progresses forward in the method again) on that particular line. Any thoughts? Am I doing something cluelessly wrong? I tried it on a linux server as well to make sure it didn't have anything to do with me and the same crap happened.
public static boolean authorize(String n, DatagramPacket packet) {
DatagramPacket send = new DatagramPacket(new byte[4096], 4096);
try {
System.out.println("in auth");
String[] t1 = n.split("%#");
String name = t1[1];
int k = genKey(name);
clients.put(name, k);
send.setAddress(packet.getAddress());
System.out.println("set add");
send.setPort(packet.getPort());
System.out.println("set port");
send.setData(("l-succeed%#" + Integer.toString(k)).getBytes());
System.out.println("set data");
main.dispathcer(send);
System.out.println("called send");
return true;
} catch(Exception e) {
send.setData("l-failed".getBytes());
main.dispathcer(send);
return false;
}
}
EDIT: it took 6 minutes before the authorization token was received by the client. So obviously the setAddress() works but is taking far too long...
It's possible that the process is hanging because there's an issue doing DNS resolution on the address for packet when you call .getAddress(). A few DNS calls are made in order to create the InetAddress object. On these machines, are you able to do a reverse DNS lookup on the IP that the packet packet came from? Try setting an entry for this IP in your /etc/hosts file.
I have given the received message length as 1000000 but still message gets truncated completed code is below
import java.io.FileInputStream;
import javax.jms.JMSException;
import javax.jms.Session;
import com.ibm.jms.JMSMessage;
import com.ibm.jms.JMSTextMessage;
import com.ibm.mq.jms.JMSC;
import com.ibm.mq.jms.MQQueue;
import com.ibm.mq.jms.MQQueueConnection;
import com.ibm.mq.jms.MQQueueConnectionFactory;
import com.ibm.mq.jms.MQQueueReceiver;
import com.ibm.mq.jms.MQQueueSender;
import com.ibm.mq.jms.MQQueueSession;
public class SimplePTP {
public static void main(String[] args) {
try {
MQQueueConnectionFactory cf = new MQQueueConnectionFactory();
String request= null;
// Config
cf.setHostName("CTMQ9000");
cf.setPort(1414);
cf.setTransportType(JMSC.MQJMS_TP_CLIENT_MQ_TCPIP);
cf.setQueueManager("CTMQTST01");
cf.setChannel("SYSTEM.ADMIN.SVRCONN");
MQQueueConnection connection = (MQQueueConnection) cf.createQueueConnection();
MQQueueSession session = (MQQueueSession) connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
MQQueue queue = (MQQueue) session.createQueue("CONTPLAT.CPS.DELIVERYPREP.REQUEST.TEST");
MQQueueSender sender = (MQQueueSender) session.createSender(queue);
MQQueue queue1 = (MQQueue) session.createQueue("CONTPLAT.CPS.DELIVERYPREP.RESPONSE.TEST");
MQQueueReceiver receiver = (MQQueueReceiver) session.createReceiver(queue1);
String request ="sdfHelp Me Name name for Photo Studio!I'm opening a portrait studio in a my town and am stuck on what to name it. I will be photographing (Portrait, wedding Photography)) children and families both at the studio and on location.sdffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffsdf"
long uniqueNumber = System.currentTimeMillis() % 1000;
JMSTextMessage message = (JMSTextMessage) session.createTextMessage(request);
// Start the connection
connection.start();
sender.send(message);
System.out.println("Sent message:\\n" + message +"lol");
Thread.sleep(2000);
JMSMessage receivedMessage = (JMSMessage) receiver.receive(10000000);
System.out.println("\\nReceived message:\\n" + receivedMessage);
sender.close();
receiver.close();
session.close();
connection.close();
System.out.println("\\nSUCCESS\\n");
}
catch (JMSException jmsex) {
System.out.println(jmsex);
System.out.println("\\nFAILURE\\n");
}
catch (Exception ex) {
System.out.println(ex);
System.out.println("\\nFAILURE\\n");
}
}
}
If the length of the message exceeds some limit i don't know how much , but its getting truncated , any way to increase or workaround to display complete message.
As Germann pointed out the parameter that you pass to receive method is not the message size, it is actually the wait time in milli seconds. Meaning how long the call must wait for a message to arrive. For example if you have set the wait time as 1000, then the receive call will wait for 1 second for message to arrive. If a message arrives before 1 second, the call will return immediately and give the message to application. If a message does not arrive even after a second, then the call will return with a timeout and no message is given to application. In MQ terms you will see a 2033 reason code.
How are you determining that the message is truncated? Are you getting a MQRC_TRUNCATED_MESSAGE_FAILED exception? This exception will be thrown if the application supplied buffer is not enough to fill the incoming message. MQ JMS is not expected to throw a MQRC_TRUNCATED_MESSAGE_FAILED exception as it internally handles the buffer size required and returns message to application.
I am guessing that the issue could be because you are printing a JMSMessage whereas the sent message is a JMSTextMessage. JMSMessage.ToString may not be printing the whole message.
Where do you set the length to 1000000?
If you read MQ docs you will find that receiver.receive(10000000); does NOT set the message size.
This is in Java, but I can always revert to C via JNI if needed.
I have a system with two NICs, each connected to a distinct subnet. I want to use multicast (in particular, SDP) to discover other hosts on both networks.
One network is easy: create a MulticastSocket on the specified port, joinGroup it, and I get packets. Simplicity.
Two networks: so far impossible. I've tried:
1) creating two sockets, binding to the same port and using setInterface() or setNetworkInterface() to "connect" to the right interface. No luck, even after various permutations of setReuseAddress().
2) create a single socket, and then attempt to join twice, with two calls to joinGroup(SocketAddress mcastaddr, NetworkInterface netIf). The second join call fails.
Solutions outside of Java would be great. In particular, if I could set up multicast routes that would effectively 'combine' the two interfaces (I could then look at each packet to determine which network) that would be fine. As I mentioned before, any amount of native code is usable in this environment (Linux, with the Apache "luni" java infrastructure).
Thanks!
Coincidentally, I was working on a similar problem recently.
Here's some Java code which does what you want -- it picks up SDP packets on several interfaces. joinGroup is used to "attach" to the specified interfaces.
/**
* Demonstrate multi-homed multicast listening
*
* usage: java Multihome eth0 eth1 lo <etc>
*/
import java.io.IOException;
import java.net.DatagramPacket;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.MulticastSocket;
import java.net.NetworkInterface;
import java.net.SocketException;
import java.util.Enumeration;
import java.util.HashSet;
import java.util.Set;
public class Multihome {
// SDP constants
public static final String MULTICAST_ADDRESS = "239.255.255.250";
public static final int MULTICAST_PORT = 1900;
// args: each arg is the name of an interface.
public void doMain(Set<String> args)
throws Exception
{
InetSocketAddress socketAddress =
new InetSocketAddress(MULTICAST_ADDRESS, MULTICAST_PORT);
MulticastSocket socket = new MulticastSocket(MULTICAST_PORT);
Enumeration<NetworkInterface> ifs =
NetworkInterface.getNetworkInterfaces();
while (ifs.hasMoreElements()) {
NetworkInterface xface = ifs.nextElement();
Enumeration<InetAddress> addrs = xface.getInetAddresses();
String name = xface.getName();
while (addrs.hasMoreElements()) {
InetAddress addr = addrs.nextElement();
System.out.println(name + " ... has addr " + addr);
}
if (args.contains(name)) {
System.out.println("Adding " + name + " to our interface set");
socket.joinGroup(socketAddress, xface);
}
}
byte[] buffer = new byte[1500];
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
while (true) {
try {
packet.setData(buffer, 0, buffer.length);
socket.receive(packet);
System.out.println("Received pkt from " + packet.getAddress() +
" of length " + packet.getLength());
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
public static void main(String[] args)
throws Exception
{
Set<String> argSet = new HashSet<String>();
Multihome multi = new Multihome();
for (String arg : args) {
argSet.add(arg);
}
multi.doMain(argSet);
}
}
I'd recommend using JGroups, which abstracts everything you're trying to do, if I understand your needs correctly. it's an elegant and well-made framework for multicast (and multicast-like semantics, emulated when necessary).
I have no reasonable setup to try this here, but receiving multicast messages should not require the MulticastSocket to be bound to the port number from the multicast address and setNetworkInterface is used to set the interface used for outbound messages.
What I would try to create two different MulticastSockets (on any free port) and then use joinGroup(SocketAddress mcastaddr, NetworkInterface netIf) on each of them using the same multicast address, but different network interfaces.
Have you considered using ZeroConf for this?
The jmdns project has a pure java implementation which should work very well.