I'm building a real-time GPS tracking system, which will receive GPS data sent from a couple of Arduino devices using UDP. I have this code so far:
PreparedStatement stmt ...
DatagramSocket serverSocket = new DatagramSocket(9876);
byte[] receiveData = new byte[1024];
while(true){
DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);
serverSocket.receive(receivePacket);
String received = new String( receivePacket.getData());
System.out.println("RECEIVED: " + received);
stmt.set...
stmt.execute();
}
1 - Anyone with more knowledge could tell me if there's a better way of doing this? I really don't know how the JVM handles this, but I don't like that infinite loop.
2 - Lets say that I have 50 Arduinos sending data. I need to use threads or something like this?
3 - It's best to use a thread per "connection" (UDP is connectionless) like an answer below or use frameworks/libs like Apache Mina or Netty?
There is no problem using an infinite loop in this case. Calling receive waits until a new datagram is delivered:
This method blocks until a datagram is received. T...
So no CPU power is wasted here, it simply waits until new data is available.
If you have many clients or if processing the packet isn't completely trivial, you should start a new thread for processing each one, so that the main thread that receives the datagrams doesn't get blocked. Probably the best approach is to use thread pools that will create threads for you, and at the same time prevent creating too many threads if your application is overloaded by requests.
I'd proceed as follows:
Create a dedicated thread for receiving the datagrams. It could also create a thread pool for dispatching processing the requests. Something like:
int maxNumberOfThreads = ...; // your choice
int bufSize = ...; // your choice
ExecutorService exec = Executors.newFixedThreadPool(maxNumberOfThreads);
DatagramSocket serverSocket = new DatagramSocket(9876);
while (true) {
// we need to create a new buffer every time because
// multiple threads will be working with the data
DatagramPacket receivePacket =
new DatagramPacket(new byte[bufSize], bufSize);
serverSocket.receive(receivePacket);
exec.submit(new YourTask(receivePacket));
}
Create class YourTask that processes the datagrams:
// We don't use return values for anything here, so
// we just use Object.
public class YourTask extends Callable<Object> {
private DatagramPacket received;
public YourTask(DatagramPacket received) {
this.received = received;
}
public Object call() {
// do your processing here
System.out.println("RECEIVED from " +
received.getAddress() +
": " + new String(received.getData(),
0, received.getLength()));
return null;
}
}
I recommend you look at Apache MINA (http://mina.apache.org/), a great framework for network applications. With MINA you don't need to implement a loop or worry about threading.
The actual problem I see in your question is the "Real Time" term. What do you mean with that? Do you need a highly predictable (in terms of timing) application, is it safety/mission critical? If so there may be problem in using Java, as it is for many reasons (i.e. garbage collector, etc.) not realtime. There are however some realtime JVM as http://www.atego.com/products/aonix-perc/.
I do like Java but I guess in this case, if you really need a RT system, C++ would be a better choice.
Related
I have a problem. I want to create a network connection so that I can make a multiplayer game. I know and understand sockets.
My problem is that if I press two keys on the keyboard at the same time it writes to the socket at the same time which causes an error. What I then did was create multiple sockets for one client to writ to and only writes to a socket that isn't busy. By for some reason seems to overload it or something. Any ideas for simultaneously sending messages to server and vice versa.
Is your code doing like this in server?
while (true) {
try {
socket = serverSocket.accept();
} catch (IOException e) {
System.out.println("I/O error: " + e);
}
// new threa for a client
new MyThread(socket).start();
}
then, inside MyThread#run
inp = socket.getInputStream();
brinp = new BufferedReader(new InputStreamReader(inp));
out = new DataOutputStream(socket.getOutputStream());
line = brinp.readLine(); // read the data.
out.writeBytes(line + "\n\r"); //write back to client
out.flush(); // flush socket
Java NIO would faster and efficient. Please refer open source netty servers as well.
I have used a queueing method and has fixed the problem.
So after looking around for a suitable library, I've decided to make my own and here is part of it.
try {
DatagramSocket serverSocket = new DatagramSocket(client_port);
byte[] receiveData = new byte[1024];
byte[] finalData;
while (true) {
DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);
erverSocket.receive(receivePacket);
if (this.address == receivePacket.getAddress() && this.server_port == receivePacket.getPort()) {
** handler.onMessage(receivePacket.getData()); **
}
}
} catch (IOException ex) {
Logger.getLogger(PacketHandler.class.getName()).log(Level.SEVERE, null, ex);
This code is obviously ran asynchronously.
The receiveData is set to 1024, which is terribly wasteful. It also means that anything bigger than 1024 gets split into two or more "Events" fired by the library.
I was wondering how it is possible to make a buffer, as I've gone completely blank. Essentially, you'd have to count the number of bytes somehow and push it into a new array.
So I'm basically asking for an efficient method, which doesn't have to include code.
Are you sure that you need to use UDP for it?
Seems like you should change it to TCP impelementation. Do you want to use netty for it ? See here.
1024 bytes it's normal safe value for UDP datagaram, making it bigger can bring issues with router. (See https://stackoverflow.com/a/13089776/3502543).
If you don't want to use netty, you should provide something similar to DelimiterBasedFrameDecoder.
edited.
I have another device & application transmitting data in real-time (every few ms) and on my receiving device, I want to:
1) read/receive this data, and
2) use it to update a UI element (a dynamic graph in this case)
The data sender uses a socket in a background service, using AsyncTasks every few ms to send data. To initialize, it does the following:
echoSocket = new Socket(HOST, PORT);
out = new PrintWriter(echoSocket.getOutputStream(), true);
And to periodically send data it does:
static class sendDataTask extends AsyncTask<Float, Void, Void> {
#Override
protected Void doInBackground(Float... params) {
try {
JSONObject j = new JSONObject();
j.put("x", params[0]);
j.put("y", params[1]);
j.put("z", params[2]);
String jString = j.toString();
out.println(jString);
} catch (Exception e) {
Log.e("sendDataTask", e.toString());
}
return null;
}
}
How should I go about receiving this data in my application? Should I also use a background service with AsyncTasks that try to read from the socket every few ms? How about communicating with the UI thread?
There are many ways to do this. The simplest would be to use blocking reads in the doInBackground method of an AsyncTask and call publishProgress() to forward the new data to the UI thread.
Then implement onProgressUpdate with code (running in the UI thread) that updates the screen.
You should be aware that your read may not receive the entire message that you sent -- you may need to read more data and append it to the input received so far until you have an entire JSON message.
By blocking reads, I mean something like this (in pseudo code):
open a socket connected to the sender
is = socket.getInputStream()
initialize buffer, offset, and length
while the socket is good
bytesRead = is.read(buffer, offset, length)
if(bytesRead <= 0)
bail out you have an error
offset += bytesRead;
length -= bytesRead
if(you have a complete message)
copy the message out of the buffer (or parse the message here into
some other data structure)
publishProgress(the message)
reset buffer offset and length for the next message.
(remember you may have received part of the following message)
end-if
end-while
The copy-out-of-the-buffer is necessary because the onProgressUpdate does not happen immediately so you need to be sure the next message does not overwrite the current one before it gets handled.
Local on Linux. It's about 10 seconds for a 20k message. My guess is my Java is bad and Python is fine.
py client:
def scan(self, msg):
try:
print 'begin scan'
HOST = 'localhost'
PORT = 33000
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT));
s.sendall(msg)
data = s.recv(1024)
s.close()
print 'Received', repr(data)
except Exception, e:
print "error: " + str(e)
Java server:
ServerSocket service = new ServerSocket(33000);
while(true) {
debug("Begin waiting for connection");
//this spins
Socket connection = service.accept();
debug("Connection received from " + connection.getInetAddress().getHostName());
OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream());
BufferedInputStream in = new BufferedInputStream(connection.getInputStream());
ScanResultsHeader results = new ScanResultsHeader();
Scanner scanner = new Scanner();
results = scanner.scan("scannerfake#gmail.com", "123", in);
and
public ScanResultsHeader scan (String userEmail,
String imapRetrievalId,
BufferedInputStream mimeEmail)
throws IOException, FileNotFoundException, MimeException, ScannerException {
//how fast would it be to just slurp up stream?
debug("slurp!");
String slurp = IOUtils.toString(mimeEmail);
debug("slurped " + slurp.length() + " characters");
slurp = slurp.toLowerCase();
debug("lc'ed it");
//...
My guess is I'm juggling the input streams wrong. One catch is the "BufferedInputStream mimeEmail" signature is required by the library API scan is using, so I'll need to get to that form eventually. But I noticed the simple act of slurping up a string takes ludicrously long so I'm already doing something incorrect.
Revising my answer....
If you are reading efficiently, and it appears you are, it will only be taking a lot time because either
You are creating a new connection every time you send a message which can be very expensive.
You are not sending the data as fast as you think.
The message is very large (unlikely but it could be)
There are plenty of examples on how to do this and a good library you can use is IOUtils which makes it simpler.
You should be able to send about 200K/s messages over a single socket in Java.
If you have a sends X bytes protocol using Big Endian you can do this.
DataInputStream dis = new DataInputStream( ...
int len = dis.readInt();
byte[] bytes = new byte[len];
dis.readFully(bytes);
String text = new String(bytes, "UTF-8");
Original problem was that the client isn't sending an end-of-input so the "slurp" operation keeps waiting for more stuff to cross the connection.
Solution was to implement an application-layer protocol to send the size of the message in advance, then stop listening for more message after that many bytes. I would have preferred a standard library -- something like, FiniteInputStream extends BufferedInputStream and takes a size as an argument, but wrote my own.
I'm trying to create a basic multiplayer game for android, using a Java TCP server and Android client. The problem is slow speed when sending TCP packets. When I put Thread.sleep(100) then it works.
server side:
for(int i = 0; i<50; i++) {
client.send("test_" + i);
}
client just received (~3 packet)
test_0
test_1
server with sleep:
for(int i = 0; i<50; i++) {
client.send("test_" + i);
Thread.sleep(100);
}
client received ~45
EDIT: client side:
while (true) {
if (!running)
break;
inFromServer = new BufferedReader(new InputStreamReader(socket.getInputStream(), "UTF-8"), 2 * 1024);
String rawRecervied = inFromServer.readLine();
if (rawRecervied == null) {
close();
break;
}
final String recervied = rawRecervied.substring(2); // for skip utf bom chars
new Thread() {
public void run() {
listener.dataRecervied(recervied);
Log.e("TCP recervied", recervied); // debug
}
}.start();
}
Maybe the key is in the BufferedReader. You're in a loop, and constantly create a BufferedReader to check if something has been sent from the server. Once data is detected, you start processing it, but data keeps coming, and is buffered in the BufferedReader. After processing the initially detected data, you create again a BufferedReader but, what happens with all the data that was already buffered in the BufferedReader created before? Maybe it's lost.
Could you try creating the BufferedReader outside the loop?
If it is a one-way protocol where packet loss is acceptable then use UDP instead of TCP as it is cheaper in terms of network resources. I think this is not your case however. If TCP, then implement a basic flow control where the client acknowledges the received packet with echoing its ID back to the server.
You should also revise your client and server code because this behaviour might be in the way you implemented that client.sent(..). Do you always close and reopen the connection? Or what?