Im writing a JavaFX application that has to comunicate with my Arduino UNO. For this im using the jSerialComm library.
Just for testing puroposes I've upload a very simple sketch to my Arduino that prints to the Serial a "Hello" word every 2 seconds:
void setup() {
//put your setup code here, to run once:
Serial.begin(9600);
}
void loop() {
// put your main code here, to run repeatedly:
delay(2000);
Serial.print("Hello");
}
And in my JavaFX scene im reading the incoming data with this:
public void setDevice(SerialPort device) {
this.device = device;
device.openPort();
device.addDataListener(new SerialPortDataListener() {
#Override
public int getListeningEvents() {
return SerialPort.LISTENING_EVENT_DATA_RECEIVED;
}
#Override
public void serialEvent(SerialPortEvent serialPortEvent) {
if (serialPortEvent.getEventType() == SerialPort.LISTENING_EVENT_DATA_RECEIVED){
byte [] data = serialPortEvent.getReceivedData();
String msg = new String(data);
System.out.println(msg);
}
}
});
}
I can read the data from Arduino, but it comes in a strange way. Like the string is sended in 2 diffent strings. Here is an image of the console output:
Am I doing something wrong? Thank you very much!
You are using jSerialComm in default = NonBlocking mode. So what happens is (as pseudo code steps)
LISTENING_EVENT_DATA_RECEIVED triggered
get the char H
get the char e
Because we are nonBlocking we have to move on in the program
Print what we have so far -> HE
.... do other stuff or check if other stuff has to be processed
LISTENING_EVENT_DATA_RECEIVED triggered
get the char L
get the char L
get the char O
Because we are nonBlocking we have to move on in the program
Print what we have so far -> LLO
.... do other stuff or check if other stuff has to be processed
So you could do 2 things -> change the mode to e.g. blocking while receiving (ensure a proper timeout to prevent deadlocks) or rewrite the function (my preferred way) to check for end terminators in the communication flow and then process the content of the buffer (which should be implemented in a non-blocking way)
Related
I have to write a java program that receives G-Code commands via network and sends them to a 3D printer via serial communication. In principle everything seems to be okay, as long as the printer needs more than 300ms to execute a command. If execution time is shorter than that, it takes too much time for the printer to receive the next command and that results in a delay between command execution (printer nozzle standing still for about 100-200ms). This can become a problem in 3d printing so i have to eliminate that delay.
For comparison: Software like Repetier Host or Cura can send the same commands via seial without any delay between command execution, so it has to be possible somehow.
I use jSerialComm library for serial communication.
This is the Thread that sends commands to the printer:
#Override
public void run() {
if(printer == null) return;
log("Printer Thread started!");
//wait just in case
Main.sleep(3000);
long last = 0;
while(true) {
String cmd = printer.cmdQueue.poll();
if (cmd != null && !cmd.equals("") && !cmd.equals("\n")) {
log(cmd+" last: "+(System.currentTimeMillis()-last)+"ms");
last = System.currentTimeMillis();
send(cmd + "\n", 0);
}
}
}
private void send(String cmd, int timeout) {
printer.serialWrite(cmd);
waitForBuffer(timeout);
}
private void waitForBuffer(int timeout) {
if(!blockForOK(timeout))
log("OK Timeout ("+timeout+"ms)");
}
public boolean blockForOK(int timeoutMillis) {
long millis = System.currentTimeMillis();
while(!printer.bufferAvailable) {
if(timeoutMillis != 0)
if(millis + timeoutMillis < System.currentTimeMillis()) return false;
try {
sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
printer.bufferAvailable = false;
return true;
}
this is printer.serialWrite: ("Inspired" by Arduino Java Lib)
public void serialWrite(String s){
comPort.setComPortTimeouts(SerialPort.TIMEOUT_SCANNER, 0, 500);
try{Thread.sleep(5);} catch(Exception e){}
PrintWriter pout = new PrintWriter(comPort.getOutputStream());
pout.print(s);
pout.flush();
}
printer is an Object of class Printer which implements com.fazecast.jSerialComm.SerialPortDataListener
relevant functions of Printer
#Override
public int getListeningEvents() {
return SerialPort.LISTENING_EVENT_DATA_AVAILABLE;
}
#Override
public void serialEvent(SerialPortEvent serialPortEvent) {
byte[] newData = new byte[comPort.bytesAvailable()];
int numRead = comPort.readBytes(newData, newData.length);
handleData(new String(newData));
}
private void handleData(String line) {
//log("RX: "+line);
if(line.contains("ok")) {
bufferAvailable = true;
}
if(line.contains("T:")) {
printerThread.printer.temperature[0] = Utils.readFloat(line.substring(line.indexOf("T:")+2));
}
if(line.contains("T0:")) {
printerThread.printer.temperature[0] = Utils.readFloat(line.substring(line.indexOf("T0:")+3));
}
if(line.contains("T1:")) {
printerThread.printer.temperature[1] = Utils.readFloat(line.substring(line.indexOf("T1:")+3));
}
if(line.contains("T2:")) {
printerThread.printer.temperature[2] = Utils.readFloat(line.substring(line.indexOf("T2:")+3));
}
}
Printer.bufferAvailable is declared volatile
I also tried blocking functions of jserialcomm in another thread, same result.
Where is my bottleneck? Is there a bottleneck in my code at all or does jserialcomm produce too much overhead?
For those who do not have experience in 3d-printing:
When the printer receives a valid command, it will put that command into an internal buffer to minimize delay. As long as there is free space in the internal buffer it replies with ok. When the buffer is full, the ok is delayed until there is free space again.
So basicly you just have to send a command, wait for the ok, send another one immediately.
#Override
public void serialEvent(SerialPortEvent serialPortEvent) {
byte[] newData = new byte[comPort.bytesAvailable()];
int numRead = comPort.readBytes(newData, newData.length);
handleData(new String(newData));
}
This part is problematic, the event may have been triggered before a full line was read, so potentially only half an ok has been received yet. You need to buffer (over multiple events) and reassamble into messages first before attempting to parse this as full messages.
Worst case, this may have resulted in entirely loosing temperature readings or ok messages as they have been ripped in half.
See the InputStream example and wrap it in a BufferedReader to get access to BufferedReader::readLine(). With the BufferedReader in place, you can that just use that to poll directly in the main thread and process the response synchronously.
try{Thread.sleep(5);} catch(Exception e){}
sleep(1);
You don't want to sleep. Depending on your system environment (and I strongly assume that this isn't running on Windows on x86, but rather Linux on an embedded platform), a sleep can be much longer than anticipated. Up to 30ms or 100ms, depending on the Kernel configuration.
The sleep before write doesn't make much sense in the first place, you know that the serial port is ready to write as you already had received an ok confirming reception of the previously sent command.
The sleep during receive becomes pointless when using the BufferedReader.
comPort.setComPortTimeouts(SerialPort.TIMEOUT_SCANNER, 0, 500);
And this is actually causing your problems. SerialPort.TIMEOUT_SCANNER activates a wait period on read. After receiving the first byte it will wait at least for another 100ms to see if it will become part of a message. So after it has seen the ok it then waits 100ms internally on the OS side before it assumes that this was all there is.
You need SerialPort.TIMEOUT_READ_SEMI_BLOCKING for low latency, but then the problem predicted in the first paragraph will occur unless buffered.
Setting repeatedly also causes yet another problem, because there is a 200ms sleep in Serialport::setComPortTimeouts internally. Set it per serial connection once, no more than that.
Check the manual of the printer (or tell us the model) not sure you actually need to wait for the ok, and therefore you can read/write concurrently. Some of the time there's a hardware flow control handling this stuff for you, with large enough buffers. Try just send the commands without waiting for ok, see what happens.
If you just want to pipe commands from the network to serial port, you can use ready-made solution like socat. For example running the following:
socat TCP-LISTEN:8888,fork,reuseaddr FILE:/dev/ttyUSB0,b115200,raw
would pipe all bytes coming from clients connected to the 8888 port directly to the /dev/ttyUSB0 at baud rate of 115200 (and vice-versa).
I've encountered a specific problem with my implementation and can't find a solution for it.
I have a two-part application. One part is a Java swing GUI. The second part is a C++ application that does all the (time consuming) calculation logic. The two processes communicate (in both directions) with their output and input streams. My problem now is that in one part of the C++ program I have to wait for user input coming from the Java program. However, waiting seems to block.
What works perfectly when I call the program in a shell is:
std::string inputLine1;
std::cin >> inputLine1;
When working with the Java UI, this does not work (naturally), because reading from std::cin is blocking, so when the C++ application waits for input, the Java application can't do anything.
Therefore I made another way of reading from std::cin that should (at least in my mind) work, but I can't make it work. It is:
std::string inputLine1;
while (true)
{
int c = std::cin.peek();
if (c != EOF)
{
std::cin >> inputLine1;
break;
}
std::this_thread::yield();
}
I also tried to replace the line with yield() with
std::this_thread::sleep_for(std::chrono::milliseconds(500));
In my mind, this code should work as following: I peek on std::cin. If something is there, I read it from cin. If nothing is there, I yield and try again later.
I know, yielding is considered to not be a very clean way of working, but I want to keep the communication between those two applications as simple as possible. No third party libraries, no more complex concepts (as Sockets), if possible.
However, this approach doesn't work, it gives the same behaviour as the first approach with just reading in from std::cin. The Java program becomes unresponsive and none of the two applications seem to do anything.
The C++ application works perfectly if called in a shell and if I provide the same input from the keyboard, so the problem shouldn't be there. If I delete all these given code snippets from the C++ application, the Java application is responsive and works - although it doesn't get the input it needs, obviously.
After trying for a long time to implement non-blocking input from cin, I'm pretty sure it's impossible to getting it to work consistently.
My current solution is to put the blocking cin into it's own tiny thread and let it do it's thing.
I've simplified my implementation a little bit for this example, as you need a thread safe storage system of some kind.
#include <iostream>
#include <thread>
#include <mutex>
#include <queue>
// Super simple thread safe storage
std::queue<std::string> Database;
std::mutex Padlock;
void PushLine(std::string Line) {
std::unique_lock<std::mutex> Lock(Padlock); (void)Lock;
Database.push(Line);
}
bool IsLineAvailable(void) {
std::unique_lock<std::mutex> Lock(Padlock); (void)Lock;
return !Database.empty();
}
std::string PopLine(void) {
std::unique_lock<std::mutex> Lock(Padlock); (void)Lock;
std::string Line(std::move(Database.front()));
Database.pop();
return Line;
}
// Main function with "non-blocking" input from cin
int main(int argc, char *argv[]) {
(void)argc;
(void)argv;
std::thread InputThread = std::thread([](){
do {
// Ensure the input is as clean as possible
if (std::cin.rdbuf()->in_avail()) {
std::cin.ignore(std::cin.rdbuf()->in_avail());
}
std::cin.clear();
// Get a line, cin will block here.
std::string Line;
std::getline(std::cin, Line);
// If the line is not empty attempt to store it.
if (!Line.empty()) {
PushLine(Line);
}
} while (1);
});
// Detach from the thread, it will never end.
InputThread.detach();
// A job to do.
unsigned int Counter = 0;
// Run your program.
bool Running = true;
while(Running) {
// Perform a job, in this case counting.
Counter++;
// Check for available input
if (IsLineAvailable()) {
// If there is input available, first get it
std::string Line = PopLine();
// Echo it to the terminal
std::cout << "Command: " << Line << std::endl;
// Perform actions based on the command
if (Line == "quit") {
Running = false;
}
else if (Line == "count") {
std::cout << " Count: " << Counter << std::endl;
}
}
// Sleep for a while
std::this_thread::sleep_for(std::chrono::milliseconds(100));
}
// Done.
return 0;
}
It only happens in Androids, not PCs and my server is made using C++
Lets say I made a function for sending packet consists of 3 int and one char
void SendTestPacket(int num1, int num2, int num3) {
System.out.println("Function called");
if(true) {
clientOut.writeInt(1); // clientOut is a DataOutputStream
clientOut.writeInt(2);
clientOut.writeInt(3);
clientOut.writeChar('\n');
System.out.println("Data sent");
}
System.out.println("Function fully executed");
}
I tried using my android client to send one packet on every finger pan function called which has been called more than 1000~2000 times in 5 secs when dragging finger around the screen.
Somehow, during the process of dragging my finger, the function stop executing properly especially starting from the first clientOut.writeInt(int); until the end of if section. It's like 'break'or 'continue' statement has been called.
In the early part, I always got these kind of output
Function called
Data sent
Function fully executed
After a few moment, it became these kind of output every loop
Function called
Function fully executed
If I tested it on pc java client, it will never occurs no matter how I do it
Sorry I forgot to mention it, its on Wirelesss Lan connection.
Edit
Actual Function Code
void SendTestPacket(int packettype, int num1, int num2) {
System.out.println("Function called");
try {
clientOut.writeInt(packettype); // clientOut is a DataOutputStream
clientOut.writeInt(num1);
clientOut.writeInt(num2);
clientOut.writeChar('\n');
System.out.println("Data sent");
} catch (IOException e) {
e.printStackTrace();
}
System.out.println("Function fully executed");
}
We have a Java application which is acting as a server. Client applications (written in C#) are communicating with it using ZeroMQ. We are (mostly) following the Lazy Pirate pattern.
The server has a Router socket, implemented as follows (using JeroMQ):
ZContext context = new ZContext();
Socket socket = context.createSocket(ZMQ.ROUTER);
socket.bind("tcp://*:5555");
The clients connect and send messages like this:
ZContext context = ZContext.Create();
ZSocket socket = ZSocket.Create(context, ZSocketType.REQ);
socket.Identity = Encoding.UTF8.GetBytes("Some identity");
socket.Connect("tcp://my_host:5555");
socket.Send(new ZFrame("request data"));
We have experienced lost messages when multiple clients are sending messages at the same time. With a single client, there doesn't appear to be any problem.
Are we implementing this the right way for a multiple-client-single-server setup?
Update: Example client and server exhibiting this behaviour:
Server:
import org.zeromq.ZContext;
import org.zeromq.ZMQ;
import org.zeromq.ZMQ.PollItem;
import org.zeromq.ZMQ.Poller;
import org.zeromq.ZMQ.Socket;
import org.zeromq.ZMsg;
public class SimpleServer
{
public static void main(String[] args) throws InterruptedException
{
ZContext context = new ZContext();
Socket socket = context.createSocket(ZMQ.ROUTER);
socket.setRouterMandatory(true);
socket.bind("tcp://*:5559");
PollItem pollItem = new PollItem(socket, Poller.POLLIN);
int messagesReceived = 0;
int pollCount = 0;
while ((pollCount = ZMQ.poll(new PollItem[]{pollItem}, 3000)) > -1)
{
messagesReceived += pollCount;
for (int i = 0 ; i < pollCount ; i++)
{
ZMsg msg = ZMsg.recvMsg(socket);
System.out.println(String.format("Received message: %s. Total messages received: %d", msg, messagesReceived));
}
if (pollCount == 0)
{
System.out.println(String.format("No messages on socket. Total messages received: %d", messagesReceived));
}
}
}
}
Client:
using NetMQ;
using System;
using System.Text;
namespace SimpleClient
{
class Program
{
static byte[] identity = Encoding.UTF8.GetBytes("id" + DateTime.UtcNow.Ticks);
static void Main(string[] args)
{
for (int i = 0; i < 100; i++)
{
SendMessage();
}
}
private static void SendMessage()
{
using (NetMQContext context = NetMQContext.Create())
{
using (NetMQSocket socket = context.CreateRequestSocket())
{
socket.Options.Identity = identity;
socket.Connect("tcp://localhost:5559");
socket.Send(Encoding.UTF8.GetBytes("hello!"));
}
}
}
}
}
If I run the server and a single client, I can see all my 100 messages arrive. If I run, say, 5 clients simultaneously, I only get around 200 -> 300 messages arrive, instead of the full 500. As an aside, it appears that closing the socket in the client is somehow stopping the router socket on the server from receiving messages briefly, although this is just a theory.
Part 1 - poll may return more than one event
ZMQ.poll() returns the number of events that were found:
int rc = ZMQ.poll(new PollItem[]{pollItem}, 3000);
You currently assume that one return from poll is one event. Instead, you should loop over ZMsg msg = ZMsg.recvMsg(socket); for the number of events that are indicated by the return of ZMQ.Poll().
From the source of JeroMQ:
/**
* Polling on items. This has very poor performance.
* Try to use zmq_poll with selector
* CAUTION: This could be affected by jdk epoll bug
*
* #param items
* #param timeout
* #return number of events
*/
public static int zmq_poll(PollItem[] items, long timeout)
{
return zmq_poll(items, items.length, timeout);
}
Part 2 - ZMsg.receive() may return multiple frames
When you receive a ZMsg from ZMsg msg = ZMsg.recvMsg(socket);, the ZMsg may contain multiple ZFrames, each containing client data.
From the comments of the ZMsg class in JeroMQ's source:
* // Receive message from ZMQSocket "input" socket object and iterate over frames
* ZMsg receivedMessage = ZMsg.recvMsg(input);
* for (ZFrame f : receivedMessage) {
* // Do something with frame f (of type ZFrame)
* }
Part 3 - messages can be split across multiple ZFrames
From ZFrame's source in JeroMQ:
* The ZFrame class provides methods to send and receive single message
* frames across 0MQ sockets. A 'frame' corresponds to one underlying zmq_msg_t in the libzmq code.
* When you read a frame from a socket, the more() method indicates if the frame is part of an
* unfinished multipart message.
If I'm understanding this correctly, then for each event you may get multiple frames, and one client message may map to 1..N frames (if the message is big?).
So to summarize:
One return from poll may indicate multiple events.
One event and thus one ZMsg.receive() may contain multiple frames
One frame could contain one complete client message or only part of a client message; one client message maps to 1..N frames.
Unfortunately we couldn't solve this particular issue, and have moved away from using ZeroMQ for this interface. In case it helps anyone else, the only things we worked out for definite is that rapidly opening/closing the request sockets caused undesirable behaviour (lost messages) on the router socket end. The problem was exacerbated by a poorly performing server CPU, and didn't appear at all when the server was on a fast multi-core machine.
Unfortunatley I was not even close working with ZMQ at the time this question was active. But I had the same problem today and found this page. And your answer (not using ZMQ) was not satisfying for me. So I searched a bit more and finally found out what to do.
Just as a reminder: this works with the "POLLER" in ZMQ [1]
If you use "PAIR" connection you will for sure do NOT lose nay files, BUT send/recive takes approx. the same time. So you can not speed up and was not a solution for me.
Solution:
in zmq_setsockopt (python: zmq.setsockopt) you can set ZMQ_HWM (zmq.SNDHWM, zmq.RCVHWM) to '0' [2]
in python: sock.setsockopt(zmq.SNDHWM , 0) resp. sock.setsockopt(zmq.RCVHWM, 0) for the Sender resp. Reciver
note: i think notation changed from HWM to SNDWHM/RCVHWM
HWM = 0 means that there is "NO limit" for the number of messages (so be careful, maybe set a (hvery high) limit)
there is also ZMQ_SNDBUF/ ZMQ_RCVBUF (python: zmq.SNDBUF/zmq.RCVBUF) which you can give as well, ie. sock.setsockopt(zmq.RCVBUF, 0) resp. ..... [2]
so this will set the operating system "SO_RCVBUF" to default (here my knowledge ends)
setting this parameter or not did NOT influence my case but I think it might
Performance:
So with this I could "send" 100'000 files with 98kB in ~8s (~10GB): this will fill your RAM (if this is full I think your program will slow down), see also picture
in the mean time I "recived" and saved the files in about ~enter image description here118s and freeing the RAM again
Also, with that I NERVER lost a file up to now. (you might if you hit the limits of your PC)
data loss is "GOOD":
if you realy NEED all the data you should use this method
if you can regard some losses are fine (e.g. live plotting: as long as your FPS > ~50 you will smoothly see the plots and you do not care if you lose something)
--> you can save RAM and avoid blocking your whole PC!
Hope this post helps for the next person coming by...
[1]: https://learning-0mq-with-pyzmq.readthedocs.io/en/latest/pyzmq/multisocket/zmqpoller.htm
[2]: http://api.zeromq.org/2-1:zmq-setsockopt
You find a Picture of the RAM her:
RAM is loading in about 8s. Afterwords the disk is saving the files from the buffer
I am working on a system that rings a bell when someone decides to ring it or it is timed. This also includes emergency bells, such as when there is an earthquake threat or an armed intruder. On the armed intruder bell it rings for .5 seconds (500ms) and then waits 1.5s (1500ms) and repeats. However, on the fourth cycle the bell gets stuck on and will stay on until I turn the arduino off. I have tried many ways of trying to fix it but it will not work. I will post my code below and could someone have a look and see what is wrong with it?
Thank you!
Java for loop:
for(int i = 0; i < 21; i++) {
out.write("500".getBytes("UTF-8"));
out.flush();
Thread.sleep(2000);
}
Arduino Code:
int Relay = 13;
//The pin that the relay is attached to
//int time;
//Creates temp variable
void setup() {
Serial.begin(9600);
pinMode(Relay, OUTPUT);
}
void loop() {
while(true) {
//Check if data has been sent from the computer:
if (Serial.available()) {
int time;
//Assign serial value to temp
time = Serial.parseInt();
//Output value to relay
delay(1000);
digitalWrite(Relay, HIGH);
delay(time);
digitalWrite(Relay, LOW);
}
}
}
You might try putting a small delay into the if-loop, after checking and before reading from the Serial (just a few ms. Maybe put your 1000ms delay at the beginning). As the data arrive serialized, and you check with a high frequency, it might be, that the program starts reading out data before everything is received.
I don't know how exactly Serial.parseInt() works, but maybe try:
out.write("500\n".getBytes("UTF-8"));
in your Java-loop