I'm working on an Android app that runs on remotely-sited, unattended devices. It has a Service to periodically retrieve settings from a fixed URL. It does this by daisy-chaining posts of a Runnable to a Handler obtained from a HandlerThread.getLooper(): it does an initial post() after creating the Handler in the Service's startup, and thereafter it does a postDelayed() at the end of each run through the Runnable.
This works "forever" on some devices in the project (fetching the settings successfully every 30 seconds for thousands of hours), but on other devices it stops working after a variable amount of time (hours to tens of hours) and thereafter never works again. I've even added code to monitor the Service's activity and restart the process by issuing a new post() if it's been more than 5 minutes since the last run, and that code kicks in when as intended when the process stalls, but still the Runnable no longer runs.
Some pared-down code snippets, and then further discussion of the problem/symptoms:
public class SettingsMonitor extends Service {
private HandlerThread mHandlerThread = new HandlerThread("SettingsMonitorHandler");
private Handler mSettingsMonitorHandler;
private long mLastTryMillis = 0;
#Override
public int onStartCommand(Intent intent, int flags, int startId) {
mHandlerThread.start();
mSettingsMonitorHandler = new Handler(mHandlerThread.getLooper());
mSettingsMonitorHandler.post(checkSettings);
return START_STICKY;
}
final Runnable checkSettings = new Runnable()
{
#Override
public void run() {
Log.d(TAG, "checking latest settings");
mLastTryMillis = System.currentTimeMillis();
getSettingsFromUrl(SETTINGS_URL);
// queue up next check
mSettingsMonitorHandler.postDelayed(checkSettings, appSettings.getSettingsMonitorIntervalMillis());
}
};
public void restartCheckingIfStalled() {
long settingsMonitorIntervalMillis = appSettings.getSettingsMonitorIntervalMillis();
long maxStalledMillis = 10 * settingsMonitorIntervalMillis;
long millisSinceLastCheck = System.currentTimeMillis() - mLastTryMillis;
if (millisSinceLastCheck > maxStalledMillis) {
mSettingsMonitorHandler.removeCallbacksAndMessages(null);
Log.d(TAG, "checkSettings stalled -- restarting...");
mSettingsMonitorHandler.post(checkSettings);
}
}
}
The app logs extensive debug info, including every exception caught in local catch blocks (e.g., inside getSettingsFromUrl()) as well as any exception caught in a global app-level UncaughtExceptionHandler -- there are no exceptions being thrown by the app. There are log entries made at many points in getSettingsFromUrl(), as well as the first line of the Runnable, but at some point all of these log entries cease to appear, so I infer that the Runnable itself is no longer actually running.
I monitor the Service from elsewhere in the app and so I know that it is running throughout the lifetime of the app (even after the Runnable no longer runs); at the same time, I call restartCheckingIfStalled() and I do see the log entries indicating that the re-start is being requested, but apparently the post() it issues does not cause the Runnable to run again.
It's a bit difficult to quantify, but it appears that this problem is occurring on devices that are showing other signs of having network connection problems (some are on wi-fi, some are on sim cards). I don't really know what to make of that possible correlation, though.
I am mystified that this process goes off the rails without any sign of trouble -- no exception is thrown, and the last successful run through the Runnable and getSettingsFromUrl() look completely normal. I am looking for suggestions about ways to isolate, identify, and ultimately fix the problem, because I've exhausted a bunch of ideas/tests and I'm still no closer to understanding it.
#x-code: Ok, here's getSettingsFromUrl() -- I don't (explicitly) set a timeout (I'll go search now for whether there is any kind of default). How might lack of timeout relate to or explain the symptoms I'm describing?
private boolean getSettingsFromUrl(String settingsUrl) {
Log.d(TAG, "trying settings file: " + settingsUrl);
try {
URL url = new URL(settingsUrl);
Scanner scanner = new Scanner(url.openStream());
while (scanner.hasNextLine()) {
String s = scanner.nextLine().split(SETTINGS_FILE_COMMENT_CHAR)[0].trim(); // remove any comments and trim leading/trailing whitespace
if (s.length() > 0) { // skip blank lines
AdminRequest req = new AdminRequest(s);
if (req.isSetReq()) { // only apply SET requests, not GET (or other nonsense)
String reqType = req.getReqType();
if (settingsToApply.containsKey(reqType)) { // override earlier settings with later
settingsToApply.remove(reqType);
}
settingsToApply.put(reqType, req);
}
}
}
scanner.close();
return true;
}
catch (FileNotFoundException e) {
Log.d(TAG, "settings file not found at " + settingsUrl);
}
catch (Exception e) {
Log.d(TAG, "error checking settings: " + Log.getStackTraceString(e));
}
return false;
}
This may beg further questions (about whether/how I use this function's return value -- which I do; about what these AdminRequests are and what I'm doing with them in the settingsToApply HashMap, etc). I'm willing to continue providing more info if it seems likely to lead to insight, but it's not clear to me why, at a minimum, I don't see the first "checking latest settings" message at the top of checkSettings()... how would a problem (e.g., lack of a timeout in accessing the URL) affect subsequent calls to checkSettings()? Do all invocations of checkSettings() run on the same thread, and it's tied up waiting "forever" to access the URL? If that's the case, why is restartCheckingIfStalled() not also blocked? I do get the Log.d() messages from that.
Edited to show the changes I made to add timeouts, which solved the problem per the responses I received: in getSettingsFromUrl() change these two lines
URL url = new URL(settingsUrl);
Scanner scanner = new Scanner(url.openStream());
to these:
URL url = new URL(settingsUrl);
URLConnection cxn = url.openConnection();
cxn.setConnectTimeout(URL_CONNECT_TIMEOUT_MILLIS);
cxn.setReadTimeout(URL_READ_TIMEOUT_MILLIS);
Scanner scanner = new Scanner(cxn.getInputStream());
Do all invocations of checkSettings() run on the same thread
Yes.
If that's the case, why is restartCheckingIfStalled() not also blocked?
Because it runs on your app's main (ui) thread.
The advantage of running a HandlerThread in a Service is that each message or Runnable that you post to it is placed in a queue, so you don't have to write thread-safe code.
One disadvantage is that this kind of blocking can occur. As GreyBeardedGeek points out, you should set a timeout other than 0 on your URL connection.
I suspect that the default connectionTimeout is 0 (wait forever).
And if the connection blocks 'forever', your handler thread will block as well - your subsequent posts will have no effect.
You should probably set both the connection timeout and the read timeout.
Related
I have to write a java program that receives G-Code commands via network and sends them to a 3D printer via serial communication. In principle everything seems to be okay, as long as the printer needs more than 300ms to execute a command. If execution time is shorter than that, it takes too much time for the printer to receive the next command and that results in a delay between command execution (printer nozzle standing still for about 100-200ms). This can become a problem in 3d printing so i have to eliminate that delay.
For comparison: Software like Repetier Host or Cura can send the same commands via seial without any delay between command execution, so it has to be possible somehow.
I use jSerialComm library for serial communication.
This is the Thread that sends commands to the printer:
#Override
public void run() {
if(printer == null) return;
log("Printer Thread started!");
//wait just in case
Main.sleep(3000);
long last = 0;
while(true) {
String cmd = printer.cmdQueue.poll();
if (cmd != null && !cmd.equals("") && !cmd.equals("\n")) {
log(cmd+" last: "+(System.currentTimeMillis()-last)+"ms");
last = System.currentTimeMillis();
send(cmd + "\n", 0);
}
}
}
private void send(String cmd, int timeout) {
printer.serialWrite(cmd);
waitForBuffer(timeout);
}
private void waitForBuffer(int timeout) {
if(!blockForOK(timeout))
log("OK Timeout ("+timeout+"ms)");
}
public boolean blockForOK(int timeoutMillis) {
long millis = System.currentTimeMillis();
while(!printer.bufferAvailable) {
if(timeoutMillis != 0)
if(millis + timeoutMillis < System.currentTimeMillis()) return false;
try {
sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
printer.bufferAvailable = false;
return true;
}
this is printer.serialWrite: ("Inspired" by Arduino Java Lib)
public void serialWrite(String s){
comPort.setComPortTimeouts(SerialPort.TIMEOUT_SCANNER, 0, 500);
try{Thread.sleep(5);} catch(Exception e){}
PrintWriter pout = new PrintWriter(comPort.getOutputStream());
pout.print(s);
pout.flush();
}
printer is an Object of class Printer which implements com.fazecast.jSerialComm.SerialPortDataListener
relevant functions of Printer
#Override
public int getListeningEvents() {
return SerialPort.LISTENING_EVENT_DATA_AVAILABLE;
}
#Override
public void serialEvent(SerialPortEvent serialPortEvent) {
byte[] newData = new byte[comPort.bytesAvailable()];
int numRead = comPort.readBytes(newData, newData.length);
handleData(new String(newData));
}
private void handleData(String line) {
//log("RX: "+line);
if(line.contains("ok")) {
bufferAvailable = true;
}
if(line.contains("T:")) {
printerThread.printer.temperature[0] = Utils.readFloat(line.substring(line.indexOf("T:")+2));
}
if(line.contains("T0:")) {
printerThread.printer.temperature[0] = Utils.readFloat(line.substring(line.indexOf("T0:")+3));
}
if(line.contains("T1:")) {
printerThread.printer.temperature[1] = Utils.readFloat(line.substring(line.indexOf("T1:")+3));
}
if(line.contains("T2:")) {
printerThread.printer.temperature[2] = Utils.readFloat(line.substring(line.indexOf("T2:")+3));
}
}
Printer.bufferAvailable is declared volatile
I also tried blocking functions of jserialcomm in another thread, same result.
Where is my bottleneck? Is there a bottleneck in my code at all or does jserialcomm produce too much overhead?
For those who do not have experience in 3d-printing:
When the printer receives a valid command, it will put that command into an internal buffer to minimize delay. As long as there is free space in the internal buffer it replies with ok. When the buffer is full, the ok is delayed until there is free space again.
So basicly you just have to send a command, wait for the ok, send another one immediately.
#Override
public void serialEvent(SerialPortEvent serialPortEvent) {
byte[] newData = new byte[comPort.bytesAvailable()];
int numRead = comPort.readBytes(newData, newData.length);
handleData(new String(newData));
}
This part is problematic, the event may have been triggered before a full line was read, so potentially only half an ok has been received yet. You need to buffer (over multiple events) and reassamble into messages first before attempting to parse this as full messages.
Worst case, this may have resulted in entirely loosing temperature readings or ok messages as they have been ripped in half.
See the InputStream example and wrap it in a BufferedReader to get access to BufferedReader::readLine(). With the BufferedReader in place, you can that just use that to poll directly in the main thread and process the response synchronously.
try{Thread.sleep(5);} catch(Exception e){}
sleep(1);
You don't want to sleep. Depending on your system environment (and I strongly assume that this isn't running on Windows on x86, but rather Linux on an embedded platform), a sleep can be much longer than anticipated. Up to 30ms or 100ms, depending on the Kernel configuration.
The sleep before write doesn't make much sense in the first place, you know that the serial port is ready to write as you already had received an ok confirming reception of the previously sent command.
The sleep during receive becomes pointless when using the BufferedReader.
comPort.setComPortTimeouts(SerialPort.TIMEOUT_SCANNER, 0, 500);
And this is actually causing your problems. SerialPort.TIMEOUT_SCANNER activates a wait period on read. After receiving the first byte it will wait at least for another 100ms to see if it will become part of a message. So after it has seen the ok it then waits 100ms internally on the OS side before it assumes that this was all there is.
You need SerialPort.TIMEOUT_READ_SEMI_BLOCKING for low latency, but then the problem predicted in the first paragraph will occur unless buffered.
Setting repeatedly also causes yet another problem, because there is a 200ms sleep in Serialport::setComPortTimeouts internally. Set it per serial connection once, no more than that.
Check the manual of the printer (or tell us the model) not sure you actually need to wait for the ok, and therefore you can read/write concurrently. Some of the time there's a hardware flow control handling this stuff for you, with large enough buffers. Try just send the commands without waiting for ok, see what happens.
If you just want to pipe commands from the network to serial port, you can use ready-made solution like socat. For example running the following:
socat TCP-LISTEN:8888,fork,reuseaddr FILE:/dev/ttyUSB0,b115200,raw
would pipe all bytes coming from clients connected to the 8888 port directly to the /dev/ttyUSB0 at baud rate of 115200 (and vice-versa).
I have been wondering this since my early school grades, now I think I have faced a real life scenario regarding this. There is an unexplainable delay in 2 consective log statements, here is the detail:
I am using slf4j with logback and here is what I got in my logs:
INFO 2014-01-15 01:01:12,215 [Thread-38005034] 185 - REQ Rcvd:68506950
INFO 2014-01-15 01:01:34,183 [Thread-38005035] 123 - [MGW150114010112fc1b][OC]923029563761->8803026062143 rDialogID: 68506950
These two lines should be printed with no time delay but it takes more than 20 secs for the second line to be printed.
Here is my code:
public void onRequest(final DPRequest arg0) {
new Thread() {
#Override
public void run() {
Request idp = new Request();
idp.arg0 = arg0;
Thread dpThread = new Thread(idp);
dpThread.start();
logger.info("REQ Rcvd:" + arg0.getId());
}
}.start();
}
This is the request class
#Override
public void run() {
processdp();
}
public void processdp() {
OCHandling ocsiHandler = new OCHandling(this.arg0);
ocHandler.run();
}
This is in the OCHandling class
public void run() {
logger.info("[" + refId + "][OC]" + PNumber + "->" + CNumber + " rDialogID: " + arg0.getId());
}
onRequest function recieves almost 50 requests per second, this is the first time I have seen this much delay, and it only happened for less than a minute, where do you think could be the issue
Thread count: I have checked was normal
Logger not writting real time logs? Edit-> I found out this is not an issue
Any blocking request, this process wasnt given time in this slot?
Java does not pass parameters by reference so its not possible that the API which calls onRequest method is meddling with the parameter or what ever I dont know
This program is running on a VM which is on ESXI server
turns out it was a compaction problem
http://knucle.blogspot.com/2011/05/compaction-in-java-garbage-collection.html
never heard of that before, I could think of no reason why gc needs to stop all other threads while collecting unreferenced memory, but reading about this, it solves the mystery. Thanks for pointing to gc, which lead to this compaction thingy.
I am running a java application in a TimerTask using the schedule method, i.e. it runs at certain intervals and if the timertask gets delayed it does not try to play catch-up. I want the program to run for as long as possible if not indefinitely. I am using Netbeans 7.3.
However, I find that after a few hours (sometimes only 1 hour and sometimes over 24 hours) the program stops printing to the console and in fact stops running altogether. I know it isn't simply that it stops printing to the console because the program also sends an email regularly, and after it stops the emails stop coming as well. It does not always say "build complete" or throw an exception, though sometimes it will simply say "build complete" and stop running.
I have tried looking around the web, but I cannot seem to find any conclusive answer. I saw somewhere that the TimerTask will stop running if it throws an unchecked excpetion, but since it does not always stop at the end of a TimerTask but rather at some random point in the middle of a task, I don't think that is what's happening.
Does anyone have any idea what is going wrong and how I can work around it?
Any pointers would be much appreciated.
Thanks,
Paul
EDIT: after I tried putting a logger into my timertask, the code errored out with the following exception:
java.io.IOException: Couldn't get lock for Logging.txt. This may very well be a separate problem because I have never used a logger. I read somewhere that this exception probably means I am missing some permissions to write to the file in java (not sure if that is the case or how to fix in Netbeans). If I fix the logger it and it's unrelated I will delete this edit
EDIT: I forgot to mention that the Broadcast function involves a lot of communication with various servers. I think the problem might be that the program simply gets stuck on a line and never completes. In this case I need to figure out a way to have the broadcast method timeout after a certain amount of time and try again.
EDIT: here is an example of my code:
public void RUN(){
exchanges.add(Bitstamp);
exchanges.add(BTCE);
exchanges.add(CampBX);
t.schedule(
new TimerTask()
{
public void run()
{
Broadcast(exchanges);
UseLogger.Log();
}
}, 0,15000);
}
public static void Broadcast(ArrayList<Exchange> exchanges){
for (int i=0; i<exchanges.size();i++){
exchanges.get(i).prepareGetUpdate();
}
System.out.println("************** TICKERS *****************");
try{
for (int i=0; i<exchanges.size();i++){
System.out.println(exchanges.get(i).getPrices());
}
tickerException=false;
} catch (Exception e){
tickerException=true;
}
System.out.println("");
System.out.println("************** BALANCES *****************");
try{
for (int i=0;i<exchanges.size();i++){
System.out.println("i: " + i);
System.out.println(exchanges.get(i).getBalance());
}
balanceException=false;
} catch (Exception e){
balanceException=true;
}
System.out.println("************ ORDERS *****************");
try{
for (int i=0; i< exchanges.size();i++){
exchanges.get(i).getOrders();
if (exchanges.get(i).exchangeName.contentEquals("BTCE")){
exchanges.get(i).Total_btc+=exchanges.get(i).btcInOrders;
exchanges.get(i).Total_usd+=exchanges.get(i).usdInOrders;
}
}
orderException=false;
} catch (Exception e){
orderException = true;
}
for (int i=0; i< exchanges.size();i++){
exchanges.get(i).getUpdate();
}
btcDiff = totalBTC-originalTotal;
btcNeutral=true;
}
I want to check an huge amount (thousands) of Websites, if they are still running. Because I want to get rid of unececarry entries in my HostFile Wikipage about Hostfiles.
I want to do it in a 2 Stage process.
Check if something is running on Port 80
Check the HTTP response code (if it's not 200 I have to check the site)
I want to multithread, because if I want to check thousands of addresses, I cant wait for timeouts.
This question is just about Step one.
I have the problem, that ~1/4 of my connect attempts don't work. If I retry the not working ones about ~3/4 work? Do I not close the Sockets correctly? Do I run into a limit of open Sockets?
Default I run 16 threads, but I have the same problems with 8 or 4.
Is there something I'm missing
I have simplified the code a little.
Here is the code of the Thread
public class SocketThread extends Thread{
int tn;
int n;
String[] s;
private ArrayList<String> good;
private ArrayList<String> bad;
public SocketThread(int tn, int n, String[] s) {
this.tn = tn;
this.n = n;
this.s = s;
good = new ArrayList<String>();
bad = new ArrayList<String>();
}
#Override
public void run() {
int answer;
for (int i = tn * (s.length / n); i < ((tn + 1) * (s.length / n)) - 1; i++) {
answer = checkPort80(s[i]);
if (answer == 1) {
good.add(s[i]);
} else {
bad.add(s[i]);
}
System.out.println(s[i] + " | " + answer);
}
}
}
And here is the checkPort80 Method
public static int checkPort80(String host)
Socket socket = null;
int reachable = -1;
try {
//One way of doing it
//socket = new Socket(host, 80);
//socket.close();
//Another way I've tried
socket = new Socket();
InetSocketAddress ina = new InetSocketAddress(host, 80);
socket.connect(ina, 30000);
socket.close();
return reachable = 1;
} catch (Exception e) {
} finally {
if (socket != null) {
if (socket.isBound()) {
try {
socket.close();
return reachable;
} catch (Exception e) {
e.getMessage();
return reachable;
}
}
}
}
}
About Threads, I make a ArrayList of Threads, create them and .start() them and right afterwards I .join() them, get the "God" and the "Bad" save them to files.
Help is appreciated.
PS: I rename the Hosts-file first so that it doesn't affect the process, so this is not an issue.
Edit:
Thanks to Marcelo Hernández Rishr I discovered, that HttpURLConnection seems to be the better solution. It works faster and I can also get the HttpResponseCode, which I was also interested anyways (just thought it would be much slower, then just checking Port 80). I still after a while suddenly get Errors, I guess this has to do with the DNS server thinking this is a DOS-Attack ^^ (but I should examine futher if the error lies somewhere else) also fyi I use OpenDNS, so maybe they just don't like me ^^.
x4u suggested adding a sleep() to the Threads, which seems to make things a little better, but will it help me raise entries/second i don't know.
Still, I can't (by far) get to the speed I wanted (10+ entries/second), even 6 entries per second doesn't seem to work.
Here are a few scenarios I tested (until now all without any sleep()).
number of time i get first round how many entries where entries/second
threads of errors processed until then
10 1 minute 17 seconds ~770 entries 10
8 3 minute 55 seconds ~2000 entries 8,51
6 6 minute 30 seconds ~2270 entries 5,82
I will try to find a sweet spot with Threads and sleep (or maybe simply pause all for one minute if I get many errors).
Problem is, there are Hostfiles with one million entries, which at one entry per second would take 11 Days, which I guess all understand, is not expectable.
Are there ways to switch DNS-Servers on the fly?
Any other suggestions?
Should I post the new questions as separate questions?
Thanks for the help until now.
I'll post new results in about a week.
I have 3 suggestions that may help you in your task.
Maybe you can use the class HttpURLConnection
Use a maximum of 10 threads because you are still limited by cpu, bandwidth, etc.
The lists good and bad shouldn't be part of your thread class, maybe they can be static members of the class were you have your main method and do static synchronized methods to add members to both lists from any thread.
Sockets usually try to shut down gracefully and wait for a response from the destination port. While they are waiting they are still blocking resources which can make successive connection attempts fail if they were executed while there have still been too many open sockets.
To avoid this you can turn off the lingering before you connect the socket:
socket.setSoLinger(false, 0);
I was just wondering how i would let my java program continue running, but always be ready to take input.
Right now i am using a bufferreader and reading in a line, but that makes my program halt and wait for input. I want the program to continue running but be ready to take input whenever needed, is there a way to do this?
I would expect that you're going to have to look into multithreading your application in order to get this working as you desire.
Edit: Of course, while this functionality can be achieved by a purely CLI interface, you would probably be better off exploring other options (i.e. GUI options) if you intend on having a full response/event-driven system running in a multithreaded fashion.
Here is a quick example of how a multi-threaded command line interface application may work. This will not require polling for input, nor a GUI interface in order to perform tasks in the background while waiting for input from a user in a console.
In this example, a Thread is running in the background, which can be told to output a greeting in a specified number of seconds later.
public class CommandLineMultiThread
{
public static void main(String[] args)
{
Scanner s = new Scanner(System.in);
// Makes and runs the background thread.
MyThread myThread = new MyThread();
Thread t = new Thread(myThread);
t.start();
// Get the number of seconds to wait from the console.
// Exit when "0" is entered.
int waitDuration;
do
{
waitDuration = s.nextInt();
myThread.setGreetIn(waitDuration);
} while (waitDuration != 0);
myThread.quit();
}
static class MyThread implements Runnable
{
boolean running = true; // Boolean to keep thread alive.
long greetingTime = Long.MAX_VALUE; // Time to greet at.
public void setGreetIn(int i)
{
greetingTime = System.currentTimeMillis() + (i * 1000);
}
public void quit()
{
running = false;
}
public void run()
{
while (running)
{
if (System.currentTimeMillis() > greetingTime)
{
System.out.println("Hello!");
greetingTime = Long.MAX_VALUE;
}
try
{
Thread.sleep(100);
}
catch (InterruptedException e) {}
}
}
}
}
The Scanner class can be used to accept user input from the command line interface by routing the input from the System.in object.
Then, while the background thread myThread is running, the main thread is waiting for an input from System.in via the Scanner.nextInt method. Once the seconds to wait has been accepted, the background thread is told to wait until a certain time, and once that time arrives, the greeting Hello! is output to the console.
I think your program will have to occasionally poll for user input.
Give it a nice multi-threaded GUI instead of a CLI :)
I agree with James, another alternative is "faking" continuous program running. This won't work with all scenarios, but you can set a timer right before you display the user input, then calculate the time between stop and "start" again when the user inputs something.
Use that time to perform a repeated function a certain number of times. This is only helpful if you've got something on a timer itself already, like a constantly draining integer every few seconds.
An example:
You ask the user a question but only want to give them 5 seconds to answer. When the user answers (hits enter) the program will compute the time it took them to enter, if too long, throw one message, if under the time limit throw another.
I'm only suggesting this method because threading, which is what you really want to get into, is quite advanced.