I need to know what could cause the InputStream to hang indefinitely on close. Here's my code.
URL url = new URL("ftp://..");
InputSream is = url.openStream();
BufferedReader reader = new InputStreamReader(new GZIPInputStream(is));
try{
while(true){
if(reader.readline() == null){
break;
}
}
}catch(Exception e){
e.printStackTrace();
} finally{
System.out.println("Closing reader");
is.close(); // sometimes hangs indefinitely
System.out.println("Reader closed");
}
Closing InputStream or BufferedReader has the same effect.
I need to know what could cause the InputStream to hang indefinitely on close
I think it is the nature of the particular input stream that you are using.
You have a stream open to a read a file from an FTP server. Depending on the protocol driver that is being used on the Java client side, when you close() the file it may be attempting to close an active FTP session. This could involve exchanging network packets with the remote server, and if there is a network problem or the server has died, then that could take "a long time" ... depending on how long network timeouts have been set to, etc.
It could also be something like this:
Java HttpURLConnection InputStream.close() hangs (or works too long?)
My recommendation you be to capture a thread stacktrace while a connection is hung in close() and then delve into the Java codebase to figure out where exactly it is hung. (There are too many possible places for a problem to occur to investigate this without concrete evidence.)
Also try removing the GZIPInputStream from the stack to see if that makes any difference.
Finally, if you come back to us with more evidence, please tell us the precise Java version you are using and the real URL of the FTP server.
I am currently working on a java project which implements web-scraping and I am facing a weird issue so far.
Here is what I do :
Get an URL Connection with a page of a website
Parse the HTML code to get some content (OpenData)
Add the content in my database
Move onto the next page and go back to Step 1
This is actually very long and it can last for days so I need to let the script running. The problem is that sometimes, it stops for no reason (no errors, no messages, no window close ; It just litterally stops and I need to press one of my button to restart it). I have implemented a short code which restarts the application from where it stopped. I believe it's a connection problem to the database so I would like to know how could I fix it.
I use a static class which creates an instance of this class at the beginning of the application and then I use static methods from this class to run my queries like this for example :
ConnexionBDD.con.prepareStatement(query);
public static Connection loadDriver() {
try {
Class.forName(Driver);
con = DriverManager.getConnection(ConnectionString, user, pwd);
} catch (ClassNotFoundException e) {
System.err.println("Classe not found : Class.forName(...)");
} catch (SQLException e) {
System.err.println(e.getMessage());
}
return con;
}
I am not sure I am doing the right thing to make my connection lasts forever (in theory) and eventually close it when It has finished to iterate over my links.
You're jumping the gun a bit here. There's no evidence that the database connection is actually the problem. Usually if you were having DB connection issues you'd be getting an exception from the connection when you try to perform operations on it, a timeout, etc.
You need to:
Add detailed logging to your application, so you can see what it's doing as it progresses, and what it's trying to do when it stops; and
Run it with -Xdebug and other suitable options for remote debugging, so you can attach a debugger to it when it stops and examine its state to see what it is doing at the time. Use the debugger user interface from NetBeans, Eclipse, or whatever you prefer to attach to the program when the logging indicates that it's stopped progressing.
For logging, you can use java.util.logging. See the javadoc and the logging overview docs.
Here's an example of how to do remote debugging with Eclipse. You'll be able to find similar guides for your chosen IDE. Java also has a command line debugger, but it's pretty painful.
You also need to check to see whether the program might be crashing or exiting, rather than just stopping working. You should capture any standard error output from the program and check the program's error return code from the shell. Also look for hs_error files in the directory the program runs in, in case there's a JVM crash, though that should generate output on stderr as well.
You should also:
Set an application_name when you establish a connection to PostgreSQL, so you can easily see what your client is doing with the database. You can specify application_name as a JDBC connection parameter, or run a SET application_name = 'blah' statement after connecting.
When logging (or however you currently tell that your program is no longer progressing) indicates that the program has stopped working, examine pg_stat_activity in the server, looking at the entry/entries for your application. See if the connection is idle, idle in transaction, or running a statement, and what that statement is. If it's running a statement, query against pg_locks to see if it's blocked on an ungranted lock.
Let me explain the problem by a scenario :
1) User visit my page "www.proxy4html.com" , fills up the form:
web-address: |www.google.co.in |
2) click submit.
3) Gets google home page with "www.proxy4html.com" in browser address bar.
Now to fetch the contents from web I am using java.net.HttpURLConnection (although to resolve this I have tried several other options too). The code works as expected if it runs stand alone
(i.e while running through public static void main(String..).. thread, it fetched whatever the html is on given web address).
But the same when runs under the Servlet environment it throws
java.net.ConnectException: Connection timed out: connect
Here is the code:
public void write(String urlString, PrintWriter writer) {
URL url;
try {
url = new URL(urlString);
HttpURLConnection huc = (HttpURLConnection) url.openConnection();
HttpURLConnection.setFollowRedirects(false);
huc.setConnectTimeout(15 * 1000 * 60);
huc.setReadTimeout(15 * 1000 * 60);
huc.setRequestMethod("GET");
huc.connect();
InputStream input = url.openStream();
logger.info("got input stream");//I never reach here in servlet env :(
int i = 0;
while((i = input.read()) != -1) {
writer.write(i);
System.out.print((char)i);
}
input.close();
} catch (ConnectException e) {
logger.log(Level.SEVERE, "", e);
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
The code example given may not appear clean but point to be taken is that when the above method is invoked from Servlet passing a proper URL (and whatever outputstream), the connection is never made.
I am running all this on my tomcat which is on my local machine, which is in some corporate network (you may not want get any conclusion concerning firewall because same code is running under a p.s.v.main).
please help....
Thanks in advance.
Edit: I tried somewhat same at home and it worked fine as expected. I guess this question falls more into operation system domain rather than java. So I will like to rephrase my question again:
Is operating system (Window 7) is somehow preventing tomcat to make Http requests?
Is their any configuration under Window 7 that doesn't allow application to make Http Request other than browser?
I have seen in times that Mozilla Firefox (which I am running from a directory which is copied instead of being installed i.e may not have admin privileges and doesn't have a window registry) never able to do regular update to itself.
I understand that you prefer to insist that this is not a firewall issue (as the same code runs well as a standalone Java program), but I'm willing to take a risk here.
If you're absolutely sure that the very same code is being run in both cases (standalone case, and Tomcat case), there still is a possibility that it's a firewall issue.
The built-in firewall in Windows 7 assigns outbound rules per process per port. Is it possible that your standalone program runs with a JVM that has been allowed outbound access, whereas your Tomcat server runs with a JVM that hasn't been allowed outbound access?
In other words, are you absolutely confident that your standalone program, and your Tomcat server, are being run by the very same JVM? Very same java.exe? Under Windows, it's possible that your standalone program is actually being run by javaw.exe.
(You didn't mention whether you tried to completely disable the firewall. If you can disable the firewall completely, I'd suggest you do so, for the purpose of troubleshooting)
I am currently trying to make a litlle handy tool, you see I am a network administrator and my boss told me that he wanted me to monitor the network and block certain sites and ip for some game servers, so for the monitoring part we are going to redirect all traffic on the network to a server where we can monitor the traffic before sending it to the gateway.
For this we are going to use arpspoof in linux and I have finished a solution for the blocking of sites and servers, and what I am going to make is a GUI that makes it easier for me to handle and control these things and when I tried running arpspoof from java using a ProcessBuilder it does not work and I get no output?
It also does not enter the while loop. I can't really think of more to write atm, but if I can think of more I will update this thread.
My code:
new Thread() {
public void run() {
try {
System.out.println("running arpspoof...");
Process prb = new ProcessBuilder("gksudo", "arpspoof", "-i", "wlan0", Gateway).start();
InputStream is = prb.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line;
while ((line = br.readLine()) != null) {
System.out.println("Output: " + line);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}.start();
I have never used gksudo, but I googled it and it says it's a GUI version of sudo. I'm guessing that you just launched a GUI app which does not write anything to stdout and which does not return. If so, then the code is doing what I would expect. It is blocking until the process writes a line of text that it can read - which never occurs so it blocks indefinitely.
First test your ProcessBuilder code using a trivial command like "echo" to make sure your Java code is working as expected. Then work your way back. Try running your program as root so you don't need the sudo argument and see if that works. Then finally try to run it using sudo instead of gksudo.
I think #user is on the right track, but there are a couple of other possible explanations.
The gksudo command could be asking for a password. I'm not sure where it would ask, but there's a good chance that it won't be the "stdout" stream of the "gksudo" process.
If "gksudo" or the command that you are "gksudo"-ing fails to launch, there is a good chance that it will write an error message to its "stderr" stream. But you are not reading "stderr".
To help diagnose this, you need to try the following:
Look in the log file that for "sudo" - it is "/var/log/secure" on my box.
Use "ps -efl" (or similar) to see what processes exist while your application is blocked waiting for output. (If that is happening ...)
Look to see if "gksudo" is prompting for a password in an unexpected place.
Try temporarily tweaking the "sudoers" file to allow the "arpspoof" command to be "sudo"-ed without a password.
I've been using RXTX for about a year now, without too many problems. I just started a new program to interact with a new piece of hardware, so I reused the connect() method I've used on my other projects, but I have a weird problem I've never seen before.
The Problem
The device works fine, because when I connect with HyperTerminal, I send things and receive what I expect, and Serial Port Monitor(SPM) reflects this.
However, when I run the simple HyperTerminal-clone I wrote to diagnose the problem I'm having with my main app, bytes are sent, according to SPM, but nothing is received, and my SerialPortEventListener never fires. Even when I check for available data in the main loop, reader.ready() returns false. If I ignore this check, then I get an exception, details below.
Relevant section of connect() method
// Configure and open port
port = (SerialPort) CommPortIdentifier.getPortIdentifier(name)
.open(owner,1000)
port.setSerialPortParams(baud, databits, stopbits, parity);
port.setFlowControlMode(fc_mode);
final BufferedReader br = new BufferedReader(
new InputStreamReader(
port.getInputStream(),
"US-ASCII"));
// Add listener to print received characters to screen
port.addEventListener(new SerialPortEventListener(){
public void serialEvent(SerialPortEvent ev) {
try {
System.out.println("Received: "+br.readLine());
} catch (IOException e) { e.printStackTrace(); }
}
});
port.notifyOnDataAvailable();
Exception
java.io.IOException: Underlying input stream returned zero bytes
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:268)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
at java.io.InputStreamReader.read(InputStreamReader.java:167)
at java.io.BufferedReader.fill(BufferedReader.java:136)
at java.io.BufferedReader.read(BufferedReader.java:157)
at <my code>
The big question (again)
I think I've eliminated all possible hardware problems, so what could be wrong with my code, or the RXTX library?
Edit: something interesting
When I open HyperTerminal after sending a bunch of commands from java that should have gotten responses, all of the responses appear immediately, as if they had been put in the buffer somewhere, but unavailable.
Edit 2: Tried something new, same results
I ran the code example found here, with the same results. No data came in, but when I switched to a new program, it came all at once.
Edit 3
The hardware is fine, and even a different computer has the same problem. I am not using any sort of USB adapter.
I've started using PortMon, too, and it's giving me some interesting results. HyperTerminal and RXTX are not using the same settings, and RXTX always polls the port, unlike HyperTerminal, but I still can't see what settings would affect this. As soon as I can isolate the configuration from the constant polling, I'll post my PortMon logs.
Edit 4
Is it possible that some sort of Windows update in the last 3 months could have caused this? It has screwed up one of my MATLAB mex-based programs once.
Edit 5
I've also noticed some things that are different between HyperTerminal, RXTX, and a separate program I found that communicates with the device (but doesn't do what I want, which is why I'm rolling my own program)
HyperTerminal - set to no flow control, but Serial Port Monitor's RTS and DTR indicators are green
Other program - not sure what settings it thinks it's using, but only SPM's RTS indicator is green
RXTX - no matter what flow control I set, only SPM's CTS and DTR indicators are on.
From Serial Port Monitor's help files (paraphrased):
the indicators display the state of the serial control lines
RTS - Request To Send
CTS - Clear To Send
DTR - Data Terminal Ready
OK, sorry it's taken me so long to come back to this question. Here's how I got things working.
Note: This method will NOT work for everyone, please read below before copy/pasting into your own code
public void connect(CommPortIdentifier portId) throws Failure {
if (portId == null)
throw new Failure("No port set");
try { port = (SerialPort) portId.open(getClass().getName(), 10000); }
catch (PortInUseException e) {
throw new Failure("Port in use by " + e.currentOwner,e); }
try {
port.setSerialPortParams(9600, SerialPort.DATABITS_8,
SerialPort.STOPBITS_1, SerialPort.PARITY_NONE);
port.setFlowControlMode(SerialPort.FLOWCONTROL_RTSCTS_IN
| SerialPort.FLOWCONTROL_RTSCTS_OUT);
} catch (UnsupportedCommOperationException e) { throw new Failure(e); }
port.setRTS(true);
// More setup
}
So, in my case, the problem was that my particular device requires RTS flow control. Other devices may require different things (CTS, XON/XOFF), so check that device's manual. By default, RXTX disables all flow control mechanisms (unlike Hypertrm or other programs). Enabling each one is a two-step process.
Once you have a SerialPort object, call the setFlowControlMode() method, and bitwise-OR ('|') the necessary SerialPort.FLOWCONTROL_ constants
Set the appropriate flow control to true or false (like I did with port.setRTS(true))
For the others with similar problems, if this doesn't work, I suggest
Using a serial port monitoring program like Serial Port Monitor and/or PortMon (both Windows) to see what is actually going on.
Emailing the RXTX developers at rxtx#qbang.org (they are very helpful)
There is a simpler solution to this problem. This is what I did:
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String line;
while (keepRunning) {
try {
while ((br.ready()) && (line = br.readLine()) != null) {
....
}
If you check that the buffer "is ready" before you read it there should be no problem.
Ok, I do realize this thread is extremely old, but none of these solutions worked for me. I had the same problem and I tried everything to fix it, to no avail. Then I did some research on what causes the problem, and, when not dealing with Serial Communication, it happens at the end of a file. So, I figured I needed to add an ending to whatever is being received by the Java Application, specifically, a line return (\n). And sure enough, it fixed the problem for me! Hopefully this helps someone new, as I'm not expecting this to help anyone already on this thread...
(might be too simple, but might as well start somewhere...)
Is the port in use? Rather than:
port = (SerialPort) CommPortIdentifier.getPortIdentifier(name)
.open(owner,1000)
what about:
CommPortIdentifier portIdentifier;
try {
portIdentifier = CommPortIdentifier.getPortIdentifier(name);
} catch (NoSuchPortException nspe) {
// handle?
}
if (portIdentifier.isCurrentlyOwned()) {
// handle?
}
port = portIdentifier.open(owner, 1000);
if (!(port instanceof SerialPort)) {
// handle?
}
Are you swallowing any exceptions?
I tried RXTX a few months ago and ran into similar problems. I suggest two things:
Create a virtual comport using com0com. Enable trace logging. Compare the logs for when you use Hyperterminal versus when you run your own program. The difference will highlight what you are doing wrong.
In my humble opinion, RXTX's design is flawed and its implementation is quite buggy (take a look at its source-code, what a mess!). I've published an alternative library at http://kenai.com/projects/jperipheral with the following caveats: It's Windows-only and there are no pre-built binaries. Both of these will change in the near future. If you are interested in trying it out send me an email using http://desktopbeautifier.com/Main/contactus and I'll send you a pre-built version.
If anyone is still getting java.io.IOException: Underlying input stream returned zero bytes after you've read your characters using br.readline() for RXTX (even when you are checking first to see if br.readline() == null), just do this simple fix with a try/catch:
String line;
while (true){
try{
line = br.readLine();
}catch(IOException e){
System.out.println("No more characters received");
break;
}
//Print the line read
if (line.length() != 0)
System.out.println(line);
}
I've done some searching and it appears that this is the best/easiest way to get around this problem.
EDIT : I take that back. I tried this and still ended up having some problems. I'd recommend working with the raw InputStream directly, and implementing your own read/readLine method using InputStream.read(). That worked for me.