Develop Reference

No data on socket input stream even though there is data

I have a problem with my Java socket and connection. It seems like my input stream does not read any data even though there is data on the stream. Here is an example on how we read data:
InputStream is = socket.getInputStream();
StringBuffer buf = new StringBuffer();
final int SO_TIME = 2500;
socket.setSoTimeout(SO_TIME);
long readTime = 0;
boolean remoteSocketClosed = false;
//Read data while the Remote-Socket is opened and the timeout is OK
while (!remoteSocketClosed && readTime < 30000)
{
try
{
int c = is.read();
if (c != -1)
{
buf.append((char)c);
}
else //c==-1
{
remoteSocketClosed = true;
}
}
catch (SocketTimeoutException socketTimeout)
{
readTime += SO_TIME;
}
}
if (readTime >= 30000)
{
throw new IOException("No answer from server after request (" + readTime +"ms )");
}
else
{
if (buf.length() > 0)
{
return buf;
}
else
{
return null;
}
}
finally
{
try
{
if (socket!=null)
socket.close();
}
catch (Exception e)
{
l.error("Socket not closed", e);
}
try{Thread.sleep(250);}catch(Exception e){}
}
Now this looks fine to me. However when there is data on the stream, it will not get picked up. I can use telnet and see that data is coming in on that connection, yet Java just throws the IOException (timeout).
The server is running on the same machine (localhost), and we can send data to the server, yet the answer is lost somehow (only in Java not in telnet). Is this some weird setting I don't know about? We are using Windows 10 and Java 8, firewalls are all disabled. I have never experienced something like this so any help is appreciated.

Sometimes you just don't see the forest for the trees and have to sleep over it...
The problem is my condition
if (c != -1)
{
buf.append((char)c);
}
else //c==-1
{
remoteSocketClosed = true;
}
But the server never closes the connection... So I never exit my loop and im stuck for 30 seconds running into socket timeouts. After the 30 seconds of waiting my answer is in the StringBuffer but of course my readTime is 30000 so I throw the exception rather providing an answer... I will have to think of an EndOfPackage Byte or something to avoid this.
Thanks for any help tho

Continue the program execution in multiple try-catch statements scenario after catching exception

I have the following code running in my project:
HashMap<String, DeviceData> deviceMap = getAllDevices();
int status = 0;
DeviceHandle devHandle = null;
for (LicenseData licenseData:listLicenses) {
Map<String, String> licenseMap = licenseData.getLicenseKeyValues();
if ((licenseMap != null && !licenseMap.isEmpty())) {
String keyDecrypt = licenseMap.get("key");
Date expiryDate = new Date(Long.parseLong(licenseMap.get("expiryDate")));
boolean allowForeign = Boolean.parseBoolean(licenseMap.get("allowForeign"));
String ipDecrypt = licenseMap.get("ipAddress");
if (expiryDate.compareTo(new Date()) > 0 || keyDecrypt.equals(licenseData.getKey().getCurrentValueAsString()))
{
try {
DeviceData device = deviceMap.get(ipDecrypt);
devHandle = (DeviceHandle)device.getHandle();
if(device != null && devHandle != null) {
deviceMap.remove(ipDecrypt, device);
System.out.println("After deletion device map.");
System.out.println(deviceMap);
createUser(devHandle);
try {
if (allowForeign) {
Process pr = Runtime.getRuntime().exec(SomeOperation);
status = pr.waitFor();
if (status == 0)
//Debug Statement
else
//Error Debug Statemnt
deleteUser(devHandle);
}
else {
Process pr = Runtime.getRuntime().exec(SomeOperation);
status = pr.waitFor();
if (status == 0)
//Debug Statement
else
//Error Debug Statement
deleteUser(devHandle);
}
} catch(Exception e) {
//Exception statement
deleteUser(devHandle);
}
} catch(Exception e) {
e.printStackTrace();
}
}
}
}
Explanation: I have a list of licenses for my application in listLicenses. All the devices present in the server are in deviceMap. For each license, I am decrypting it and getting the values. If license for a device is present, I get a handle on that device and doing some operations.
The issue is:
If I am not able to get a handle on device(getHandle()), or if I am not able to create a user after getting the device handle(createUser()), an exception is thrown. These methods are very hierarchical, i.e I am calling them from here, they are in another class throwing own exceptions and for their operation, they call other methods.
If there are three devices in the map, and three licenses, and if for the first one I am not able to get a handle or create a user, device is removed from deviceMap but no further execution happens i.e. for the next two devices.
If exception occurs for on device, I want to continue the exception for other two devices. I tried using return but couldn't get it to work.
Please help.Also, please forgive for the syntax and if any mismatch is there in the code.

Make use of first try's catch block.
This is how I handled when I faced same kind of situation.
catch (Exception exp) {
if (exp instanceof NullPointerException) {
log.info"Invalid/ Inactive ");
} else if (exp instanceof NonUniqueResultException) {
log.info("Multiple records existed");
} else {
exp.printStackTrace();
errorMsgs.append("Unexpected Error Occured. Please contact Admin.");
}
}

Too many open files error

Introduction
I've found plenty of information about the Too many open files exception on the Web but I couldn't solve this strange case. As I've read, the exception is thrown when the number of opened file descriptors by process defined in the OS is exceeded. The nature of these files is diverse. Files could be sockets, documents, etc. And I've found robust and secure ways to open files that I have implemented in my Java application.
The application is a short program that downloads Web pages using the Boilerpipe algorithm. This way I get the most representative contents of that site. Then, I write it in an appropriate format (TREC format) to disk. The URLs of these websites are taken from a MySQL database that I access using the JDBC connector.
So, I think that the exception can be thrown form three different places:
Connection to database
HTTP Connection to the web sites
Opening and writing files
Although, as I said, I think that I use a correct way of opening and writing those files.
Problem
There are thousands of URL's to process and the exception is thrown after a while (what makes it also very difficult to debug...). I don't know if that matters, but URLs are classified into different categories and I run different instances of the program to speed up the whole process. Categories don't overlap so there shouldn't be any problem.
Code
To make it more readable I'm going to show just those three parts of my code simplified:
Database access
// Connect to database
Connection dbconn = null;
try {
String dbUrl = "jdbc:mysql://" + dbServer + "/" + dbName;
Class.forName ("com.mysql.jdbc.Driver").newInstance ();
dbconn = DriverManager.getConnection(dbUrl, dbUser, dbPass);
System.out.println ("Database connection established");
} catch (Exception e) {
e.printStackTrace();
System.err.println ("Cannot connect to database server");
System.exit(-1);
}
System.out.println(" Downloading category: " + category);
Statement s = null;
try {
s = dbconn.createStatement();
} catch (SQLException e) {
System.err.println ("Error on creating the statement");
System.exit(-1);
e.printStackTrace();
}
String q = "SELECT resource,topic FROM " +
"content_links " +
"WHERE topic LIKE 'Top/" + category + "%';";
try {
s.executeQuery(q);
} catch(Exception e) {
System.err.println ("Error on executing the SQL statement");
System.exit(-1);
e.printStackTrace();
}
ResultSet rs = null;
try {
rs = s.getResultSet ();
} catch (SQLException e) {
System.err.println ("Error on getting the result set");
System.exit(-1);
e.printStackTrace();
}
int count = 0, webError = 0;
// work with the result set
try {
while (rs.next ()) {
// MAIN LOOP
}
} catch (SQLException e) {
System.err.println ("Error on getting next item");
System.exit(-1);
e.printStackTrace();
}
// Close connection to database
if (dbconn != null) {
try {
dbconn.close ();
System.out.println (" Database connection terminated");
} catch (Exception e) { /* ignore close errors */ }
}
HTTP connection, extract site's title and boilerpipe filter
try {
String title = "";
org.jsoup.nodes.Document doc = Jsoup.connect(urlVal).get();
for (Element element : doc.select("*")) {
if (element.tagName().equalsIgnoreCase("title")) {
title = element.text();
}
if (!element.hasText() && element.isBlock()) {
element.remove();
}
}
String contents = "";
contents = NumWordsRulesExtractor.INSTANCE.getText(doc.text());
storeFile(id, urlVal, catVal, title, contents);
}
} catch (BoilerpipeProcessingException e) {
System.err.println("Connection failed to: " + urlVal);
} catch (MalformedURLException e1) {
System.err.println("Malformed URL: " + urlVal);
} catch(Exception e2) {
System.err.println("Exception: " + e2.getMessage());
e2.getStackTrace();
}
Writing file
private static void storeFile(String id, String url, String cat, String title, String contents) {
BufferedWriter out = null;
try {
out = new BufferedWriter(
new OutputStreamWriter(
new FileOutputStream(
new File(path + "/" + id + ".webtrec")),"UTF8"));
// write in TREC format
out.write("...");
} catch (IOException e) {
System.err.println("Error: " + e.getMessage());
e.printStackTrace();
} finally {
try {
out.close();
} catch (IOException e) {
System.err.println("Error: " + e.getMessage());
e.printStackTrace();
}
}

Yup. You are leaking file descriptors.
In the first case you open a DB connection and never close it. The connection will typically use a socket or something to talk to the database. Since you don't close the connection, the socket won't be closed, and you will leak a file descriptor.
In the second case, I suspect that the call to Jsoup.connect(urlVal) is opening a connection, which you don't then close. That will result in a file descriptor leak.
Correction - there is no close() method on the Connection interface. It looks like the actual connection must be created and then closed internally by the get method. Assuming that is so, there is no file descriptor leak in the second case.
The third case does not leak file descriptors. However, if you fail to open the file, out.close(); statement will attempt to call a method on null ... and will throw a NPE.
The solution is to find all of the places where you open files, database connection, http connections, and make sure that the handle is always closed.
One way to do it is to put the close() call (or equivalent) in a finally block ... but make sure that you don't accidentally call close() on null.
The other way to do it is to use the Java 7 "try with resource" syntax. For example:
private static void storeFile(String id, String url, String cat,
String title, String contents) {
try (BufferedWriter out = new BufferedWriter(
new OutputStreamWriter(
new FileOutputStream(
new File(path + "/" + id + ".webtrec")),"UTF8"))) {
// write in TREC format
out.write("...");
out.close();
} catch (IOException e) {
System.err.println("Error: " + e.getMessage());
e.printStackTrace();
}
}
(Note however that the Java 7 syntax can only be used with resources that implement the new Closeable interface.)

To add to Stephen's comprehensive analysis. I recommend using a connection pool for the database, although, as Stephen has pointed, unless you're closing these connections, you'll drain the pool close, but at least it will be easier to discover why...
I've not seen any evidence, but you should be using some kind of Thread pool to download the pages, this will help to maximize the resources of the system. Some of executor service would suffice. Like I say, you're probably already doing this, but you didn't show any code (or comment) for it.

java's executeUpdate is not working properly

I have the following java code:
if (ps.executeUpdate() != 1)
{
// Error - did not insert one row
String err = "insert unable to insert LocalUsage data: " + usg.toString();
Logger.log(err, _MODULE_CLASS, Logger.DEBUG);
throw new DaoException(err);
}
The problem if the query had a foreign key exception, then it will be thrown but it will never get to inside the if. what should I do so that it will get inside the if and output the log I have?
The problem is that this if condition is inside a try catch block, and it is going to the catch and never enters to the if condition.

executeUpdate() might throw an SQLException, as is described in its API documentation. You might want to catch that exception.
int count;
try {
count = ps.executeUpdate();
} catch (SQLException e) {
throw new DaoException("Exception while executing update: " + e.getMessage());
}
if (count != 1) {
// ...
}

As the docs states executeUpdate() may throw an exception so your code flow will fail and you will not be able to do any processing afterwards incase your exception handling is not proper.
Which I think is happening in your code right now.
While doing database call I would suggest you do it like this:
int operationStatus;
try {
operationStatus = ps.executeUpdate();
} catch(SQLException exp) {
final String message = "SQL Exception while calling executeUpdate()";
logger.error(message, exp);
throw new DAOException(message, logger);
} catch(Exception exp) {
final String message = "Exception while calling executeUpdate()";
logger.error(message, exp);
throw new DAOException(message, logger);
} finally {
//you may wish to clean up resources if they are not going to be used after this point.
}
if(operationStatus < 0) {
//Next steps
}

Limited sockets to midp?

It seems to me that there is some kind of limitation in socket creation in MIDP.
I need to make lots of connections to a server (none concourrent) and in the forth or fith try my app crashes. It crashes in the simulator and in my real device as well.
To isolate any possibility of it being influenced by my code, I isolated the following code:
try {
StreamConnection c;
StringBuffer sb = new StringBuffer();
c = (StreamConnection) Connector.open(
"http://www.cnn.com.br/", Connector.READ_WRITE);
InputStreamReader r = new InputStreamReader(c.openInputStream(), "UTF-8");
System.out.println(r.read());
c.close();
} catch (IOException ex) {
ex.printStackTrace();
}
This code crashes in the 13th try.
I've tryed to add a sleep of 10 seconds inside a while loop and, it crashes at the 13th try too.
The crash message is:
java.io.IOException: Resource limit exceeded for TCP client sockets
- com.sun.midp.io.j2me.socket.Protocol.open0(), bci=0
- com.sun.midp.io.j2me.socket.Protocol.connect(), bci=124
- com.sun.midp.io.j2me.socket.Protocol.open(), bci=125

While c.close() inside the try should be adequate, I am wondering if you have other issues that are triggering this. The code really should be closing the connection AND inputstream inside of a finally. Something like this:
StreamConnection c = null;
InputStream is = null;
try {
StringBuffer sb = new StringBuffer();
c = (StreamConnection) Connector.open(
"http://www.cnn.com.br/", Connector.READ_WRITE);
is = c.openInputStream();
InputStreamReader r = new InputStreamReader(is, "UTF-8");
System.out.println(r.read());
} catch (IOException ex) {
ex.printStackTrace();
} finally {
if (is != null) {
try {
is.close();
} catch (Exception ex) {
System.out.println("Failed to close is!");
}
}
if (c != null) {
try {
c.close();
} catch (Exception ex) {
System.out.println("Failed to close conn!");
}
}
}

The reason, why the c.close() did not actually close was because the inputstream was not closed. Some devices require that both the stream and the connection be closed. Also the connections do not close immediately, on some devices, when the close() method is called. You might have to do a gc too