I am currently trying to connect to an amazon RDS from a lambda function. The problem I am facing is that I do not get an error but neither I manage to get a connection, so I am not sure how to proceed. I do not have a timeout, or an error with the parameters, but the object "con" is always null.
To do this I am doing the following:
Connection con = null;
String jdbcUrl2 = "jdbc:mysql://" + "connectionValueExample-zone.rds.amazonaws.com" + ":" + "3306" + "/" + "botdb" + "?user=" + "bot" + "&password=" + "PWvalueExample";
con = DriverManager.getConnection(jdbcUrl2);
After every step I check with prints() how it's going. After I try to stablish the connection I try to catch it, because while I am not getting exceptions right now, the object con never stops being null, so I do not stablish a connection.
catch (SQLException e) {
e.toString();
log.warn(e.toString());
} catch (Exception e) {
System.out.println("excepcion no capturada ");
e.toString();
}
The second catch is just in case I was missing something, originally it was not there.
All in all, the code looks something like:
#Override
public String handleRequest(Object input, Context context) {
context.getLogger().log("Input: " + input);
Connection con = null;
try {
String jdbcUrl2 = "jdbc:mysql://" + "value-zone.rds.amazonaws.com" + ":" + "3306" + "/" + "botdb" + "?user=" + "bot" + "&password=" + "PW";
con = DriverManager.getConnection(jdbcUrl2);
} catch (SQLException e) {
e.toString();
log.warn(e.toString());
} catch (Exception e) {
System.out.println("excepcion no capturada ");
e.toString();
}
String status = null;
if (con != null) {
status = "conexion establecida";
System.out.println("connection stablished");
} else status = "connection failed";
return status;
}
Edit:
I am now capturing slightly better the exceptions and get the following message:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
At this point I am unsure if the Strin jdbcURL is not properly formed OR why is the connection failing.
Related
I have below code:
public synchronized InputStream getResourceStream(final String name)
throws ResourceNotFoundException
{
if (org.apache.commons.lang.StringUtils.isEmpty(name))
{
throw new ResourceNotFoundException("DataSourceResourceLoader: Template name was empty or null");
}
Connection conn = null;
ResultSet rs = null;
PreparedStatement ps = null;
try
{
conn = openDbConnection();
ps = getStatement(conn, templateColumn, name);
rs = ps.executeQuery();
if (rs.next())
{
InputStream stream = rs.getBinaryStream(templateColumn);
if (stream == null)
{
throw new ResourceNotFoundException("DataSourceResourceLoader: "
+ "template column for '"
+ name + "' is null");
}
return new BufferedInputStream(stream);
}
else
{
throw new ResourceNotFoundException("DataSourceResourceLoader: "
+ "could not find resource '"
+ name + "'");
}
}
catch (SQLException sqle)
{
String msg = "DataSourceResourceLoader: database problem while getting resource '"
+ name + "': ";
log.error(msg, sqle);
throw new ResourceNotFoundException(msg);
}
catch (NamingException ne)
{
String msg = "DataSourceResourceLoader: database problem while getting resource '"
+ name + "': ";
log.error(msg, ne);
throw new ResourceNotFoundException(msg);
}
finally
{
closeResultSet(rs);
closeStatement(ps);
closeDbConnection(conn);
}
}
Above method return type is InputStream. in above i am getting column value as InputStream stream = rs.getBinaryStream(templateColumn); and returning the same.
Now my requirement is i have to retrieve the value of one more column and return in the same stream along with templateColumn. How can i do that?
Basically in above logic i have to add one more line as below.
InputStream stream2 = rs.getBinaryStream(oneMoreColumn);
Now is it possible to return both the values in a single Stream?
Thanks!
You could read both streams completely into a byte[] and return a new ByteArrayInputStream, but that wouldn't make any real sense because the client code wouldn't know what part of the array is your first and which part is the second stream.
Instead change your method to return an object with two InputStream fields.
Introduction
I've found plenty of information about the Too many open files exception on the Web but I couldn't solve this strange case. As I've read, the exception is thrown when the number of opened file descriptors by process defined in the OS is exceeded. The nature of these files is diverse. Files could be sockets, documents, etc. And I've found robust and secure ways to open files that I have implemented in my Java application.
The application is a short program that downloads Web pages using the Boilerpipe algorithm. This way I get the most representative contents of that site. Then, I write it in an appropriate format (TREC format) to disk. The URLs of these websites are taken from a MySQL database that I access using the JDBC connector.
So, I think that the exception can be thrown form three different places:
Connection to database
HTTP Connection to the web sites
Opening and writing files
Although, as I said, I think that I use a correct way of opening and writing those files.
Problem
There are thousands of URL's to process and the exception is thrown after a while (what makes it also very difficult to debug...). I don't know if that matters, but URLs are classified into different categories and I run different instances of the program to speed up the whole process. Categories don't overlap so there shouldn't be any problem.
Code
To make it more readable I'm going to show just those three parts of my code simplified:
Database access
// Connect to database
Connection dbconn = null;
try {
String dbUrl = "jdbc:mysql://" + dbServer + "/" + dbName;
Class.forName ("com.mysql.jdbc.Driver").newInstance ();
dbconn = DriverManager.getConnection(dbUrl, dbUser, dbPass);
System.out.println ("Database connection established");
} catch (Exception e) {
e.printStackTrace();
System.err.println ("Cannot connect to database server");
System.exit(-1);
}
System.out.println(" Downloading category: " + category);
Statement s = null;
try {
s = dbconn.createStatement();
} catch (SQLException e) {
System.err.println ("Error on creating the statement");
System.exit(-1);
e.printStackTrace();
}
String q = "SELECT resource,topic FROM " +
"content_links " +
"WHERE topic LIKE 'Top/" + category + "%';";
try {
s.executeQuery(q);
} catch(Exception e) {
System.err.println ("Error on executing the SQL statement");
System.exit(-1);
e.printStackTrace();
}
ResultSet rs = null;
try {
rs = s.getResultSet ();
} catch (SQLException e) {
System.err.println ("Error on getting the result set");
System.exit(-1);
e.printStackTrace();
}
int count = 0, webError = 0;
// work with the result set
try {
while (rs.next ()) {
// MAIN LOOP
}
} catch (SQLException e) {
System.err.println ("Error on getting next item");
System.exit(-1);
e.printStackTrace();
}
// Close connection to database
if (dbconn != null) {
try {
dbconn.close ();
System.out.println (" Database connection terminated");
} catch (Exception e) { /* ignore close errors */ }
}
HTTP connection, extract site's title and boilerpipe filter
try {
String title = "";
org.jsoup.nodes.Document doc = Jsoup.connect(urlVal).get();
for (Element element : doc.select("*")) {
if (element.tagName().equalsIgnoreCase("title")) {
title = element.text();
}
if (!element.hasText() && element.isBlock()) {
element.remove();
}
}
String contents = "";
contents = NumWordsRulesExtractor.INSTANCE.getText(doc.text());
storeFile(id, urlVal, catVal, title, contents);
}
} catch (BoilerpipeProcessingException e) {
System.err.println("Connection failed to: " + urlVal);
} catch (MalformedURLException e1) {
System.err.println("Malformed URL: " + urlVal);
} catch(Exception e2) {
System.err.println("Exception: " + e2.getMessage());
e2.getStackTrace();
}
Writing file
private static void storeFile(String id, String url, String cat, String title, String contents) {
BufferedWriter out = null;
try {
out = new BufferedWriter(
new OutputStreamWriter(
new FileOutputStream(
new File(path + "/" + id + ".webtrec")),"UTF8"));
// write in TREC format
out.write("...");
} catch (IOException e) {
System.err.println("Error: " + e.getMessage());
e.printStackTrace();
} finally {
try {
out.close();
} catch (IOException e) {
System.err.println("Error: " + e.getMessage());
e.printStackTrace();
}
}
Yup. You are leaking file descriptors.
In the first case you open a DB connection and never close it. The connection will typically use a socket or something to talk to the database. Since you don't close the connection, the socket won't be closed, and you will leak a file descriptor.
In the second case, I suspect that the call to Jsoup.connect(urlVal) is opening a connection, which you don't then close. That will result in a file descriptor leak.
Correction - there is no close() method on the Connection interface. It looks like the actual connection must be created and then closed internally by the get method. Assuming that is so, there is no file descriptor leak in the second case.
The third case does not leak file descriptors. However, if you fail to open the file, out.close(); statement will attempt to call a method on null ... and will throw a NPE.
The solution is to find all of the places where you open files, database connection, http connections, and make sure that the handle is always closed.
One way to do it is to put the close() call (or equivalent) in a finally block ... but make sure that you don't accidentally call close() on null.
The other way to do it is to use the Java 7 "try with resource" syntax. For example:
private static void storeFile(String id, String url, String cat,
String title, String contents) {
try (BufferedWriter out = new BufferedWriter(
new OutputStreamWriter(
new FileOutputStream(
new File(path + "/" + id + ".webtrec")),"UTF8"))) {
// write in TREC format
out.write("...");
out.close();
} catch (IOException e) {
System.err.println("Error: " + e.getMessage());
e.printStackTrace();
}
}
(Note however that the Java 7 syntax can only be used with resources that implement the new Closeable interface.)
To add to Stephen's comprehensive analysis. I recommend using a connection pool for the database, although, as Stephen has pointed, unless you're closing these connections, you'll drain the pool close, but at least it will be easier to discover why...
I've not seen any evidence, but you should be using some kind of Thread pool to download the pages, this will help to maximize the resources of the system. Some of executor service would suffice. Like I say, you're probably already doing this, but you didn't show any code (or comment) for it.
My android program isn't working. I am using normal client-server sockets. I have tested my server with telnet and it works fine, but when I try it with my android program, it doesn't work (more details in a second). Here's my code:
Socket s = null;
try
{
String SocketServerAddress = db.getPhSsServerAddress();
Integer SocketServerPort = db.getPhSsServerPort();
s = new Socket(SocketServerAddress, SocketServerPort);
Log.d(MY_DEBUG_TAG, "Setting up Socket: " + SocketServerAddress + ":" + SocketServerPort);
DataOutputStream out = new DataOutputStream(s.getOutputStream());
DataInputStream in = new DataInputStream(s.getInputStream());
Log.d(MY_DEBUG_TAG, "Connected to: " + s.getInetAddress() + " on port " + s.getPort());
out.writeUTF("Helo, Server");
out.flush();
Log.d(MY_DEBUG_TAG, "Bytes written: " + out.size());
String st = in.readUTF();
Log.d(MY_DEBUG_TAG, "SocketServerResponse: " + st);
}
catch (UnknownHostException e)
{
Log.e(MY_ERROR_TAG, "UnknownHostException: " + e.getMessage() + "; " + e.getCause());
}
catch (IOException e)
{
Log.e(MY_ERROR_TAG, "IOException: " + e.getMessage() + "; " + e.getCause() + "; " + e.getLocalizedMessage());
}
finally
{
try {
s.close();
} catch (IOException e) {
Log.e(MY_ERROR_TAG, "IOException on socket.close(): " + e.getMessage() + "; " + e.getCause());
}
}
All I ever get here is a thrown IOException with no message or cause attached. The specific line causing the error is the String st = in.readUTF(). If I comment out that line, my code runs fine (no exceptions thrown), but my server does not acknowledge that any data has been sent to it. And of course I don't get any data back since that line is commented out.
So, how can I figure out what the problem is? Tonight I am going to try and see what is being passed with wireshark to see if that gives any insight.
Is the server using readUTF() and writeUTF() too? writeUTF() writes data in a unique format that can only be understood by readUTF(), which won't understand anything else.
EDIT EOFException means that there is no more data. You should catch it separately and handle it by closing the socket etc. It can certainly be caused spuriously by readUTF() trying to read data that wasn't written with writeUTF().
And deciding it was an IOException when it was really an EOFException means you didn't print out or log the exception itself, just its message. Always use the log methods provided for exceptions, or at least use Exception.toString().
As I remember I had a problem with DataInpuStream some day... try doing so:
in = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
I'am trying to set a timeout when a client try to connect to a server, if the server is down, the connection will wait 10 sec befor throwing the timeout exception.
In my case the code bellow throw the IOException without waiting, I really don't get it !
public boolean establishConnection()
{
System.out.println ("Connecting to " +
this.getServerHostname() + " au port " + this.getServerPort()+ " ...");
try {
SocketAddress sockaddr= new InetSocketAddress(_serverHostname, _serverPort);
_echoSocket = new Socket();
_echoSocket.connect(sockaddr,10000);
return _echoSocket.isConnected();
} catch (UnknownHostException e) {
System.err.println("Unknown Host: " + this.getServerHostname());
return false;
} catch (SocketTimeoutException e) {
System.err.println("Timeout");
return false;
} catch (IOException e) {
System.err.println("IOException : " +
this.getServerHostname() + ":" + this.getServerPort());
return false;
}
}
You'll only get a timeout if your connection request is not answered. If the server immediately rejects it, or if the server doesn't exist, you'll get an exception immediately.
But is this what you want to achieve?
If your intention is, in case the server is temporarily down, then try again after 10 sec then your approach is wrong.
You should try to do a connection to the server and if you get an exception because the server is down, you can sleep for 10 seconds and try the request again.
Otherwise Ernest's answer is correct
Good afternoon, I wrote a project to Get Park Queue Info from the IBM MQ, it has producing an error when attempting to close the connection though. It is written in java.
Under application in Event Viewer on the MQ machine it displays two errors. They are:
“Channel program ended abnormally.
Channel program ‘system.def.surconn’ ended abnormally. Look at previous error messages for channel program ‘system.def.surconn’ in the error files to determine the cause of the failure.
The other message states:
“Error on receive from host rnanaj (10.10.12.34)
An error occurred receiving data from rnanaj (10.10.12.34) over tcp/ip. This may be due to a communications failure. The return code from tcp/ip recv() call was 10054 (X’2746’). Record these values.”
This must be something how I try to connect or close the connection, below I have my code to connect and close, any ideas??
Connect:
_logger.info("Start");
File outputFile = new File(System.getProperty("PROJECT_HOME"), "run/" + this.getClass().getSimpleName() + "." + System.getProperty("qmgr") + ".txt");
FileUtils.mkdirs(outputFile.getParentFile());
Connection jmsConn = null;
Session jmsSession = null;
QueueBrowser queueBrowser = null;
BufferedWriter commandsBw = null;
try {
// get queue connection
MQConnectionFactory MQConn = new MQConnectionFactory();
MQConn.setHostName(System.getProperty("host"));
MQConn.setPort(Integer.valueOf(System.getProperty("port")));
MQConn.setQueueManager(System.getProperty("qmgr"));
MQConn.setChannel("SYSTEM.DEF.SVRCONN");
MQConn.setTransportType(JMSC.MQJMS_TP_CLIENT_MQ_TCPIP);
jmsConn = (Connection) MQConn.createConnection();
jmsSession = jmsConn.createSession(false, Session.AUTO_ACKNOWLEDGE);
Queue jmsQueue = jmsSession.createQueue("PARK");
// browse thru messages
queueBrowser = jmsSession.createBrowser(jmsQueue);
Enumeration msgEnum = queueBrowser.getEnumeration();
commandsBw = new BufferedWriter(new FileWriter(outputFile));
//
String line = "DateTime\tMsgID\tOrigMsgID\tCorrelationID\tComputerName\tSubsystem\tDispatcherName\tProcessor\tJobID\tErrorMsg";
commandsBw.write(line);
commandsBw.newLine();
while (msgEnum.hasMoreElements()) {
Message message = (Message) msgEnum.nextElement();
line = dateFormatter.format(new Date(message.getJMSTimestamp()))
+ "\t" + message.getJMSMessageID()
+ "\t" + message.getStringProperty("pkd_orig_jms_msg_id")
+ "\t" + message.getJMSCorrelationID()
+ "\t" + message.getStringProperty("pkd_computer_name")
+ "\t" + message.getStringProperty("pkd_subsystem")
+ "\t" + message.getStringProperty("pkd_dispatcher_name")
+ "\t" + message.getStringProperty("pkd_processor")
+ "\t" + message.getStringProperty("pkd_job_id")
+ "\t" + message.getStringProperty("pkd_sysex_msg");
_logger.info(line);
commandsBw.write(line);
commandsBw.newLine();
}
}
Close:
finally {
IO.close(commandsBw);
if (queueBrowser != null) { try { queueBrowser.close();} catch (Exception ignore) {}}
if (jmsSession != null) { try { jmsSession.close();} catch (Exception ignore) {}}
if (jmsConn != null) { try { jmsConn.stop();} catch (Exception ignore) {}}
}
As per the Javadoc for the connection object, the function of the stop() method is...
Temporarily stops a connection's
delivery of incoming messages.
So stop() doesn't actually sever the connection. You want the close() method.