Develop Reference

AWS Lambda Java, connect to MySQL RDS

I need to develop an AWS Lambda Java function to retrieve some records from RDS MySQL database.
Should I use JDBC? Should I use the standard JDBC example:
try {
String url = "jdbc:msql://200.210.220.1:1114/Demo";
Connection conn = DriverManager.getConnection(url,"","");
Statement stmt = conn.createStatement();
ResultSet rs;
rs = stmt.executeQuery("SELECT Lname FROM Customers WHERE Snum = 2001");
while ( rs.next() ) {
String lastName = rs.getString("Lname");
System.out.println(lastName);
}
conn.close();
} catch (Exception e) {
System.err.println("Got an exception! ");
System.err.println(e.getMessage());
}

Step 1:
login IAM console
roles -> create new roles
role name :lambda-vpc-execution-role
AWS service roles ->
a) Select aws lambda
b) Attach policy "AWSLambdaFullAccess"
Step 2:
Get code from https://github.com/vinayselvaraj/lambda-jdbc-sample (note this is maven project)
Right click on project select Run as --->5.maven build...
for goal provide name package shade:shade
Go to project folder and target/lamda-0.0.1-SNAPSHOT-shaded.jar
Step 3:
Login to lambda console(skip blueprint)
Create new lambda
name: time-test
a) runtime-java
b) upload .zip(.jar) file (target/lamda-0.0.1-SNAPSHOT-shaded.jar)
Provide package.class-name::myhandler -> Handler
Role -> lambda-vpc-exceution-role
vpc provide rds-vpc details (this should work in same vpc group)
Create function
In the Action drop down list select configure test event
result will shown down like this "Execution result: succeeded (logs)"

Yes, you need to use standard JDBC code in your lambda function class. The code you provided looks okay. There are a few more things you need to do when accessing RDS or any other RDBMS through a Lamda function -
Create a jar or a zip file for your Lambda function
Your zip file needs to have a lib folder in which your JDBC driver file goes. The Lambda function doc says this is one of the two standard ways, but it didn't work for me.
You can create a jar file in which the driver classes are put in. This works. The best way to do it is through the Maven Shade plugin, which extracts the JDBC drivers and packs the classes in one single jar file.
Setup the handler function and specify it at the time of Lambda deployment
Define execution role and VPC as needed.
Upload and publish your jar or zip file.
You can test the Lambda function through the console, and see the actual output in the CloudWatch logs.

You could use this kinda implementation:
public static DataSource getDataSource(){
Utils._logger.log("Get data source");
MysqlDataSource mysqlDs = null;
try{
mysqlDs = new MysqlDataSource();
mysqlDs.setURL('jdbc:msql://'+'url');
mysqlDs.setUser('user');
mysqlDs.setPassword('pwd');
Utils._logger.log("Object "+mysqlDs.getUrl()+" "+mysqlDs.getUser()+" ");
return mysqlDs;
}
catch(Exception e) {
Utils._logger.log("No se pudo abrir el archivo de properties");
e.printStackTrace();
}
return mysqlDs;
}

One thing I am particularly noticing in your codebase is that even when you use this Lambda function for connecting to the specific RDS you have, the hostname may not be the correct one for Amazon RDS.
It needs to be the endpoint of the RDS you are trying to connect to and your complete connection url would look something like this below -
//jdbc:mysql://hostname (endpoint of RDS):port/databasename
String url = "jdbc:mysql://"+dbHost+":3306/"+dbName;
Since those endpoints can change for different databases and servers, you can make them as environment variables within Lambda and refer using
String dbHost = System.getenv("dbHost");
String dbName = System.getenv("dbName");
for a much cleaner and stateless design that Lambda supports.

Connecting to MongoDB using jdbc driver

Purpose is to connect MongoDB remote server through JAVA:
URL = "jdbc:mongo://" + serverIP + ":"
+ port+ "/" +databaseName;
Class.forName("mongodb.jdbc.MongoDriver");
dbConn = getConnection(URL,mongo1, mongo1);
Tried Unity_trial.Jar, mongo_version.jar files but the error comes is 'mongodb.jdbc.MongoDriver' classNameNotFound.
If I comment the class.forname line, the next error is
URL = "jdbc:mongo://" + serverIP + ":" + port
+ "/" +databaseName;
is not in correct format.
Not sure about where I am making the mistake.
Thanks for your help in advance.

You can checkout this project:
https://github.com/erh/mongo-jdbc
There are two examples given.
But in general I would recommend to use the MongoDB Client or some Spring Data abstraction.

If you are getting a ClassNotFoundException, the issue is that the jar containing the mongodb.jdbc.MongoDriver class is not on your classpath. If you're not sure what JAR this class is in, I would reccomend getting 7-Zip so that you can inspect the contents of the jar and see for yourself if the class is there.
The correct way to connect to MongoDB with your approach is:
Class.forName("mongodb.jdbc.MongoDriver");
String URL = "jdbc:mongo://<servername>:<port>/<databaseName>";
Connection jdbcConn = DriverManager.getConnection(url,"user","pass");
But MongoDB isn't really meant to be used with JDBC, so if your requirements allow, I would reccomend getting a connection the "mongodb" way.
MongoClient client = new MongoClient("localhost");
For details on how to do it this way, see the MongoDB docs

I know its very late to answer but might help someone else. If you are compiling and running your code from cmd then before compilation set classpath for mongo.jar like below :
set classpath=C:\DemoProject\java db\Mongo\mongo.jar;
then run your code.
or if you are using editor like eclipse then add this jar to your lib folder.

I met this question today morning.
The key is missing mongo-java-driver.jar.
when I add the jar, the project can run normal.

DbSchema database designer is providing an Open Source MongoDb JDBC driver which does support native MongoDb queries, including find(), projections, aggregate, etc..
The driver is using an internal embedded JavaScript engine.
The driver is Open Source on GitHub.
Few of the driver features:
Support native MongoDb queries
Calling DatabaseMetaData methods can 'guess' the collection structure, so a 'virtual schema' is created. This is used by the Designer for MongoDB to represent the MongoDb database structure in diagrams like below.
Implement most of the JDBC driver methods. Use the native MongoDB JDBC URL to connect, which means full functionality regarding connectivity.
And one snippet of code about how to use the driver
Class.forName("com.dbschema.MongoDbJdbcDriver");
Properties properties = new Properties();
properties.put("user", "someuser");
properties.put("password", "somepassword" );
Connection con = DriverManager.getConnection("jdbc:mongodb://host1:9160/keyspace1", properties);
// OTHER URL (SAME AS FOR MONGODB NATIVE DRIVER): mongodb://db1.example.net,db2.example.net:2500/?replicaSet=test&connectTimeoutMS=300000
String query = "db.sampleCollection().find()";
Statement statement = con.createStatement();
ResultSet rs = statement.executeQuery( query );
Object json = rs.getObject(1);

The first option
MongoClient mongoClient = new MongoClient( "1.2.3.4",27017 );
MongoDatabase database = mongoClient.getDatabase(dataBase);
MongoCollection<Document> collection = database.getCollection(DBcollection);
another option
MongoClientURI connectionString = new MongoClientURI("mongodb://1.2.3.4:27017");
MongoClient mongoClient = new MongoClient(connectionString);
MongoDatabase database = mongoClient.getDatabase(dataBase);
MongoCollection collection = database.getCollection(DBcollection);

Jython and standalone DB connection pool: How to confirm the pool is a Singleton

Question: Can you use the zxJDBC.connectx method, outside of an application server container?
I have my own server application in Jython, and I'm looking to upgrade to using a database connection pool (since I'm building and destroying individual connections manually, at this point). I have found some sample code and have gotten it to work (using Tomcat's connection pool), but something about the way it works bothers me. To me, it looks like the pool is getting created over and over again. Here's my working example:
from __future__ import with_statement
from com.ziclix.python.sql import zxJDBC
params = { }
params['url'] = 'jdbc:mysql://localhost:3306/my_database'
params['driverClassName'] = 'com.mysql.jdbc.Driver'
params['username'] = 'mario'
params['password'] = 'myP#ssw0rd'
params['validationQuery'] = 'SELECT 1'
params['jdbcInterceptors'] = \
'org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;' + \
'org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer'
# This is the line that worries me!
conn = zxJDBC.connectx('org.apache.tomcat.jdbc.pool.DataSource', **params)
with conn.cursor() as cursor:
cursor.execute('SELECT * FROM MyTable')
data = cursor.fetchall()
print data
conn.close()
Take a look at "the line that worries me." I would like to use the zxJDBC connection object, but if each time I obtain it I have to give the DataSource class name along with the setup parameters, then the first thing I think is that a connection pool is being created anew each time. That's obviously not what I want.
Does anyone know for sure what's going on, or how I might go about confirming what's going on—with a little experimentation, perhaps? Am I supposed to duplicate the JNDI infrastructure of a servlet container or something if I want to use this in my server? Am I failing to understand what exactly a DataSource is and how it works? I don't know how to get to the bottom of it. Thanks!
Edit: Jython source code
I took a look at the Jython source code. The connectx method is backed by the com.ziclix.python.sql.connect.Connectx class. The relevant snippet looks like this:
/**
* Construct a javax.sql.DataSource or javax.sql.ConnectionPooledDataSource
*/
#Override
public PyObject __call__(PyObject[] args, String[] keywords) {
Connection c = null;
PyConnection pc = null;
Object datasource = null;
PyArgParser parser = new PyArgParser(args, keywords);
try {
String klass = (String) parser.arg(0).__tojava__(String.class);
datasource = Class.forName(klass).newInstance();
} catch (Exception e) {
throw zxJDBC.makeException(zxJDBC.DatabaseError, "unable to instantiate datasource");
}
/*
* The code continues on, setting up the connection pool's parameters,
* handling errors, etc., and obtaining a connection (variable: c).
*/
try {
if (c == null || c.isClosed()) {
throw zxJDBC.makeException(zxJDBC.DatabaseError, "unable to establish connection");
}
pc = new PyConnection(c);
} catch (SQLException e) {
throw zxJDBC.makeException(zxJDBC.DatabaseError, e);
}
return pc;
}
Building Jython from source isn't all that easy (there appear to be not well-documented dependencies), or I would put in some debugging statements to compare the datasource objects. But when I try to duplicate the creation part on my own...
datasource = Class.forName(klass).newInstance();
...it looks to me like unique DataSource instances (and therefore, presumably, unique pool instances) are being created with each call.
Does anyone have any experience with Jython and know for sure? Thanks.

Based on my research, I have come up with the following solution.
Instead of using the more direct approach shown in most Jython example code, I instantiate a DataSource using Java methods. (I took this from example code on the Apache Tomcat site.) Then, instead of calling a zxJDBC method to obtain a connection object, I make use of what that method relies on: namely, the com.ziclix.python.sql.PyConnection class.
The result is a traditional Java connection object drawn from a pool, which is then wrapped in a PyConnection object, allowing for the convenience of a Jython connection and its cursor object.
from __future__ import with_statement
from com.ziclix.python.sql import PyConnection
import org.apache.tomcat.jdbc.pool as pool
# https://people.apache.org/~fhanik/jdbc-pool/jdbc-pool.html
p = pool.PoolProperties()
p.setUrl('jdbc:mysql://localhost:3306/my_database')
p.setDriverClassName('com.mysql.jdbc.Driver')
p.setUsername('mario')
p.setPassword('myP#ssw0rd')
p.setValidationQuery("SELECT 1")
p.setJdbcInterceptors('org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;' +
'org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer')
datasource = pool.DataSource()
datasource.setPoolProperties(p)
# http://www.jython.org/javadoc/com/ziclix/python/sql/PyConnection.html
conn = PyConnection(datasource.getConnection())
with conn.cursor() as cursor:
cursor.execute('SELECT * FROM MyTable')
data = cursor.fetchall()
print data
conn.close()
I have more confidence in this approach. Is there any reason why I shouldn't?

DriverManager always returns my custom driver regardless of the connection URL

I am writing a driver to act as a wrapper around two separate MySQL connections (to distributed databases). Basically, the goal is to enable interaction with my driver for all applications instead of requiring the application to sort out which database holds the desired data.
Most of the code for this is in place, but I'm having a problem in that when I attempt to create connections via the MySQL Driver, the DriverManager is returning an instance of my driver instead of the MySQL Driver. I'd appreciate any tips on what could be causing this and what could be done to fix it!
Below is a few relevant snippets of code. I can provide more, but there's a lot, so I'd need to know what else you want to see.
First, from MyDriver.java:
public MyDriver() throws SQLException
{
DriverManager.registerDriver(this);
}
public Connection connect(String url, Properties info)
throws SQLException
{
try { return new MyConnection(info); }
catch (Exception e) { return null; }
}
public boolean acceptsURL(String url)
throws SQLException
{
if (url.contains("jdbc:jgb://"))
{ return true; }
return false;
}
It is my understanding that this acceptsURL function will dictate whether or not the DriverManager deems my driver a suitable fit for a given URL. Hence it should only be passing connections from my driver if the URL contains "jdbc:jgb://" right?
Here's code from MyConnection.java:
Connection c1 = null;
Connection c2 = null;
/**
*Constructors
*/
public DDBSConnection (Properties info)
throws SQLException, Exception
{
info.list(System.out); //included for testing
Class.forName("com.mysql.jdbc.Driver").newInstance();
String url1 = "jdbc:mysql://server1.com/jgb";
String url2 = "jdbc:mysql://server2.com/jgb";
this.c1 = DriverManager.getConnection(
url1, info.getProperty("username"), info.getProperty("password"));
this.c2 = DriverManager.getConnection(
url2, info.getProperty("username"), info.getProperty("password"));
}
And this tells me two things. First, the info.list() call confirms that the correct user and password are being sent. Second, because we enter an infinite loop, we see that the DriverManager is providing new instances of my connection as matches for the mysql URLs instead of the desired mysql driver/connection.
FWIW, I have separately tested implementations that go straight to the mysql driver using this exact syntax (al beit only one at a time), and was able to successfully interact with each database individually from a test application outside of my driver.

IMO, the main problem with this code is that it uses DriverManager. Avoid statics and stick with instances.
The specific problem is the DriverManager.getConnection goes directly to attempting to connect rather than the acceptsURL that getDriver does. Therefore, you connect implementation should do the same check as the acceptsURL implementation (it may even be stricter and possibly fail at runtime).
As a relatively minor point, the acceptsURL implementation is a little odd.
if (url.contains("jdbc:jgb://"))
{ return true; }
return false;
contains should be startsWith. The little dance with if and returns does not help clarity. It can be written as:
return url.startsWith("jdbc:jgb://");

integrate ms access and mysql in java

I have a problem connecting to MS Access and MySQL using Java. My problem is that I cannot find the driver for MySQL. Here is my code:
<%# page import="java.sql.*" %>
<%
Connection odbcconn = null;
Connection jdbcconn = null;
PreparedStatement readsms = null;
PreparedStatement updsms = null;
ResultSet rsread = null;
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver"); //load database driver
odbcconn = DriverManager.getConnection("jdbc:odbc:SMS"); //connect to database
readsms = odbcconn.prepareStatement("select * from inbox where Status='New'");
rsread = readsms.executeQuery();
while(rsread.next()){
Class.forName("com.mysql.jdbc.Driver");
jdbcconn = DriverManager.getConnection("jdbc:mysql://localhost:3306/bakme", "root", ""); //connect to database
updsms = jdbcconn.prepareStatement("insert into inbox(sms,phone) values (?,?)");
updsms.setString(1, rsread.getString("Message"));
updsms.setString(2, rsread.getString("Phone"));
updsms.executeUpdate();
}
%>

Thus, you get a ClassNotFoundException on the MySQL JDBC driver class? Then you need to put the MySQL JDBC driver JAR file containing that class in the classpath. In case of a JSP/Servlet application, the classpath covers under each the webapplication's /WEB-INF/lib folder. Just drop the JAR file in there. Its JDBC driver is also known as Connector/J. You can download it here.
That said, that's really not the way how to use JDBC and JSP together. This doesn't belong in a JSP file. You should be doing this in a real Java class. The JDBC code should also be more robust written, now it's leaking resources.

BalusC is spot on: this is not the way you should write something like this.
Connection, Statement, and ResultSet all represent finite resources. They are not like memory allocation; the garbage collector does not clean them up. You have to do that in your code, like this:
// Inside a method
Connection connection = null;
Statement statement = null;
ResultSet rs = null;
try
{
// interact with the database using connection, statement, and rs
}
finally
{
// clean up resources in a finally block using static methods,
// called in reverse order of creation, that don't throw exceptions
close(rs);
close(statement);
close(connection);
}
But if you do decide to move this to a server-side component, you're bound to have huge problems with code like this:
while(rsread.next())
{
Class.forName("com.mysql.jdbc.Driver");
jdbcconn = DriverManager.getConnection("jdbc:mysql://localhost:3306/bakme", "root", ""); //connect to database
updsms = jdbcconn.prepareStatement("insert into inbox(sms,phone) values (?,?)");
updsms.setString(1, rsread.getString("Message"));
updsms.setString(2, rsread.getString("Phone"));
updsms.executeUpdate();
}
Registering the driver, creating a connection, without closing it, and repeatedly preparing a statement inside a loop for every row that you get out of Access shows a serious misunderstanding of relational databases and JDBC.
You should register the driver and create connections once, do what needs to be done, and clean up all the resources you've used.
If you absolutely MUST do this in a JSP, I'd recommend using JNDI data sources for both databases so you don't have to set up connections inside the page. You should not be writing scriptlet code - better to learn JSTL and use its <sql> tags.

You can use this link to download the MySql Driver. Once you download it, you need to make sure it is on the class path for the web server that you are using. The particulars of configuring JDBC drivers for a server vary from server to server. You may want to edit your question to include more details to get a better answer.