AWS Lambda Java, connect to MySQL RDS - java

I need to develop an AWS Lambda Java function to retrieve some records from RDS MySQL database.
Should I use JDBC? Should I use the standard JDBC example:
try {
String url = "jdbc:msql://200.210.220.1:1114/Demo";
Connection conn = DriverManager.getConnection(url,"","");
Statement stmt = conn.createStatement();
ResultSet rs;
rs = stmt.executeQuery("SELECT Lname FROM Customers WHERE Snum = 2001");
while ( rs.next() ) {
String lastName = rs.getString("Lname");
System.out.println(lastName);
}
conn.close();
} catch (Exception e) {
System.err.println("Got an exception! ");
System.err.println(e.getMessage());
}

Step 1:
login IAM console
roles -> create new roles
role name :lambda-vpc-execution-role
AWS service roles ->
a) Select aws lambda
b) Attach policy "AWSLambdaFullAccess"
Step 2:
Get code from https://github.com/vinayselvaraj/lambda-jdbc-sample (note this is maven project)
Right click on project select Run as --->5.maven build...
for goal provide name package shade:shade
Go to project folder and target/lamda-0.0.1-SNAPSHOT-shaded.jar
Step 3:
Login to lambda console(skip blueprint)
Create new lambda
name: time-test
a) runtime-java
b) upload .zip(.jar) file (target/lamda-0.0.1-SNAPSHOT-shaded.jar)
Provide package.class-name::myhandler -> Handler
Role -> lambda-vpc-exceution-role
vpc provide rds-vpc details (this should work in same vpc group)
Create function
In the Action drop down list select configure test event
result will shown down like this "Execution result: succeeded (logs)"

Yes, you need to use standard JDBC code in your lambda function class. The code you provided looks okay. There are a few more things you need to do when accessing RDS or any other RDBMS through a Lamda function -
Create a jar or a zip file for your Lambda function
Your zip file needs to have a lib folder in which your JDBC driver file goes. The Lambda function doc says this is one of the two standard ways, but it didn't work for me.
You can create a jar file in which the driver classes are put in. This works. The best way to do it is through the Maven Shade plugin, which extracts the JDBC drivers and packs the classes in one single jar file.
Setup the handler function and specify it at the time of Lambda deployment
Define execution role and VPC as needed.
Upload and publish your jar or zip file.
You can test the Lambda function through the console, and see the actual output in the CloudWatch logs.

You could use this kinda implementation:
public static DataSource getDataSource(){
Utils._logger.log("Get data source");
MysqlDataSource mysqlDs = null;
try{
mysqlDs = new MysqlDataSource();
mysqlDs.setURL('jdbc:msql://'+'url');
mysqlDs.setUser('user');
mysqlDs.setPassword('pwd');
Utils._logger.log("Object "+mysqlDs.getUrl()+" "+mysqlDs.getUser()+" ");
return mysqlDs;
}
catch(Exception e) {
Utils._logger.log("No se pudo abrir el archivo de properties");
e.printStackTrace();
}
return mysqlDs;
}

One thing I am particularly noticing in your codebase is that even when you use this Lambda function for connecting to the specific RDS you have, the hostname may not be the correct one for Amazon RDS.
It needs to be the endpoint of the RDS you are trying to connect to and your complete connection url would look something like this below -
//jdbc:mysql://hostname (endpoint of RDS):port/databasename
String url = "jdbc:mysql://"+dbHost+":3306/"+dbName;
Since those endpoints can change for different databases and servers, you can make them as environment variables within Lambda and refer using
String dbHost = System.getenv("dbHost");
String dbName = System.getenv("dbName");
for a much cleaner and stateless design that Lambda supports.

Related

Connection to oracle database

Hi I am new enough to java and I am trying to create a connection from it to my sql database. using Netbeans I managed to set up a connection easily enough. The issue is when I try to connect using my code I get driver not found. Is there something wrong with what I have?
//function to execute the insert update delete query
public void theQuery(String query){
Connection con = null;
Statement st = null;
try{
con = DriverManager.getConnection("jdbc:oracle:thin:#redwood.ict.ad.dit.ie:1521:pdb12c.ict.ad.dit.ie", "eocribin","");
st = con.createStatement();
st.executeUpdate(query);
JOptionPane.showMessageDialog(null,"Query Executed");
}catch(Exception ex){
JOptionPane.showMessageDialog(null,ex.getMessage());
}
}
you have to add oracle thin driver into your projects library and then restart the IDE to take notice of changes. Make sure that oracle's server is running when you run this code. Also check the username and password.
Java Persistence in netbeans is a much better option then using Oracle thin driver in your code. Because it saves much of your time typing and harcoding. JPA(Java Persistence API) allows you to easily add the oracle's thin driver from its menu and reduces much of your time. It is used in enterprise applications . You should try this and watch its tutorials on the following site.
These are called JPA tutorials.
http://www.programming-simplified.com

Connecting to MongoDB using jdbc driver

Purpose is to connect MongoDB remote server through JAVA:
URL = "jdbc:mongo://" + serverIP + ":"
+ port+ "/" +databaseName;
Class.forName("mongodb.jdbc.MongoDriver");
dbConn = getConnection(URL,mongo1, mongo1);
Tried Unity_trial.Jar, mongo_version.jar files but the error comes is 'mongodb.jdbc.MongoDriver' classNameNotFound.
If I comment the class.forname line, the next error is
URL = "jdbc:mongo://" + serverIP + ":" + port
+ "/" +databaseName;
is not in correct format.
Not sure about where I am making the mistake.
Thanks for your help in advance.
You can checkout this project:
https://github.com/erh/mongo-jdbc
There are two examples given.
But in general I would recommend to use the MongoDB Client or some Spring Data abstraction.
If you are getting a ClassNotFoundException, the issue is that the jar containing the mongodb.jdbc.MongoDriver class is not on your classpath. If you're not sure what JAR this class is in, I would reccomend getting 7-Zip so that you can inspect the contents of the jar and see for yourself if the class is there.
The correct way to connect to MongoDB with your approach is:
Class.forName("mongodb.jdbc.MongoDriver");
String URL = "jdbc:mongo://<servername>:<port>/<databaseName>";
Connection jdbcConn = DriverManager.getConnection(url,"user","pass");
But MongoDB isn't really meant to be used with JDBC, so if your requirements allow, I would reccomend getting a connection the "mongodb" way.
MongoClient client = new MongoClient("localhost");
For details on how to do it this way, see the MongoDB docs
I know its very late to answer but might help someone else. If you are compiling and running your code from cmd then before compilation set classpath for mongo.jar like below :
set classpath=C:\DemoProject\java db\Mongo\mongo.jar;
then run your code.
or if you are using editor like eclipse then add this jar to your lib folder.
I met this question today morning.
The key is missing mongo-java-driver.jar.
when I add the jar, the project can run normal.
DbSchema database designer is providing an Open Source MongoDb JDBC driver which does support native MongoDb queries, including find(), projections, aggregate, etc..
The driver is using an internal embedded JavaScript engine.
The driver is Open Source on GitHub.
Few of the driver features:
Support native MongoDb queries
Calling DatabaseMetaData methods can 'guess' the collection structure, so a 'virtual schema' is created. This is used by the Designer for MongoDB to represent the MongoDb database structure in diagrams like below.
Implement most of the JDBC driver methods. Use the native MongoDB JDBC URL to connect, which means full functionality regarding connectivity.
And one snippet of code about how to use the driver
Class.forName("com.dbschema.MongoDbJdbcDriver");
Properties properties = new Properties();
properties.put("user", "someuser");
properties.put("password", "somepassword" );
Connection con = DriverManager.getConnection("jdbc:mongodb://host1:9160/keyspace1", properties);
// OTHER URL (SAME AS FOR MONGODB NATIVE DRIVER): mongodb://db1.example.net,db2.example.net:2500/?replicaSet=test&connectTimeoutMS=300000
String query = "db.sampleCollection().find()";
Statement statement = con.createStatement();
ResultSet rs = statement.executeQuery( query );
Object json = rs.getObject(1);
The first option
MongoClient mongoClient = new MongoClient( "1.2.3.4",27017 );
MongoDatabase database = mongoClient.getDatabase(dataBase);
MongoCollection<Document> collection = database.getCollection(DBcollection);
another option
MongoClientURI connectionString = new MongoClientURI("mongodb://1.2.3.4:27017");
MongoClient mongoClient = new MongoClient(connectionString);
MongoDatabase database = mongoClient.getDatabase(dataBase);
MongoCollection collection = database.getCollection(DBcollection);

Jython and standalone DB connection pool: How to confirm the pool is a Singleton

Question: Can you use the zxJDBC.connectx method, outside of an application server container?
I have my own server application in Jython, and I'm looking to upgrade to using a database connection pool (since I'm building and destroying individual connections manually, at this point). I have found some sample code and have gotten it to work (using Tomcat's connection pool), but something about the way it works bothers me. To me, it looks like the pool is getting created over and over again. Here's my working example:
from __future__ import with_statement
from com.ziclix.python.sql import zxJDBC
params = { }
params['url'] = 'jdbc:mysql://localhost:3306/my_database'
params['driverClassName'] = 'com.mysql.jdbc.Driver'
params['username'] = 'mario'
params['password'] = 'myP#ssw0rd'
params['validationQuery'] = 'SELECT 1'
params['jdbcInterceptors'] = \
'org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;' + \
'org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer'
# This is the line that worries me!
conn = zxJDBC.connectx('org.apache.tomcat.jdbc.pool.DataSource', **params)
with conn.cursor() as cursor:
cursor.execute('SELECT * FROM MyTable')
data = cursor.fetchall()
print data
conn.close()
Take a look at "the line that worries me." I would like to use the zxJDBC connection object, but if each time I obtain it I have to give the DataSource class name along with the setup parameters, then the first thing I think is that a connection pool is being created anew each time. That's obviously not what I want.
Does anyone know for sure what's going on, or how I might go about confirming what's going on—with a little experimentation, perhaps? Am I supposed to duplicate the JNDI infrastructure of a servlet container or something if I want to use this in my server? Am I failing to understand what exactly a DataSource is and how it works? I don't know how to get to the bottom of it. Thanks!
Edit: Jython source code
I took a look at the Jython source code. The connectx method is backed by the com.ziclix.python.sql.connect.Connectx class. The relevant snippet looks like this:
/**
* Construct a javax.sql.DataSource or javax.sql.ConnectionPooledDataSource
*/
#Override
public PyObject __call__(PyObject[] args, String[] keywords) {
Connection c = null;
PyConnection pc = null;
Object datasource = null;
PyArgParser parser = new PyArgParser(args, keywords);
try {
String klass = (String) parser.arg(0).__tojava__(String.class);
datasource = Class.forName(klass).newInstance();
} catch (Exception e) {
throw zxJDBC.makeException(zxJDBC.DatabaseError, "unable to instantiate datasource");
}
/*
* The code continues on, setting up the connection pool's parameters,
* handling errors, etc., and obtaining a connection (variable: c).
*/
try {
if (c == null || c.isClosed()) {
throw zxJDBC.makeException(zxJDBC.DatabaseError, "unable to establish connection");
}
pc = new PyConnection(c);
} catch (SQLException e) {
throw zxJDBC.makeException(zxJDBC.DatabaseError, e);
}
return pc;
}
Building Jython from source isn't all that easy (there appear to be not well-documented dependencies), or I would put in some debugging statements to compare the datasource objects. But when I try to duplicate the creation part on my own...
datasource = Class.forName(klass).newInstance();
...it looks to me like unique DataSource instances (and therefore, presumably, unique pool instances) are being created with each call.
Does anyone have any experience with Jython and know for sure? Thanks.
Based on my research, I have come up with the following solution.
Instead of using the more direct approach shown in most Jython example code, I instantiate a DataSource using Java methods. (I took this from example code on the Apache Tomcat site.) Then, instead of calling a zxJDBC method to obtain a connection object, I make use of what that method relies on: namely, the com.ziclix.python.sql.PyConnection class.
The result is a traditional Java connection object drawn from a pool, which is then wrapped in a PyConnection object, allowing for the convenience of a Jython connection and its cursor object.
from __future__ import with_statement
from com.ziclix.python.sql import PyConnection
import org.apache.tomcat.jdbc.pool as pool
# https://people.apache.org/~fhanik/jdbc-pool/jdbc-pool.html
p = pool.PoolProperties()
p.setUrl('jdbc:mysql://localhost:3306/my_database')
p.setDriverClassName('com.mysql.jdbc.Driver')
p.setUsername('mario')
p.setPassword('myP#ssw0rd')
p.setValidationQuery("SELECT 1")
p.setJdbcInterceptors('org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;' +
'org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer')
datasource = pool.DataSource()
datasource.setPoolProperties(p)
# http://www.jython.org/javadoc/com/ziclix/python/sql/PyConnection.html
conn = PyConnection(datasource.getConnection())
with conn.cursor() as cursor:
cursor.execute('SELECT * FROM MyTable')
data = cursor.fetchall()
print data
conn.close()
I have more confidence in this approach. Is there any reason why I shouldn't?

Use Database in java application

I am beginner in Java application programming.
I've created a database application in Java. I use an MS access database with the JDBC-ODBC driver. My application's create-connection code is below:
private void connection() {
try {
String driverurl = "jdbc:odbc:dharti_data";
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
con = DriverManager.getConnection(driverurl,"","");
} catch (SQLException e) {
JOptionPane.showMessageDialog(frm,e.getSQLState(),"Database Access Error",JOptionPane.ERROR_MESSAGE);
} catch (Exception e) {
JOptionPane.showMessageDialog(null,e.getMessage(),"Database Access Error",JOptionPane.ERROR_MESSAGE);
}
}
This code works perfectly, but this code uses a datasource name I declared in Control Panel > Administrative Tools > Data Sources (ODBC) > System DSN > Add Data Source, with a Microsoft Access Driver (*.mdb).
But when I run the application on another PC, it can't run and instead it generates a database error.
I know that I can declare a driver in Data Sources (ODBC) > System DSN, and then it will run. But I don't want to do this on every machine I run my application on. My application should be able to pick up the database connection automatically. How can I make my application not require a data-source name?
String filename = "C:/Lab/northwind.mdb"; // this the path to mdb file
String database = "jdbc:odbc:Driver={Microsoft Access Driver (*.mdb)};DBQ=";
database+= filename.trim() + ";DriverID=22;READONLY=true}"; // add on to the end
// now we can get the connection from the DriverManager
Connection con = DriverManager.getConnection( database ,"","");
Im not sure if I got this, but are you shipping the jdbc driver along with your application? It has to be in your classpath and needs to be deployed with your application.
You will have to programmatically modify these registry sections:
HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI and
HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\ODBC
Ice Engineering offer a public domain api that allows you to do that. Beside jars it has a DLL that you have to ship with your application. It is fairly straight forward and will work.
In order to get a better idea of what you have to do, use regedit in order to see the values before installing anything, then install an ODBC database manually, finally compare the new values with the old.
I used the sun.jdbc.odbc.JdbcOdbcDriver to connect to a MS Access database. Have that in the same directory as the class file and it should work. Although it should come already installed in the Java SDK.
This is a sample of a practice program I made a while ago.
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
System.out.println("Driver loaded");
// Establish a connection
Connection connection = DriverManager.getConnection
("jdbc:odbc:Driver={Microsoft Access Driver (*.mdb)};DBQ=(MS ACCESS DATABASE DIRECTORY)");
System.out.println("Database connected");
// Create a statement
Statement statement = connection.createStatement();
// Execute a statement
ResultSet resultSet = statement.executeQuery
("select f_name, l_name from Test where f_name = 'Luke'"); // For example
// Iterate through the result and print the results
while (resultSet.next())
System.out.println(resultSet.getString(1) + "\t" + resultSet.getString(2) );

integrate ms access and mysql in java

I have a problem connecting to MS Access and MySQL using Java. My problem is that I cannot find the driver for MySQL. Here is my code:
<%# page import="java.sql.*" %>
<%
Connection odbcconn = null;
Connection jdbcconn = null;
PreparedStatement readsms = null;
PreparedStatement updsms = null;
ResultSet rsread = null;
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver"); //load database driver
odbcconn = DriverManager.getConnection("jdbc:odbc:SMS"); //connect to database
readsms = odbcconn.prepareStatement("select * from inbox where Status='New'");
rsread = readsms.executeQuery();
while(rsread.next()){
Class.forName("com.mysql.jdbc.Driver");
jdbcconn = DriverManager.getConnection("jdbc:mysql://localhost:3306/bakme", "root", ""); //connect to database
updsms = jdbcconn.prepareStatement("insert into inbox(sms,phone) values (?,?)");
updsms.setString(1, rsread.getString("Message"));
updsms.setString(2, rsread.getString("Phone"));
updsms.executeUpdate();
}
%>
Thus, you get a ClassNotFoundException on the MySQL JDBC driver class? Then you need to put the MySQL JDBC driver JAR file containing that class in the classpath. In case of a JSP/Servlet application, the classpath covers under each the webapplication's /WEB-INF/lib folder. Just drop the JAR file in there. Its JDBC driver is also known as Connector/J. You can download it here.
That said, that's really not the way how to use JDBC and JSP together. This doesn't belong in a JSP file. You should be doing this in a real Java class. The JDBC code should also be more robust written, now it's leaking resources.
BalusC is spot on: this is not the way you should write something like this.
Connection, Statement, and ResultSet all represent finite resources. They are not like memory allocation; the garbage collector does not clean them up. You have to do that in your code, like this:
// Inside a method
Connection connection = null;
Statement statement = null;
ResultSet rs = null;
try
{
// interact with the database using connection, statement, and rs
}
finally
{
// clean up resources in a finally block using static methods,
// called in reverse order of creation, that don't throw exceptions
close(rs);
close(statement);
close(connection);
}
But if you do decide to move this to a server-side component, you're bound to have huge problems with code like this:
while(rsread.next())
{
Class.forName("com.mysql.jdbc.Driver");
jdbcconn = DriverManager.getConnection("jdbc:mysql://localhost:3306/bakme", "root", ""); //connect to database
updsms = jdbcconn.prepareStatement("insert into inbox(sms,phone) values (?,?)");
updsms.setString(1, rsread.getString("Message"));
updsms.setString(2, rsread.getString("Phone"));
updsms.executeUpdate();
}
Registering the driver, creating a connection, without closing it, and repeatedly preparing a statement inside a loop for every row that you get out of Access shows a serious misunderstanding of relational databases and JDBC.
You should register the driver and create connections once, do what needs to be done, and clean up all the resources you've used.
If you absolutely MUST do this in a JSP, I'd recommend using JNDI data sources for both databases so you don't have to set up connections inside the page. You should not be writing scriptlet code - better to learn JSTL and use its <sql> tags.
You can use this link to download the MySql Driver. Once you download it, you need to make sure it is on the class path for the web server that you are using. The particulars of configuring JDBC drivers for a server vary from server to server. You may want to edit your question to include more details to get a better answer.

Categories

Resources