Develop Reference

Accessing AWS RDS using IAM Authentication and Spring JDBC (DataSource and JdbcTemplace)

I am not able to figure out how to implement this. Any help and/or pointers will be greatly appreciated.
Currently, my Java/Spring application backend is deployed on EC2 and accessing MySQL on RDS successfully using the regular Spring JDBC setup. That is, storing database info in application.properties and configuring DataSource and JdbcTemplate in #Configuration class. Everything works fine.
Now, I need to access MySQL on RDS securely. RDS instance has IAM Authentication enabled. I have also successfully created IAM role and applied inline policy. Then, following the AWS RDS documentation and Java example on this link, I am able to access the database from a standalone Java class successfully using Authentication Token and the user I created instead of regular db username and password. This standalone Java class is dealing with "Connection" object directly.
The place I am stuck is how I translate this to Spring JDBC configuration. That is, setting up DataSource and JdbcTemplate beans for this in my #Configuration class.
What would be a correct/right approach to implement this?
----- EDIT - Start -----
I am trying to implement this as a library that can be used for multiple projects. That is, it will be used as a JAR and declared as a dependency in a project's POM file. This library is going to include configurable AWS Services like this RDS access using general DB username and password, RDS access using IAM Authentication, KMS (CMK/data keys) for data encryption, etc.
Idea is to use this library on any web/app server depending on the project.
Hope this clarifies my need more.
----- EDIT - End -----
DataSource internally has getConnection() so I can basically create my own DataSource implementation to achieve what I want. But is this a good approach?
Something like:
public class MyDataSource implements DataSource {
#Override
public Connection getConnection() throws SQLException {
Connection conn = null;
// get a connection using IAM Authentication Token for accessing AWS RDS, etc. as in the AWS docs
return conn;
}
#Override
public Connection getConnection(String username, String password) throws SQLException {
return getConnection();
}
//other methods
}

You can use the following snippet as a replacement for the default connection-pool provided by SpringBoot/Tomcat. It will refresh the token password every 10 minutes, since the token is valid for 15 minutes. Also, it assumes the region can be extracted from the DNS hostname. If this is not the case, you'll need to specify the region to use.
public class RdsIamAuthDataSource extends org.apache.tomcat.jdbc.pool.DataSource {
private static final Logger LOG = LoggerFactory.getLogger(RdsIamAuthDataSource.class);
/**
* The Java KeyStore (JKS) file that contains the Amazon root CAs
*/
public static final String RDS_CACERTS = "/rds-cacerts";
/**
* Password for the ca-certs file.
*/
public static final String PASSWORD = "changeit";
public static final int DEFAULT_PORT = 3306;
#Override
public ConnectionPool createPool() throws SQLException {
return pool != null ? pool : createPoolImpl();
}
protected synchronized ConnectionPool createPoolImpl() throws SQLException {
return pool = new RdsIamAuthConnectionPool(poolProperties);
}
public static class RdsIamAuthConnectionPool extends ConnectionPool implements Runnable {
private RdsIamAuthTokenGenerator rdsIamAuthTokenGenerator;
private String host;
private String region;
private int port;
private String username;
private Thread tokenThread;
public RdsIamAuthConnectionPool(PoolConfiguration prop) throws SQLException {
super(prop);
}
#Override
protected void init(PoolConfiguration prop) throws SQLException {
try {
URI uri = new URI(prop.getUrl().substring(5));
this.host = uri.getHost();
this.port = uri.getPort();
if (this.port < 0) {
this.port = DEFAULT_PORT;
}
this.region = StringUtils.split(this.host,'.')[2]; // extract region from rds hostname
this.username = prop.getUsername();
this.rdsIamAuthTokenGenerator = RdsIamAuthTokenGenerator.builder().credentials(new DefaultAWSCredentialsProviderChain()).region(this.region).build();
updatePassword(prop);
final Properties props = prop.getDbProperties();
props.setProperty("useSSL","true");
props.setProperty("requireSSL","true");
props.setProperty("trustCertificateKeyStoreUrl",getClass().getResource(RDS_CACERTS).toString());
props.setProperty("trustCertificateKeyStorePassword", PASSWORD);
super.init(prop);
this.tokenThread = new Thread(this, "RdsIamAuthDataSourceTokenThread");
this.tokenThread.setDaemon(true);
this.tokenThread.start();
} catch (URISyntaxException e) {
throw new RuntimeException(e.getMessage());
}
}
#Override
public void run() {
try {
while (this.tokenThread != null) {
Thread.sleep(10 * 60 * 1000); // wait for 10 minutes, then recreate the token
updatePassword(getPoolProperties());
}
} catch (InterruptedException e) {
LOG.debug("Background token thread interrupted");
}
}
#Override
protected void close(boolean force) {
super.close(force);
Thread t = tokenThread;
tokenThread = null;
if (t != null) {
t.interrupt();
}
}
private void updatePassword(PoolConfiguration props) {
String token = rdsIamAuthTokenGenerator.getAuthToken(GetIamAuthTokenRequest.builder().hostname(host).port(port).userName(this.username).build());
LOG.debug("Updated IAM token for connection pool");
props.setPassword(token);
}
}
}
Please note that you'll need to import Amazon's root/intermediate certificates to establish a trusted connection. The example code above assumes that the certificates have been imported into a file called 'rds-cacert' and is available on the classpath. Alternatively, you can also import them into the JVM 'cacerts' file.
To use this data-source, you can use the following properties for Spring:
datasource:
url: jdbc:mysql://dbhost.xyz123abc.us-east-1.rds.amazonaws.com/dbname
username: iam_app_user
driver-class-name: com.mysql.cj.jdbc.Driver
type: com.mydomain.jdbc.RdsIamAuthDataSource
Using Spring Java config:
#Bean public DataSource dataSource() {
PoolConfiguration props = new PoolProperties();
props.setUrl("jdbc:mysql://dbname.abc123xyz.us-east-1.rds.amazonaws.com/dbschema");
props.setUsername("iam_dbuser_app");
props.setDriverClassName("com.mysql.jdbc.Driver");
return new RdsIamAuthDataSource(props);
}
UPDATE: When using MySQL, you can also decide to use the MariaDB JDBC driver, which has builtin support for IAM authentication:
spring:
datasource:
host: dbhost.cluster-xxx.eu-west-1.rds.amazonaws.com
url: jdbc:mariadb:aurora//${spring.datasource.host}/db?user=xxx&credentialType=AWS-IAM&useSsl&serverSslCert=classpath:rds-combined-ca-bundle.pem
type: org.mariadb.jdbc.MariaDbPoolDataSource
The above requires MariaDB and AWS SDK libraries, and needs the CA-bundle in the classpath

I know this is an older question, but after a some searching I found a pretty easy way you can now do this using the MariaDB driver. In version 2.5 they added an AWS IAM credential plugin to the driver. It will handle generating, caching and refreshing the token automatically.
I've tested using Spring Boot 2.3 with the default HikariCP connection pool and it is working fine for me with these settings:
spring.datasource.url=jdbc:mariadb://host/db?credentialType=AWS-IAM&useSsl&serverSslCert=classpath:rds-combined-ca-bundle.pem
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
spring.datasource.username=iam_username
#spring.datasource.password=dont-need-this
spring.datasource.hikari.maxLifetime=600000
Download rds-combined-ca-bundle.pem and put it in src/main/resources so you can connect via SSL.
You will need these dependencies on the classpath as well:
runtime 'org.mariadb.jdbc:mariadb-java-client'
runtime 'com.amazonaws:aws-java-sdk-rds:1.11.880'
The driver uses the standard DefaultAWSCredentialsProviderChain so make sure you have credentials with policy allowing IAM DB access available wherever you are running your app.
Hope this helps someone else - most examples I found online involved custom code, background threads, etc - but using the new driver feature is much easier!

There is a library that can make this easy. Effectively you just override the getPassword() method in the HikariDataSource. You use STS to assume the role and send a "password" for that role.
<dependency>
<groupId>io.volcanolabs</groupId>
<artifactId>rds-iam-hikari-datasource</artifactId>
<version>1.0.4</version>
</dependency>

How to use JMX MBean for HikariCP in Spring boot application?

How to use JMX MBean for HikariCP in Spring boot application? I have a code like this:
#SpringBootApplication
public class App() { ... }
And other class:
#Configuration
public class DatabaseCfg() {
#Bean
#ManagedOperation
public DataSource ds (#Value("${hikari.proprerties}") String config) {
HikariConfig hikariConfig = new HikariConfig(config);
return new HikariDataSource(hikariConfig);
}
In Java Mission Control (or JMX Console) a saw only Datasource managed bean, not JMX MBean for HikariCP (link). Is it possible to add it too?

In Spring Boot 2.0+ you can set the register-mbeans property in your application.properties file
spring.datasource.hikari.register-mbeans = true
If you are using an earlier version of Spring Boot you will also have to set the datasource
spring.datasource.type = com.zaxxer.hikari.HikariDataSource

I believe on your hikariConfig you need to set a few additional settings. You need to register the MBeans and set a pool name on the configuration.
HikariConfig hiakriConfig = new HikariConfig(config);
hikariConfig.setRegisterMbeans(true);
kikariConfig.setPoolName("my-pool-1");
Yes you obviously could drive these through the properties as well. I'm not sure if you are including these in your properties file as they are not listed. Also please note you are spelling properties wrong (#Value("${ds.proprerties}") should probably should be (#Value("${ds.properties}") but I'm not sure how you actually have named variables and property files. You may want to double check if that is where you want to set all of the properties.

Try this. Exclude your Hiakri DataSource Bean from being registered by Spring.
#Resource
private ObjectProvider<MBeanExporter> mBeanExporter;
#Bean("dataSource")
public DataSource createDataSource() {
String url = hikariDataSourceConfig.getUrl();
String username = hikariDataSourceConfig.getUsername();
String password = hikariDataSourceConfig.getPassword();
long idleTimeoutInMilliSeconds =
hikariDataSourceConfig.getIdleTimeOutInMilliseconds();
long maxLifetimeInMilliseconds =
hikariDataSourceConfig.getMaxLifetimeInMilliseconds();
int maximumPoolSize = hikariDataSourceConfig.getMaximumPoolSize();
int minimumIdle = hikariDataSourceConfig.getMinimumIdle();
String poolName = "HikariDataSource";
HikariConfig hikariConfig = new HikariConfig();
hikariConfig.setRegisterMbeans(true);
hikariConfig.setJdbcUrl(url);
hikariConfig.setUsername(username);
hikariConfig.setPassword(password);
hikariConfig.setIdleTimeout(idleTimeoutInMilliSeconds);
hikariConfig.setMaxLifetime(maxLifetimeInMilliseconds);
hikariConfig.setMaximumPoolSize(maximumPoolSize);
hikariConfig.setMinimumIdle(minimumIdle);
hikariConfig.setPoolName(poolName);
HikariDataSource dataSource = new HikariDataSource(hikariConfig);
mBeanExporter
.ifUnique((exporter) -> exporter.addExcludedBean("dataSource"));
return dataSource;
}

Properties file on HikariCP

I tried unsuccessfully configure hikaricp and I don't see error in the code please help.
public class DatabaseManager {
private DatabaseClient[] databaseClients;
private HikariDataSource hikariDataSource;
public DatabaseManager(String absoluteFilePath) {
final HikariConfig hikariConfig = new HikariConfig(absoluteFilePath);
this.hikariDataSource = new HikariDataSource(hikariConfig);
System.out.println(hikariConfig.getUsername()); // null u-u
}
}
Properties file:
## Database Settings
dataSourceClassName=org.mariadb.jdbc.MySQLDataSource
dataSource.user=root
dataSource.password=
dataSource.databaseName=imagine-db
dataSource.portNumber=3306
dataSource.serverName=localhost

You've set the username on the data source not on the config itself. This will still work fine but you can't access it using hikariConfig.getUsername().
Try adding this to your properties file if you really need to access the user like that, although I suspect you don't.
username=root
password=

How to use SchemaExportTool with JPA and Hibernate 4.3

At Hibernate 4.3 Ejb3Configuration class was removed. This class was commonly used for creating hibernate configuration file from a persistence unit (persistence.xml file) to SchemaExport tool.
As a simple alternative to export schema to .sql file I'm using the following code:
public static void export(String persistenceUnit, String exportFileName) {
Map<String, String> hash = new HashMap<String, String>();
hash.put("hibernate.hbm2ddl.auto", "create-drop");
EntityManagerFactory factory = Persistence.createEntityManagerFactory(
persistenceUnit, hash);
org.hibernate.jpa.internal.EntityManagerFactoryImpl hibFactory = (org.hibernate.jpa.internal.EntityManagerFactoryImpl) factory;
SessionFactoryImpl hibSessionFactory = hibFactory.getSessionFactory();
SchemaExport schema = ReflectUtils.getPrivateFieldValue(
hibSessionFactory, "schemaExport");
schema.setOutputFile(exportFileName);
schema.setFormat(false);
schema.setDelimiter(";");
schema.drop(true, false);
schema.create(true, false);
}
At this piece of code, i'm basically using schemaexport object created by HibernateSessionFactoryImpl. The drawback is that every time it´s executed the database schema is recreated. Is there any other simple way to use SchemaExporTool with Hibernate 4.3 and JPA? It seems that the real question is how to create Hibernate Configuration Object from a persistenceunit?

I ran into the same Problem. I ended up by using the internal PersistenceXmlParser of Hibernate to access information in the persistence.xml file and creating the Configuration object manually:
public static void main(String[] args) {
PersistenceXmlParser parser = new PersistenceXmlParser(new ClassLoaderServiceImpl(), PersistenceUnitTransactionType.RESOURCE_LOCAL);
List<ParsedPersistenceXmlDescriptor> allDescriptors = parser.doResolve(new HashMap<>());
for (ParsedPersistenceXmlDescriptor descriptor : allDescriptors) {
Configuration cfg = new Configuration();
cfg.setProperty("hibernate.hbm2ddl.auto", "create");
cfg.setProperty("hibernate.dialect", "org.hibernate.dialect.Oracle10gDialect");
cfg.setProperty("hibernate.id.new_generator_mappings", "true");
List<String> managedClassNames = descriptor.getManagedClassNames();
for (String className : managedClassNames) {
try {
cfg.addAnnotatedClass(Class.forName(className));
} catch (ClassNotFoundException e) {
System.out.println("Class not found: " + className);
}
}
SchemaExport export = new SchemaExport(cfg);
export.setDelimiter(";");
export.setOutputFile("C:\\dev\\" + descriptor.getName() + "_create_schema.sql");
export.setFormat(true);
export.execute(true, false, false, false);
}
}

Can an EJB be called from a desktop application?

I am new in Java EJB 3.0. It is possible to call a (session) bean—deployed on JBoss—from a desktop application client?
Thanks in advance.

Yes you can. Some specifics are here (references EJB2 but it the same for EJB3 when it comes to remote clients): http://www.theserverside.com/discussions/thread.tss?thread_id=9197
Paraphrased:
Hashtable env = new Hashtable();
env.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory");
env.put("java.naming.provider.url", "jnp://localhost:1099");
env.put("java.naming.factory.url.pkgs", "org.jboss.naming:org.jnp.interfaces");
Context ctx = new InitialContext(env);
// name is whatever JNDI name you gave it
Object o = ctx.lookup("home name");
EJBHome ejbHome = (EJBHome) PortableRemoteObject.narrow(o,EJBHome.class);
// This is userID should be the one passed.
EJB ejb = ejbHome.create(..);

Yes.
public static void main(String args[]) throws Exception {
InitialContext ctx = new InitialContext();
YourService yourService = (YourService) ctx.lookup("com.example.session.YourService");
String time = yourService.getTime();
System.out.println("Time is: " + time);
}
For client configuration you must provide jndi.properties file with contents
java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces
java.naming.provider.url=localhost
If you are looking for working examples on JBoss try download source code of Enterprise JavaBeans 3.0, Fifth Edition

Let's assume you have the following remote interface:
#Remote
public interface HelloBeanRemote {
public String sayHello();
}
And a session bean implementing it:
#Stateless
public class HelloBean implements HelloBeanRemote {
...
}
And that this EJB is correctly packaged and deployed on JBoss.
On the client side, create a jndi.properties with the following content and put it on the classpath:
java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces
java.naming.provider.url=localhost:1099
Then use the following code to call your EJB:
Context context;
try {
context = new InitialContext();
HelloBeanRemote beanRemote = (HelloBeanRemote)context.lookup("HelloBean/remote");
beanRemote.test();
} catch (NamingException e) {
e.printStackTrace();
throw new RuntimeException(e);
}
Alternatively, if you don't want to provide a jndi.properties file, you can explicitly setup the JNDI environment in the code and create the context like this:
Properties properties = new Properties();
properties.put("java.naming.factory.initial","org.jnp.interfaces.NamingContextFactory");
properties.put("java.naming.factory.url.pkgs","=org.jboss.naming:org.jnp.interfaces");
properties.put("java.naming.provider.url","localhost:1099");
Context context = new InitialContext(properties);
But I'd recommend using the jndi.properties for the sake of portability.

You can also expose the bean as a web service. I believe this is available as of EJB 3. It is quite nice considering you can do it with annotations. You may wish to consider using this option to decrease coupling. Here is a link to a tutorial.