Develop Reference

Accessing AWS RDS using IAM Authentication and Spring JDBC (DataSource and JdbcTemplace)

I am not able to figure out how to implement this. Any help and/or pointers will be greatly appreciated.
Currently, my Java/Spring application backend is deployed on EC2 and accessing MySQL on RDS successfully using the regular Spring JDBC setup. That is, storing database info in application.properties and configuring DataSource and JdbcTemplate in #Configuration class. Everything works fine.
Now, I need to access MySQL on RDS securely. RDS instance has IAM Authentication enabled. I have also successfully created IAM role and applied inline policy. Then, following the AWS RDS documentation and Java example on this link, I am able to access the database from a standalone Java class successfully using Authentication Token and the user I created instead of regular db username and password. This standalone Java class is dealing with "Connection" object directly.
The place I am stuck is how I translate this to Spring JDBC configuration. That is, setting up DataSource and JdbcTemplate beans for this in my #Configuration class.
What would be a correct/right approach to implement this?
----- EDIT - Start -----
I am trying to implement this as a library that can be used for multiple projects. That is, it will be used as a JAR and declared as a dependency in a project's POM file. This library is going to include configurable AWS Services like this RDS access using general DB username and password, RDS access using IAM Authentication, KMS (CMK/data keys) for data encryption, etc.
Idea is to use this library on any web/app server depending on the project.
Hope this clarifies my need more.
----- EDIT - End -----
DataSource internally has getConnection() so I can basically create my own DataSource implementation to achieve what I want. But is this a good approach?
Something like:
public class MyDataSource implements DataSource {
#Override
public Connection getConnection() throws SQLException {
Connection conn = null;
// get a connection using IAM Authentication Token for accessing AWS RDS, etc. as in the AWS docs
return conn;
}
#Override
public Connection getConnection(String username, String password) throws SQLException {
return getConnection();
}
//other methods
}

You can use the following snippet as a replacement for the default connection-pool provided by SpringBoot/Tomcat. It will refresh the token password every 10 minutes, since the token is valid for 15 minutes. Also, it assumes the region can be extracted from the DNS hostname. If this is not the case, you'll need to specify the region to use.
public class RdsIamAuthDataSource extends org.apache.tomcat.jdbc.pool.DataSource {
private static final Logger LOG = LoggerFactory.getLogger(RdsIamAuthDataSource.class);
/**
* The Java KeyStore (JKS) file that contains the Amazon root CAs
*/
public static final String RDS_CACERTS = "/rds-cacerts";
/**
* Password for the ca-certs file.
*/
public static final String PASSWORD = "changeit";
public static final int DEFAULT_PORT = 3306;
#Override
public ConnectionPool createPool() throws SQLException {
return pool != null ? pool : createPoolImpl();
}
protected synchronized ConnectionPool createPoolImpl() throws SQLException {
return pool = new RdsIamAuthConnectionPool(poolProperties);
}
public static class RdsIamAuthConnectionPool extends ConnectionPool implements Runnable {
private RdsIamAuthTokenGenerator rdsIamAuthTokenGenerator;
private String host;
private String region;
private int port;
private String username;
private Thread tokenThread;
public RdsIamAuthConnectionPool(PoolConfiguration prop) throws SQLException {
super(prop);
}
#Override
protected void init(PoolConfiguration prop) throws SQLException {
try {
URI uri = new URI(prop.getUrl().substring(5));
this.host = uri.getHost();
this.port = uri.getPort();
if (this.port < 0) {
this.port = DEFAULT_PORT;
}
this.region = StringUtils.split(this.host,'.')[2]; // extract region from rds hostname
this.username = prop.getUsername();
this.rdsIamAuthTokenGenerator = RdsIamAuthTokenGenerator.builder().credentials(new DefaultAWSCredentialsProviderChain()).region(this.region).build();
updatePassword(prop);
final Properties props = prop.getDbProperties();
props.setProperty("useSSL","true");
props.setProperty("requireSSL","true");
props.setProperty("trustCertificateKeyStoreUrl",getClass().getResource(RDS_CACERTS).toString());
props.setProperty("trustCertificateKeyStorePassword", PASSWORD);
super.init(prop);
this.tokenThread = new Thread(this, "RdsIamAuthDataSourceTokenThread");
this.tokenThread.setDaemon(true);
this.tokenThread.start();
} catch (URISyntaxException e) {
throw new RuntimeException(e.getMessage());
}
}
#Override
public void run() {
try {
while (this.tokenThread != null) {
Thread.sleep(10 * 60 * 1000); // wait for 10 minutes, then recreate the token
updatePassword(getPoolProperties());
}
} catch (InterruptedException e) {
LOG.debug("Background token thread interrupted");
}
}
#Override
protected void close(boolean force) {
super.close(force);
Thread t = tokenThread;
tokenThread = null;
if (t != null) {
t.interrupt();
}
}
private void updatePassword(PoolConfiguration props) {
String token = rdsIamAuthTokenGenerator.getAuthToken(GetIamAuthTokenRequest.builder().hostname(host).port(port).userName(this.username).build());
LOG.debug("Updated IAM token for connection pool");
props.setPassword(token);
}
}
}
Please note that you'll need to import Amazon's root/intermediate certificates to establish a trusted connection. The example code above assumes that the certificates have been imported into a file called 'rds-cacert' and is available on the classpath. Alternatively, you can also import them into the JVM 'cacerts' file.
To use this data-source, you can use the following properties for Spring:
datasource:
url: jdbc:mysql://dbhost.xyz123abc.us-east-1.rds.amazonaws.com/dbname
username: iam_app_user
driver-class-name: com.mysql.cj.jdbc.Driver
type: com.mydomain.jdbc.RdsIamAuthDataSource
Using Spring Java config:
#Bean public DataSource dataSource() {
PoolConfiguration props = new PoolProperties();
props.setUrl("jdbc:mysql://dbname.abc123xyz.us-east-1.rds.amazonaws.com/dbschema");
props.setUsername("iam_dbuser_app");
props.setDriverClassName("com.mysql.jdbc.Driver");
return new RdsIamAuthDataSource(props);
}
UPDATE: When using MySQL, you can also decide to use the MariaDB JDBC driver, which has builtin support for IAM authentication:
spring:
datasource:
host: dbhost.cluster-xxx.eu-west-1.rds.amazonaws.com
url: jdbc:mariadb:aurora//${spring.datasource.host}/db?user=xxx&credentialType=AWS-IAM&useSsl&serverSslCert=classpath:rds-combined-ca-bundle.pem
type: org.mariadb.jdbc.MariaDbPoolDataSource
The above requires MariaDB and AWS SDK libraries, and needs the CA-bundle in the classpath

I know this is an older question, but after a some searching I found a pretty easy way you can now do this using the MariaDB driver. In version 2.5 they added an AWS IAM credential plugin to the driver. It will handle generating, caching and refreshing the token automatically.
I've tested using Spring Boot 2.3 with the default HikariCP connection pool and it is working fine for me with these settings:
spring.datasource.url=jdbc:mariadb://host/db?credentialType=AWS-IAM&useSsl&serverSslCert=classpath:rds-combined-ca-bundle.pem
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
spring.datasource.username=iam_username
#spring.datasource.password=dont-need-this
spring.datasource.hikari.maxLifetime=600000
Download rds-combined-ca-bundle.pem and put it in src/main/resources so you can connect via SSL.
You will need these dependencies on the classpath as well:
runtime 'org.mariadb.jdbc:mariadb-java-client'
runtime 'com.amazonaws:aws-java-sdk-rds:1.11.880'
The driver uses the standard DefaultAWSCredentialsProviderChain so make sure you have credentials with policy allowing IAM DB access available wherever you are running your app.
Hope this helps someone else - most examples I found online involved custom code, background threads, etc - but using the new driver feature is much easier!

There is a library that can make this easy. Effectively you just override the getPassword() method in the HikariDataSource. You use STS to assume the role and send a "password" for that role.
<dependency>
<groupId>io.volcanolabs</groupId>
<artifactId>rds-iam-hikari-datasource</artifactId>
<version>1.0.4</version>
</dependency>

Java/Android Bouncy castle ClassNotFoundException

I am writing an application for the Android platform which makes use of cryptography. It's a secure chat, client->server->client with diffie hellman keys.
Clients are android phones and the server is running on my PC.
I am trying to send the public key via sockets but i am getting this error:
Server Up and Running
Bouncy Castle provider is available
Aug 21, 2016 4:47:03 PM Connection run
SEVERE: null
java.lang.ClassNotFoundException: com.android.org.bouncycastle.jcajce.provider.asymmetric.dh.BCDHPublicKey
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:626)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1613)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
at Connection.run(Connection.java:124)
As you can see bouncy castle provider is available.
Here is my code with the problem:
Server:
if (Security.getProvider("BC") == null){
System.out.println("Bouncy Castle provider is NOT available");
}
else{
System.out.println("Bouncy Castle provider is available");
}
try {
pubkey = (PublicKey) instream.readObject();
}catch (IOException ex) {}
Android:
PublicKey pubkey;
kp23 = genDHKeyPair();
pubkey= kp23.getPublic();
//System.out.println(pubkey);
outstream.writeObject(pubkey);
outstream.flush();

The version of BouncyCastle available on android os is very limited and apparently it does't include the com.android.org.bouncycastle.jcajce.provider.asymmetric.dh.BCDHPublicKey.
If you want full BouncyCastle functionality, you need to include the version that was repackaged with Android in mind, which is the SpongyCastle cryptography provider.
Follow this link for more explanations.

ClassNotFoundException is thrown because the package com.android.org.bouncycastle.jcajce.provider.asymmetric.dh.BCDHPublicKey is not available in the java version of BouncyCastle which includes the equivalent org.bouncycastle.jcajce.provider.asymmetric.dh.BCDHPublicKey.
I suggest you to create a new class with the following properties on both Server and Client applications.
import java.io.Serializable;
public class PublicKeyProperties implements Serializable {
private static final long serialVersionUID = 1L;
public byte[] Encoded;
public String Format;
public String Algorithm;
}
Client side
Pass the PublicKey's corresponding properties to a new instance of PublicKeyProperties class, serialize it and send it to the server.
PublicKeyProperties prop = new PublicKeyProperties();
prop.Format = pubkey.getFormat();
prop.Algorithm = pubkey.getAlgorithm();
prop.Encoded = pubkey.getEncoded();
outstream.writeObject(prop);
Server side
Deserialize it and create a new PublicKey instance as follows.
PublicKeyProperties prop = (PublicKeyProperties) instream.readObject();
pubkey = new PublicKey() {
#Override
public String getAlgorithm() {
return prop.Algorithm;
}
#Override
public String getFormat() {
return prop.Format;
}
#Override
public byte[] getEncoded() {
return prop.Encoded;
}
};
Hope that helps.

PeerUnavailableException using JAIN SIP API on Android

I'm trying to build SIP application using JAIN SIP 1.2 and the NIST implementation on android.
I have rebuilt jain-sip-api-1.2.jar and jain-sip-ri-1.2.1111.jar from source, and renamed javax -> jain_javax and gov.nist.javax -> jain_gov.nist.jain_javax. I tested the jar files on textclient example on standard java without problem. However, when I run it on Android I still get the error:
"The Peer SIP Stack: jain_gov.nist.jain_javax.sip.SipstackImpl could not be instantiated. Ensure the Path Name has been set".
Did I miss anything here?

It is not sufficient to rename the packages. JAIN-SIP has internal references to some classes by their original package name "gov.nist". You should also double check all your code to rename any "gov.nist" references such as the prefix for the stack classes.
Android has built-in an older version of JAIN-SIP which is taking over some of the existing references to those "gov.nist" classes. It's not an exported API, so not quite obvious. That's why it may behave differently on desktop machines. Post you code and full error messages/debug logs if you need more help.

Sovled. Jain Sip is using log4i-1.2.x.jar which does not work properly on Android. There are lots of discussion on Internet how to make log4j working on Android but none of them works for me. I have removed all log4j related code from Jain Sip source and now the sip stack is working properly on Android.

I am using JAIN-SIP-1-2-164. Here is the app code:
import java.io.UnsupportedEncodingException;
import java.net.InetAddress;
import java.text.ParseException;
import java.util.*;
import android.os.Handler;
import jain_javax.sip.*;
import jain_javax.sip.address.*;
import jain_javax.sip.header.*;
import jain_javax.sip.message.*;
public class SipLayer implements SipListener {
private SipStack sipStack;
private SipFactory sipFactory;
private Properties properties;
private String local_ip;
int listen_port;
/** Here we initialize the SIP stack. */
public SipLayer(int listen_port) {
try {
setUsername(username);
this.local_ip = InetAddress.getLocalHost().getHostAddress();;
this.listen_port = listen_port;
// Create the SIP factory and set the path name.
this.sipFactory = SipFactory.getInstance();
this.sipFactory.setPathName("jain_gov.nist");
// Create and set the SIP stack properties.
this.properties = new Properties();
this.properties.setProperty("jain_javax.sip.STACK_NAME", "stack");
this.properties.setProperty("jain_javax.sip.IP_ADDRESS", local_ip);
if(proxy != null)
this.properties.setProperty("jain_javax.sip.OUTBOUND_PROXY", proxy + ':' + server_port + '/' + protocol);
//DEBUGGING: Information will go to files textclient.log and textclientdebug.log
this.properties.setProperty("jain_gov.nist.javax.sip.TRACE_LEVEL", "32");
// this.properties.setProperty("jain_gov.nist.javax.sip.SERVER_LOG", "textclient.txt");
// this.properties.setProperty("jain_gov.nist.javax.sip.DEBUG_LOG", "textclientdebug.log");
// Create the SIP stack.
this.sipStack = this.sipFactory.createSipStack(properties);
}
catch (Exception e) {
msgProc.processError("SipLayer failed: " + e.getMessage() + "\n");
}
}
}
Same code runs ok on java on a windows machine, but android emulator I got above mentioned error message.
I found that it failed in following Jain SIP 1.2 routine at "SipStack sipStack = (SipStack) sipStackConstructor.newInstance(conArgs);"
private SipStack createStack(Properties properties)
throws PeerUnavailableException {
try {
// create parameters argument to identify constructor
Class[] paramTypes = new Class[1];
paramTypes[0] = Class.forName("java.util.Properties");
// get constructor of SipStack in order to instantiate
Constructor sipStackConstructor = Class.forName(
getPathName() + ".jain_javax.sip.SipStackImpl").getConstructor(
paramTypes);
// Wrap properties object in order to pass to constructor of
// SipSatck
Object[] conArgs = new Object[1];
conArgs[0] = properties;
// Creates a new instance of SipStack Class with the supplied
// properties.
SipStack sipStack = (SipStack) sipStackConstructor.newInstance(conArgs);
sipStackList.add(sipStack);
String name = properties.getProperty("jain_javax.sip.STACK_NAME");
this.sipStackByName.put(name, sipStack);
return sipStack;
} catch (Exception e) {
String errmsg = "The Peer SIP Stack: "
+ getPathName()
+ ".jain_javax.sip.SipStackImpl"
+ " could not be instantiated. Ensure the Path Name has been set.";
throw new PeerUnavailableException(errmsg, e);
}
}
Any suggestion or how to debug further?

nosuchelementexception in calling a dynamic web service

I am trying to call a web service dynamically, I don't have any problem in calling a method with input parameters , but when I use a method without arguments, I got nosuchelementexception error ,My source code is described below:
I have an interface :
public interface SmsParam extends java.rmi.Remote {
public String getSmsIncomePackServices() throws java.rmi.RemoteException;
public String getCustMobileNo(Integer intCustId) throws java.rmi.RemoteException;
}
and I use below code for calling getSmsIncomePackServices method:
String UrlString = " ?wsdl";
String nameSpaceUri = " ";
String serviceName = " ";
String portName = "coreBankingSMSWebServiceHttpSoap11Endpoint";//"inactiveMobileSMSHttpSoap11Endpoint";
ServiceFactory serviceFactory = ServiceFactory.newInstance();
Service helloService = serviceFactory.createService(helloWsdlUrl, new QName(nameSpaceUri, serviceName));
SmsParam myProxy = (SmsParam)helloService.getPort(
new QName(nameSpaceUri, portName),
SmsParam.class);
myProxy.getCustMobileNo(12); //Runs successfuly
myProxy.getSmsFreePackServices();//I got nosuchelementexception error
Is there any way to solve my problem?

I dont understand how the code compiled... There is no getSmsFreePackServices() method defined in your interface SmsParam.
Your SmsParam contains getSmsIncomePackServices and getCustMobileNo methods only

running an axis2 client version 1.5

So I'm running out of ideas to try to actually get a client to connect to the SOAP service I'm running through axis2.
I tried two methods, one was to use wsdl2java to build the stub and associated client side classes, and then write a Client class that build the requests messages and sends them through the Stub. The other way was to use the ServiceClient to connect..
Both are failing in their own way..
Option #1, every time a message is sent through the stub I get this back:
org.apache.axis2.AxisFault: The input stream for an incoming message is null.
at org.apache.axis2.transport.TransportUtils.createSOAPMessage(TransportUtils.java:87)
at org.apache.axis2.transport.TransportUtils.createSOAPMessage(TransportUtils.java:67)
at org.apache.axis2.description.OutInAxisOperationClient.handleResponse(OutInAxisOperation.java:354)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:417)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:229)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:165)
Option #2, everytime I run it I get this Exception:
org.apache.axis2.deployment.DeploymentException: org.apache.axis2.transport.local.LocalTransportSender
Option #2 source:
import javax.xml.stream.XMLStreamException;
import org.apache.axiom.om.OMAbstractFactory;
import org.apache.axiom.om.OMElement;
import org.apache.axiom.om.OMFactory;
import org.apache.axiom.om.OMNamespace;
import org.apache.axis2.addressing.EndpointReference;
import org.apache.axis2.client.Options;
import org.apache.axis2.Constants;
import org.apache.axis2.client.ServiceClient;
public class loyaltyClient {
private static EndpointReference targetEPR =
new EndpointReference(
"http://localhost:8080/axis2/services/service");
public static OMElement verifyCustomer(String customer_id) {
OMFactory fac = OMAbstractFactory.getOMFactory();
OMNamespace omNs = fac.createOMNamespace(
"http://localhost/", "service");
OMElement method = fac.createOMElement("VerifyCustomer", omNs);
OMElement value1 = fac.createOMElement("customer_id",omNs);
OMElement value2 = fac.createOMElement("source_id",omNs);
OMElement value3 = fac.createOMElement("source_password",omNs);
OMElement value4 = fac.createOMElement("source_txnid",omNs);
OMElement value5 = fac.createOMElement("timestamp",omNs);
value1.addChild(fac.createOMText(value1, customer_id));
value2.addChild(fac.createOMText(value2, "source"));
value3.addChild(fac.createOMText(value3, "1234"));
value4.addChild(fac.createOMText(value4, "123"));
value5.addChild(fac.createOMText(value5, "06-01-2010 12:01:01"));
method.addChild(value1);
method.addChild(value2);
method.addChild(value3);
method.addChild(value4);
method.addChild(value5);
return method;
}
public static void main(String[] args) {
try {
OMElement vctest = loyaltyClient.verifyCustomer("6177740603");
Options options = new Options();
options.setTo(targetEPR);
options.setTransportInProtocol(Constants.TRANSPORT_HTTP);
ServiceClient sender = new ServiceClient();
sender.setOptions(options);
OMElement result = sender.sendReceive(vctest);
String response = result.getFirstElement().getText();
System.out.println(response);
} catch (Exception e) { //(XMLStreamException e) {
System.out.println(e.toString());
}
}
}

I've also encountered the error "The input stream for an incoming message is null" while using Axis to connect to a .Net service provider.
The problem is that .Net doesn't not support a feature called "chunked encoding", by default Axis will break its request header in chunks which is suppose to be a HTTP 1.1 compliant thing.
Anyway, you can turn this feature off in Axis by doing the following:
// Turn off the Axsis Chunked feature, some service providers (like .Net) don't support chunked headers.
Options options = serviceClient.getOptions();
options.setProperty(HTTPConstants.CHUNKED, Constants.VALUE_FALSE);
serviceClient.setOptions(options);
This worked for me. Another thing to make sure of when dealing with .Net services is to be able to specify the port name and make sure your message payload has the namespace prefix for each element.
Hope this info helps somebody.
Cheers,
DC

With the caveat that Axis2 is a buggy pile of crap, I recently had to write an Axis2 client, and found that using the default ServiceClient() constructor didn't work well -- I had to manually create a ConfigurationContext, etc. I found that using ServiceClient.getOptions() instead of creating new Options() preserved some default data. I'd also recommend dropping the options.setTransportInProtocol(...) unless you really need it -- everything should work fine via HTTP without this. Also, you may need to set options.setAction(...) to correspond with the "operation" in your WSDL.
I've included the bulk of my client (with sensitive information stripped out), in hopes that it will help. You can probably safely ignore the portions regarding addressing unless you plan to use WS-Addressing.
ConfigurationContext cfgCtx = null;
try {
/* Passing null to both params causes an AxisConfiguration to be created that uses
* the default axis2.xml file, which is included in the axis2 distribution jar.
* This is ideal for our case, since we cannot pass a full file path (relative
* paths are not allowed) because we do not know where the customer will deploy
* the application. This also allows engaging modules from the classpath. */
cfgCtx = ConfigurationContextFactory.createConfigurationContextFromFileSystem(null , null);
} catch (AxisFault e) {
// Bubble up the error
}
ServiceClient svcClient = null;
try {
svcClient = new ServiceClient(cfgCtx, null);
} catch (AxisFault e) {
// Bubble up the error
}
try {
/* This will work with the above ConfigurationContext as long as the module
* (addressing-1.5.1.mar) is on the classpath, e.g. in shared/lib. */
svcClient.engageModule("addressing");
} catch (AxisFault e) {
// Bubble up the error
}
Options opts = svcClient.getOptions();
opts.setTo(new EndpointReference("http://myservername:8080/axis2/services/MyService"));
opts.setAction("urn:doSomething"); // Corresponds to the "operation" in MyService's WSDL
opts.setSoapVersionURI(SOAP12Constants.SOAP_ENVELOPE_NAMESPACE_URI); // Set output to SOAP 1.2
SOAPFactory factory = OMAbstractFactory.getSOAP12Factory();
svcClient.addHeader(createSOAPSecurityHeader(factory, response)); // CreateSOAPHeader just creates an OMElement
try {
svcClient.sendReceive(createSOAPBody(factory, response)); // CreateSOAPBody just creates an OMElement
} catch (AxisFault e) {
throw new ResponseDeliveryException(1, "Error sending SOAP payload.", e);
}