Develop Reference

It´s possible retrive or refresh a password by secretmanagerid in AWS, the password has been rotated by a policy every five minutes

well that´s the question, It´s possible to retrieve or refresh a password by secretmanagerid in AWS?, the password has been rotated by a policy every five minutes. I don´t want to restart my microservice to retrieve the pass, I was looking for a solution and I found something like this:
<groupId>com.amazonaws.secretsmanager</groupId>
<artifactId>aws-secretsmanager-jdbc</artifactId>
<version>1.0.5</version>
spring:
datasource:
url: jdbc-secretsmanager:mysql://database-host:3306/rotate_db
username: secret/rotation
driver-class-name: com.amazonaws.secretsmanager.sql.AWSSecretsManagerMySQLDriver
But I don't want to use the configuration in the application.yml or .properties, I want to keep these values in the parameter store as secrets, currently my code looks like this:
#Bean
public DataSource dataSource() {
AwsSecrets secrets = getSecret();
if(Objects.nonNull(secrets)){
log.info("Getting parameters: host: {}, port: {}, Db: {}, user: {}, pass: {}", secrets.getHost(), secrets.getPort(), secrets.getDatabase(),secrets.getUsername(), secrets.getPassword());
DataSource dataSource = DataSourceBuilder
.create()
.url("jdbc:postgresql://" + secrets.getHost() + ":" + secrets.getPort() + "/" + secrets.getDatabase())
.username(secrets.getUsername())
.password(secrets.getPassword())
.build();
return new TracingDataSource(dataSource);
}
log.debug("Unable to get secrets");
return null;
}
#Bean
public Filter tracingFilter() {
return new AWSXRayServletFilter("back-microservice");
}
private AwsSecrets getSecret() {
AWSSecretsManager client = AWSSecretsManagerClientBuilder.standard()
.withRegion(amazonRegion)
.withCredentials(dynamoDBConfig.accountAmazonAWSCredentials())
.build();
String secret;
GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest()
.withSecretId(secretmanagerId);
GetSecretValueResult getSecretValueResult = null;
try {
getSecretValueResult = client.getSecretValue(getSecretValueRequest);
} catch (Exception e) {
log.debug("Unable to get secrets values");
throw e;
}
if (getSecretValueResult.getSecretString() != null) {
secret = getSecretValueResult.getSecretString();
return gson.fromJson(secret, AwsSecrets.class);
}
return null;
}

Yes, it's possible to do. The exact implementation will vary by framework/ORM, but you would need to cache the credentials, but also check if a new password is needed every time a new connection is acquired.
In your code, you are setting the password once statically:
DataSource dataSource = DataSourceBuilder
.create()
.url("jdbc:postgresql://" + secrets.getHost() + ":" + secrets.getPort() + "/" + secrets.getDatabase())
.username(secrets.getUsername())
.password(secrets.getPassword())
.build();
The getPassword() function is only ever called once when you build the datasource and is reused for additional connections from that data source. Instead, you need to retrieve (from cache/secretsmanger) every time a connection is created.
AWS provides a java caching client for AWS secretsmanager (clients for other languages are also available). You can adapt that into your data source to use that to retrieve the password for every connection. You can read the official guidance documentation on that here: Rotate database credentials without restarting containers.
In the context of Spring, that means implementing this pattern in your data source driver, which is exactly what the com.amazonaws.secretsmanager.sql.AWSSecretsManagerMySQLDriver class provides. Because you are rotating the secret so often, you probably want to configure the secret cache to refresh more often than the default of 1 hour or adjust your rotation frequency.
If you really need to be rotating the password that often and you're using RDS for your database, you might consider just using IAM-based authentication instead. I can't imagine why you would want to have a password that rotates so frequently, keeping in mind you are billed for secret retrieval API calls.

creating a mongodb healthcheck (in dropwizard)

Not necessarily specific to dropwizard, but for the life of me I can't figure out how to easily create a healthcheck for mongodb. This is in java, using version 3.3.0 of mongodb's own java driver.
I was hoping there would be a method that doesn't change the state of the database if it succeeds, but also throws an Exception when the query (or connection, or whatever) fails in order to return a health or unhealthy state. Ideally I'd perform a find, but this doesn't throw an Exception as far as I can tell.

I would just list all collections in database like:
MongoClient client = new MongoClient(addr, opts);
MongoDatabase db = client.getDatabase(database);
try {
MongoIterable<String> allCollections = db.listCollectionNames();
for (String collection : allCollections) {
System.out.println("MongoDB collection: " + collection);
}
} catch (Exception me) {
// problems with mongodb
}

JDBC-JobStoreCMT lock when scheduling

I am using Weblogic + Spring + quartz.
Quartz is configured to use JobStoreCMT.
I noticed that JobStoreCMT is aquireing a DB lock on the quartz tables when jobs are scheduled.
Below is the JobStoreCMT snippet
protected Object executeInLock(
String lockName,
TransactionCallback txCallback) throws JobPersistenceException {
boolean transOwner = false;
Connection conn = null;
try {
if (lockName != null) {
// If we aren't using db locks, then delay getting DB connection
// until after aquiring the lock since it isn't needed.
if (getLockHandler().requiresConnection()) {
conn = getConnection();
}
transOwner = getLockHandler().obtainLock(conn, lockName);
}
if (conn == null) {
conn = getConnection();
}
return txCallback.execute(conn);
} finally {
try {
releaseLock(conn, LOCK_TRIGGER_ACCESS, transOwner);
} finally {
cleanupConnection(conn);
}
}
}
After this method I see in the quartz tables in the DB inserted the triggers and jobs i scheduled.
My question is why Quartz needs lock on the DB level at this phase ?
I would see a need to have the lock when the jobs are started to be executed , finished etc.
Thanks

I found some setting which solved my issue:
setLockOnInsert to false because it is true by default.
public void setLockOnInsert(boolean lockOnInsert)
Whether or not to obtain locks when inserting new jobs/triggers. Defaults to true, which is safest - some db's (such as MS SQLServer) seem to require this to avoid deadlocks under high load, while others seem to do fine without.
Setting this property to false will provide a significant performance increase during the addition of new jobs and triggers.
+org.quartz.jobStore.acquireTriggersWithinLock i set it to false (as default ) not to true as i configured initially.

"(User has exceeded the 'max_user_connections' resource (current value: 10))" error on heroku

This question is related to Resque Mysql2::Error: User has exceeded the 'max_user_connections' resource (current value: 10)
I'm receiving the error "(User has exceeded the 'max_user_connections' resource (current value: 10))" when trying to insert a table into a MySql database.
I'm using this code to connect to the database :
#Configuration
public class MainConfig {
#Bean
public BasicDataSource dataSource() throws URISyntaxException {
URI dbUri = new URI(System.getenv("DATABASE_URL"));
String username = dbUri.getUserInfo().split(":")[0];
String password = dbUri.getUserInfo().split(":")[1];
String dbUrl = "jdbc:postgresql://" + dbUri.getHost() + ':' + dbUri.getPort() + dbUri.getPath();
BasicDataSource basicDataSource = new BasicDataSource();
basicDataSource.setUrl(dbUrl);
basicDataSource.setUsername(username);
basicDataSource.setPassword(password);
return basicDataSource;
}
}
whch is based on :
https://devcenter.heroku.com/articles/connecting-to-relational-databases-on-heroku-with-java#using-the-database_url-in-plain-jdbc
To fix this issue can I re-use this connection ?
I've tried updating the code above to re-use the same BasicDataSource but I receive error.
When I check the client connections here is what I see :
The amount of client connection decreases over time, it seems there is max number of connection that can be made within a specified time span ?

I may be wrong but I believe it is the database its self telling you that you have reached the limit. You can change it in the database options.
For a project I was working we used a connection pool and had a limit of 110 connections. This was based on a small size of users though.

cassandra stop writing after some time

I have a problem: i write entries from java code to cassandra database, it works for a while, and then stops writing. (nodetool cfstats keyspace.users -H on all nodes show no changes in Number of keys (estimate))
Configuration : 4 nodes (4GB, 4GB, 4GB, and 6GB RAM).
I am using datastax driver, and connection like
private Cluster cluster = Cluster.builder()
.addContactPoints(<points>)
.build();
private Session session = cluster.connect("keyspace");
private MappingManager mappingManager = new MappingManager(session);
...
I do insert in database like
public void writeUser(User user) {
Mapper<User> mapper = mappingManager.mapper(User.class);
mapper.saveAsync(user, Mapper.Option.timestamp(TimeUnit.NANOSECONDS.toMicros(System.nanoTime())));
}
I also tried
public void writeUser(User user) {
Mapper<User> mapper = mappingManager.mapper(User.class);
mapper.save(user);
}
And two variants between.
In debug.log from server i see
DEBUG [GossipStage:1] 2016-05-11 12:21:14,565 FailureDetector.java:456 - Ignoring interval time of 2000380153 for /node
Maybe the problem is, that server in another country? But why it is writing entities at the beginning? How can i fix my problem?
Another update: session.execute on mapper.save returns ResultSet[ exhausted: true, Columns[]]