Develop Reference

How does the postgresql (set role user) command use in SSM projects?

Now the project is using springmvc+ spring + mybatis + druid + postgresql
The users in the project correspond to the users in the database, so each time you run SQL, you switch the users with the (set role user) command and then perform the crud operations of the database.
My question:
Because there are many connections in the connection pool, the first step is to get the connection of the database, then switch users, and then perform the operation of business SQL on the database. But I don't know which part of the project this logic should be processed, because the connection of the connection pool and the execution of SQL are implemented by the underlying code. Do you have any good plans?
Can you provide me with a complete demo, such as the following operations:
Step 1, get the user's name from spring security (or shiro).
Step 2, Get the connection currently using the database from the connection pool.
Step 3, execute SQL (set role user) to switch roles.
Step 4, perform crud operation.
Step 5, Reset the database connection(reset role)

Here is a simple way to do what you need with the help of mybatis-spring.
Unless you already use mybatis-spring the first step would be to change the configuration of your project so that you obtain SqlSessionFactory using org.mybatis.spring.SqlSessionFactoryBean provided by mybatis-spring.
The next step is the implementation of setting/resetting the user role for the connection. In mybatis the connection lifecycle is controlled by the class implementing org.apache.ibatis.transaction.Transaction interface. The instance of this class is used by the query executor to get the connection.
In a nutshell you need to create your own implementation of this class and configure mybatis to use it.
Your implementation can be based on the SpringManagedTransaction from mybatis-spring and would look something like:
import org.springframework.security.core.Authentication;
class UserRoleAwareSpringManagedTransaction extends SpringManagedTransaction {
public UserRoleAwareSpringManagedTransaction(DataSource dataSource) {
super(dataSource);
}
#Override
public Connection getConnection() throws SQLException {
Connection connection = getCurrentConnection();
setUserRole(connection);
return connection;
}
private Connection getCurrentConnection() {
return super.getConnection();
}
#Override
public void close() throws SQLException {
resetUserRole(getCurrentConnection());
super.close();
}
private void setUserRole(Connection connection) {
Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
String username = authentication.getName();
Statement statement = connection.createStatement();
try {
// note that this direct usage of usernmae is a subject for SQL injection
// so you need to use the suggestion from
// https://stackoverflow.com/questions/2998597/switch-role-after-connecting-to-database
// about encoding of the username
statement.execute("set role '" + username + "'");
} finally {
statement.close();
}
}
private void resetUserRole(Connection connection) {
Statement statement = connection.createStatement();
try {
statement.execute("reset role");
} finally {
statement.close();
}
}
}
Now you need to configure mybatis to use you Transaction implementation. For this you need to implement TransactionFactory similar to org.mybatis.spring.transaction.SpringManagedTransactionFactory provided by mybatis-spring:
public class UserRoleAwareSpringManagedTransactionFactory implements TransactionFactory {
#Override
public Transaction newTransaction(DataSource dataSource, TransactionIsolationLevel level, boolean autoCommit) {
return new UserRoleAwareSpringManagedTransaction(dataSource);
}
#Override
public Transaction newTransaction(Connection conn) {
throw new UnsupportedOperationException("New Spring transactions require a DataSource");
}
#Override
public void setProperties(Properties props) {
}
}
And then define a bean of type UserRoleAwareSpringManagedTransactionFactory in your spring context and inject it into transactionFactory property of the SqlSessionFactoryBeen in your spring context.
Now every time mybatis obtains a Connection the implementation of Transaction will set the current spring security user to set the role.

Best practice is that database users are applications. Application users' access to particular data/resource should be controlled in the application. Applications should not rely on database to restrict data/resource access. Therefore, application users should not have different roles in database. An application should use only a single database user account.
Spring is manifestation of best practices. Therefore, Spring does not implement this functionality. If you want such functionality, you need to hack.
Referring to this, your best bet is to:
#Autowired JdbcTemplate jdbcTemplate;
// ...
public runPerUserSql() {
jdbcTemplate.execute("set role user 'user_1';");
jdbcTemplate.execute("SELECT 1;");
}
I still do not have much confidence in this. Unless you are writing a pgAdmin webapp for multiple users, you should re-consider your approach and design.

Mocking a Nested Groovy Method With Mockito

I'm trying to do some testing on a piece of legacy code and I have hit a bit of a wall. The code is part of a backend server for one of our angular web applications. The particular code I need to test is responsible for managing the creation of a sales rep account using data passed in from the client. In addition to saving the new rep to our mongo db, the code also has to handle saving to an external sql db. The work flow looks something like this:
Receive 'put' request from client.
Create a new 'rep' object with the passed in data.
Save the rep to a mongo db.
Call a Groovy class that will do an 'insert' on a remote db and return the id of the remote record.
Save the remote id into the mongo data.
Normally, I would use Mockito to mock out the sql connection, but I couldn't get that to work for this case. My next thought was to try mocking the Groovy class instead. I don't actually care about the internals of the groovy method, I just need an Id back. So far that has not worked either. The reason I suspect is that the groovy method is getting called from a protected method inside my service class. I don't have any control over the signature of this method, it is an override from another library.
Is there anything I can do to be able to test this code without having to set up an actual connection to the Sql db?
Web Service:
#Override
protected void beforeInsert() {
super.beforeInsert();
final Rep weakRep = this;
dwPhase1 = injector.getInstance(DwPhase1.class);
return dwPhase1.insertRepDetail(weakRep);
}
Mocking code:
DwPhase1 dwGroovy = Mockito.mock(DwPhase1.class);
Mockito.when(dwGroovy.insertRepDetail(Mockito.any(Rep.class))).then(new Answer<Object>() {
#Override
public Object answer(InvocationOnMock invocation) throws Throwable {
// TODO Auto-generated method stub
return "hello";
}
});
}
Groovy snippet:
class DwPhase1 extends SQL{
public Number insertRepDetail(Rep r){
String insert="""insert into REP_DETAIL (some values)""";
List<List<Object>> rows =null;
sql.withTransaction {
if(update!=null){
sql.executeUpdate(update);
}
rows = sql.executeInsert(params,insert.toString());
}
Number dwId = null;
if(rows!=null && !rows.isEmpty()){
List columns=rows.get(0);
if(columns!=null && !columns.isEmpty()){
dwId = columns.get(0);
}
}
return dwId;
}
}

Override default Spring-Boot application.properties settings in Junit Test with dynamic value

I want to override properties defined in application.properties in tests, but #TestPropertySource only allows to provide predefined values.
What I need is to start a server on a random port N, then pass this port to spring-boot application. The port has to be ephemeral to allow running multiple tests on the same host at the same time.
I don't mean the embedded http server (jetty), but some different server that is started at the beginning of the test (e.g. zookeeper) and the application being tested has to connect to it.
What's the best way to achieve this?
(here's a similar question, but answers do not mention a solution for ephemeral ports - Override default Spring-Boot application.properties settings in Junit Test)

As of Spring Framework 5.2.5 and Spring Boot 2.2.6 you can use Dynamic Properties in tests:
#DynamicPropertySource
static void dynamicProperties(DynamicPropertyRegistry registry) {
registry.add("property.name", "value");
}

Thanks to the changes made in Spring Framework 5.2.5, the use of #ContextConfiguration and the ApplicationContextInitializer can be replaced with a static #DynamicPropertySource method that serves the same purpose.
#SpringBootTest
#Testcontainers
class SomeSprintTest {
#Container
static LocalStackContainer localStack =
new LocalStackContainer().withServices(LocalStackContainer.Service.S3);
#DynamicPropertySource
static void initialize(DynamicPropertyRegistry registry) {
AwsClientBuilder.EndpointConfiguration endpointConfiguration =
localStack.getEndpointConfiguration(LocalStackContainer.Service.S3);
registry.add("cloud.aws.s3.default-endpoint", endpointConfiguration::getServiceEndpoint);
}
}

You could override the value of the port property in the #BeforeClass like this:
#BeforeClass
public static void beforeClass() {
System.setProperty("zookeeper.port", getRandomPort());
}

The "clean" solution is to use an ApplicationContextInitializer.
See this answer to a similar question.
See also this github issue asking a similar question.
To summarize the above mentioned posts using a real-world example that's been sanitized to protect copyright holders (I have a REST endpoint which uses an #Autowired DataSource which needs to use the dynamic properties to know which port the in-memory MySQL database is using):
Your test must declare the initializer (see the #ContextConfiguration line below).
// standard spring-boot test stuff
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#ActiveProfiles("local")
#ContextConfiguration(
classes = Application.class,
// declare the initializer to use
initializers = SpringTestDatabaseInitializer.class)
// use random management port as well so we don't conflict with other running tests
#TestPropertySource(properties = {"management.port=0"})
public class SomeSprintTest {
#LocalServerPort
private int randomLocalPort;
#Value("${local.management.port}")
private int randomManagementPort;
#Test
public void testThatDoesSomethingUseful() {
// now ping your service that talks to the dynamic resource
}
}
Your initializer needs to add the dynamic properties to your environment. Don't forget to add a shutdown hook for any cleanup that needs to run. Following is an example that sets up an in-memory database using a custom DatabaseObject class.
public class SpringTestDatabaseInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
private static final int INITIAL_PORT = 0; // bind to an ephemeral port
private static final String DB_USERNAME = "username";
private static final String DB_PASSWORD = "password-to-use";
private static final String DB_SCHEMA_NAME = "default-schema";
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
DatabaseObject databaseObject = new InMemoryDatabaseObject(INITIAL_PORT, DB_USERNAME, DB_PASSWORD, DB_SCHEMA_NAME);
registerShutdownHook(databaseObject);
int databasePort = startDatabase(databaseObject);
addDatabasePropertiesToEnvironment(applicationContext, databasePort);
}
private static void addDatabasePropertiesToEnvironment(ConfigurableApplicationContext applicationContext, int databasePort) {
String url = String.format("jdbc:mysql://localhost:%s/%s", databasePort, DB_SCHEMA_NAME);
System.out.println("Adding db props to environment for url: " + url);
TestPropertySourceUtils.addInlinedPropertiesToEnvironment(
applicationContext,
"db.port=" + databasePort,
"db.schema=" + DB_SCHEMA_NAME,
"db.url=" + url,
"db.username=" + DB_USERNAME,
"db.password=" + DB_PASSWORD);
}
private static int startDatabase(DatabaseObject database) {
try {
database.start();
return database.getBoundPort();
} catch (Exception e) {
throw new IllegalStateException("Failed to start database", e);
}
}
private static void registerShutdownHook(DatabaseObject databaseObject) {
Runnable shutdownTask = () -> {
try {
int boundPort = databaseObject.getBoundPort();
System.out.println("Shutting down database at port: " + boundPort);
databaseObject.stop();
} catch (Exception e) {
// nothing to do here
}
};
Thread shutdownThread = new Thread(shutdownTask, "Database Shutdown Thread");
Runtime.getRuntime().addShutdownHook(shutdownThread);
}
}
When I look at the logs, it shows that for both of my tests that use this initializer class, they use the same object (the initialize method only gets called once, as does the shutdown hook). So it starts up a database, and leaves it running until both tests finish, then shuts the database down.

Updating Dropwizard config at runtime

Is it possible to have my app update the config settings at runtime? I can easily expose the settings I want in my UI but is there a way to allow the user to update settings and make them permanent ie save them to the config.yaml file? The only way I can see it to update the file by hand then restart the server which seems a bit limiting.

Yes. It is possible to reload the service classes at runtime.
Dropwizard by itself does not have the way to reload the app, but jersey has.
Jersey uses a container object internally to maintain the running application. Dropwizard uses the ServletContainer class of Jersey to run the application.
How to reload the app without restarting it -
Get a handle to the container used internally by jersey
You can do this by registering a AbstractContainerLifeCycleListener in Dropwizard Environment before starting the app. and implement its onStartup method as below -
In your main method where you start the app -
//getting the container instance
environment.jersey().register(new AbstractContainerLifecycleListener() {
#Override
public void onStartup(Container container) {
//initializing container - which will be used to reload the app
_container = container;
}
});
Add a method to your app to reload the app. It will take in the list of string which are the names of the service classes you want to reload. This method will call the reload method of the container with the new custom DropWizardConfiguration instance.
In your Application class
public static synchronized void reloadApp(List<String> reloadClasses) {
DropwizardResourceConfig dropwizardResourceConfig = new DropwizardResourceConfig();
for (String className : reloadClasses) {
try {
Class<?> serviceClass = Class.forName(className);
dropwizardResourceConfig.registerClasses(serviceClass);
System.out.printf(" + loaded class %s.\n", className);
} catch (ClassNotFoundException ex) {
System.out.printf(" ! class %s not found.\n", className);
}
}
_container.reload(dropwizardResourceConfig);
}
For more details see the example documentation of jersey - jersey example for reload
Consider going through the code and documentation of following files in Dropwizard/Jersey for a better understanding -
Container.java
ContainerLifeCycleListener.java
ServletContainer.java
AbstractContainerLifeCycleListener.java
DropWizardResourceConfig.java
ResourceConfig.java

No.
Yaml file is parsed at startup and given to the application as Configuration object once and for all. I believe you can change the file after that but it wouldn't affect your application until you restart it.
Possible follow up question: Can one restart the service programmatically?
AFAIK, no. I've researched and read the code somewhat for that but couldn't find a way to do that yet. If there is, I'd love to hear that :).

I made a task that reloads the main yaml file (it would be useful if something in the file changes). However, it is not reloading the environment. After researching this, Dropwizard uses a lot of final variables and it's quite hard to reload these on the go, without restarting the app.
class ReloadYAMLTask extends Task {
private String yamlFileName;
ReloadYAMLTask(String yamlFileName) {
super("reloadYaml");
this.yamlFileName = yamlFileName;
}
#Override
public void execute(ImmutableMultimap<String, String> parameters, PrintWriter output) throws Exception {
if (yamlFileName != null) {
ConfigurationFactoryFactory configurationFactoryFactory = new DefaultConfigurationFactoryFactory<ReportingServiceConfiguration>();
ValidatorFactory validatorFactory = Validation.buildDefaultValidatorFactory();
Validator validator = validatorFactory.getValidator();
ObjectMapper objectMapper = Jackson.newObjectMapper();
final ConfigurationFactory<ServiceConfiguration> configurationFactory = configurationFactoryFactory.create(ServiceConfiguration.class, validator, objectMapper, "dw");
File confFile = new File(yamlFileName);
configurationFactory.build(new File(confFile.toURI()));
}
}
}

You can change the configuration in the YAML and read it while your application is running. This will not however restart the server or change any server configurations. You will be able to read any changed custom configurations and use them. For example, you can change the logging level at runtime or reload other custom settings.
My solution -
Define a custom server command. You should use this command to start your application instead of the "server" command.
ArgsServerCommand.java
public class ArgsServerCommand<WC extends WebConfiguration> extends EnvironmentCommand<WC> {
private static final Logger LOGGER = LoggerFactory.getLogger(ArgsServerCommand.class);
private final Class<WC> configurationClass;
private Namespace _namespace;
public static String COMMAND_NAME = "args-server";
public ArgsServerCommand(Application<WC> application) {
super(application, "args-server", "Runs the Dropwizard application as an HTTP server specific to my settings");
this.configurationClass = application.getConfigurationClass();
}
/*
* Since we don't subclass ServerCommand, we need a concrete reference to the configuration
* class.
*/
#Override
protected Class<WC> getConfigurationClass() {
return configurationClass;
}
public Namespace getNamespace() {
return _namespace;
}
#Override
protected void run(Environment environment, Namespace namespace, WC configuration) throws Exception {
_namespace = namespace;
final Server server = configuration.getServerFactory().build(environment);
try {
server.addLifeCycleListener(new LifeCycleListener());
cleanupAsynchronously();
server.start();
} catch (Exception e) {
LOGGER.error("Unable to start server, shutting down", e);
server.stop();
cleanup();
throw e;
}
}
private class LifeCycleListener extends AbstractLifeCycle.AbstractLifeCycleListener {
#Override
public void lifeCycleStopped(LifeCycle event) {
cleanup();
}
}
}
Method to reload in your Application -
_ymlFilePath = null; //class variable
public static boolean reloadConfiguration() throws IOException, ConfigurationException {
boolean reloaded = false;
if (_ymlFilePath == null) {
List<Command> commands = _configurationBootstrap.getCommands();
for (Command command : commands) {
String commandName = command.getName();
if (commandName.equals(ArgsServerCommand.COMMAND_NAME)) {
Namespace namespace = ((ArgsServerCommand) command).getNamespace();
if (namespace != null) {
_ymlFilePath = namespace.getString("file");
}
}
}
}
ConfigurationFactoryFactory configurationFactoryFactory = _configurationBootstrap.getConfigurationFactoryFactory();
ValidatorFactory validatorFactory = _configurationBootstrap.getValidatorFactory();
Validator validator = validatorFactory.getValidator();
ObjectMapper objectMapper = _configurationBootstrap.getObjectMapper();
ConfigurationSourceProvider provider = _configurationBootstrap.getConfigurationSourceProvider();
final ConfigurationFactory<CustomWebConfiguration> configurationFactory = configurationFactoryFactory.create(CustomWebConfiguration.class, validator, objectMapper, "dw");
if (_ymlFilePath != null) {
// Refresh logging level.
CustomWebConfiguration webConfiguration = configurationFactory.build(provider, _ymlFilePath);
LoggingFactory loggingFactory = webConfiguration.getLoggingFactory();
loggingFactory.configure(_configurationBootstrap.getMetricRegistry(), _configurationBootstrap.getApplication().getName());
// Get my defined custom settings
CustomSettings customSettings = webConfiguration.getCustomSettings();
reloaded = true;
}
return reloaded;
}

Although this feature isn't supported out of the box by dropwizard, you're able to accomplish this fairly easy with the tools they give you.
Before I get started, note that this isn't a complete solution for the question asked as it doesn't persist the updated config values to the config.yml. However, this would be easy enough to implement yourself simply by writing to the config file from the application. If anyone would like to write this implementation feel free to open a PR on the example project I've linked below.
Code
Start off with a minimal config:
config.yml
myConfigValue: "hello"
And it's corresponding configuration file:
ExampleConfiguration.java
public class ExampleConfiguration extends Configuration {
private String myConfigValue;
public String getMyConfigValue() {
return myConfigValue;
}
public void setMyConfigValue(String value) {
myConfigValue = value;
}
}
Then create a task which updates the config:
UpdateConfigTask.java
public class UpdateConfigTask extends Task {
ExampleConfiguration config;
public UpdateConfigTask(ExampleConfiguration config) {
super("updateconfig");
this.config = config;
}
#Override
public void execute(Map<String, List<String>> parameters, PrintWriter output) {
config.setMyConfigValue("goodbye");
}
}
Also for demonstration purposes, create a resource which allows you to get the config value:
ConfigResource.java
#Path("/config")
public class ConfigResource {
private final ExampleConfiguration config;
public ConfigResource(ExampleConfiguration config) {
this.config = config;
}
#GET
public Response handleGet() {
return Response.ok().entity(config.getMyConfigValue()).build();
}
}
Finally wire everything up in your application:
ExampleApplication.java (exerpt)
environment.jersey().register(new ConfigResource(configuration));
environment.admin().addTask(new UpdateConfigTask(configuration));
Usage
Start up the application then run:
$ curl 'http://localhost:8080/config'
hello
$ curl -X POST 'http://localhost:8081/tasks/updateconfig'
$ curl 'http://localhost:8080/config'
goodbye
How it works
This works simply by passing the same reference to the constructor of ConfigResource.java and UpdateConfigTask.java. If you aren't familiar with the concept see here:
Is Java "pass-by-reference" or "pass-by-value"?
The linked classes above are to a project I've created which demonstrates this as a complete solution. Here's a link to the project:
scottg489/dropwizard-runtime-config-example
Footnote: I haven't verified this works with the built in configuration. However, the dropwizard Configuration class which you need to extend for your own configuration does have various "setters" for internal configuration, but it may not be safe to update those outside of run().
Disclaimer: The project I've linked here was created by me.

How to implement transaction in Spring Data Redis in a clean way?

I am following RetwisJ tutorial available here. In this I don't think Redis transactions are implemented. For example, in the following function, if some exception occurs in between, the data will be left in an inconsistent state.
I want to know how a function like the following can be implemented in Spring Data Redis as a single transaction:
public String addUser(String name, String password) {
String uid = String.valueOf(userIdCounter.incrementAndGet());
// save user as hash
// uid -> user
BoundHashOperations<String, String, String> userOps = template.boundHashOps(KeyUtils.uid(uid));
userOps.put("name", name);
userOps.put("pass", password);
valueOps.set(KeyUtils.user(name), uid);
users.addFirst(name);
return addAuth(name);
}
Here userIdCounter, valueOps and users are initialized in the constructor. I have come across this in the documentation(section 4.8), but I can't figure out how to fit that into this function where some variables are initialized outside the function(please don't tell I have to initialize these variables in each and every function where I need transactions!).
PS: Also is there any #Transaction annotation or transaction manager available for Spring Data Redis?
UPDATE: I have tried using MULTI, EXEC. The code which I have written is for another project, but when its applied to this problem it'll be as follows:
public String addMyUser(String name, String password) {
String uid = String.valueOf(userIdCounter.incrementAndGet());
template.execute(new SessionCallback<Object>() {
#Override
public <K, V> Object execute(RedisOperations<K, V> operations)
throws DataAccessException {
operations.multi();
getUserOps(operations, KeyUtils.uid(uid)).put("name", name);
getUserOps(operations, KeyUtils.uid(uid)).put("pass", password);
getValueOps(operations).set(KeyUtils.user(name), uid);
getUserList(operations, KeyUtils.users()).leftPush(name);
operations.exec();
return null;
}
});
return addAuth(name);
}
private ValueOperations<String, String> getValueOps(RedisOperations operations) {
return operations.opsForValue();
}
private BoundHashOperations<String, String, String> getUserOps(RedisOperations operations, String key) {
return operations.boundHashOps(key);
}
private BoundListOperations<String, String> getUserList(RedisOperations operations, String key) {
return operations.boundListOps(key);
}
Please tell whether this way of using MULTI, EXEC is recommended or not.

Up to SD Redis 1.2 you will have to take care of tansaction handling yourself using TransactionSynchronisationManager
The snipplet above could then look something like this:
public String addUser(String name, String password) {
String uid = String.valueOf(userIdCounter.incrementAndGet());
// start the transaction
template.multi();
// register synchronisation
if(TransactionSynchronisationManager.isActualTransactionActive()) {
TransactionSynchronisationManager.registerSynchronisation(new TransactionSynchronizationAdapter()) {
#Override
public void afterCompletion(int status) {
switch(status) {
case STATUS_COMMITTED : template.exec(); break;
case STATUS_ROLLED_BACK : template.discard(); break;
default : template.discard();
}
}
}
}
BoundHashOperations<String, String, String> userOps = template.boundHashOps(KeyUtils.uid(uid));
userOps.put("name", name);
userOps.put("pass", password);
valueOps.set(KeyUtils.user(name), uid);
users.addFirst(name);
return addAuth(name);
}
Please note that once in multi, read operations will also be part of the transaction which means you'll likely not be able to read data from redis server.
The setup might differ form the above one as you could want to additionally call WATCH. Further on you'll also have to take care of multiple callbacks do not sending MULTI and/or EXEC more than once.
The upcoming 1.3 RELEASE of Spring Data Redis will ship with support for spring managed transactions in a way of taking care of MULTi|EXEC|DISCARD plus allowing read operations (on already existing keys) while transaction synchronization is active. You could already give the BUILD-SNAPSHOT a spin and turn this on by setting template.setEnableTransactionSupport(true).

By default, RedisTemplate does not participate in managed Spring
transactions. If you want RedisTemplate to make use of Redis
transaction when using #Transactional or TransactionTemplate, you need
to be explicitly enable transaction support for each RedisTemplate by
setting setEnableTransactionSupport(true). Enabling transaction
support binds RedisConnection to the current transaction backed by a
ThreadLocal. If the transaction finishes without errors, the Redis
transaction gets commited with EXEC, otherwise rolled back with
DISCARD. Redis transactions are batch-oriented. Commands issued during
an ongoing transaction are queued and only applied when committing the
transaction.
Spring Data Redis distinguishes between read-only and write commands
in an ongoing transaction. Read-only commands, such as KEYS, are piped
to a fresh (non-thread-bound) RedisConnection to allow reads. Write
commands are queued by RedisTemplate and applied upon commit.
https://docs.spring.io/spring-data/data-redis/docs/current/reference/html/#tx.spring