I'm new using Jedis and I'm not sure if the configuration i used is correct.
I'm trying to use Jedis in a API server application as a cache used by my handlers. For this reason the environment is a multi-threaded one and i'm looking for the correct configuration for this scenario.
I found lot of example mentioning the "JedisPool" such as the following:
JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
ShardedJedisPool pool = new ShardedJedisPool(jedisPoolConfig, shards);
try (ShardedJedis jedis = pool.getResource()) {
jedis.set("a", "foo");
}
pool.close();
But in my case i need to use a UnifiedJadis object in order to have access to jsonSet and jsonGet methods, which "normal" Jedis does not provide.
This is what i've done.
This is the class i use to manage a UnifiedJedis object as singleton.
public class MyDB{
//singleton
public static UnifiedJedis jedisPooled;
//called at startup
public void init() throws DBConnectionException {
ConnectionPoolConfig poolConfig = RedisConfig.getRediConfiguration();
jedisPooled = new JedisPooled(poolConfig, dbUrl, dbPort);
}
}
At application startup i initialize the connection and inject the singleton everywhere is needed (e.g. in my handlers).
public class App{
public static void main(String[] args) throws Exception {
MyDB db = new MyDB();
db.init();
//...inject db where needed
}
}
Finally in my handlers I use the variable injected to use the jedisPooled methods jsonGet and jsonSet.
public class MyHandler{
// in the MyHandler constructor i receive the db variable
#Override
public void handle() throws Exception {
//...
db.jedisPooled.jsonSet(redisKey, Path.of("."), myPojo);
}
}
My question is: since there is no method "getResource" in the UnifiedJedis object, i can i get a connection from the pool and then release it after i send a command? Is it automatically done by the UnifiedJedis methods?
Is my solution correct?
Thank you
UnifiedJedis could act differently depending on parameters.
In your object creation, I can see new JedisPooled(poolConfig, dbUrl, dbPort);, which means you are actually creating and using JedisPooled[1]. This narrows down the possible behaviors of UnifiedJedis and so I am providing answer to your questions only for JedisPooled.
The Answer:
In JedisPooled you don't have to get a connection from pool and release it after sending a command. It is automatically done JedisPooled methods.
Note [1]: JedisPooled is a sub-class of UnifiedJedis. That's why you could set a JedisPooled object in a UnifiedJedis variable.
Related
Currently I am specifying the hibernate multi tenant connection provider and resolver in a properties file using this.
hibernate.multi_tenant_connection_provider: com.app.MapMultiTenantConnectionProvider
hibernate.tenant_identifier_resolver: com.app.MultiTenantCurrentTenantIdentifierResolver
Hibernate is using reflection to load these classes. The problem is that I need these classes to have access to certain variables. E.g. The DropWizard config file and users organisation to know what the database URL is and the tenant id. Currently I'm having to make variables static so that the provider has access to it.
The tutorials like this have all the required info specified in the properties file while I need mine to be dynamic depending on the currently connected user.
Here is an example of the kind of thing I'm having to do with static variables. Overall it makes the code quite messy.
public class MapMultiTenantConnectionProvider extends AbstractMultiTenantConnectionProvider {
private static Logger log = LoggerFactory.getLogger(MapMultiTenantConnectionProvider.class);
private Map<String, ConnectionProvider> connectionProviderMap = new HashMap<>();
public MapMultiTenantConnectionProvider() throws IOException {
}
#Override
protected ConnectionProvider getAnyConnectionProvider() {
return getConnectionProviderForTenant("chorus");
}
#Override
protected ConnectionProvider selectConnectionProvider(final String tenantIdentifier) {
return getConnectionProviderForTenant(tenantIdentifier);
}
private ConnectionProvider getConnectionProviderForTenant(final String tenantId) {
final ConnectionProvider connectionProvider;
if (!connectionProviderMap.containsKey(tenantId)) {
// Access a static variable here that contains the database URL, username, etc
final MyConfig config = MyApp.CONFIGURATION;
final Properties properties = new Properties();
properties.setProperty("hibernate.connection.url", config.connectionUrl);
properties.setProperty("hibernate.connection.username", config.username);
properties.setProperty("hibernate.connection.password", config.password);
properties.setProperty("hibernate.dialect", config.databaseConfig.getHibernateDialect());
final DriverManagerConnectionProviderImpl newConnectionProvider = new DriverManagerConnectionProviderImpl();
newConnectionProvider.configure(properties);
this.connectionProviderMap.put(tenantId, newConnectionProvider);
connectionProvider = newConnectionProvider;
} else {
connectionProvider = connectionProviderMap.get(tenantId);
}
return connectionProvider;
}
}
I would like to create an instance and pass in the configuration (or anything else needed) like this.
new MapMultiTenantConnectionProvider(configuration)
Is it possible to specify the provider by creating an instance of it instead of defining it in a properties file?
You could implement the ServiceRegistryAwareService interface to get access to the Hibernate ServiceRegistry which provides you access to almost all Hibernate configurations. I don't know how you'd normally access this dropwizard configuration, but in case it is available through a managed bean, you could access the ManagedBeanRegistry and access the bean that provides this information. Other than that, there is not much you can do. Please note though, that you current implementation is not thread safe. You should be using a ConcurrentHashMap and use putIfAbsent, or even better, computeIfAbsent.
I have this code in my Main() function:
DataStream<OutputObject> asyncResultStream = AsyncDataStream.orderedWait(
listOfData,
new CustomAsyncConnector(),
5,
TimeUnit.SECONDS,
10).setParallelism(3).startNewChain().uid("customUid");
Which is the simple format for using AsyncDataStreams in 1.2. And the code in the CustomAsyncConnector is just like every example you will find at its core:
public class CustomAsyncConnector extends RichAsyncFunction<CustomObject, ResultObject> {
private transient Session client;
#Override
public void open(Configuration parameters) throws Exception {
client = Cluster.builder().addContactPoint("<CustomUrl>")
.withPort(1234)
.build()
.connect("<thisKeyspace>");
}
#Override
public void close() throws Exception {
client.close();
}
#Override
public void asyncInvoke(final CustomObject ctsa, final AsyncCollector<ResultObject> asyncCollector) throws Exception {
//Custom code here...
}
}
Now here are my questions:
1.) What is the proper way to pass "parameters" to the open() function in CustomAsyncConnector() from where it is called in my Main() function.
2.) How are the parameters supposed to be used to set up the connection to the client in the open() function?
My guess on the first question is to create a new CustomAsyncConnector() object instance in main and then call the open() function directly and pass the parameters object to it and then put that instance in the AsysDataStream's code. However I am not sure if this is the best way or, and more importantly, the proper way to set the fields in a Configuration type object (again, assuming that doing "configParameters.setString("contactPointUrl", "127.0.0.1")" is right, but am not sure). And this leads to my second, and honestly most important, question.
So regarding my second question, the parameters I want to pass to the open() function are the contactPointUrl, the portNumber, and the keyspace to be put in .connect(). However I cannot seem to access them by doing something like ".addContactPoint(parameters.getString("contactPointUrl"))". I also tried seeing if or where I should do Cluster.builder().getConfiguration(parameters) but I am shooting in the dark where that even belongs or if at all and if the parameter names have to be something specific and so on.
So I hope I didn't word that too poorly, but any and all help would be greatly appreciated.
Thanks in advance!
Here is what ended up working. Still not sure how to pass the configuration parameters to the .open() method, but oh well.
Added this to the CustomAsyncConnector class:
private final CustomProps props;
public CustomAsyncConnector(CustomProps props) {
super();
this.props = props;
}
And what I pass in the main() method:
AsyncDataStream
.unorderedWait(
dataToProcess,
new CustomAsyncConnector(props),
5,
TimeUnit.SECONDS,
10);
And utilized the props in the .open() method like how I wanted to use the parameters.
I have a question about closing resources (AutoCloseable or not) that are static members of a Java class. In my particular situation I have a class that manages connections to a MongoDB instance using a static MongoClient instance variable. How would I ensure that this client is closed as the documentation recommends when my application terminates? I am using this class as part of the back end to a Java webapp which is run in a container (Tomcat 7). I could not override the Object's finalize() method to close the client because that is called on an instance of the class and would have no effect on static members correct? Here is my example code:
public class MyManager {
//This needs to be closed when the application terminates
private static MongoClient CLIENT;
static {
...
CLIENT = new MongoClient("localhost", 27017);
...
}
public static DB getSomeDB(String dbName) {
return CLIENT.getDB(dbName);
}
//more factory methods
...
//Would this work?
#Override
protected void finalize() throws Throwable {
CLIENT.close();
}
}
Can someone tell me how to best handle this situation and in general with resources such as a database connection or JDBC driver? Thanks!
We are using Spring and simply create a bean which calls the close() method once it's being destroyed:
#Bean(name = "mongoClient", destroyMethod = "close")
public MongoClient mongoClient() throws MongoException, UnknownHostException {
...
I'm using OrmLite to handle persistence in an Android application.
The OrmLite documentation discusses DAO Enabled Objects, along with providing a class you can extend to tell OrmLite you want to have the Dao set on instances of the class that are retrieved from the database.
This has some nice properties, like letting object.update() and object.refresh() DTRT.
For non-database-generated objects, there is an object.setDao(Dao) method you can use.
Would it be problematic to instead just initialise a Dao as a static member variable on the class at start?
public class Order extends BaseDaoEnabled<Order, Integer> {
protected static globalDao = null;
public Order() {
// Set non-static dao used by parent BaseDaoEnabled
this.dao = globalDao;
}
In the main class of the program I would initialise globalDao once with a Dao appropriate for the object.
This would have the nice property of allowing us to do database operations given an instance of the class even without access to OrmLiteSqliteOpenHelper.getDao().
I think this is threadsafe, since my reading of DaoManager indicates there is generally only one Dao per class anyway.
[ Sorry for the late response. ]
Would it be problematic to instead just initialise a Dao as a static member variable on the class at start?
Yes and no. You need to make sure when the application closes that the DAO is set to null so it gets reinitialized when it comes back. The problem is that I've see applications that are stopped but the classes are still in memory. Then if the user re-runs the application the static initializers will not be reinstantiated and old versions of the DAO with now dead connections to the database will be in place.
The right thing to do is to mirror the behavior that the DatabaseHelper class uses in the HelloAndroid project. To paraphrase it:
public class DatabaseHelper extends OrmLiteSqliteOpenHelper {
private Dao<SimpleData, Integer> simpleDao = null;
public Dao<SimpleData, Integer> getDao() throws SQLException {
if (simpleDao == null) {
simpleDao = getDao(SimpleData.class);
}
return simpleDao;
}
#Override
public void close() {
super.close();
simpleDao = null;
}
If I have a Java class defined below that is injected in my web application via dependency injection:
public AccountDao
{
private NamedParameterJdbcTemplate njt;
private List<Account> accounts;
public AccountDao(Datasource ds)
{
this.njt = new NamedParameterJdbcTemplate(ds);
refreshAccounts();
}
/*called at creation, and then via API calls to inform service new users have
been added to the database by a separate program*/
public void refreshAccounts()
{
this.accounts = /*call to database to get list of accounts*/
}
//called by every request to web service
public boolean isActiveAccount(String accountId)
{
Account a = map.get(accountId);
return a == null ? false : a.isActive();
}
}
I am concerned about thread safety. Does the Spring framework not handle cases where one request is reading from the list and it is currently being updated by another? I have used read/write locks before in other applications, but I have never thought about a case such as above before.
I was planning on using the bean as a singleton so I could reduce database load.
By the way, this is a follow up of the below question:
Java Memory Storage to Reduce Database Load - Safe?
EDIT:
So would code like this solve this problem:
/*called at creation, and then via API calls to inform service new users have
been added to the database by a separate program*/
public void refreshAccounts()
{
//java.util.concurrent.locks.Lock
final Lock w = lock.writeLock();
w.lock();
try{
this.accounts = /*call to database to get list of accounts*/
}
finally{
w.unlock();
}
}
//called by every request to web service
public boolean isActiveAccount(String accountId)
{
final Lock r = lock.readLock();
r.lock();
try{
Account a = map.get(accountId);
}
finally{
r.unlock();
}
return a == null ? false : a.isActive();
}
Spring framework does not do anything under the hood concerning the multithreaded behavior of a singleton bean. It is the developer's responsibility to deal with concurrency issue and thread safety of the singleton bean.
I would suggest reading the below article: Spring Singleton, Request, Session Beans and Thread Safety
You could have asked for clarification on my initial answer. Spring does not synchronize access to a bean. If you have a bean in the default scope (singleton), there will only be a single object for that bean, and all concurrent requests will access that object, requiring that object to the thread safe.
Most spring beans have no mutable state, and as such are trivially thread safe. Your bean has mutable state, so you need to ensure no thread sees a list of accounts the other thread is currently assembling.
The easiest way to do that is to make the accounts field volatile. That assumes that you assign the new list to the field after having filled it (as you appear to be doing).
private volatile List<Accounts> accounts;
As a singleton and non-synchronized, Spring will allow any number of threads to concurrently invoke isActiveAccount and refreshAccounts. So, no this class is not going to be thread-safe and will not reduce the database load.
we have many such meta data and have some 11 nodes running. on each app node we have static maps for such data so its one instance only, init from db at start up once in off peak hour every day or when support person triggers it. have an interal simple http post based API to send updates from 1 node to others for some of the data which we need updates in real time.
public AccountDao
{
private static List<Account> accounts;
private static List<String> activeAccounts;
private NamedParameterJdbcTemplate njt;
static {
try{
refreshAccounts();
}catch(Exception e){
//log but do not throw. any uncaught exceptions in static means your class is un-usable
}
}
public AccountDao(Datasource ds)
{
this.njt = new NamedParameterJdbcTemplate(ds);
//refreshAccounts();
}
/*called at creation, and then via API calls to inform service new users have
been added to the database by a separate program*/
public void refreshAccounts()
{
this.accounts = /*call to database to get list of accounts*/
}
public void addAccount(Account acEditedOrAdded)
{
//add or reove from map onr row
//can be called from this node or other node
//meaning if you have 2 nodes, keep IP port of each or use a internal web service or the like to tell
//node B when a account id added or changed in node A ...
}
//called by every request to web service
public static boolean isActiveAccount(String accountId)
{
Account a = map.get(accountId);
return a == null ? false : a.isActive();
}
}