I have a class that uses a static cache that is shared between all instances of the class. I'd like to be able to set the timeout of the cache at runtime.
To provide a concrete use case: I cache values fetched from cloud storage. I'd like to refresh the values much quicker in development environments than in prod. When deploying the code, it takes an argument for the config file corresponding to that environment. This config file can contain a value for the cache refresh time.
public class Pipeline {
private static final LoadingCache<BlobId, Definitions> CACHE =
CacheBuilder.newBuilder()
.refreshAfterWrite(VALUE, TimeUnit.MINUTES) // <-- how do I set VALUE from a config file?
.build(
new CacheLoader<BlobId, Definitions>() {
public Definitions load(BlobId key) throws Exception {
return DefinitionsLoader.load(key);
}
});
...
}
To dynamically load different configurations at runtime, you can use a .properties file. In the example below I load the properties file in a static block, but you can also implement the logic in a static method that initializes the cache.
https://docs.oracle.com/javase/tutorial/essential/environment/properties.html
private static final LoadingCache<BlobId, Definitions> CACHE;
static {
Properties prop = new Properties();
try {
prop.load(new FileInputStream("config.properties"));
} catch (IOException e) {
// handle exception
}
Long value = Long.parseLong(prop.getProperty("value", "5"));
CACHE = CacheBuilder.newBuilder()
.refreshAfterWrite(value, TimeUnit.MINUTES)
.build(new CacheLoader<Integer, Definitions>() {
public Definitions load(BlobId key) throws Exception {
return DefinitionsLoader.load(key);
}
});
}
Static field initialized in their declaration are not designed to be parameterized as you want to do.
Besides, your cache loading is not flexible.
If tomorrow you change your mind or you want to use multiple of them, you cannot.
At last, it is not testable either.
If you want to provide a specific behavior to the cache loading, the most natural way to do it is changing the API of the class that contains the field.
You could provide a Pipeline constructor with a long delayInMnToRefresh parameter and you could use this parameter to set the refresh time of the cache.
public Pipeline(int delayInMnToRefresh){
CacheBuilder.newBuilder()
.refreshAfterWrite(delayInMnToRefresh, TimeUnit.MINUTES)
....
}
If you use Spring, your could using a #Autowired constructor that uses a property defined at runtime when the Spring context is loaded :
#Autowired
public Pipeline(#Value("${clould.cache.delayInMnToRefresh}") int delayInMnToRefresh){
....
}
With a property defined in this way for example in env :
clould.cache.delayInMnToRefresh=5
And another property defined in this way for example in prod :
clould.cache.delayInMnToRefresh=15
Of course, you can implement this requirement without Spring (and Spring Boot) but you should simply perform more boiler plate tasks (load the properties file before to call this method and handle yourself the notion of environment).
Related
I have built a spring boot application for the first time. I have about 10-12 user specific caches and 1 global cache in my application. User level caches are HashMap based.
Class userCache1 {
private Map<Integer, MyObject> dataMap = new HashMap<>();
public Map<Integer, MyObject> getDataMap() {
return dataMap;
}
public void setDataMap(Map<Integer, MyData> dataMap) {
this.dataMap = dataMap;
}
public dataMap get(int key) {
if(dataMap.containsKey(key))
return dataMap.get(key);
else
return null;
}
public void setData(int dataMap, MyData data) {
dataMap.put(key, data);
}
}
Now there has to be one instance of userCache1 for each registered user. I have similarly objects each holding various kinds of data that are required by multiple objects for the application to process the user requests. Some of the classes would required 5-10 of these caches such as userCache1...userCache5.
Below are my questions:
How would I tie the userCache1 to userId which is the primary key of the user of this application? Do I need another class that holds userId and userCache1? Wouldn't it create one such class for each of these user level caches just for the purpose of associating the cache with userId? Are there better options that I can use?
Based on the approach I take in #1, I would like to use DI to inject the required userCacheX objects to the constructor of the classes that require these caches to process the biz logic. I don't think that having a constructor with 5+ such caches is an elegant approach. What makes the most sense in this example as below
public class UserCache {
private UserCache1 cache1;
private UserCache2 cache2;
...
private UserCacheN cacheN;
UserCache(UserCache1...UserCacheN) {
this.cache1 = cache1;
....
this.cacheN = cacheN;
}
/*
use the cache1...cacheN as needed in the body of this class
*/
}
In addition to user specific cache I have 1 or 2 global cache as well. I understand that spring enables caching which I would be using once I decide on the high level design.
All my user specific caches are HashMap based at this time. The content of most of these caches would NOT change during lifetime of the user once registered but I would assume that i should expire the caches to manage memory efficiently and optimize performance which can happen once I successfully build the application.
I am planning to use Spring Boot, Hibernate to persist these objects into the datastore.
What are my recommended design options for this problem?
I'm using the #PropertyInject annotation to get properties from the application.properties file to use in my beans.
This normally works fine, but now I need to be able to change the injected property based on a header value.
In my head it looks something like this:
#PropertyInject(PROPERTY_NAME)
private String property;
public void pickProperty(String msgVersion) {
if (msgVersion.equals("A")) {
PROPERTY_NAME = "property.firstType.name";
} else {
PROPERTY_NAME = "property.secondType.name";
}
}
I've considered just injecting both properties and deciding in the main method which one to use, but that seems like a roundabout way of doing things and will get a bit bloated if more versions are added.
Is there an easy way this can be done?
now I need to be able to change the injected property based on a header value
Properties and Beans are created on application startup and typically do not change while the application is running. They both have application scope.
Header values on the other hand can change for every message that is processed by your application.
As you suggested yourself: You can inject both properties into the Bean and provide a method that is called once per message to get the correct value
#PropertyInject(PROPERTY_A)
private String propertyA;
#PropertyInject(PROPERTY_B)
private String propertyB;
// called for every message processing
public String pickProperty(#Header("msgVersion") String msgVersion) {
if (msgVersion.equals("A")) {
return propertyA;
} else {
return propertyB;
}
}
This is not at all a workaround, but simply a method that returns a different result based on the input.
I have essentially the same question as here but am hoping to get a less vague, more informative answer.
I'm looking for a way to configure DropWizard programmatically, or at the very least, to be able to tweak configs at runtime. Specifically I have a use case where I'd like to configure metrics in the YAML file to be published with a frequency of, say, 2 minutes. This would be the "normal" default. However, under certain circumstances, I may want to speed that up to, say, every 10 seconds, and then throttle it back to the normal/default.
How can I do this, and not just for the metrics.frequency property, but for any config that might be present inside the YAML config file?
Dropwizard reads the YAML config file and configures all the components only once on startup. Neither the YAML file nor the Configuration object is used ever again. That means there is no direct way to configure on run-time.
It also doesn't provide special interfaces/delegates where you can manipulate the components. However, you can access the objects of the components (usually; if not you can always send a pull request) and configure them manually as you see fit. You may need to read the source code a bit but it's usually easy to navigate.
In the case of metrics.frequency you can see that MetricsFactory class creates ScheduledReporterManager objects per metric type using the frequency setting and doesn't look like you can change them on runtime. But you can probably work around it somehow or even better, modify the code and send a Pull Request to dropwizard community.
Although this feature isn't supported out of the box by dropwizard, you're able to accomplish this fairly easy with the tools they give you. Note that the below solution definitely works on config values you've provided, but it may not work for built in configuration values.
Also note that this doesn't persist the updated config values to the config.yml. However, this would be easy enough to implement yourself simply by writing to the config file from the application. If anyone would like to write this implementation feel free to open a PR on the example project I've linked below.
Code
Start off with a minimal config:
config.yml
myConfigValue: "hello"
And it's corresponding configuration file:
ExampleConfiguration.java
public class ExampleConfiguration extends Configuration {
private String myConfigValue;
public String getMyConfigValue() {
return myConfigValue;
}
public void setMyConfigValue(String value) {
myConfigValue = value;
}
}
Then create a task which updates the config:
UpdateConfigTask.java
public class UpdateConfigTask extends Task {
ExampleConfiguration config;
public UpdateConfigTask(ExampleConfiguration config) {
super("updateconfig");
this.config = config;
}
#Override
public void execute(Map<String, List<String>> parameters, PrintWriter output) {
config.setMyConfigValue("goodbye");
}
}
Also for demonstration purposes, create a resource which allows you to get the config value:
ConfigResource.java
#Path("/config")
public class ConfigResource {
private final ExampleConfiguration config;
public ConfigResource(ExampleConfiguration config) {
this.config = config;
}
#GET
public Response handleGet() {
return Response.ok().entity(config.getMyConfigValue()).build();
}
}
Finally wire everything up in your application:
ExampleApplication.java (exerpt)
environment.jersey().register(new ConfigResource(configuration));
environment.admin().addTask(new UpdateConfigTask(configuration));
Usage
Start up the application then run:
$ curl 'http://localhost:8080/config'
hello
$ curl -X POST 'http://localhost:8081/tasks/updateconfig'
$ curl 'http://localhost:8080/config'
goodbye
How it works
This works simply by passing the same reference to the constructor of ConfigResource.java and UpdateConfigTask.java. If you aren't familiar with the concept see here:
Is Java "pass-by-reference" or "pass-by-value"?
The linked classes above are to a project I've created which demonstrates this as a complete solution. Here's a link to the project:
scottg489/dropwizard-runtime-config-example
Footnote: I haven't verified this works with the built in configuration. However, the dropwizard Configuration class which you need to extend for your own configuration does have various "setters" for internal configuration, but it may not be safe to update those outside of run().
Disclaimer: The project I've linked here was created by me.
I solved this with bytecode manipulation via Javassist
In my case, I wanted to change the "influx" reporter
and modifyInfluxDbReporterFactory should be ran BEFORE dropwizard starts
private static void modifyInfluxDbReporterFactory() throws Exception {
ClassPool cp = ClassPool.getDefault();
CtClass cc = cp.get("com.izettle.metrics.dw.InfluxDbReporterFactory"); // do NOT use InfluxDbReporterFactory.class.getName() as this will force the class into the classloader
CtMethod m = cc.getDeclaredMethod("setTags");
m.insertAfter(
"if (tags.get(\"cloud\") != null) tags.put(\"cloud_host\", tags.get(\"cloud\") + \"_\" + host);tags.put(\"app\", \"sam\");");
cc.toClass();
}
What I want to do is load key/value pairs from a file (excel file using Apache poi) into a static map that will be used as a lookup table. Once loaded the table will not change.
public final class LookupTable
{
private final static Map<String, String> map;
static {
map = new HashMap<String, String>();
// should do initialization here
// InputStream is = new FileInputStream(new File("pathToFile"));
// not sure how to pass pathToFile without hardcoding it?
}
private LookupTable() {
}
public static void loadTable(InputStream is) {
// read table from file
// load it into map
map.put("regex", "value");
}
public static String getValue(String key) {
return map.get(key);
}
}
Ideally I want to load the map within the static initialization block, but how would I pass the stream in without hard coding it? The problem I see using the loadTable static method is it might not be called before calling the other static methods.
// LookupTable.loadTable(stream);
LookupTable.getValue("regex"); // null since map was never populated.
Is there a better approach to this?
Anything you use will have to be accessible at startup. As far as I know, your options are:
Hard-code the path. This is bad for obvious reasons.
A static variable or static method. This presents a bit of a chicken-and-egg problem; ultimately it gets hard-coded, but at least you can do a search with a static method.
Use a variable, either Java or Environment. So, you'd use something System.getProperty("filename", "/default/filename"). Better because it's at least customizable using the environment or -D parameters at JVM startup.
Use the ClassLoader getResource* methods. This is probably The Right Answer. Specifically, you'll probably want to use the getResourceAsStream() method on the current thread's context ClassLoader Thread.currentThread().getContextClassLoader(). (So, Thread.currentThread().getContextClassLoader().getResourceAsStream("filename") in total.) The ClassLoader will then find your resource for you (as long as you put it somewhere sane in your CLASSPATH).
Yes, there is a better approach, use the Factory design pattern to initialise the object before you have to use it:
http://www.oodesign.com/factory-pattern.html
You cannot pass information into static initialization blocks - they are supposed to work in isolation. Since the stream that you are planning to pass needs to be known before the program begins execution, presumably your LookupTable should be able to find it too. For example, this could be some sort of configuration utility that provides the stream for you. Then you can write your initializer like this:
static {
InputStream exelStream = MyConfigUtil.getExcelStreamForLookup();
loadTable(exelStream);
}
Presumably, there is a class in the system that could get your Excel stream from a source that is known to it. The source does not need to be hard-coded: it could read the location from a configuration file, or receive the data from a predefined network location on your server. In all cases the process of getting the Excel stream has to "bottom out" somewhere, in the sense that something in your system needs to be able to find it without additional parameters.
This is not directly answering your question, but I don't see why map has to be static. You could change map to non-static and change the constructor to public LookupTable(File file) {...fill map...}. You could then even have many instances of that class if you have different excel files; it might not be the case now, but it would "future-proof" your code.
This is probably a case for using lazy loading of the map.
But you will need to set the inputFileName before calling getValue() the first time. This would be done in your initialization code for the applications. (Or you could have a static method to set it.)
This points out the advantage of lazy loading. You don't have to have the file name available until you call getValue() the first time. With a static initializer, you have to get the file name stored somewhere outside the class so it can be used to load the data when the class loads (but after the static fields have been initialized.
public static String inputFileName = null;
public static String getValue(String key) {
if (map == null) {
map = = new HashMap<String, String>();
// open the file using 'inputFileName'
loadTable(InputStream is);
}
return map.get(key);
}
If your code is multithreaded, let me know and I'll comment on the synchronization issues.
Alternate
You could also use Spring to inject the map and build it in some other class -- MapBuilder for example.
Try using System.getProperty() and pass parameter with -D in command line.
static String prop;
static {
prop = System.getProperty("java.home");
}
public static void main(String... args) {
System.out.println(prop);
}
I would suggest a singleton enum approach if it suits your case.
public enum LookupTable {
INSTANCE(FileManager.getFileName());
LookupTable(String fileName){
props = new HashMap<String,String>();
//Read from excel and fill the hashmap
}
private final Map<String, String> props;
public String getMapValue(String key){
return props.get(key);
}
}
Which can be called by
LookupTable.INSTANCE.getMapValue("mykey");
This will call these methods in order
Get filename from a filemanager class, which is parametrized on your needs
Call the constructor (it is private) and load properties from the excel file
getMapValue for the key and return
A subsequent call LookupTable.INSTANCE.getMapValue("mysecondkey") will only call getMapValue as the INSTANCE is initialized beforehand.
I have resource bundle as Java class that read values from database. When i update db i need to reload bundle, but i don't know how. Anybody helps ?
package model.helpers;
public class Messages_en extends ListResourceBundle {
protected Object[][] getContents() {
// from DB
// ...
}
}
In view i use bundle as below:
<f:loadBundle basename="model.helpers.Messages" var="m" />
This is not exactly trivial.
For one just clearing the ResourceBundle via clearCache() doesn't always yield the desired results. Often you need at least also try to clear using the context class loader:
ResourceBundle.clearCache(Thread.currentThread().getContextClassLoader());
This however will still not reload the resource bundle defined in a faces-config.xml file. At least the Mojarra JSF 1.2 implementation privately caches the resource bundle internally. This happens in:
FacesContext -> Application -> associate (ApplicationAssociate) -> resourceBundles (Map<String, ApplicationResourceBundle>()) -> resources (Map<Locale, ResourceBundle>)
It's possible to clear this cache via reflection (at the end of the day, it's just an entry in a Map), or you might wanna replace the Application. Both are not things you normally do lightheartedly.
Purely for development you could use JRebel, which probably already has knowledge of Mojarra and most likely does the reflection trick mentioned above.
After some experimenting, I came to the following code which does the trick on JBoss AS 5/JSF 1.2. It does tie your code to Mojarra (imports sun packages) and can break with any upgrade because of reflective tricks being used. But anyway, this is the code:
public static void reloadBundle() {
ResourceBundle.clearCache(Thread.currentThread().getContextClassLoader());
ApplicationResourceBundle appBundle = ApplicationAssociate.getCurrentInstance().getResourceBundles().get("your_bundle_name");
Map<Locale, ResourceBundle> resources = getFieldValue(appBundle, "resources");
resources.clear();
}
#SuppressWarnings("unchecked")
private static <T> T getFieldValue(Object object, String fieldName) {
try {
Field field = object.getClass().getDeclaredField(fieldName);
field.setAccessible(true);
return (T) field.get(object);
} catch (Exception e) {
return null;
}
}
(replace the getFieldValue helper method with your own favorite reflective util if necessary and sprinkle with exception and null handlers where appropriate)
ResourceBundle.clearCache();
OR
Messages_en .clearCache();
Calling this method will reload the resources, it will refresh the bundle
Reference
You can even avoid to have to import weld and jsf-impl classes in your module with some more lines of reflection:
Class<?> applicationAssociateClass = Class.forName("com.sun.faces.application.ApplicationAssociate");
Method getCurrentInstance = applicationAssociateClass.getMethod("getCurrentInstance");
Object applicationAssociate = getCurrentInstance.invoke(null);
Method getResourceBundles = applicationAssociate.getClass().getMethod("getResourceBundles");
Map<String, ?> resourceBundles = (Map<String, ?>)getResourceBundles.invoke(applicationAssociate);
Object appBundle = resourceBundles.get(name);
Map<Locale, ResourceBundle> resources = getFieldValue(appBundle, "resources");
resources.clear();
(works well with Wildfly 10)