I have essentially the same question as here but am hoping to get a less vague, more informative answer.
I'm looking for a way to configure DropWizard programmatically, or at the very least, to be able to tweak configs at runtime. Specifically I have a use case where I'd like to configure metrics in the YAML file to be published with a frequency of, say, 2 minutes. This would be the "normal" default. However, under certain circumstances, I may want to speed that up to, say, every 10 seconds, and then throttle it back to the normal/default.
How can I do this, and not just for the metrics.frequency property, but for any config that might be present inside the YAML config file?
Dropwizard reads the YAML config file and configures all the components only once on startup. Neither the YAML file nor the Configuration object is used ever again. That means there is no direct way to configure on run-time.
It also doesn't provide special interfaces/delegates where you can manipulate the components. However, you can access the objects of the components (usually; if not you can always send a pull request) and configure them manually as you see fit. You may need to read the source code a bit but it's usually easy to navigate.
In the case of metrics.frequency you can see that MetricsFactory class creates ScheduledReporterManager objects per metric type using the frequency setting and doesn't look like you can change them on runtime. But you can probably work around it somehow or even better, modify the code and send a Pull Request to dropwizard community.
Although this feature isn't supported out of the box by dropwizard, you're able to accomplish this fairly easy with the tools they give you. Note that the below solution definitely works on config values you've provided, but it may not work for built in configuration values.
Also note that this doesn't persist the updated config values to the config.yml. However, this would be easy enough to implement yourself simply by writing to the config file from the application. If anyone would like to write this implementation feel free to open a PR on the example project I've linked below.
Code
Start off with a minimal config:
config.yml
myConfigValue: "hello"
And it's corresponding configuration file:
ExampleConfiguration.java
public class ExampleConfiguration extends Configuration {
private String myConfigValue;
public String getMyConfigValue() {
return myConfigValue;
}
public void setMyConfigValue(String value) {
myConfigValue = value;
}
}
Then create a task which updates the config:
UpdateConfigTask.java
public class UpdateConfigTask extends Task {
ExampleConfiguration config;
public UpdateConfigTask(ExampleConfiguration config) {
super("updateconfig");
this.config = config;
}
#Override
public void execute(Map<String, List<String>> parameters, PrintWriter output) {
config.setMyConfigValue("goodbye");
}
}
Also for demonstration purposes, create a resource which allows you to get the config value:
ConfigResource.java
#Path("/config")
public class ConfigResource {
private final ExampleConfiguration config;
public ConfigResource(ExampleConfiguration config) {
this.config = config;
}
#GET
public Response handleGet() {
return Response.ok().entity(config.getMyConfigValue()).build();
}
}
Finally wire everything up in your application:
ExampleApplication.java (exerpt)
environment.jersey().register(new ConfigResource(configuration));
environment.admin().addTask(new UpdateConfigTask(configuration));
Usage
Start up the application then run:
$ curl 'http://localhost:8080/config'
hello
$ curl -X POST 'http://localhost:8081/tasks/updateconfig'
$ curl 'http://localhost:8080/config'
goodbye
How it works
This works simply by passing the same reference to the constructor of ConfigResource.java and UpdateConfigTask.java. If you aren't familiar with the concept see here:
Is Java "pass-by-reference" or "pass-by-value"?
The linked classes above are to a project I've created which demonstrates this as a complete solution. Here's a link to the project:
scottg489/dropwizard-runtime-config-example
Footnote: I haven't verified this works with the built in configuration. However, the dropwizard Configuration class which you need to extend for your own configuration does have various "setters" for internal configuration, but it may not be safe to update those outside of run().
Disclaimer: The project I've linked here was created by me.
I solved this with bytecode manipulation via Javassist
In my case, I wanted to change the "influx" reporter
and modifyInfluxDbReporterFactory should be ran BEFORE dropwizard starts
private static void modifyInfluxDbReporterFactory() throws Exception {
ClassPool cp = ClassPool.getDefault();
CtClass cc = cp.get("com.izettle.metrics.dw.InfluxDbReporterFactory"); // do NOT use InfluxDbReporterFactory.class.getName() as this will force the class into the classloader
CtMethod m = cc.getDeclaredMethod("setTags");
m.insertAfter(
"if (tags.get(\"cloud\") != null) tags.put(\"cloud_host\", tags.get(\"cloud\") + \"_\" + host);tags.put(\"app\", \"sam\");");
cc.toClass();
}
Related
I have a Wicket application and I'm trying to implement separate configuration that can be changed remotely. That's the end goal, anyway.
What I'm trying to do is set up Cayenne to work by starting it manually, rather than using the web.xml file. I have tried a bunch of different things, but I'm not sure I fully understand how the context is applied to all threads.
I have tried creating a ServerRuntime in my Application class. I've also tried on my custom BasePage class that each page uses. I can get it to kind of work by doing the following on the BasePage, but it is inconsistent:
public class BasePage ....
public static ServerRuntime runtime = new ServerRuntime("cayenne-config.xml");//This is in my BasePage class, but I've also tried this in the Application class
#Override
protected void init() {
BaseContext.bindThreadObjectContext(Application.runtime.getContext());//This is in my BasePage class
}
Like I said, that kind of works, but it isn't consistent. I keep getting errors on
BaseContext.getThreadObjectContext();
Error is this:
java.lang.IllegalStateException: Current thread has no bound ObjectContext.
I can't seem to find much information on this. I tried doing stuff like this, and accessing the runtime using these as well, but nothing is working consistently.
WebUtil.setCayenneRuntime(this.getServletContext(), runtime);
BaseContext.bindThreadObjectContext(WebUtil.getCayenneRuntime(((Application)getApplication()).getServletContext()).getContext());
Any help would be greatly appreciated.
I figured out a way to do this on my own.
I'm extending CayenneFilter and overriding the init method.
In there I copied nearly their exact code. I will be able to check for any config here and load the proper xml file. This obviously isn't the ironed out solution, but is definitely a step forward, and could be the way I end up doing this.
Either way, here's what I have tested to be working.
#WebFilter(filterName = "cayenne-config", displayName = "cayenne-config", urlPatterns = {"/*"})
public class TestFilter extends CayenneFilter
{
#Override
public void init(FilterConfig config) throws ServletException
{
this.checkAlreadyConfigured(config.getServletContext());
this.servletContext = config.getServletContext();
WebConfiguration configAdapter = new WebConfiguration(config);
Collection modules = configAdapter.createModules(new Module[]{new WebModule()});
ServerRuntime runtime = new ServerRuntime("cayenne-test.xml", (Module[])modules.toArray(new Module[modules.size()]));
WebUtil.setCayenneRuntime(config.getServletContext(), runtime);
}
}
I don't think the annotation is needed (I am specifying it all in the web.xml file), but I thought I would leave it here so you could see that it is changing.
If I could find a way to change the config (FilterConfig) values (the init parameters), then I could just change that to the name of the xml file I want to use and not override this entire method. I couldn't figure out how to do that, but I'll look further later.
If anyone has another better answer, I would love to hear it.
I have a (web-)application that needs special configurations and/or extensions based on the customer using the application. I call these additions "plugins" and they are auto discovered by classpath scanning when the application starts. For extensions that is incredibly easy. Let's say I want to have a plugin which adds an API that prints "hello world" when the URL /myplugin/greet is called: I just create a #Controller annotated class with the according #RequestMapping, put this in a myplugin.jar, copy that on the classpath and that's it.
Problems come up when I want to change some defaults and especially if I want to do this multiple times. Let's say my core application has a config like this:
#Configuration
public class CoreConfiguration {
#Bean
public Set<String> availableModules() {
return Collections.singleton("core");
}
}
Now I have two plugins that don't know about each other (but they do know the CoreConfig), but they both want to add themselves to the list of available modules. How would I do that? If I only had a single plugin that wants to override the module list I could override the existing bean from CoreConfiguration, but with two plugins that becomes a problem. What I imagine is something like this:
#Configuration
public class FirstPluginConfiguration {
#Bean
public Set<String> availableModules(Set<String> availableModules) {
Set<String> extendedSet = new HashSet<>(availableModules);
extendedSet.add("FirstPlugin");
return extendedSet;
}
}
Of course a SecondPluginConfiguration would look nearly exactly like this, except that the Set is not extended by "FirstPlugin", but by "SecondPlugin". I tested it to check what would happen and spring will just never call the First/SecondPluginConfiguration "availableModules" methods but it does not show an error either.
Now of course in this case this could easily be solved by using a mutable Set in the CoreConfiguration and then autowiring and extending the set in the other configurations, but for example I also want to be able to add method interceptors to some beans. So for example I might have an interface CrashLogger which has a logCrash(Throwable t) method and in CoreConfiguration a ToFileCrashLogger is created that writes stack traces to files as the name suggests. Now a plugin could say that he also wants to get notified about crashes, for example the plugin wants to ADDITIONALLY send the stacktrace to someone by email. For that matter that plugin could wrap the CrashLogger configured by the CoreConfiguration and fire BOTH. A second plugin could wrap the wrapper again and do something totally different with the stacktrace and still call both of the other CrashLoggers.
The later does sound somewhat like AOP and if I'd just let ALL my beans be proxied (I did not test that) I could autowire them into my plugin configurations, cast them to org.springframework.aop.framework.Advised and then add advices that manipulate behaviour. However it does seem like a huge overkill to generate proxies for each and everyone of my beans just so that that plugin can potentially add one or two advices one one or two beans.
While creating new scenarios I only want to test the scenario I am currently working with. For this purpose I want to use the Meta: #skip tag before my scenarios. As I found out I have to use the embedder to configure the used meta tags, so I tried:
configuredEmbedder().useMetaFilters(Arrays.asList("-skip"));
but actually this still has no effect on my test scenarios. I used it in the constructor of my SerenityStories test suite definition. Here is the complete code of this class:
public class AcceptanceTestSuite extends SerenityStories {
#Managed
WebDriver driver;
public AcceptanceTestSuite() {
System.setProperty("webdriver.chrome.driver", "D:/files/chromedriver/chromedriver.exe");
System.setProperty("chrome.switches", "--lang=en");
System.setProperty("restart.browser.each.scenario", "true");
configuredEmbedder().useMetaFilters(Arrays.asList("-skip"));
runSerenity().withDriver("chrome");
}
#Override
public Configuration configuration() {
Configuration configuration = super.configuration();
Keywords keywords = new LocalizedKeywords(DEFAULTSTORYLANGUAGE);
Properties properties = configuration.storyReporterBuilder().viewResources();
properties.setProperty("encoding", "UTF-8");
configuration.useKeywords(keywords)
.useStoryParser(new RegexStoryParser(keywords, new ExamplesTableFactory(new LoadFromClasspath(this.getClass()))))
.useStoryLoader(new UTF8StoryLoader()).useStepCollector(new MarkUnmatchedStepsAsPending(keywords))
.useDefaultStoryReporter(new ConsoleOutput(keywords)).storyReporterBuilder().withKeywords(keywords).withViewResources(properties);
return configuration;
}
}
Is this the wrong place or have I missed something? Still all scenarios are executed.
EDIT:
I changed following classes and now I think that it "works"
public AcceptanceTestSuite() {
System.setProperty("webdriver.chrome.driver", "D:/files/chromedriver/chromedriver.exe");
System.setProperty("chrome.switches", "--lang=de");
System.setProperty("restart.browser.each.scenario", "true");
this.useEmbedder(configuredEmbedder());
runSerenity().withDriver("chrome");
}
#Override
public Embedder configuredEmbedder() {
final Embedder embedder = new Embedder();
embedder.embedderControls()
.useThreads(1)
.doGenerateViewAfterStories(true)
.doIgnoreFailureInStories(false)
.doIgnoreFailureInView(false)
.doVerboseFailures(true);
final Configuration configuration = configuration();
embedder.useConfiguration(configuration);
embedder.useStepsFactory(stepsFactory());
embedder.useMetaFilters(Arrays.asList("-skip"));
return embedder;
}
But now I get the message [pool-1-thread-1] INFO net.serenitybdd.core.Serenity - TEST IGNORED but the scenario is still executed. Only in the result page I get the info that this scenario is ignored (but still executed). Is there a way to SKIP the scenario so it won't run?
I could not make it run with using configuredEmbedder() but by adding -Dmetafilter="+working -finished" as goals in my mvn run configurations and using the tags #working for scenarios I'm working with and which I want to run and #finsihed for scenarios I don't want to execute. Still I have to change the run configuration if I want to change the meta tags so it is not very comfortable but still I get what I was looking for.
As long as you document it well (some doc in https://github.com/serenity-bdd/the-serenity-book would be brilliant), I think as a JBehave/Serenity user you are well enough placed to decide which option makes the most sense.
Investigation
I debugged the serenity-jbehave classes, trying to understand why setting
configuredEmbedder().useMetaFilters(Collections.singletonList("-skip"))
is not working in all the possible places I put it within my class extending the SerenityStories, I found the strategic code place where metaFilters in ExtendedEmbedder#embedder are overwritten with what we define in our class into settings from serenity-jbehave.
This method is SerenityReportingRunner#createPerformableTree:
private PerformableTree createPerformableTree(List<CandidateSteps> candidateSteps, List<String> storyPaths) {
ExtendedEmbedder configuredEmbedder = this.getConfiguredEmbedder();
configuredEmbedder.useMetaFilters(getMetaFilters());
BatchFailures failures = new BatchFailures(configuredEmbedder.embedderControls().verboseFailures());
PerformableTree performableTree = configuredEmbedder.performableTree();
RunContext context = performableTree.newRunContext(getConfiguration(), candidateSteps,
configuredEmbedder.embedderMonitor(), configuredEmbedder.metaFilter(), failures);
performableTree.addStories(context, configuredEmbedder.storyManager().storiesOfPaths(storyPaths));
return performableTree;
}
This line changes the set metaFilters:
configuredEmbedder.useMetaFilters(getMetaFilters());
It overrides the current metaFilters value.
Going further the call chain, we get to the logic that defines from where it gets metaFilters, i.e. where we can actually set it.
SerenityReportingRunner#createPerformableTree
↓
SerenityReportingRunner#getMetaFilters
↓
SerenityReportingRunner#getMetafilterSetting
This is the method we need!
private String getMetafilterSetting() {
Optional<String> environmentMetafilters = getEnvironmentMetafilters();
Optional<String> annotatedMetafilters = getAnnotatedMetafilters(testClass);
Optional<String> thucAnnotatedMetafilters = getThucAnnotatedMetafilters(testClass);
return environmentMetafilters.orElse(annotatedMetafilters.orElse(thucAnnotatedMetafilters.orElse("")));
}
As we see here, the metaFilters can be defined in three places, and they override each other. In the priority lowering order, they are:
Value of metafilter (exactly all lowercase!) VM property.
Value of on net.serenitybdd.jbehave.annotations.Metafilter annotation on our SerenityStories class.
Value of on net.thucydides.jbehave.annotations.Metafilter annotation on our SerenityStories class. This annotation is deprecated, but left in place for backwards-compatibility.
Solution that is working with the current serenity-jbehave version
I've tried/debugged all these three options, they work and override each other as described above.
1. Use environment metafilter property
Added this to my JVM run arguments:
-Dmetafilter=skip
2. Use the modern #Metafilter annotation
import net.serenitybdd.jbehave.SerenityStories;
import net.serenitybdd.jbehave.annotations.Metafilter;
#Metafilter("-skip")
public class Acceptance extends SerenityStories {
3. Use the deprecated #Metafilter annotation
import net.serenitybdd.jbehave.SerenityStories;
import net.thucydides.jbehave.annotations.Metafilter;
#Metafilter("-skip") // warned as deprecated
public class Acceptance extends SerenityStories {
Solution for my current project is to use the current #Metafilter("-skip") annotation on my test class, to not depend on/have to change VM properties of the particular Jenkins/local dev execution.
Possible pull request to make
https://github.com/serenity-bdd/serenity-core/issues/95 — here Serenity guys have suggested me to do a PR with this fix, since they are not concentrated on Serenity + JBehave now.
I understand where to make the changes (in the code chain described above), but I don't know what overriding logic should be:
— MetaFilters from configuredEmbedder override any of ENV/annotation MetaFilters.
OR
— Any ENV/annotation MetaFilters override Metafilters from configuredEmbedder
OR
— MetaFilters from configuredEmbedder are merged with ENV/annotation MetaFilters. This option required merging priority.
Any suggestions?
In any type of fix, I would prefer add the explicit logs about how the overriding is now working into SerenityReportingRunner#getMetafilterSetting, since the current behaviour is really non-obvious and took lots of time to investigate.
We're developing a Java application that reads a Config file at runtime. My question is that which of the following scenario is efficient for reading a Config file.
Scenario #1: Retrieves a value from config by opening the file, get the value, then close the file.
So this means that file will be open and close every time retrieving a value.
Scenario #2: Open the file during initialization, then expose the object statically across the runtime.
File will be open once, then retrieving a value using the Config object.
Honestly, we currently using the scenario #2. A reason why we choose it because Config file will be open once. Opening file needs syncrhonization, which may lead to untimely retrieval of value. Also, scenario #1 may cause runtime error if the Config file is moved from it's absolute path (anything is possible). But scenario #1 is efficient when it comes in modifying the Config values during runtime.
So which is efficient?
Is your application a mulit-threaded application? if yes, you need to make sure that you have scenario #1 with necessary synchronisation and semaphores. The reason is that your config file is a general item in your application which will be used by all the threads. You don't want the config file reading operation to be interleaved. Even with single threaded operation, it is good to do #1 because you will have less chances of having IO Error. The RTE that you mentioned in your question can happen with anything if you decide to move the file (? Why would you if you need it at a certain place?).
If I should have to access the file more than once, probably I would create a class with a field for each configuration, I would read the file once (only to initialize the class fields) and then I would close the file. In this way you read the file once, during the init, and then you could simply access the configuration by reading the field values stored in the class.
public class Configuration
{
private static String confOne;
private static int confTwo;
private static boolean confThree;
public static init(File configFile) {
/* read the file and init fields */
}
public static String getConfigOne() {
return configOne;
}
public static int getConfigTwo() {
return configOne;
}
public static boolean getConfigThree() {
return configOne;
}
}
A little advice: tries to never hold open a file (or resources) unless absolutely necessary.
I think neither of the scenarios is the best one:
As you you have already stated in sencario #1 you may have a lot of unnecessary IO-Workload and in #2 you cannot change config parameters dynamically at runtime.
I suggest to use a mixture of both. You can load your configuration statically once and then check periodically, if changes have happend. If so, reload your config.
You can see how it may be done by inspecting Java's ResourceBundle-Class. In fact, if the config can be placed as properties-file in your classpath, you can (ab?)use this implementation for your purpose:
private static Control MY_CONTROL = new Control()
{
#Override
public long getTimeToLive(String baseName, Locale locale)
{
return MY_TTL; //make shure, that changes are checked periodically
}
};
public static String getParamter(String name)
{
ResourceBundle config = ResourceBundle.getBundle("config", MY_CONTROL);
return config.getString(name);
}
I am working on a project that has been through multiple hands with a sometimes rushed development. Over time the message.properties file has become out of sync with the jsps that use it. Now I don't know which properties are used and which aren't. Is there a tool (eclipse plugin perhaps) that can root out dead messages?
The problem is that messages may be accessed by JSP or Java, and resource names may be constructed rather than literal strings.
Simple grepping may be able to identify "obvious" resource access. The other solution, a resource lookup mechanism that tracks what's used, is only semi-reliable as well since code paths may determine which resources are used, and unless every path is traveled, you may miss some.
A combination of the two will catch most everything (over time).
Alternatively you can hide the functionality of ResourceBundle behind another façade ResourceBundle, which should generally pipe all calls to original one, but add logging and/or statistics collection on the top.
The example can be as following:
import java.util.Collection;
import java.util.Enumeration;
import java.util.HashSet;
import java.util.NoSuchElementException;
import java.util.ResourceBundle;
public class WrapResourceBundle {
static class LoggingResourceBundle extends ResourceBundle {
private Collection<String> usedKeys = new HashSet<String>();
public LoggingResourceBundle(ResourceBundle parentResourceBundle) {
setParent(parentResourceBundle);
}
#Override
protected Object handleGetObject(String key) {
Object value = parent.getObject(key);
if (value != null) {
usedKeys.add(key);
return value;
}
return null;
}
#Override
public Enumeration<String> getKeys() {
return EMPTY_ENUMERATOR;
}
public Collection<String> getUsedKeys() {
return usedKeys;
}
private static EmptyEnumerator EMPTY_ENUMERATOR = new EmptyEnumerator();
private static class EmptyEnumerator implements Enumeration<String> {
EmptyEnumerator() {
}
public boolean hasMoreElements() {
return false;
}
public String nextElement() {
throw new NoSuchElementException("Empty Enumerator");
}
}
}
public static void main(String[] args) {
LoggingResourceBundle bundle = new LoggingResourceBundle(ResourceBundle.getBundle("test"));
bundle.getString("key1");
System.out.println("Used keys: " + bundle.getUsedKeys());
}
}
Considering that some of your keys are run-time generated, I don't think you'll ever be able to find a tool to validate which keys are in use and which ones are not.
Given the problem you posed, I would probably write an AOP aspect which wraps the MessageSource.getMessage() implementation and log all the requested codes that are being retrieved from the resource bundle. Given that MessageSource is an interface, you would need to know the implementation that you are using, but I suspect that you must know that already.
Given that you would be writing the aspect yourself, you can create a format that is easily correlated against your resource bundle and once you are confident that it contains all the keys required, it becomes a trivial task to compare the two files and eliminate any superfluous lines.
If you really want to be thorough about this, if you already have Spring configured for annotation scan, you could even package up your aspect as its own jar (or .class) and drop it in a production WEB-INF/lib (WEB-INF/classes) folder, restart the webapp and let it run for a while. The great thing about annotations is that it can all be self contained. Once you are sure that you have accumulated enough data you just delete the jar (.class) and you're good to go.
I know that at least two of the major java IDEs can offer this functionality.
IntelliJ IDEA has a (disabled, by default) Inspection that you can
use to do this:
go to Settings -> Inspections -> Properties files -> ... and enable
the 'Unused property'
..Only problem I had was that it didn't pick up some usages of the property from a custom tag library I had written, which I was using in a few JSPs.
Eclipse also has something like this ( http://help.eclipse.org/helios/index.jsp?topic=%2Forg.eclipse.jdt.doc.user%2Ftasks%2Ftasks-202.htm ) but I haven't really exhausted the how well it works.