Develop Reference

Updating Dropwizard config at runtime

Is it possible to have my app update the config settings at runtime? I can easily expose the settings I want in my UI but is there a way to allow the user to update settings and make them permanent ie save them to the config.yaml file? The only way I can see it to update the file by hand then restart the server which seems a bit limiting.

Yes. It is possible to reload the service classes at runtime.
Dropwizard by itself does not have the way to reload the app, but jersey has.
Jersey uses a container object internally to maintain the running application. Dropwizard uses the ServletContainer class of Jersey to run the application.
How to reload the app without restarting it -
Get a handle to the container used internally by jersey
You can do this by registering a AbstractContainerLifeCycleListener in Dropwizard Environment before starting the app. and implement its onStartup method as below -
In your main method where you start the app -
//getting the container instance
environment.jersey().register(new AbstractContainerLifecycleListener() {
#Override
public void onStartup(Container container) {
//initializing container - which will be used to reload the app
_container = container;
}
});
Add a method to your app to reload the app. It will take in the list of string which are the names of the service classes you want to reload. This method will call the reload method of the container with the new custom DropWizardConfiguration instance.
In your Application class
public static synchronized void reloadApp(List<String> reloadClasses) {
DropwizardResourceConfig dropwizardResourceConfig = new DropwizardResourceConfig();
for (String className : reloadClasses) {
try {
Class<?> serviceClass = Class.forName(className);
dropwizardResourceConfig.registerClasses(serviceClass);
System.out.printf(" + loaded class %s.\n", className);
} catch (ClassNotFoundException ex) {
System.out.printf(" ! class %s not found.\n", className);
}
}
_container.reload(dropwizardResourceConfig);
}
For more details see the example documentation of jersey - jersey example for reload
Consider going through the code and documentation of following files in Dropwizard/Jersey for a better understanding -
Container.java
ContainerLifeCycleListener.java
ServletContainer.java
AbstractContainerLifeCycleListener.java
DropWizardResourceConfig.java
ResourceConfig.java

No.
Yaml file is parsed at startup and given to the application as Configuration object once and for all. I believe you can change the file after that but it wouldn't affect your application until you restart it.
Possible follow up question: Can one restart the service programmatically?
AFAIK, no. I've researched and read the code somewhat for that but couldn't find a way to do that yet. If there is, I'd love to hear that :).

I made a task that reloads the main yaml file (it would be useful if something in the file changes). However, it is not reloading the environment. After researching this, Dropwizard uses a lot of final variables and it's quite hard to reload these on the go, without restarting the app.
class ReloadYAMLTask extends Task {
private String yamlFileName;
ReloadYAMLTask(String yamlFileName) {
super("reloadYaml");
this.yamlFileName = yamlFileName;
}
#Override
public void execute(ImmutableMultimap<String, String> parameters, PrintWriter output) throws Exception {
if (yamlFileName != null) {
ConfigurationFactoryFactory configurationFactoryFactory = new DefaultConfigurationFactoryFactory<ReportingServiceConfiguration>();
ValidatorFactory validatorFactory = Validation.buildDefaultValidatorFactory();
Validator validator = validatorFactory.getValidator();
ObjectMapper objectMapper = Jackson.newObjectMapper();
final ConfigurationFactory<ServiceConfiguration> configurationFactory = configurationFactoryFactory.create(ServiceConfiguration.class, validator, objectMapper, "dw");
File confFile = new File(yamlFileName);
configurationFactory.build(new File(confFile.toURI()));
}
}
}

You can change the configuration in the YAML and read it while your application is running. This will not however restart the server or change any server configurations. You will be able to read any changed custom configurations and use them. For example, you can change the logging level at runtime or reload other custom settings.
My solution -
Define a custom server command. You should use this command to start your application instead of the "server" command.
ArgsServerCommand.java
public class ArgsServerCommand<WC extends WebConfiguration> extends EnvironmentCommand<WC> {
private static final Logger LOGGER = LoggerFactory.getLogger(ArgsServerCommand.class);
private final Class<WC> configurationClass;
private Namespace _namespace;
public static String COMMAND_NAME = "args-server";
public ArgsServerCommand(Application<WC> application) {
super(application, "args-server", "Runs the Dropwizard application as an HTTP server specific to my settings");
this.configurationClass = application.getConfigurationClass();
}
/*
* Since we don't subclass ServerCommand, we need a concrete reference to the configuration
* class.
*/
#Override
protected Class<WC> getConfigurationClass() {
return configurationClass;
}
public Namespace getNamespace() {
return _namespace;
}
#Override
protected void run(Environment environment, Namespace namespace, WC configuration) throws Exception {
_namespace = namespace;
final Server server = configuration.getServerFactory().build(environment);
try {
server.addLifeCycleListener(new LifeCycleListener());
cleanupAsynchronously();
server.start();
} catch (Exception e) {
LOGGER.error("Unable to start server, shutting down", e);
server.stop();
cleanup();
throw e;
}
}
private class LifeCycleListener extends AbstractLifeCycle.AbstractLifeCycleListener {
#Override
public void lifeCycleStopped(LifeCycle event) {
cleanup();
}
}
}
Method to reload in your Application -
_ymlFilePath = null; //class variable
public static boolean reloadConfiguration() throws IOException, ConfigurationException {
boolean reloaded = false;
if (_ymlFilePath == null) {
List<Command> commands = _configurationBootstrap.getCommands();
for (Command command : commands) {
String commandName = command.getName();
if (commandName.equals(ArgsServerCommand.COMMAND_NAME)) {
Namespace namespace = ((ArgsServerCommand) command).getNamespace();
if (namespace != null) {
_ymlFilePath = namespace.getString("file");
}
}
}
}
ConfigurationFactoryFactory configurationFactoryFactory = _configurationBootstrap.getConfigurationFactoryFactory();
ValidatorFactory validatorFactory = _configurationBootstrap.getValidatorFactory();
Validator validator = validatorFactory.getValidator();
ObjectMapper objectMapper = _configurationBootstrap.getObjectMapper();
ConfigurationSourceProvider provider = _configurationBootstrap.getConfigurationSourceProvider();
final ConfigurationFactory<CustomWebConfiguration> configurationFactory = configurationFactoryFactory.create(CustomWebConfiguration.class, validator, objectMapper, "dw");
if (_ymlFilePath != null) {
// Refresh logging level.
CustomWebConfiguration webConfiguration = configurationFactory.build(provider, _ymlFilePath);
LoggingFactory loggingFactory = webConfiguration.getLoggingFactory();
loggingFactory.configure(_configurationBootstrap.getMetricRegistry(), _configurationBootstrap.getApplication().getName());
// Get my defined custom settings
CustomSettings customSettings = webConfiguration.getCustomSettings();
reloaded = true;
}
return reloaded;
}

Although this feature isn't supported out of the box by dropwizard, you're able to accomplish this fairly easy with the tools they give you.
Before I get started, note that this isn't a complete solution for the question asked as it doesn't persist the updated config values to the config.yml. However, this would be easy enough to implement yourself simply by writing to the config file from the application. If anyone would like to write this implementation feel free to open a PR on the example project I've linked below.
Code
Start off with a minimal config:
config.yml
myConfigValue: "hello"
And it's corresponding configuration file:
ExampleConfiguration.java
public class ExampleConfiguration extends Configuration {
private String myConfigValue;
public String getMyConfigValue() {
return myConfigValue;
}
public void setMyConfigValue(String value) {
myConfigValue = value;
}
}
Then create a task which updates the config:
UpdateConfigTask.java
public class UpdateConfigTask extends Task {
ExampleConfiguration config;
public UpdateConfigTask(ExampleConfiguration config) {
super("updateconfig");
this.config = config;
}
#Override
public void execute(Map<String, List<String>> parameters, PrintWriter output) {
config.setMyConfigValue("goodbye");
}
}
Also for demonstration purposes, create a resource which allows you to get the config value:
ConfigResource.java
#Path("/config")
public class ConfigResource {
private final ExampleConfiguration config;
public ConfigResource(ExampleConfiguration config) {
this.config = config;
}
#GET
public Response handleGet() {
return Response.ok().entity(config.getMyConfigValue()).build();
}
}
Finally wire everything up in your application:
ExampleApplication.java (exerpt)
environment.jersey().register(new ConfigResource(configuration));
environment.admin().addTask(new UpdateConfigTask(configuration));
Usage
Start up the application then run:
$ curl 'http://localhost:8080/config'
hello
$ curl -X POST 'http://localhost:8081/tasks/updateconfig'
$ curl 'http://localhost:8080/config'
goodbye
How it works
This works simply by passing the same reference to the constructor of ConfigResource.java and UpdateConfigTask.java. If you aren't familiar with the concept see here:
Is Java "pass-by-reference" or "pass-by-value"?
The linked classes above are to a project I've created which demonstrates this as a complete solution. Here's a link to the project:
scottg489/dropwizard-runtime-config-example
Footnote: I haven't verified this works with the built in configuration. However, the dropwizard Configuration class which you need to extend for your own configuration does have various "setters" for internal configuration, but it may not be safe to update those outside of run().
Disclaimer: The project I've linked here was created by me.

Pig UDF Maxmind GeoIP Database Data File Loading Issue

The following code works when I execute the Pig script locally while specifying a local GeoIPASNum.dat file. However, it does not work when run in MapReduce distributed mode. What am I missing?
Pig job
DEFINE AsnResolver AsnResolver('/hdfs/location/of/GeoIPASNum.dat');
loaded = LOAD 'log_file' Using PigStorage() AS (ip:chararray);
columned = FOREACH loaded GENERATE AsnResolver(ip);
STORE columned INTO 'output/' USING PigStorage();
AsnResolver.java
public class AsnResolver extends EvalFunc<String> {
String ipAsnFile = null;
#Override
public String exec(Tuple input) throws IOException {
try {
LookupService lus = new LookupService(ipAsnFile,
LookupService.GEOIP_MEMORY_CACHE);
return lus.getOrg((String) input.get(0));
} catch (IOException e) {
}
return null;
}
public AsnResolver(String file) {
ipAsnFile = file;
}
...
}

The problem is that you are using a string reference to an HDFS path and the LookupService constructor can't resolve the file. It probably works when you run it locally since the LookupService has no problem with a file in your local FS.
Override the getCacheFiles method:
#Override
public List<String> getCacheFiles() {
List<String> list = new ArrayList<String>(1);
list.add(ipAsnFile + "#GeoIPASNum.dat");
return list;
}
Then change your LookupService constructor to use the Distributed Cache reference to "GeoIPASNum.dat" :
LookupService lus = new LookupService("GeoIPASNum.dat", LookupService.GEOIP_MEMORY_CACHE);

Search for "Distributed Cache" in this page of the Pig docs: http://pig.apache.org/docs/r0.11.0/udf.html
The example it shows using the getCacheFiles() method should ensure that the file is accessible to all the nodes in the cluster.

Eclipse - JAR creation failed "Class files on classpath not found or not accessible for..."

I have a project in Eclipse that has a red cross on it and will not export to a runnable JAR. I can't remember if I have looked at it since I reinstalled Windows on my laptop, but I know that I haven't changed any code. There are no errors in any of the classes, however the error I get points to the following class that deals with the menu items on Mac OSx:
import java.lang.reflect.*;
public class osxhandler implements InvocationHandler {
protected Object targetObject;
protected Method targetMethod;
protected String proxySignature;
static Object macOSXApplication;
// Pass this method an Object and Method equipped to perform application shutdown logic
// The method passed should return a boolean stating whether or not the quit should occur
public static void setQuitHandler(Object target, Method quitHandler) {
setHandler(new HOsx("handleQuit", target, quitHandler));
}
public static void setAboutHandler(Object target, Method aboutHandler) {
boolean enableAboutMenu = (target != null && aboutHandler != null);
if (enableAboutMenu) {
setHandler(new HOsx("handleAbout", target, aboutHandler));
}
// If we're setting a handler, enable the About menu item by calling
// com.apple.eawt.Application reflectively
try {
Method enableAboutMethod = macOSXApplication.getClass().getDeclaredMethod("setEnabledAboutMenu", new Class[] { boolean.class });
enableAboutMethod.invoke(macOSXApplication, new Object[] { Boolean.valueOf(enableAboutMenu) });
} catch (Exception ex) {
System.err.println("MacOSHandler could not access the About Menu");
ex.printStackTrace();
}
}
public static void setPreferencesHandler(Object target, Method prefsHandler) {
boolean enablePrefsMenu = (target != null && prefsHandler != null);
if (enablePrefsMenu) {
setHandler(new HOsx("handlePreferences", target, prefsHandler));
}
// If we're setting a handler, enable the Preferences menu item by calling
// com.apple.eawt.Application reflectively
try {
Method enablePrefsMethod = macOSXApplication.getClass().getDeclaredMethod("setEnabledPreferencesMenu", new Class[] { boolean.class });
enablePrefsMethod.invoke(macOSXApplication, new Object[] { Boolean.valueOf(enablePrefsMenu) });
} catch (Exception ex) {
System.err.println("MacOSHandler could not access the About Menu");
ex.printStackTrace();
}
}
// Pass this method an Object and a Method equipped to handle document events from the Finder
// Documents are registered with the Finder via the CFBundleDocumentTypes dictionary in the
// application bundle's Info.plist
public static void setFileHandler(Object target, Method fileHandler) {
setHandler(new HOsx("handleOpenFile", target, fileHandler) {
// Override MacOSHandler.callTarget to send information on the
// file to be opened
public boolean callTarget(Object appleEvent) {
if (appleEvent != null) {
try {
Method getFilenameMethod = appleEvent.getClass().getDeclaredMethod("getFilename", (Class[])null);
String filename = (String) getFilenameMethod.invoke(appleEvent, (Object[])null);
this.targetMethod.invoke(this.targetObject, new Object[] { filename });
} catch (Exception ex) {
}
}
return true;
}
});
}
// setHandler creates a Proxy object from the passed MacOSHandler and adds it as an ApplicationListener
#SuppressWarnings({ "unchecked", "rawtypes" })
public static void setHandler(HOsx adapter) {
try {
Class applicationClass = Class.forName("com.apple.eawt.Application");
if (macOSXApplication == null) {
macOSXApplication = applicationClass.getConstructor((Class[])null).newInstance((Object[])null);
}
Class applicationListenerClass = Class.forName("com.apple.eawt.ApplicationListener");
Method addListenerMethod = applicationClass.getDeclaredMethod("addApplicationListener", new Class[] { applicationListenerClass });
// Create a proxy object around this handler that can be reflectively added as an Apple ApplicationListener
Object MacOSHandlerProxy = Proxy.newProxyInstance(HOsx.class.getClassLoader(), new Class[] { applicationListenerClass }, adapter);
addListenerMethod.invoke(macOSXApplication, new Object[] { MacOSHandlerProxy });
} catch (ClassNotFoundException cnfe) {
System.err.println("This version of Mac OS X does not support the Apple EAWT. ApplicationEvent handling has been disabled (" + cnfe + ")");
} catch (Exception ex) { // Likely a NoSuchMethodException or an IllegalAccessException loading/invoking eawt.Application methods
System.err.println("Mac OS X Adapter could not talk to EAWT:");
ex.printStackTrace();
}
}
// Each MacOSHandler has the name of the EAWT method it intends to listen for (handleAbout, for example),
// the Object that will ultimately perform the task, and the Method to be called on that Object
protected HOsx(String proxySignature, Object target, Method handler) {
this.proxySignature = proxySignature;
this.targetObject = target;
this.targetMethod = handler;
}
// Override this method to perform any operations on the event
// that comes with the various callbacks
// See setFileHandler above for an example
public boolean callTarget(Object appleEvent) throws InvocationTargetException, IllegalAccessException {
Object result = targetMethod.invoke(targetObject, (Object[])null);
if (result == null) {
return true;
}
return Boolean.valueOf(result.toString()).booleanValue();
}
// InvocationHandler implementation
// This is the entry point for our proxy object; it is called every time an ApplicationListener method is invoked
public Object invoke (Object proxy, Method method, Object[] args) throws Throwable {
if (isCorrectMethod(method, args)) {
boolean handled = callTarget(args[0]);
setApplicationEventHandled(args[0], handled);
}
// All of the ApplicationListener methods are void; return null regardless of what happens
return null;
}
// Compare the method that was called to the intended method when the MacOSHandler instance was created
// (e.g. handleAbout, handleQuit, handleOpenFile, etc.)
protected boolean isCorrectMethod(Method method, Object[] args) {
return (targetMethod != null && proxySignature.equals(method.getName()) && args.length == 1);
}
// It is important to mark the ApplicationEvent as handled and cancel the default behavior
// This method checks for a boolean result from the proxy method and sets the event accordingly
protected void setApplicationEventHandled(Object event, boolean handled) {
if (event != null) {
try {
Method setHandledMethod = event.getClass().getDeclaredMethod("setHandled", new Class[] { boolean.class });
// If the target method returns a boolean, use that as a hint
setHandledMethod.invoke(event, new Object[] { Boolean.valueOf(handled) });
} catch (Exception ex) {
System.err.println("MacOSHandler was unable to handle an ApplicationEvent: " + event);
ex.printStackTrace();
}
}
}
}
Any ideas as to why I can't export/compile? I've never had this issue before.

Just do a clean and/or rebuild on the project.
You can find it under the Project menu of Eclipse.

I also had a different, degenerate case of this problem. Turned out, we had a class in our project that had a file (so Eclipse kept it on the classpath) but no actual class defined in the file (the file only had imports and a class comment... probably a merge gone wrong). Anyway, deleting the file solved the issue.

It’s quite hateful that Eclipse always generates hidden files .project
and .classpath in project folder. Sometimes you’re not aware if
something goes wrong in these files.
After upgrading your Eclipse and if you found the following compile
error, I’d suggest you to check .classpath in your project folder.
The project was not built since its build path is incomplete. Cannot
find the class file for java.lang.Object. Fix the build path then try
building this project
Most likely you would see a line like this.
<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/ org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/j2re1.4.2_03"/>
The stupid Eclipse appended this for no reason. Just simply remove it
to make it work again. ;)
/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/j2re1.4.2_xx
Source: http://hochit.com/2006/07/06/eclipse-upgrading-problem-javalangobject-not-found/
In addition, you can check your project settings in eclipse. Right click on your project and choose properties. Go to Java Build Path and there should be more specific information of the problem. Most likely you set the JDK to an Version which doesn't exist on the new System.
If this doesn't help too, select your project and then use the menu entry Source->Clean Up.

In my case, the classes were empty, and the compiler whined:
Class files on classpath not found or not accessible for: 'ibDemo/src/com/ib/controller/LocationCode.java'
Class files on classpath not found or not accessible for: 'ibDemo/src/com/ib/controller/PairPanel.java'
To solve this I'd to add a class declaration:
public class LocationCode
{
}
and
public class PairPanel
{
}

I got referred here, because I had the same error.
I am using maven on eclipse. I did right click on repo, chose build path->Conifgure build->Project References and checked the project references for my repo. This worked for me.

I was also getting the same error. In my case problem was, I had put same jar multiple times once through "user library" & next time through "build path" on the same Project. Just deleted the repeated jars from the classpath & got ride of the above error.

I had the same error and after trying out multiple recommendations, nothing had worked out. So I created a new workspace and refer to this project. After that, it got successfully built and exported the JAR without errors.

Not sure this might be the best possible solution, but do check java build path. I had it pointing to a wrong location because of which I was facing class not found error.
Once java build path was fixed, the problem was resolved.

I came here on same error. In my case, nothing was compiling (building?) and Eclipse didn't tell me there was any issue with the build other than these cryptic messages. I eventually unzipped the jar file and saw that it had no classes in it. It was because because the project I referenced in my build path wasn't built. In my case, the project would not compile in a million years, but I had access to jar files from R&D dept who could and did compile it in their own way. So I referenced those jar files instead. Now my classes compile and the error went away. I'm sure I would have done that in the first place but "Helpful" Eclipse suggested for me to reference the unbuilt project so I went along with the bad suggestion!

I closed all tabs with files in Eclipse, and it's fixed problem.

In my case, I was getting the same problem and I noticed I mvn clean and tried to export the jar and end-up getting the same error.
It worked for me after mvn install.

Organise Imports Eclipse Method

I need to know which method is called inside eclipse when I press "CTRL+ SHIFT + O" (Organise Imports), in order to invoke it after a code generation. What the name of this method and where can I find it (Package.Interface)
Thanks

"Organize Imports" action is contributed by org.eclipse.jdt.ui.actions.OrganizeImportsAction, which, in turn, calls org.eclipse.jdt.internal.corext.codemanipulation.OrganizeImportsOperation.

Finaly Managed to do it with this code (targetSite is a IWorkbench site initialized at the ame time as shell):
#Override
public void postLaunchAction(final IProject project, final IProgressMonitor monitor) throws CoreException {
super.postLaunchAction(project, monitor);
Runnable job = new Runnable() {
#Override
public void run() {
OrganizeImportsAction org = new OrganizeImportsAction(SpringServicesAction.this.targetSite);
try {
IJavaProject prj = null;
if (project.hasNature("org.eclipse.jdt.core.javanature")) {
prj = JavaCore.create(project);
}
IStructuredSelection selection = new StructuredSelection(prj);
org.run(selection);
} catch (CoreException ce) {
ce.printStackTrace();
}
}
};
this.shell.getDisplay().syncExec(job);
}

For reference, this is how I did it:
I made a large automated refactor in the codebase in our project. Due to a (i think so) bug in eclipse with refactoring static methods which are staticly imported in another file, i had to call organize imports after each refactor (also because I commit every move to git automaticly):
private void organizeImports(ICompilationUnit cu)
throws OperationCanceledException, CoreException {
cu.becomeWorkingCopy(null);
CompilationUnit unit = cu.reconcile(AST.JLS4, false, null, pm);
NullProgressMonitor pm = new NullProgressMonitor();
OrganizeImportsOperation op = new OrganizeImportsOperation(cu, unit,
true, true, true, null);
TextEdit edit = op.createTextEdit(pm);
if (edit == null) {
return;
}
JavaModelUtil.applyEdit(cu, edit, true, pm);
cu.commitWorkingCopy(true, pm);
cu.save(pm, true);
}
Disadvantagde: Discouraged access. If somebody has an idea to call this action properly without creating a new runnable and without using a shell etc., please comment.

Junit create report without ant/maven

Is it possible to create a report from JUnit without Ant or Maven? Because I call the tests with velocitycode, and the velocitycodes calls a method. And that method calls all the tests. So I can get a response from it, the failures/errors/runs etc. But I want to create a report with it.. Or do I need to create html stuff by myself?
I created the methods and testmethods in Java, so I will do everything in Java, except the call, thats in Velocity code.
Velocitycode:
${custom.test}
Java code:
public void getTest(){
junit.textui.TestRunner runner = new junit.textui.TestRunner();
TestResult testresult = Junit.textui.TestRunner.run(runner.getTest(MyTestClass.class.getName()));
}

You will need the ant library. But with this code you can create an XML report and use it in other pograms. Such as Jenkins.
public static void getTest(){
String pathToReports = "C:\\path\\to\\the\\Reports";
Project project = new Project();
try {
new File(pathToReports).mkdir();
JUnitTask task = new JUnitTask();
project.setProperty("java.io.tmpdir",pathToReports);
task.setProject(project);
FormatterElement.TypeAttribute type = new FormatterElement.TypeAttribute();
type.setValue("xml");
FormatterElement formater = new FormatterElement();
formater.setType(type);
task.addFormatter(formater);
JUnitTest test = new JUnitTest(YOURTEST.class.getName());
test.setTodir(new File(pathToReports));
task.addTest(test);
task.execute();
} catch (Exception e) {
}
}

Don't think so. But you might be able to use ant as a library instead of a tool, and use the same code that the tool uses to generate these reports.