I'm quite confused on how to properly setup a logger in an Java EE OSGi environment. Here are my requirements:
Only 1 log file per EBA (grouping of bundles)
Multiple log files per application server (due to multiple EBAs)
Do not want to perform ClassLoader magic (its fine if a library I use does this, I just don't want to have to write it)
Must rotate the log file at the end of the day and only maintain 7 log files at once
Preferably does not require creating a logging bundle per EBA. No other developer will buy in if I tell them to write their own logging interop for each application.
Must work with WebSphere Application Server 8.5.5
First I tried to use SLF4j on top of log4j like our other Java EE applications do, however nothing could find my log4j.properties. I tried variations of importing SLF4j, and also had issues where loading it in 1 bundle prevented it from loading in another.
Next I looked into PAX logger, but it appears to log globally, not per EBA.
Attempting to use the OSGi LogService prevents my bundle from deploying to WebSphere, plus I'm not sure how I could get it to meet my requirements anyway.
I'm at the point where the only option I can see is to write my own bundle that keeps a registry for bundle -> log file (using FrameworkUtil.getBundle on the client class) and implement a full logging framework within it. If that has classloader isolation issues then possibly push to an EJB to do the actual logging. I'm really hoping that's not my only solution.
Can anyone point me to some documentation that will help me out??
Thank you!
People are in general confused about the Log Service ... The Log Service does not store any logs, it just acts as a dispatcher. I understand the confusion since the Log Service is mandated to have a small buffer for the initial start up and provides an API to get the buffer.
For what you want, you should add a Log Listener with the Log Reader services. What you want is quite easy with Declarative Services. This is a component that implements your requirements :
#Component(provide = {}, immediate = true) public class Logger extends Thread
implements LogListener {
final BlockingQueue<LogEntry> queue = new ArrayBlockingQueue<LogEntry>(1000);
File root;
#Reference void setLR(LogReaderService lr) {
lr.addLogListener(this);
}
#Activate void activate(BundleContext context, Map<String,Object> props) {
if ( props.containsKey("root"))
root = new File((String) props.get("root"));
else
root = context.getDataFile("logs");
root.mkdirs();
start();
}
#Deactivate void deactivate() {
interrupt();
}
#Override public void logged(LogEntry entry) {
queue.offer(entry); // ignore full silently
}
public void run() {
while (true)
try {
LogEntry entry = queue.take();
File file = getPath(entry);
if (file.isFile()) {
long days = TimeUnit.MILLISECONDS.toDays(System.currentTimeMillis()
- file.lastModified());
if (days > 2) file.delete();
}
try (OutputStream raf = new FileOutputStream(file, true)) {
String s = String.format("%tT [%03d] %s%n", entry.getTime(), entry
.getBundle().getBundleId(), entry.getMessage());
raf.write(s.getBytes("UTF-8"));
}
} catch (InterruptedException ie) {
return;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private File getPath(LogEntry entry) {
long rollover = TimeUnit.MILLISECONDS.toDays(System.currentTimeMillis()) % 7;
String eba = "eba"; // fill in eba from entry.getBundle()?
return new File(root, "log-" + eba + "-" + rollover + ".msg");
}
}
This could of course be done a bit more efficient, but that is left as an exercise.
Related
I've read CDI 2.0 specification (JSR 365) and found out the existence of the #Observes(during=AFTER_SUCCESS) annotation, but it actually requires a custom event to be defined in order to work.
This is what i've got:
//simple """transactional""" file system manager using command pattern
#Transactional(value = Transactional.TxType.REQUIRED)
#TransactionScoped
#Stateful
public class TransactionalFileSystemManager implements SessionSynchronization {
private final Deque<Command> commands = new ArrayDeque<>();
public void createFile(InputStream content, Path path, String name) throws IOException {
CreateFile command = CreateFile.execute(content, path, name);
commands.addLast(command);
}
public void deleteFile(Path path) throws IOException {
DeleteFile command = DeleteFile.execute(path);
commands.addLast(command);
}
private void commit() throws IOException{
for(Command c : commands){
c.confirm();
}
}
private void rollback() throws IOException{
Iterator<Command> it = commands.descendingIterator();
while (it.hasNext()) {
Command c = it.next();
c.undo();
}
}
#Override
public void afterBegin() throws EJBException{
}
#Override
public void beforeCompletion() throws EJBException{
}
#Override
public void afterCompletion(boolean commitSucceeded) throws EJBException{
if(commitSucceeded){
try {
commit();
} catch (IOException e) {
throw new EJBException(e);
}
}
else {
try {
rollback();
} catch (IOException e) {
throw new EJBException(e);
}
}
}
}
However, I want to adopt a CDI-only solution so I need to remove anything EJB related (including the SessionSynchronization interface). How can i achieve the same result using CDI?
First the facts: the authoritative source for this topic is the Java Transaction API (JTA) specification. Search for it online, I got this.
Then the bad news: In order to truly participate in a JTA transaction, you either have to implement a connector according to the Java Connector Architecture (JCA) specification or a XAResource according to JTA. Never done any of them, I am afraid both are going to be hard. Nevertheless, if you search, you may find an existing implementation of a File System Connector.
Your code above will never accomplish true 2-phase commit because, if your code fails, the transaction is already committed, so the application state is inconsistent. Or, there is a small time window when the real transaction is committed but the file system change have not beed executed, again the state is inconsistent.
Some workarounds I can think of, none of which solves the consistency problem:
Persist the File System commands in a database. This ensures that they are enqueued transactionally. A scheduled job wakes up and actually tries to execute the queued FS commands.
Register a Synchronization with the current Transaction, fire an appropriate event from there. Your TransactionalFileSystemManager observes this event, no during attribute needed I guess.
The opentelemetry-javaagent-all agent (versions 0.17.0 and 1.0.1) has been the starting point for adding trace information to my Java application. Auto-instrumentation works great.
Some of my application cannot be auto-instrumented. For this part of the application, I began by adding #WithSpan annotations to interesting spots in the code.
I now reach the limits of what seems possible with simple #WithSpan annotations. However, the framework underlying my app allows me to register callbacks to be invoked at certain points -- e.g. I can provide handlers that are notified when a client connects / disconnects.
What I think I need is to start a new Span when Foo.onConnect() is called, and set it be the parent for the Spans that correspond to each request.
public class Foo {
void onConnect() {
// called when a client connects to my app
// Here I want to create a Span that will be the parent of the Span created in
// Foo.processEachRequest().
}
#WithSpan
public void processEachRequest() {
// works, but since it is called for each request... each span is in a separate Trace
}
void onDisconnect() {
// called when the client disconnects from my app
// Here I can end the parent Span.
}
}
Other ideas - that didn't work out:
1 - The obvious solution would be to add #WithSpan annotations to the underlying framework. For various reasons, this is not going to be a practical way forward.
2 - Next choice might be to search for a way to tell the javaagent about methods in my underlying framework. (The New Relic agent can do something like this.) That doesn't seem to be a feature of the open-telemetry agent, today anyway.
So, I'm left with looking for a way to do this using the callbacks, as above.
Is there a way to do this?
That should be possible by manually instrumenting your code. You would use the Tracer interface of OpenTelemetry, as described in the OpenTelemetry Java docs.
This should give you a general idea:
public class Foo {
private Span parentSpan; // you might need a Map/List/Stack here
void onConnect() {
Tracer tracer =
openTelemetry.getTracer("instrumentation-library-name", "1.0.0");
Span span = tracer.spanBuilder("my span").startSpan();
this.parentSpan = span; // might need to store span per request/client/connection-id
}
public void processEachRequest() {
final Span parent = this.lookupParentSpan();
if (parent != null) {
try (Scope scope = span.makeCurrent()) {
yourLogic();
} catch (Throwable t) {
span.setStatus(StatusCode.ERROR, "error message");
throw t;
}
} else {
yourLogic();
}
}
void onDisconnect() {
final Span parent = this.lookupParentSpan();
if (parent != null) {
parent.end();
}
}
private Span lookupParentSpan() {
// you probably want to lookup the span by client or connection id from a (weak) map
return this.parentSpan;
}
}
NB: You must guarantee that a span is always ended and does not leak. Make sure to properly scope your spans and eventually call Span#end().
My web application has several integrations with external systems and all these integration Rest URLs are kept in a config file with in web app. My application reads this config file at start up and use the URL values while making connections to external systems. But quite often it happens that one of the external systems is down and we have to use an alternate URL. In that case, we typically will have to modify the config and redeploy the war file. Is there a way to modify config file with new value without going through a redeployment of the war file?
In my projects i usually work with Apache Commons Configuration for the management of config files (properties). This library has the capability of automatic reload the values when file changes.
This is muy suggestion of implementation:
Create a class "MyAppConfigProperties" for load the properties file and read your configuration keys:
public class MyAppConfig {
//Apache Commons library object
private PropertiesConfiguration configFile;
private void init() {
try {
//Load the file
configFile = new PropertiesConfiguration(
MyAppConfig.class.getClassLoader().getResource("configFile.properties"));
// Create refresh strategy with "FileChangedReloadingStrategy"
FileChangedReloadingStrategy fileChangedReloadingStrategy = new FileChangedReloadingStrategy();
fileChangedReloadingStrategy.setRefreshDelay(1000);
configFile.setReloadingStrategy(fileChangedReloadingStrategy);
} catch (ConfigurationException e) {
//Manage the exception
}
}
/**
* Constructor por defecto.
*/
public MyAppConfig() {
super();
init();
}
public String getKey(final String key) {
try {
if (configFile.containsKey(key)) {
return configFile.getString(key);
} else {
return null;
}
} catch (ConversionException e) {
//Manage Exception
}
}
}
Now you have to construct a instance of this class (singleton) and use it in all places in that you need to reed a config key.
Every time you use the method "getKey" you will get the last value of the key without deploy and restart.
Is it possible to have my app update the config settings at runtime? I can easily expose the settings I want in my UI but is there a way to allow the user to update settings and make them permanent ie save them to the config.yaml file? The only way I can see it to update the file by hand then restart the server which seems a bit limiting.
Yes. It is possible to reload the service classes at runtime.
Dropwizard by itself does not have the way to reload the app, but jersey has.
Jersey uses a container object internally to maintain the running application. Dropwizard uses the ServletContainer class of Jersey to run the application.
How to reload the app without restarting it -
Get a handle to the container used internally by jersey
You can do this by registering a AbstractContainerLifeCycleListener in Dropwizard Environment before starting the app. and implement its onStartup method as below -
In your main method where you start the app -
//getting the container instance
environment.jersey().register(new AbstractContainerLifecycleListener() {
#Override
public void onStartup(Container container) {
//initializing container - which will be used to reload the app
_container = container;
}
});
Add a method to your app to reload the app. It will take in the list of string which are the names of the service classes you want to reload. This method will call the reload method of the container with the new custom DropWizardConfiguration instance.
In your Application class
public static synchronized void reloadApp(List<String> reloadClasses) {
DropwizardResourceConfig dropwizardResourceConfig = new DropwizardResourceConfig();
for (String className : reloadClasses) {
try {
Class<?> serviceClass = Class.forName(className);
dropwizardResourceConfig.registerClasses(serviceClass);
System.out.printf(" + loaded class %s.\n", className);
} catch (ClassNotFoundException ex) {
System.out.printf(" ! class %s not found.\n", className);
}
}
_container.reload(dropwizardResourceConfig);
}
For more details see the example documentation of jersey - jersey example for reload
Consider going through the code and documentation of following files in Dropwizard/Jersey for a better understanding -
Container.java
ContainerLifeCycleListener.java
ServletContainer.java
AbstractContainerLifeCycleListener.java
DropWizardResourceConfig.java
ResourceConfig.java
No.
Yaml file is parsed at startup and given to the application as Configuration object once and for all. I believe you can change the file after that but it wouldn't affect your application until you restart it.
Possible follow up question: Can one restart the service programmatically?
AFAIK, no. I've researched and read the code somewhat for that but couldn't find a way to do that yet. If there is, I'd love to hear that :).
I made a task that reloads the main yaml file (it would be useful if something in the file changes). However, it is not reloading the environment. After researching this, Dropwizard uses a lot of final variables and it's quite hard to reload these on the go, without restarting the app.
class ReloadYAMLTask extends Task {
private String yamlFileName;
ReloadYAMLTask(String yamlFileName) {
super("reloadYaml");
this.yamlFileName = yamlFileName;
}
#Override
public void execute(ImmutableMultimap<String, String> parameters, PrintWriter output) throws Exception {
if (yamlFileName != null) {
ConfigurationFactoryFactory configurationFactoryFactory = new DefaultConfigurationFactoryFactory<ReportingServiceConfiguration>();
ValidatorFactory validatorFactory = Validation.buildDefaultValidatorFactory();
Validator validator = validatorFactory.getValidator();
ObjectMapper objectMapper = Jackson.newObjectMapper();
final ConfigurationFactory<ServiceConfiguration> configurationFactory = configurationFactoryFactory.create(ServiceConfiguration.class, validator, objectMapper, "dw");
File confFile = new File(yamlFileName);
configurationFactory.build(new File(confFile.toURI()));
}
}
}
You can change the configuration in the YAML and read it while your application is running. This will not however restart the server or change any server configurations. You will be able to read any changed custom configurations and use them. For example, you can change the logging level at runtime or reload other custom settings.
My solution -
Define a custom server command. You should use this command to start your application instead of the "server" command.
ArgsServerCommand.java
public class ArgsServerCommand<WC extends WebConfiguration> extends EnvironmentCommand<WC> {
private static final Logger LOGGER = LoggerFactory.getLogger(ArgsServerCommand.class);
private final Class<WC> configurationClass;
private Namespace _namespace;
public static String COMMAND_NAME = "args-server";
public ArgsServerCommand(Application<WC> application) {
super(application, "args-server", "Runs the Dropwizard application as an HTTP server specific to my settings");
this.configurationClass = application.getConfigurationClass();
}
/*
* Since we don't subclass ServerCommand, we need a concrete reference to the configuration
* class.
*/
#Override
protected Class<WC> getConfigurationClass() {
return configurationClass;
}
public Namespace getNamespace() {
return _namespace;
}
#Override
protected void run(Environment environment, Namespace namespace, WC configuration) throws Exception {
_namespace = namespace;
final Server server = configuration.getServerFactory().build(environment);
try {
server.addLifeCycleListener(new LifeCycleListener());
cleanupAsynchronously();
server.start();
} catch (Exception e) {
LOGGER.error("Unable to start server, shutting down", e);
server.stop();
cleanup();
throw e;
}
}
private class LifeCycleListener extends AbstractLifeCycle.AbstractLifeCycleListener {
#Override
public void lifeCycleStopped(LifeCycle event) {
cleanup();
}
}
}
Method to reload in your Application -
_ymlFilePath = null; //class variable
public static boolean reloadConfiguration() throws IOException, ConfigurationException {
boolean reloaded = false;
if (_ymlFilePath == null) {
List<Command> commands = _configurationBootstrap.getCommands();
for (Command command : commands) {
String commandName = command.getName();
if (commandName.equals(ArgsServerCommand.COMMAND_NAME)) {
Namespace namespace = ((ArgsServerCommand) command).getNamespace();
if (namespace != null) {
_ymlFilePath = namespace.getString("file");
}
}
}
}
ConfigurationFactoryFactory configurationFactoryFactory = _configurationBootstrap.getConfigurationFactoryFactory();
ValidatorFactory validatorFactory = _configurationBootstrap.getValidatorFactory();
Validator validator = validatorFactory.getValidator();
ObjectMapper objectMapper = _configurationBootstrap.getObjectMapper();
ConfigurationSourceProvider provider = _configurationBootstrap.getConfigurationSourceProvider();
final ConfigurationFactory<CustomWebConfiguration> configurationFactory = configurationFactoryFactory.create(CustomWebConfiguration.class, validator, objectMapper, "dw");
if (_ymlFilePath != null) {
// Refresh logging level.
CustomWebConfiguration webConfiguration = configurationFactory.build(provider, _ymlFilePath);
LoggingFactory loggingFactory = webConfiguration.getLoggingFactory();
loggingFactory.configure(_configurationBootstrap.getMetricRegistry(), _configurationBootstrap.getApplication().getName());
// Get my defined custom settings
CustomSettings customSettings = webConfiguration.getCustomSettings();
reloaded = true;
}
return reloaded;
}
Although this feature isn't supported out of the box by dropwizard, you're able to accomplish this fairly easy with the tools they give you.
Before I get started, note that this isn't a complete solution for the question asked as it doesn't persist the updated config values to the config.yml. However, this would be easy enough to implement yourself simply by writing to the config file from the application. If anyone would like to write this implementation feel free to open a PR on the example project I've linked below.
Code
Start off with a minimal config:
config.yml
myConfigValue: "hello"
And it's corresponding configuration file:
ExampleConfiguration.java
public class ExampleConfiguration extends Configuration {
private String myConfigValue;
public String getMyConfigValue() {
return myConfigValue;
}
public void setMyConfigValue(String value) {
myConfigValue = value;
}
}
Then create a task which updates the config:
UpdateConfigTask.java
public class UpdateConfigTask extends Task {
ExampleConfiguration config;
public UpdateConfigTask(ExampleConfiguration config) {
super("updateconfig");
this.config = config;
}
#Override
public void execute(Map<String, List<String>> parameters, PrintWriter output) {
config.setMyConfigValue("goodbye");
}
}
Also for demonstration purposes, create a resource which allows you to get the config value:
ConfigResource.java
#Path("/config")
public class ConfigResource {
private final ExampleConfiguration config;
public ConfigResource(ExampleConfiguration config) {
this.config = config;
}
#GET
public Response handleGet() {
return Response.ok().entity(config.getMyConfigValue()).build();
}
}
Finally wire everything up in your application:
ExampleApplication.java (exerpt)
environment.jersey().register(new ConfigResource(configuration));
environment.admin().addTask(new UpdateConfigTask(configuration));
Usage
Start up the application then run:
$ curl 'http://localhost:8080/config'
hello
$ curl -X POST 'http://localhost:8081/tasks/updateconfig'
$ curl 'http://localhost:8080/config'
goodbye
How it works
This works simply by passing the same reference to the constructor of ConfigResource.java and UpdateConfigTask.java. If you aren't familiar with the concept see here:
Is Java "pass-by-reference" or "pass-by-value"?
The linked classes above are to a project I've created which demonstrates this as a complete solution. Here's a link to the project:
scottg489/dropwizard-runtime-config-example
Footnote: I haven't verified this works with the built in configuration. However, the dropwizard Configuration class which you need to extend for your own configuration does have various "setters" for internal configuration, but it may not be safe to update those outside of run().
Disclaimer: The project I've linked here was created by me.
I'm running an embedded Jetty 8 server that loads a few *.war files at startup:
for (File aWebAppDirectory : aWebAppDirectories) {
if (aWebAppDirectory.exists() && aWebAppDirectory.isDirectory()) {
for (File warFile : aWebAppDirectory.listFiles(new WarFileFilter())) { String basename = warFile.getName().replaceFirst("\\.war$", "");
fHandlers.addHandler(new WebAppContext(warFile.getAbsolutePath(), "/" + basename));
}
}
}
These war-files have some dependencies on a few classes that may or may not exist in the classpath.
Right now if one of my servlets is missing a dependency, my entire embedded Jetty service fails. (Because of NoClassDefFoundExceptions)
I need a method that allows me to catch exceptions for failing servlets and simply doesn't activate them. I'm looking for the same thing that TomCat does when a servlet fails to load: It still loads the rest.
I haven't found any solutions after some time searching on Google.
Anyone know how I can tackle this problem using embedded Jetty 8?
If anyone is curious how I fixed this, I simply made sure that all my servlets have a wrapper servlet that basically has no dependencies. The wrapper tries to initialize a delegate with dependencies and explicitly checks for NoClassDefFountException. If this happens, the delegate is set to null, and all calls to the wrapper interface will result in an exception.
So on a high level:
public class ServletWrapper extends HttpServlet{
private ServletDelegate fDelegate;
//If this is false, the delegate does not work, and we should not forward anything to it.
private boolean fAvailable = false;
public ServletWrapper(){
try{
fDelegate = new ServletDelegate();
fAvailable = true;
} catch (NoClassDefFoundError e) {
fAvailable = false;
}
}
#Override
protected void doPost( HttpServletRequest request, HttpServletResponse response )
throws ServletException, IOException {
if ( !fAvailable || fDelegate==null ) {
response.sendError( HttpServletResponse.SC_SERVICE_UNAVAILABLE, LSP_MISSING_ERROR_MESSAGE );
return;
}
fDelegate.doPost(request,response);
}
}
It's simple enough, and it works.