I have a class that extends LayoutBase. In the doLayout method, I add a key to my logging called MSG and the value is set to ILoggingEvent event.getMessage().
I'm seeing values added to my logging but they're not consistent; some logging messages and some exception stack traces.
Can anyone tell me where ILoggingEvent event.getMessage() gets its value from?
Please refer to Logback architecture
It mentions that each log will go through series of step
Get Filter Chain Decision
If it exists, the TurboFilter chain is invoked. Turbo filters can set a context-wide threshold, or filter out certain events based on information such as Marker, Level, Logger, message, or the Throwable that are associated with each logging request.
Apply the selection rule
At this step, logback compares the effective level of the logger with the level of the request. If the logging request is disabled according to this test, then logback will drop the request without further processing.
Create a LoggingEvent object
If the request survived the previous filters, logback will create a LoggingEvent object containing all the relevant parameters of the request
Invoking appenders
After the creation of a LoggingEvent object, logback will invoke the doAppend() methods of all the applicable appenders, that is, the appenders inherited from the logger context.
Formatting the output
It is the responsibility of the invoked appender to format the logging event. However, some (but not all) appenders delegate the task of formatting the logging event to a layout.
Sending out the LoggingEvent
After the logging event is fully formatted it is sent to its destination by each appender.
I'm seeing values added to my logging but they're not consistent; some logging messages and some exception stack traces.
Now, coming to your query, it seems that you are receiving events which have exception information (Throwable) during the formatting step.
You can create a CustomFilter to filter out such events. All you have to do is extends Filter<ILoggingEvent>
public class DenyExceptionFilter extends Filter<ILoggingEvent> {
#Override
public FilterReply decide(ILoggingEvent iLoggingEvent) {
final IThrowableProxy throwableProxy = iLoggingEvent.getThrowableProxy();
if (throwableProxy != null && throwableProxy instanceof ThrowableProxy)
return FilterReply.DENY;
return FilterReply.ACCEPT;
}
}
This can be much more powerful, can filter specific types of exception. You can take this as your homework :P
Then you can add this Custom Filter to your appender as
<appender name="APPENDER_NAME" class="ch.qos.logback.classic.AsyncAppender">
<filter class="com.stackoverflow.DenyExceptionFilter" />
</appender>
Of course, add your layout as well after the filter.
Related
Let's say that I have a REST API with endpoint /user/{user_id}/foo.
Now when it is called I would like that all logs that come from handling this request contain information about {user_id}. Is it possible to achieve that without passing {user_id} to every method?
I'm using SLF4j for logging, my application is based on Spring Boot.
You could also use MDC for this, see here. It's essentially a map, you just put your contextual information in it (e.g. user id) and then you can use it in your log layout. Be aware that this only works with certain underlying frameworks like logback, where a sample layout pattern would look like this:
<Pattern>%X{user_id} %m%n</Pattern>
Check the logback manual for more details on this.
You can use Logback's Mapped Diagnotic Context to propagate the {user_id} to every log message.
There are two parts to this:
Push your {user_id} into MDC e.g. MDC.put("user_id", "Pawel");
Include the MDC entry in your log statements. You do this by specifying it in your logging pattern. So, if you store the user id in a MDC entry named "user_id" the you would set logging.pattern.level=user_id:%X{user_id} %5p to include the value of that entry in every log event.
More details in the docs
Is it possible to configure an interceptor in Spring Integration to specify a logging message?
This is in contrast to using a wire-tap to send the message to a logging channel to which a logging channel adapter subscribes. The problem with this approach is that the logging channel has the expression for what exactly to log. In my use case I'd like to use a global channel-interceptor to specify a logging message and send it to a logging channel adapter, rather than having to define a logging channel adapter for every possible logging message I might want.
For a moment I thought I could define a service activator with an SpEL expression to produce the string logging message (and with an output-channel of the deisred logging channel) inside a channel-interceptor definition, but it's looking for an input-channel.
Instead of using a wire-tap, you could make the channels you are interested in <publish-subscribe-channel/>s.
Subscribe the appropriate transformer to the channel (with an output-channel going to the logging adapter).
You can control whether the log happens before or after the real subscriber using the order attribute on the transformer and other subscriber.
Another alternative is a global interceptor that adds a header to the message
MessageBuilder.fromMessage(message).setHeader('foo', routeForThisMessageType).build()
Then send it to a <header-value-router/> which, in turn, routes to the appropriate transformer, and thence to the single logging channel adapter.
Of course, if you want, you can combine the routing and/or transforming right into the interceptor.
I'm using logback as my logging framework and have a couple of jobs that run the same main function with different parameters and would like to create a log file for each job and name the log file with the job's name.
For example, if I had jobs a,b,c that all run MyClass.main() but with different parameters, then I'd like to see a-{date}.log, b-{date}.log, c-{date}.log.
I can achieve the {date} part by specifying a <fileNamePattern>myjob-%d{yyyy-MM-dd}.log</fileNamePattern> in my logback.xml, but I'm not sure how to (or if it is even possible) create the prefix of the file names dynamically (to be the job's name).
Is there a way to dynamically name logfiles in logback? Is there another logging framework that makes this possible?
As a follow up question, am I just taking a bad approach for having multiple jobs that call the same main function with different parameters and wanting a log file named after each job? If so is there a standard/best practice solution for this case?
EDIT: The reason why I want to name each log file after the name of the job is that each job naturally defines a "unit of work" and it is easier for me to find the appropriate log file in case one of the job fails. I could simply use a rolling log file for jobs a,b,c but I found it harder for me to look through the logs and pinpoint where each job started and ended.
I would use you own logging.
public static PrintWriter getLogerFor(String prefix) {
SimpleDatFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
String filename= prefix + sdf.format(new Date());
return new PrintWriter(filename, true); // auto flush.
}
You can write a simple LRU cache e.g. with LinkedHashMap to reuse the PrintWriters.
Is there a way to dynamically name logfiles in logback? Is there another logging framework that makes this possible?
I don't believe this is possible using the out of the box appenders (File, RollingFile etc) configured by a standard logback.xml file. To do what you want, you would need to dynamically create appenders on the fly and assign loggers to different appenders. Or you would need to invent a new appender that was smart enough to write to multiple files at the same time, based on the logger name.
am I just taking a bad approach for having multiple jobs that call the same main function with different parameters and wanting a log file named after each job?
The authors of logback address this issue and slightly discourage it in the section on Mapped Diagnostic Context
A possible but slightly discouraged approach to differentiate the logging output of one client from another consists of instantiating a new and separate logger for each client. This technique promotes the proliferation of loggers and may increase their management overhead. ... A lighter technique consists of uniquely stamping each log request servicing a given client.
Then they go on to discuss mapped diagnostic contexts as a solution to this problem. They give an example of a NumberCruncherServer which is crunching numbers, for various clients in various threads simultaneously. By setting the mapped diagnostic context and an appropriate logging pattern it becomes easy to determine which log events originated from which client. Then you could simply use a grep tool to separate logging events of interest into a separate file for detailed analysis.
Yes you can.
First you have to familiarize your self with these 2 concepts: Logger and Appender. Generally speaking, your code obtains a Logger, and invoke logging method such as debug(), warn(), info() etc. Logger has Appender attached to it, and Appender presents the logging information to the user according to the configuration set to it.
Once you're familiar, what you need to do is to dynamically create a FileAppender with a different file name for each different job type, and attach it to your Logger.
I suggest you spend some time with logback manual if none of above make sense.
You can make use of the logback discriminators, as discriminators' keys can be used in the <FileNamePattern> tag. I can think of two options:
Option One:
You can use the Mapped Diagnostic Context discriminator to implement your logging separation, you'll need to set a distinct value from each job using MDC.put();
Once you've done that your appender on logback configuration would look something like:
<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator class="ch.qos.logback.classic.sift.MDCBasedDiscriminator">
<key>jobName</key> <!-- the key you used with MDC.put() -->
<defaultValue>none</defaultValue>
</discriminator>
<sift>
<appender name="jobsLogs-${jobName}" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<FileNamePattern>${jobName}.%d{dd-MM-yyyy}.log.zip</FileNamePattern>
.
.
.
</rollingPolicy>
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>...</Pattern>
</layout>
</appender>
</sift>
</appender>
Option Two:
Implement your own discriminator -implementing ch.qos.logback.core.sift.Discriminator-, to discriminate based on the thread name. Would look something like this:
public class ThreadNameDiscriminator implements Discriminator<ILoggingEvent> {
private final String THREAD_NAME_KEY = "threadName";
#Override
public String getDiscriminatingValue(ILoggingEvent event) {
return Thread.currentThread().getName();
}
#Override
public String getKey() {
return THREAD_NAME_KEY;
}
// implementation for more methods
.
.
.
}
The logging appender would look like option one with the discriminator class being ThreadNameDiscriminator and the key being threadName. In this option there is no need to set a value to the MDC from your jobs, hence, no modification on them is required.
I'm currently working on a web application using Spring MVC, and I use the #ExceptionHandler annotation in every controllers of the application.
So basically I got a method like this:
#ExceptionHandler(RuntimeException.class)
public String handleException(RuntimeException ex) {
injectedService.notifyAndLogException(ex.getMessage());
return ("error_page");
}
My idea is to log and send an email to an application administrator in the injected service.
For now, I've tried to read some documentation about logging in spring application, and all the things I've seen is setting a static logger in each controller.
Like this:
private final Logger log = LoggerFactory.getLogger(Controller.class);
#ExceptionHandler(RuntimeException.class)
public String handleException(RuntimeException ex) {
log.info("Logging error");
injectedService.notifyException(ex.getMessage());
return ("error_page");
}
I'd like to know what is the point to use a logger in each controller instead of using it in one point only (the service)?
I'd like to know what is the point to use a logger in each controller instead of using it in one point only
If you use a single logger for the whole application, then every log message will be logged as coming from the same component. By using a logger per class or component, then your log files will contain information about which component logged the message.
For example, when you do:
Logger log = LoggerFactory.getLogger(Controller.class);
This creates a logger with the name of the Controller class, which will generally be displayed in the log file, e.g.
2012-03-07:12:59:00 com.x.y.Controller Hello!
This is just a convention, but it's a good one that I advise you follow.
a logger in each of your class files enables you get 'debug' or 'info' level when you are in production, or not able to connect a debugger.
Since you can limit via package or even class file name, what is logged, you can pin point to find errors, or to see what is happening under different load situations (concurrency problems, resources used ). If you use one generic logger, then you may flood your log file.
With the logger in the class that received the exception, you may be able to get at class variables that are not being passed into your exception handler.
I would also recommend that you do not do
injectedService.notifyAndLogException(ex.getMessage());
but pass the exception into your notify. While stack traces can be notorious verbose, the messages usually are not very help full ( NullPointerException without a stacktrace ? ). In your notify service you can set the subject to ex.getMessage() and the body have the entire stack trace.
Your controllers can extend an abstract class that declares a logger like that
protected Logger logger = LoggerFactory.getLogger( getClass() );
This logger can be used in all controller and it will prefix the log message with the controller class name.
I have some doubts about the categories of log4j.
I have three categories ...
Program
Program.BUILD
Program.QUERY
When I define the following log4j.properties:
log4j.logger.program = DEBUG, stdout, file
log4j.logger.program.BUILD = DEBUG, file
and in Java I call:
Logger logger = Logger.getLogger("program.BUILD");
assume that the stdout and file are the appender to console and file respectively.
My problem is that when I specify the two categories, as shown, `program.BUILD log's are written to console and file. But he was only specified for the file appender. The log4j then makes it an inheritance?
I would like to specify three categories, but that when specified he caught the program.BUILD ONLY what was specified in that category, without taking the generic category (program).
But if not specified, the categories program.QUERY and program.BUILD, was picking up the program category, because it would represent the two that were not specified.
How can I do this?
Yes, Log4j has an inheritance system. You can disable it (that is, not let the log messages bubble up to the parent category) with the "additivity=false" flag.
Each enabled logging request for a given logger will be forwarded to
all the appenders in that logger as well as the appenders higher in
the hierarchy. In other words, appenders are inherited additively from
the logger hierarchy. For example, if a console appender is added to
the root logger, then all enabled logging requests will at least print
on the console. If in addition a file appender is added to a logger,
say C, then enabled logging requests for C and C's children will print
on a file and on the console. It is possible to override this default
behavior so that appender accumulation is no longer additive by
setting the additivity flag to false.
(See http://logging.apache.org/log4j/1.2/manual.html)