Develop Reference

Add custom value to every log message

Let's say that I have a REST API with endpoint /user/{user_id}/foo.
Now when it is called I would like that all logs that come from handling this request contain information about {user_id}. Is it possible to achieve that without passing {user_id} to every method?
I'm using SLF4j for logging, my application is based on Spring Boot.

You could also use MDC for this, see here. It's essentially a map, you just put your contextual information in it (e.g. user id) and then you can use it in your log layout. Be aware that this only works with certain underlying frameworks like logback, where a sample layout pattern would look like this:
<Pattern>%X{user_id} %m%n</Pattern>
Check the logback manual for more details on this.

You can use Logback's Mapped Diagnotic Context to propagate the {user_id} to every log message.
There are two parts to this:
Push your {user_id} into MDC e.g. MDC.put("user_id", "Pawel");
Include the MDC entry in your log statements. You do this by specifying it in your logging pattern. So, if you store the user id in a MDC entry named "user_id" the you would set logging.pattern.level=user_id:%X{user_id} %5p to include the value of that entry in every log event.
More details in the docs

Is there a way to dyamic name log files in java?

I'm using logback as my logging framework and have a couple of jobs that run the same main function with different parameters and would like to create a log file for each job and name the log file with the job's name.
For example, if I had jobs a,b,c that all run MyClass.main() but with different parameters, then I'd like to see a-{date}.log, b-{date}.log, c-{date}.log.
I can achieve the {date} part by specifying a <fileNamePattern>myjob-%d{yyyy-MM-dd}.log</fileNamePattern> in my logback.xml, but I'm not sure how to (or if it is even possible) create the prefix of the file names dynamically (to be the job's name).
Is there a way to dynamically name logfiles in logback? Is there another logging framework that makes this possible?
As a follow up question, am I just taking a bad approach for having multiple jobs that call the same main function with different parameters and wanting a log file named after each job? If so is there a standard/best practice solution for this case?
EDIT: The reason why I want to name each log file after the name of the job is that each job naturally defines a "unit of work" and it is easier for me to find the appropriate log file in case one of the job fails. I could simply use a rolling log file for jobs a,b,c but I found it harder for me to look through the logs and pinpoint where each job started and ended.

I would use you own logging.
public static PrintWriter getLogerFor(String prefix) {
SimpleDatFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
String filename= prefix + sdf.format(new Date());
return new PrintWriter(filename, true); // auto flush.
}
You can write a simple LRU cache e.g. with LinkedHashMap to reuse the PrintWriters.

Is there a way to dynamically name logfiles in logback? Is there another logging framework that makes this possible?
I don't believe this is possible using the out of the box appenders (File, RollingFile etc) configured by a standard logback.xml file. To do what you want, you would need to dynamically create appenders on the fly and assign loggers to different appenders. Or you would need to invent a new appender that was smart enough to write to multiple files at the same time, based on the logger name.
am I just taking a bad approach for having multiple jobs that call the same main function with different parameters and wanting a log file named after each job?
The authors of logback address this issue and slightly discourage it in the section on Mapped Diagnostic Context
A possible but slightly discouraged approach to differentiate the logging output of one client from another consists of instantiating a new and separate logger for each client. This technique promotes the proliferation of loggers and may increase their management overhead. ... A lighter technique consists of uniquely stamping each log request servicing a given client.
Then they go on to discuss mapped diagnostic contexts as a solution to this problem. They give an example of a NumberCruncherServer which is crunching numbers, for various clients in various threads simultaneously. By setting the mapped diagnostic context and an appropriate logging pattern it becomes easy to determine which log events originated from which client. Then you could simply use a grep tool to separate logging events of interest into a separate file for detailed analysis.

Yes you can.
First you have to familiarize your self with these 2 concepts: Logger and Appender. Generally speaking, your code obtains a Logger, and invoke logging method such as debug(), warn(), info() etc. Logger has Appender attached to it, and Appender presents the logging information to the user according to the configuration set to it.
Once you're familiar, what you need to do is to dynamically create a FileAppender with a different file name for each different job type, and attach it to your Logger.
I suggest you spend some time with logback manual if none of above make sense.

You can make use of the logback discriminators, as discriminators' keys can be used in the <FileNamePattern> tag. I can think of two options:
Option One:
You can use the Mapped Diagnostic Context discriminator to implement your logging separation, you'll need to set a distinct value from each job using MDC.put();
Once you've done that your appender on logback configuration would look something like:
<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator class="ch.qos.logback.classic.sift.MDCBasedDiscriminator">
<key>jobName</key> <!-- the key you used with MDC.put() -->
<defaultValue>none</defaultValue>
</discriminator>
<sift>
<appender name="jobsLogs-${jobName}" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<FileNamePattern>${jobName}.%d{dd-MM-yyyy}.log.zip</FileNamePattern>
.
.
.
</rollingPolicy>
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>...</Pattern>
</layout>
</appender>
</sift>
</appender>
Option Two:
Implement your own discriminator -implementing ch.qos.logback.core.sift.Discriminator-, to discriminate based on the thread name. Would look something like this:
public class ThreadNameDiscriminator implements Discriminator<ILoggingEvent> {
private final String THREAD_NAME_KEY = "threadName";
#Override
public String getDiscriminatingValue(ILoggingEvent event) {
return Thread.currentThread().getName();
}
#Override
public String getKey() {
return THREAD_NAME_KEY;
}
// implementation for more methods
.
.
.
}
The logging appender would look like option one with the discriminator class being ThreadNameDiscriminator and the key being threadName. In this option there is no need to set a value to the MDC from your jobs, hence, no modification on them is required.

Log4j: Multiple categories with specializations

I have some doubts about the categories of log4j.
I have three categories ...
Program
Program.BUILD
Program.QUERY
When I define the following log4j.properties:
log4j.logger.program = DEBUG, stdout, file
log4j.logger.program.BUILD = DEBUG, file
and in Java I call:
Logger logger = Logger.getLogger("program.BUILD");
assume that the stdout and file are the appender to console and file respectively.
My problem is that when I specify the two categories, as shown, `program.BUILD log's are written to console and file. But he was only specified for the file appender. The log4j then makes it an inheritance?
I would like to specify three categories, but that when specified he caught the program.BUILD ONLY what was specified in that category, without taking the generic category (program).
But if not specified, the categories program.QUERY and program.BUILD, was picking up the program category, because it would represent the two that were not specified.
How can I do this?

Yes, Log4j has an inheritance system. You can disable it (that is, not let the log messages bubble up to the parent category) with the "additivity=false" flag.
Each enabled logging request for a given logger will be forwarded to
all the appenders in that logger as well as the appenders higher in
the hierarchy. In other words, appenders are inherited additively from
the logger hierarchy. For example, if a console appender is added to
the root logger, then all enabled logging requests will at least print
on the console. If in addition a file appender is added to a logger,
say C, then enabled logging requests for C and C's children will print
on a file and on the console. It is possible to override this default
behavior so that appender accumulation is no longer additive by
setting the additivity flag to false.
(See http://logging.apache.org/log4j/1.2/manual.html)

java logging API, disable logging to standard output

Using the standard java logging API (import java.util.logging.Logger), after the construction:
Logger l = Logger.getLogger("mylogger");
I am already able to log something. Since it has not a FileHandler, it doesn't write anything to disk.
l.severe("test with no handler");
It writes (some, not all) the log messages to output.
How can I disable this feature?
thanks in advance
Agostino

The question arises if you don't know the default configuration of java util logging.
Architectural fact:
0)Every logger whatever its name is has the root logger as parent.
Default facts:
1) the logger property useParentHandlers is true by default
2) the root logger has a ConsoleHandler by default
So. A new logger, by default sends its log records also to his parent(point 1) that is the root logger(point 0) wich, by default, logs them to console(point 2).
Remove console logging is easy as:
Logger l0 = Logger.getLogger("");
l0.removeHandler(l0.getHandlers()[0]);

Standard Loggers in Java are in a hierarchical structure and child Loggers by default inherit the Handlers of their parents. Try this to suppress parent Handlers from being used:
l.setUseParentHandlers(false);

By disable this feature you mean you don't want to log anything at all? If that's the case you have to set the level of that logger to NONE
For instance, if you're using a logging properties file you can set:
mylogger.level=NONE

Can I turn CONSOLE logging on/off depending on a system variable?

We maintain an application wide system variable [debug=true|false], I want to disable all CONSOLE appenders in log4j upon startup when the system variable debug is false.
Is the best way just a programatic scan of the appenders? Or is there a more elegant approach that I don't know about perhaps?
Any primer on scanning the appenders would be welcome also if that's the right approach.

I would write a special filter for the console appender. Along the line of
<appender name="stdout" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.TTCCLayout">
<param name="ConversionPattern" value="%d...m%n"/>
</layout>
<filter class="OnDebugOnly"/>
</appender>
With the Filter defined as follows
import org.apache.log4j.spi.Filter;
import org.apache.log4j.spi.LoggingEvent;
public class OnDebugOnly extends Filter {
static boolean debug;
#Override
public int decide(LoggingEvent event) {
return ( debug ? Filter.NEUTRAL : Filter.DENY ) ;
}
}
Of course this needs adjustments. Like where debug is defined and how it is accessed.
The Neutral is just in case someone adds another filter...
Plus the layout is just mine, use your preferred layout here.
Caveat. I did not test it ;-)

Have you looked into setting up a .properties file for log4j? You can configure log4j to send logging messages to a file, turn off sending log messages to the console for info, warn, etc. You can even configure it on a per class basis if needed. If you go to this site and navigate to the Configuration section, it should tell you what you need to know.
http://logging.apache.org/log4j/1.2/manual.html
Hopefully this answers your question.