Develop Reference

How to select Logback appender based on property file or environment variable

I have configured logback xml for a spring boot project.
I want to configure another appender based on the property configured. We want to create an appender either for JSON logs or for text log, this will be decided either by property file or by environment variable.
So I am thinking about the best approach to do this.
Using filters to print logs to 1 of the file (either to JSON or to Txt). But this will create both of the appenders. I want to create only 1 appender.
Use "If else" blocks in logback XML file. To put if else around appenders, loggers seems untidy and error prone. So will try to avoid as much as possible.
So now exploring options where I can add appender at runtime.
So I want to know if it is possible to add appender at runtime. And will it be added before spring boots up or it could be done anytime in the project.
What could be the best approach to include this scenario.

As you're already using Spring, I suggest using Spring Profiles, lot cleaner than trying to do the same programmatically. This approach is also outlined in Spring Boot docs.
You can set an active profile from either property file:
spring.profiles.active=jsonlogs
or from environment value:
spring_profiles_active=jsonlogs
of from startup parameter:
-Dspring.profiles.active=jsonlogs
Then have separate configurations per profile:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="stdout-classic" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{dd-MM-yyyy HH:mm:ss.SSS} %magenta([%thread]) %highlight(%-5level) %logger{36}.%M - %msg%n</pattern>
</encoder>
</appender>
<appender name="stdout-json" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
<timestampFormat>yyyy-MM-dd'T'HH:mm:ss.SSSX</timestampFormat>
<timestampFormatTimezoneId>Etc/UTC</timestampFormatTimezoneId>
<jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
<prettyPrint>true</prettyPrint>
</jsonFormatter>
</layout>
</encoder>
</appender>
<!-- begin profile-specific stuff -->
<springProfile name="jsonlogs">
<root level="info">
<appender-ref ref="stdout-json" />
</root>
</springProfile>
<springProfile name="classiclogs">
<root level="info">
<appender-ref ref="stdout-classic" />
</root>
</springProfile>
</configuration>

As the previous answer states, you can set different appenders based on Spring Profiles.
However, if you do not want to rely on that feature, you can use environments variables as described in the Logback manual. I.e.:
<appender name="json" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
<jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
<prettyPrint>true</prettyPrint>
</jsonFormatter>
<appendLineSeparator>true</appendLineSeparator>
</layout>
</encoder>
</appender>
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>
%cyan(%d{HH:mm:ss.SSS}) %gray([%thread]) %highlight(%-5level) %magenta(%logger{36}) - %msg%n
</pattern>
</encoder>
</appender>
<root level="info">
<!--
! Use the content of the LOGBACK_APPENDER environment variable falling back
! to 'json' if it is not defined
-->
<appender-ref ref="${LOGBACK_APPENDER:-json}"/>
</root>

Generated One extra blank log file For one Application Run

I am Configured Logger For my application And Give the Logfilename as Current Time Stamp So It Expected to create one log File with the name as current Time stamp BuT INSTEDE IT CREATE ONE LOGFILE WITH CURRENT TIMESTAMP AND ANOTHER FILE WHICH IS BLANK CANT FIGURE OUT WHY IT CREATING EXTRA FILE??
I am using Logback logger in my application and Here is my logback.xml looks like My application is simple core java application. where i user logger to log the statements
<?xml version="1.0" encoding="UTF-8"?>
<configuration>`enter code here`
<timestamp key="byDay" datePattern="yyyy'-'MM'-'dd'''AT'''HH'-'mm'-'ss"/>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file> Myapplication${byDay}.txt </file>
<append>FALSE</append>
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="FILE" />
</root>
</configuration>

Exclude default logback file

My team is developing a telecom realtime application with high calls per second rate. We are using logback to filter log based on a key-value match (traffic live values, like Calling Party, and so on). The filtered log file is correctly created, once verified the match from live values and db values, but we would get rid of default file which is filled with logs when there is no match. It might happen that a traffic node needs to be monitored for a while before a key-value match takes place, so in the meantime the default could indefinitely increase in size and cause problems to performance and stability of node itself. What should I do in my logback.xml to avoid generation of default log file? Is it possible? Any other option to achieve same result?
logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<property scope="context" name="LOG_LEVEL" value="INFO" />
<appender name="SIFT_LOGGER" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator class="com.ericsson.jee.ngin.services.log.ServiceKeyDiscriminator">
</discriminator>
<sift>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<prudent>true</prudent>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>/var/log/tit/logback_${serviceKey}_%d{yyyy-MM-dd}_%i.log</fileNamePattern>
<maxFileSize>1MB</maxFileSize>
<maxHistory>10</maxHistory>
<totalSizeCap>2GB</totalSizeCap>
</rollingPolicy>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>${LOG_LEVEL}</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
<!-- encoders are by default assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder -->
<encoder>
<pattern> %d{yyyy-MM-dd HH:mm:ss.SSSZ} [%thread] %-5level %logger{36} %msg%n</pattern>
</encoder>
</appender>
</sift>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are by default assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder -->
<encoder>
<pattern> %d{yyyy-MM-dd HH:mm:ss.SSSZ} [%thread] %-5level %logger{36} %msg%n</pattern>
</encoder>
</appender>
<turboFilter class="ch.qos.logback.classic.turbo.DynamicThresholdFilter">
<key>serviceKey</key>
<defaultThreshold>DEBUG</defaultThreshold>
<onHigherOrEqual>ACCEPT</onHigherOrEqual>
<onLower>ACCEPT</onLower>
</turboFilter>
<root level="DEBUG">
<appender-ref ref="SIFT_LOGGER" />
<appender-ref ref="STDOUT" />
</root>
ATTACHMENTS: FILTERED LOGBACK CLASS
The provided FL class is only working for a SK which has a java Discriminator in FL module.

You must move the filter to the general sift appender.
<appender name="SIFT-TRACE"
class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator
class="ch.qos.logback.classic.sift.MDCBasedDiscriminator">
<Key>loggerFileName</Key>
<DefaultValue>unknown</DefaultValue>
</discriminator>
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator
class="ch.qos.logback.classic.boolex.JaninoEventEvaluator">
<expression>
mdc.get("loggerFileName")!=null
</expression>
</evaluator>
<OnMismatch>DENY</OnMismatch>
<OnMatch>NEUTRAL</OnMatch>
</filter>
<sift>
<appender name="TRACE-${loggerFileName}"
class="ch.qos.logback.core.FileAppender">
<File>D:/U/${loggerFileName}.log</File>
<Append>true</Append>
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>%d [%thread] %level %mdc %logger - %msg%n</Pattern>
</layout>
</appender>
</sift>
</appender>
<logger name="org.springframework" level="DEBUG" />
<root level="DEBUG">
<appender-ref ref="SIFT-TRACE" />
</root>
Also to make it work correctly, you MUST after each Thread/file/marker/etc. to put those statements:
public void handle()
{
MDC.put("loggerFileName","some value");
...
MDC.remove("loggerFileName");
}

You have defined this root logger:
<root level="DEBUG">
<appender-ref ref="SIFT_LOGGER" />
<appender-ref ref="STDOUT" />
</root>
This means that all log events with Level >= DEBUG will be directed to two appenders:
SIFT_LOGGER
STDOUT
If I understand your question correctly then you do want logs to be written via your SIFT_APPENDER but you don't want any other log output. If so, then just remove this entry:
<appender-ref ref="STDOUT" />
The STDOUT appender is a console appender so it doesn't actually write to a log file instead it writes to System.out. I suspect the reason you are seeing these log events in some file is that whatever is running your application is redirecting System.out to a file. As long as you only have your SIFT_APPENDER in the root logger definition then you can be confident that this will be the only appender in play. Note: once you remove the appender from the root logger you can probbaly remove it from logback.xml since it is unused.
Update 1: Based on your last comment I now understand that you want to discard the logs which arrive at the SiftingAppender but do not match a given condition. I suspect what's happeneing here is that some log events arrive at the sifting appender with an 'unknown' value for serviceKey and these events are then written to /var/log/tit/logback_[unknownValue]_%d{yyyy-MM-dd}_%i.log. Is this the crux of the issue? If so, then you can add a filter into the nested appender. Here are some examples:
Using Groovy to express the 'contains unknown serviceKey condition':
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<!-- GEventEvaluator requires Groovy -->
<evaluator
class="ch.qos.logback.classic.boolex.GEventEvaluator">
<expression>
serviceKey == null
</expression>
</evaluator>
<OnMismatch>NEUTRAL</OnMismatch>
<OnMatch>DENY</OnMatch>
</filter>
Using Janino to express the 'contains unknown serviceKey condition':
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<!-- JaninoEventEvaluator requires Janino -->
<evaluator
class="ch.qos.logback.classic.boolex.JaninoEventEvaluator">
<expression>
serviceKey == null
</expression>
</evaluator>
<OnMismatch>NEUTRAL</OnMismatch>
<OnMatch>DENY</OnMatch>
</filter>
With either of these filters in place any log events which arrive at the sifting appender and have the 'unknown' serviceKey will be ignored. Note: I have written the 'contains unknown serviceKey condition' as serviceKey == null your logic might differ but the above examples show should you how to tell Logback to apply this filter for you.

Just to notify to #glitch (and all others interested) the happy conclusion of this issue: I have managed to make the tag expression working was this:
<expression>mdc.get("servicekey") == null</expression>
Thanks to this expression, I have got the wanted behavior: the default file "IS_UNDEFINED is not generated when the key doesn't match with the runtime traffic values.
The reason is because the type of Event in JaninoEventEvaluator is LoggingEvent that has a reserve object "mdc" (the type is Map).
Regards,
Pierluigi

How to hide the output of a single Logback logger?

I use logback in the following case
package ninja.template;
public class TemplateEngineManagerImpl implements TemplateEngineManager {
private final Logger logger = LoggerFactory.getLogger(TemplateEngineManagerImpl.class);
...
logger.info("Registered response template engines");
...
and I want to hide all the INFO output of this TemplateEngineManagerImpl class (or all the output - the class only logs at INFO level) but this class only.
Unfortunately, the following configuration doesn't work as I can still see "Registered response template engines" in my console output.
<configuration>
<appender name="STDOUT_TERSE" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern> | %msg%n</pattern>
</encoder>
</appender>
<logger name="ninja.template.TemplateEngineManagerImpl" level="OFF"/>
<root level="info">
<appender-ref ref="STDOUT_TERSE" />
</root>
</configuration>
note I tried also the following without results
<logger name="ninja.template.TemplateEngineManagerImpl" level="WARN"/>
and
<logger name="ninja.template" level="WARN"/>
and
<logger name="TemplateEngineManagerImpl" level="WARN"/>

The provided logback configuration works
<logger name="ninja.bodyparser.BodyParserEngineManagerImpl" level="WARN" additive="false"/>
The error I had that the string was logged in two different places.
So by displaying the full logger name with the following configuration:
<appender name="STDOUT_TERSE" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern> | %logger | %msg%n</pattern>
</encoder>
I was able to copy and paste the string that should be used as the logger name!

Strange behavior of Logback - empty files

I'm refurbishing old system that used logback, some simple stuff ( 3 appenders, 2 loggers ). Now in next version of system ( or re-implementation as the previous version was stolen with notebook and backup, only config file and binaries were already on robot ), I use the same config file, but all log files stays empty.
Strange thing is that it actually creates correct files and folders by the given patterns, so it definitely do something with the config file. Besides that, loggers and appenders aren't working at all.
I also tried to use various other configuration files I found on examples - not even one worked, so I'm suspicious that there is some collision between used libraries and logback. I tried to google it, but found nothing relevant or working.
Have anyone of you came around ( or hopefully through ) such a problem? Or please point at the wrong line...
Thx in advance...
Kamil
Next code shows initialization:
public static final String LOGGER_CONFIG_FILE = "hacs.logger.conf";
public static final String LOGGER_CONFIG_FILE_DEFAULT = "./conf/logconf.xml";
public static void main( String[] args ) {
File configurationFile = new File(HACSProperties.instance().getProperty(LOGGER_CONFIG_FILE, LOGGER_CONFIG_FILE_DEFAULT));
if( configurationFile.exists() ){
LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory();
context.reset(); // When this is commented, logback works in some default configuration
try {
JoranConfigurator configurator = new JoranConfigurator();
configurator.setContext(context);
configurator.doConfigure( LOGGER_CONFIG_FILE_DEFAULT );
System.out.println("Logger successfully configured..");
Logger log = LoggerFactory.getLogger("analytics");
log.info( "Please appear in file" );
} catch (JoranException je) {
System.out.println("Error - failed to configure logger. Please check configuration.");
je.printStackTrace();
System.exit( 1 );
}
} else {
System.out.println("Error - failed to configure logger - configuration file does not exist. Please check configuration.");
System.exit( 2 );
}
}
Config file itself:
<configuration>
<timestamp key="bySecond" datePattern="dd.MM.yyyy'_T'HH.mm.ss" timeReference="contextBirth" />
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>./logs/HACS_LAST_RUN.log</file>
<append>false</append>
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>%date %level [%thread] %logger{10} [%file:%line] %msg%n
</Pattern>
</layout>
</appender>
<appender name="FILE_PER_MINUTE" class="ch.qos.logback.core.FileAppender">
<file>./logs/PER_MINUTE/HACS-RUN-${bySecond}.log</file>
<append>false</append>
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>%date %level [%thread] %logger{10} [%file:%line] %msg%n
</Pattern>
</layout>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>%d{HH:mm:ss.SSS} %level [%file:%line] %msg%n</Pattern>
</layout>
</appender>
<logger name="org.hibernate" additivity="false">
<level value="warn" />
<appender-ref ref="FILE" />
<appender-ref ref="STDOUT" />
<appender-ref ref="FILE_PER_MINUTE" />
</logger>
<logger name="FILE_ONLY" additivity="false">
<level value="INFO" />
<appender-ref ref="FILE_PER_MINUTE" />
<appender-ref ref="FILE" />
</logger>
<root>
<level value="INFO" />
<appender-ref ref="FILE" />
<appender-ref ref="STDOUT" />
<appender-ref ref="FILE_PER_MINUTE" />
</root>
</configuration>
List of libraries:
antlr-2.7.7.jar
aopalliance-1.0.jar
asm-3.3.jar
bcprov-jdk15-1.43.jar
cglib-2.2.jar
chronicle.jar
commons-collections-3.2.1.jar
commons-lang-2.5.jar
commons-logging-1.1.1.jar
commons-pool-1.5.2.jar
cxf-2.3.0.jar
cxf-manifest.jar
cxf-xjc-boolean-2.3.0.jar
cxf-xjc-bug671-2.3.0.jar
cxf-xjc-dv-2.3.0.jar
cxf-xjc-ts-2.3.0.jar
dom4j-1.6.1.jar
FastInfoset-1.2.8.jar
felix.jar
geronimo-activation_1.1_spec-1.1.jar
geronimo-annotation_1.0_spec-1.1.1.jar
geronimo-javamail_1.4_spec-1.7.1.jar
geronimo-jaxws_2.2_spec-1.0.jar
geronimo-jms_1.1_spec-1.1.1.jar
geronimo-servlet_3.0_spec-1.0.jar
geronimo-stax-api_1.0_spec-1.0.1.jar
geronimo-ws-metadata_2.0_spec-1.1.3.jar
groovy-all-1.8.5.jar
h2-1.3.154.jar
hibernate-jpa-2.0-api-1.0.0.Final.jar
hibernate3.jar
javassist-3.12.0.GA.jar
jaxb-api-2.2.1.jar
jaxb-impl-2.2.1.1.jar
jaxb-xjc-2.2.1.1.jar
jettison-1.2.jar
jetty-continuation-7.1.6.v20100715.jar
jetty-http-7.1.6.v20100715.jar
jetty-io-7.1.6.v20100715.jar
jetty-server-7.1.6.v20100715.jar
jetty-util-7.1.6.v20100715.jar
jmdns.jar
jra-1.0-alpha-4.jar
js-1.7R1.jar
jsr311-api-1.1.1.jar
jta-1.1.jar
logback-access-1.0.1.jar
logback-classic-1.0.1.jar
logback-core-1.0.1.jar
neethi-2.0.4.jar
oro-2.0.8.jar
saaj-api-1.3.jar
saaj-impl-1.3.2.jar
serializer-2.7.1.jar
slf4j-api-1.6.4.jar
spring-aop-3.0.4.RELEASE.jar
spring-asm-3.0.4.RELEASE.jar
spring-beans-3.0.4.RELEASE.jar
spring-context-3.0.4.RELEASE.jar
spring-core-3.0.4.RELEASE.jar
spring-expression-3.0.4.RELEASE.jar
spring-jms-3.0.4.RELEASE.jar
spring-tx-3.0.4.RELEASE.jar
spring-web-3.0.4.RELEASE.jar
stax2-api-3.0.2.jar
velocity-1.6.4.jar
waiter-dns.jar
woodstox-core-asl-4.0.8.jar
wsdl4j-1.6.2.jar
wss4j-1.5.9.jar
xalan-2.7.1.jar
xml-resolver-1.2.jar
xmlbeans-2.4.0.jar
XmlSchema-1.4.7.jar
xmlsec-1.4.3.jar