Develop Reference

How to select Logback appender based on property file or environment variable

I have configured logback xml for a spring boot project.
I want to configure another appender based on the property configured. We want to create an appender either for JSON logs or for text log, this will be decided either by property file or by environment variable.
So I am thinking about the best approach to do this.
Using filters to print logs to 1 of the file (either to JSON or to Txt). But this will create both of the appenders. I want to create only 1 appender.
Use "If else" blocks in logback XML file. To put if else around appenders, loggers seems untidy and error prone. So will try to avoid as much as possible.
So now exploring options where I can add appender at runtime.
So I want to know if it is possible to add appender at runtime. And will it be added before spring boots up or it could be done anytime in the project.
What could be the best approach to include this scenario.

As you're already using Spring, I suggest using Spring Profiles, lot cleaner than trying to do the same programmatically. This approach is also outlined in Spring Boot docs.
You can set an active profile from either property file:
spring.profiles.active=jsonlogs
or from environment value:
spring_profiles_active=jsonlogs
of from startup parameter:
-Dspring.profiles.active=jsonlogs
Then have separate configurations per profile:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="stdout-classic" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{dd-MM-yyyy HH:mm:ss.SSS} %magenta([%thread]) %highlight(%-5level) %logger{36}.%M - %msg%n</pattern>
</encoder>
</appender>
<appender name="stdout-json" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
<timestampFormat>yyyy-MM-dd'T'HH:mm:ss.SSSX</timestampFormat>
<timestampFormatTimezoneId>Etc/UTC</timestampFormatTimezoneId>
<jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
<prettyPrint>true</prettyPrint>
</jsonFormatter>
</layout>
</encoder>
</appender>
<!-- begin profile-specific stuff -->
<springProfile name="jsonlogs">
<root level="info">
<appender-ref ref="stdout-json" />
</root>
</springProfile>
<springProfile name="classiclogs">
<root level="info">
<appender-ref ref="stdout-classic" />
</root>
</springProfile>
</configuration>

As the previous answer states, you can set different appenders based on Spring Profiles.
However, if you do not want to rely on that feature, you can use environments variables as described in the Logback manual. I.e.:
<appender name="json" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
<jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
<prettyPrint>true</prettyPrint>
</jsonFormatter>
<appendLineSeparator>true</appendLineSeparator>
</layout>
</encoder>
</appender>
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>
%cyan(%d{HH:mm:ss.SSS}) %gray([%thread]) %highlight(%-5level) %magenta(%logger{36}) - %msg%n
</pattern>
</encoder>
</appender>
<root level="info">
<!--
! Use the content of the LOGBACK_APPENDER environment variable falling back
! to 'json' if it is not defined
-->
<appender-ref ref="${LOGBACK_APPENDER:-json}"/>
</root>

Ways to configure logs other than log4j xml, properties file, and source code (mainly java)?

I'm looking at the source code of some application. It is using Spring framework, Apache Tiles, JSP, Log4j, java, javascript, jquery, jqplot, Jsch, and etc.
I know where the logs are created. (a/b/logs) However, when I look at source code, I don't know how logs are created under the folder name 'logs'. I looked at log4j.xml, web.xml , property files. I found the code for how the path 'a/b' is created, but not logs. Also that folder has 4 types of logs. And they are name in a pattern like access.20181227001 , errors.20182111. I want to know where I have to look to find how the logs are created in this manner.
Log4J.xml
<!-- Appenders -->
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %5p [%c] %m%n" />
</layout>
</appender>
<appender name="console-infolog" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %5p %m%n" />
</layout>
</appender>
<!-- Application Loggers -->
<logger name="com.dsmentoring.chakan" additivity="false">
<level value="debug" />
<appender-ref ref="console"/>
</logger>
<!-- 3rdparty Loggers -->
<logger name="org.springframework.core">
<level value="error"/>
</logger>
<!-- Bean logger -->
<logger name="org.springframework.beans">
<level value="error"/>
</logger>
<!-- Context logger -->
<logger name="org.springframework.context">
<level value="error"/>
</logger>
<!-- Web logger -->
<logger name="org.springframework.web">
<level value="error"/>
</logger>
<logger name="org.springframework.ldap" additivity="true">
<level value="error"/>
</logger>
<!-- LDAP logger -->
<logger name="com.unboundid.ldap" additivity="true">
<level value="error"/>
</logger>
<!-- Root Logger -->
<root>
<priority value="off" />
<appender-ref ref="console" />
</root>
To sum up :
1) Is there a way to configure where logs are created and how they are created (4 types of logs) Other than log4j.xml, xml files and property files? I looked at all the java, jsp, js code but can't seem to find the configuration for logs. So I want to know if there are other ways to do that or where I should look for those configuration.
2) The 'logs' folder is possibly default for log4j?
ldap.properties
#LDAP Connection Info
ldap.host=192.168.0.17
ldap.port=22389
ldap.userName=cn=directory manager
ldap.password= 9074B18A0DE2D50C068D37B60BE5DFDE
ldap.baseDN=o=sso30root
ldap.defaultLoadSize=1000
ldap.start=start-ds
ldap.stop=stop-ds
ldap.workdir=/home/KB_openDJ // logs are created under this path
// /home/KB_openDJ/logs
In other java class, they use this.
#Value("${ldap.workdir}")
private String WORK_DIR;
// I ommited many lines in between
try{
diskUsage = sigar.getFileSystemUsage(WORK_DIR);
diskIOInfo.setDiskRead((int)(diskUsage.getDiskReadBytes()));
diskIOInfo.setDiskWrite((int)(diskUsage.getDiskWriteBytes()));
}catch(SigarException sigarEx){
log.debug("Disk Usage info load Error : " + sigarEx.getMessage());
}
I used the 'Search' feature in Eclipse many times. ( logs, WORK_DIR, and 4 types of log name, and many others. I am unable to locate the code about logging configuration. :(
My log4j version :
1.2.15

Ok, it seems that there's a solution which might help you. It requires some debugging of static initalization block of org.apache.log4j.LogManager class. This class is responsible for loading logger configuration file. Here's a documention link which thoroughly describes initalization process: link.
Here's an excerpt from a LogManager source file:
String configurationOptionStr = OptionConverter.getSystemProperty(DEFAULT_CONFIGURATION_KEY, null);
String configuratorClassName = OptionConverter.getSystemProperty(CONFIGURATOR_CLASS_KEY, null);
What I'm trying to show you here, is that your logger configuration file might be specified as a JVM option supplied to your application-server. That's why you are not able to determine actual file being used.
Even if this approach fails, I'd recommend you to investigate the appenders list retrieved at run-time. Here's a stackoverflow thread which describes how to do so: link.

How to extract output of retire.js into a file from maven execution

I am using retire.js plugin in maven pom file. The vulnerability details are listed along with the build output.
I would like to extract the retire.js output into a separate file.
Can you please suggest some ways to extract only retire.js data into a file.

Looking into the source code of retire.js Maven plugin we can notice that the log output from retirejs is redirected into Maven's stream (in initMiniLog()). And there seem to be no specific configuration for it.
However, with a bit of tweaking we can set up Maven to gather these logs. So I can suggest the following:
1) By default Maven uses slf4j-simple logger, remove its jar from {M2_HOME}/lib.
2) Place in the same folder a logging library that supports output into files, for example Logback: logback-classic-*.jar and logback-core-*.jar.
3) Define a configuration that will output everything into stdout and only the things that you are looking for into the file. logback.xml should be placed into {M2_HOME}/conf/logging. For example, I used the following draft configuration to extract the output of maven-compiler-plugin into maven.log in the current folder:
<configuration>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>./maven.log</file>
<encoder>
<pattern>%message%n</pattern>
</encoder>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%level - %message%n</pattern>
</encoder>
</appender>
<!--
specify the package of retirejs: com.h3xstream.retirejs
-->
<logger name="org.apache.maven.plugin.compiler" level="TRACE">
<appender-ref ref="FILE" />
</logger>
<root level="INFO">
<appender-ref ref="STDOUT" />
</root>
</configuration>
4) Execute your build command...

Logback: separate logging WARN / TRACE

I'm trying to configure logback to print:
- everything (level trace or debug) to the screen
- everything (level trace or debug) to the debugfile
- warnings and above to an error file
My logback.xml config is like this:
...
<logger name="be" level="TRACE">
<appender-ref ref="FILE-AUDIT" />
<appender-ref ref="STDOUT" />
</logger>
<root level="WARN">
<appender-ref ref="FILE-ERROR" />
</root>
However, the error and debug file contain exactly the same, being ALL logging (debug and error). I've already tried to play with the additivity option, but that's apparently not what I need.
The second question is that I use name "be" to have all classes under be.* but actually I want to capture everything there (com.* as well).

Found a solution by adding a filter to the appender:
<appender name="FILE-ERROR"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- deny all events with a level below WARNING -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>WARN</level>
</filter>

Run Logback in Debug

I've recently switched from log4j to logback and am wondering if there is an easy way to run logback in debug mode, similar to log4j's log4j.debug property. I need to see where it is picking up my logback.xml from.
The docs mention using a StatusPrinter to print out logback's internal status, but that would require code changes.

[EDIT]
This has been fixed in Logback 1.0.4. You can now use -Dlogback.debug=true to enable debugging of the logback setup.
-- Old Answer --
Unfortunately, there is no way to enable debugging via a System property. You have to use <configuration debug="true"> in the logback.xml. Please submit a feature request.

This is how I do it. I set a system property called 'log.level', then I reference it in logback.xml.
Edit: The downside is that you MUST have 'log.level' always set. The way I deal with this is to check in my main method and set it to INFO if not already set, be sure to do this before you first logging calls. Then I can override on the command line, and have a sensible default.
Here is how it looks in my logback.xml:
<configuration>
<logger name="com.mycompany.project" level="${log.level}" />
<logger name="httpclient" level="WARN" />
<logger name="org.apache" level="WARN" />
<logger name="org.hibernate" level="WARN" />
<logger name="org.hibernate.cfg.AnnotationBinder" level="WARN" />
<logger name="org.hibernate.cfg.annotations" level="WARN" />
<logger name="org.quartz" level="WARN" />
<logger name="org.springframework" level="WARN" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%-16thread] %-5level %-35.35logger{30} - %msg%n</pattern>
</encoder>
</appender>
<root level="${log.level:-INFO}">
<appender-ref ref="STDOUT" />
</root>
</configuration>

You can set the status listener class via system property:
java -Dlogback.statusListenerClass=ch.qos.logback.core.status.OnConsoleStatusListener ...
See: Logback manual

I could not make it work using the chosen answer. However, the following worked:
java -Dlogback.configurationFile=/path/to/config-debug.xml com.domain.Main
Just add a file (config-debug.xml in this example) somewhere on your server and leave it there when you need to debug. Like the following.
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{dd-MMM-yyyy HH:mm:ss.SSS} %-5level [%thread] %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="STDOUT" />
</root>
</configuration>
Run your application using the afore mentioned -D parameter.
When things are back to normal, remove the -D parameter and restart your application.
Source: Chapter 3: Logback configuration

Well, It's pretty easy. Either you can use
log.level = debug
inside the application.properties of Spring boot.
or you can also set this in the configuration file of logback.xml
<root level="${log.level}">
<appender-ref ref="ANY_APPENDER" />
</root>

In eclipse you can have multiple run configurations. Open your main class. Go to Debug dropdown on eclipse toolbar and select Debug configurations. Click the New launch configuration icon at the top left. Give your launch configuration a better name. Click the Arguments tab under the name and enter -Dlog.level=debug or whatever you want. Click Close or Debug
You can do this again and specify -Dlog.level=warn for example.