Develop Reference

Java logging configuration for dependencies and transitive dependencies

Scenario I have:
project --> libA --> libB
project uses libA (imported in pom), libA uses libB.
Due to other constrains, in project pom I exclude everything from libA and then I import manually libB.
Both libA and libB are maintained by me but are extracted as librarie for reusability. Project and libB have as parent the sprig-boot-parent. libA is a maven project (no parent) that contains spring libraries.
If I put logback.xml file in all of them (project, libA, libB), then I receive an warning:
13:20:26,442 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
Additional to above log, there are other lines of log that I don't desire.
If I put logback.xml in project and in libB, then I don't receive the above warning. The logs in the project and libB is formated correctly but not the one in libA
In the end I want to have the same log format for all of them without other unwanted logs.
How can I cofigure logback so that it will apply also to the library and transitive libraries ?
What are the recommendations in these types of scenarios?
Later edit:
Adding logback config that is applied in the project:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<Target>System.out</Target>
<encoder>
<pattern>D:%d{yyyy-MM-dd HH:mm:ss.SSS} L:%p C:%c{1} F:%F\(%L\) Fn:%M T:%thread R:%X{R} - %m%n</pattern>
</encoder>
</appender>
<appender name="pandaAppender" class="ch.qos.logback.core.ConsoleAppender">
<Target>System.out</Target>
<encoder>
<pattern>L:%p %m%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="stdout"/>
</root>
<logger name="com.happypanda" level="INFO" additivity="false">
<appender-ref ref="stdout"/>
</logger>
<logger name="com.happypanda.myproject" level="INFO" additivity="false">
<appender-ref ref="pandaAppender"/>
</logger>
<logger name="com.happypanda.libB" level="INFO" additivity="false">
<appender-ref ref="pandaAppender"/>
</logger>
<logger name="org.apache.http" level="ERROR">
<appender-ref ref="stdout"/>
</logger>
<logger name="org.apache.http.wire" level="ERROR">
<appender-ref ref="stdout"/>
</logger>
</configuration>
Later later edit:
I've tried to simplify the logic: to remove the libA and use directly libB in project.
Now the log level in libB is debug + there are additional logs from httpclient (see this as reference Disable HttpClient logging)
Later later edit:
In my application I'm taking the logback.xml from external source (is not in resources). In a static block I have the following config:
System.setProperty("logging.config", "file:/services/config/logback.xml");
For testing purposes I've moved the config log inside the resources file of the project. Now all the logs in both project and libB are ok.
I've tried to set them as command line properties but doing so I see debug logs in libB.
CMD java -XX:MinRAMPercentage=40.0 -XX:MaxRAMPercentage=80.0 -XX:+HeapDumpOnOutOfMemoryError -Dlogging.config=/services/config/logback.xml -jar

You should create the logback file for the Project, which is using library A and library B.
In this logback configuration files, you need to handle the log which the project requires and you need: project classes and library classes.
Since you are using logback in this project, I suggest you to specify the dependency and its version in its pom.xml, so Maven is going to use the version from the POM of Project, and not "guessing libA or libB".

Log SQL to file with logback & hibernate

I am trying to get the SQL logged directly to a file when running the dev profile.
This is my logback.xml
<configuration>
<property name="SQL_LOG_FILE" value="${LOG_PATH:-${LOG_TEMP:-${TMPDIR:-/tmp}}}/${HIBERNATE_LOG_FILE:-hibernate.log}"/>
<springProfile name="dev">
<appender name="SQLDEBUG" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${SQL_LOG_FILE}</file>
<encoder>
<charset>utf-8</charset>
<Pattern>%-5level %logger{0} - %msg%n</Pattern>
</encoder>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<charset>utf-8</charset>
<Pattern>[%highlight(%p)] [%t] %c - %m%n</Pattern>
</encoder>
</appender>
<logger name="org.hibernate.SQL" additivity="false" level="DEBUG">
<appender-ref ref="SQLDEBUG"/>
</logger>
<logger name="org.hibernate.type.descriptor.sql" additivity="false" level="TRACE">
<appender-ref ref="SQLDEBUG"/>
</logger>
</springProfile>
<root level="${logback.loglevel}">
<springProfile name="dev">
<appender-ref ref="CONSOLE"/>
</springProfile>
</root>
I have removed the prod profile settings for simplicity.
The logger for hibernate is inside the dev profile because I don't want it enabled in prod.
I have tried many combinations of these org.hibernate settings. This version generates SQL logs but only dumps them to console, not the log file. Some general startup information is added to the log file but no SQL.
If I change org.hibernate.type.descriptor.sql to org.hibernate.type there is a lot of stack trace logs that are added directly to the file, but no SQL.
Some posts recommend using org.hibernate.SQL level=TRACE but that did not seem to change anything.
I also tried putting the logger outside of the dev profile but that also did not change the results.
There is a lot of information for enabling logback & hibernate for simple console output but not for sending the SQL to its own log file.
I also tried enabling hibernate.SQL=DEBUG in IntelliJ but that makes a lot of SQL on the console, I need to not do that.
I have been try

I doubt you spring profile is being used. To get this to work rename the logback.xml file to logback-spring.xml, allowing the springProfile tag to be used. In this tag, a name can be provided that can be set via properties, environment variables, or VM options. Below is how you can set the springProfile name to dev , which has been used to represent a development environment.
To set in application.properties or as an environment variable:
spring.profiles.active=dev
Or as a VM option:
-Dspring.profiles.active=dev
Also, modify your root-level tag to be inside the spring profile tag:
<springProfile name="dev">
<root level="${logback.loglevel}">
<appender-ref ref="CONSOLE"/>
</root>
</springProfile>

Why is Spring not talking to me? (Or: How do I correctly configure Spring logging?)

My actual problem is with Spring Security and configuring OAuth2 for an authentication server, but to trace my problem there I would like Spring to tell me what actually happens, when a request reaches an endpoint. I am using Spring Boot 2.0.0-SNAPSHOT and the logging starter spring-boot-starter-log4j2and actually it seems to work quite well. My log4j2-configuration is:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configuration>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%-5level%d{MMM dd, yyyy HH:mm:ss.SSS} [%t] %X %logger{36}%n %msg%n"/>
</Console>
<File name="File" fileName="logs/application.log">
<PatternLayout pattern="%-5level%d{MMM dd, yyyy HH:mm:ss.SSS} [%t] %X %logger{36} %msg%n"/>
</File>
</Appenders>
<Loggers>
<Root level="debug">
<AppenderRef ref="Console"/>
<AppenderRef ref="File"/>
</Root>
<logger name="org.springframework" level="debug" additivity="false">
<AppenderRef ref="Console"/>
<AppenderRef ref="File"/>
</logger>
<logger name="org.springframework.security" level="debug" additivity="false">
<AppenderRef ref="Console"/>
<AppenderRef ref="File"/>
</logger>
<logger name="org.hibernate" level="info" additivity="false">
<AppenderRef ref="Console"/>
<AppenderRef ref="File"/>
</logger>
</Loggers>
</Configuration>
When booting up the application (embedded Tomcat) I can see that the configuraiton actually is used and Spring logs a lot of stuff during the process:
...
INFO Feb 02, 2018 11:31:00.295 [restartedMain] {} org.hibernate.type.BasicTypeRegistry
HHH000270: Type registration [java.util.UUID] overrides previous : org.hibernate.type.UUIDBinaryType#41229bd4
INFO Feb 02, 2018 11:31:00.345 [restartedMain] {} org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean
Initialized JPA EntityManagerFactory for persistence unit 'default'
DEBUG Feb 02, 2018 11:31:00.440 [restartedMain] {} org.springframework.data.jpa.repository.query.JpaQueryFactory
Looking up query for method findByEmail
...
But when I send a request to the application to obtain an Oauth2 access token receiving an 4xx-error I would like Spring to tell me what it does so that I can find my configuration error. But it does not log anything. I see Hiberante stuff and my own logging but Spring remains silent. I would expect a lot of messages due to the TRACE-configuration.
Can anybody imagine, what I'm doing wrong? Or is Spring simply not talkative?
UPDATE 02/08/2018:
It seems that my problem is bound to Spring Security. I did some debugging and found different loggers for Spring Security and other parts of the framework.
For example in org.springframework.data.jpa.repository.query.JpaQueryFactorySpring is using the logger org.apache.logging.slf4j.Log4jLogger (which is set to DEBUG in my configuration), but Spring security uses org.apache.commons.logging.impl.Jdk14Logger for example in org.springframework.security.web.FilterChainProxy (which is set to INFO despite my log4j2 configuration).
So I tried to debug into the code where the logger of FilterChainProxy is assigned. But unfortunately the code is either never reached or my breakpoints do not work. I guess it is the second one.

I found the cause of the problem myself:
Spring widely uses slf4J for logging. For that I included a bridge in my project, which works perfectly well and produces the log output in my question.
But Spring Security (and maybe other parts of Spring too) uses Apache Commons Logging which is no problem by itself since Spring 5 comes with spring.jcl which is among other things a bridge from Apache Commons Logging to log4J2.
But here is the devilish thing: I inluded another library that has a transitive dependency to common-logging and so inlucdes the direct implementation of "Apache Commons Logging" into my project. So I had two different org.apache.commons.logging.LogFactory classes in my classpath. Spring security decided to use the implementation in commons-logging instead of the one in spring-jcl and of course therefore my log4j-configuration did not influence the loggers of Spring security.
Actually all I had to do is exclude commons-logging in my build-file and let spring-jcl do its work again.

One option is to extend DefaultAuthenticationEventPublisher which will let you hook into auth success/failure. Of course you would replace the print statements with your own logging.
#Component
public class CustomAuthenticationEventPublisher extends DefaultAuthenticationEventPublisher {
#Override
public void publishAuthenticationSuccess(Authentication authentication) {
super.publishAuthenticationSuccess(authentication);
System.out.println("AUTH SUCCESS " + authentication);
}
#Override
public void publishAuthenticationFailure(AuthenticationException exception, Authentication authentication) {
super.publishAuthenticationFailure(exception, authentication);
System.out.println("AUTH ERROR " + exception.getLocalizedMessage());
}
}

Logback: separate logging WARN / TRACE

I'm trying to configure logback to print:
- everything (level trace or debug) to the screen
- everything (level trace or debug) to the debugfile
- warnings and above to an error file
My logback.xml config is like this:
...
<logger name="be" level="TRACE">
<appender-ref ref="FILE-AUDIT" />
<appender-ref ref="STDOUT" />
</logger>
<root level="WARN">
<appender-ref ref="FILE-ERROR" />
</root>
However, the error and debug file contain exactly the same, being ALL logging (debug and error). I've already tried to play with the additivity option, but that's apparently not what I need.
The second question is that I use name "be" to have all classes under be.* but actually I want to capture everything there (com.* as well).

Found a solution by adding a filter to the appender:
<appender name="FILE-ERROR"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- deny all events with a level below WARNING -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>WARN</level>
</filter>

Run Logback in Debug

I've recently switched from log4j to logback and am wondering if there is an easy way to run logback in debug mode, similar to log4j's log4j.debug property. I need to see where it is picking up my logback.xml from.
The docs mention using a StatusPrinter to print out logback's internal status, but that would require code changes.

[EDIT]
This has been fixed in Logback 1.0.4. You can now use -Dlogback.debug=true to enable debugging of the logback setup.
-- Old Answer --
Unfortunately, there is no way to enable debugging via a System property. You have to use <configuration debug="true"> in the logback.xml. Please submit a feature request.

This is how I do it. I set a system property called 'log.level', then I reference it in logback.xml.
Edit: The downside is that you MUST have 'log.level' always set. The way I deal with this is to check in my main method and set it to INFO if not already set, be sure to do this before you first logging calls. Then I can override on the command line, and have a sensible default.
Here is how it looks in my logback.xml:
<configuration>
<logger name="com.mycompany.project" level="${log.level}" />
<logger name="httpclient" level="WARN" />
<logger name="org.apache" level="WARN" />
<logger name="org.hibernate" level="WARN" />
<logger name="org.hibernate.cfg.AnnotationBinder" level="WARN" />
<logger name="org.hibernate.cfg.annotations" level="WARN" />
<logger name="org.quartz" level="WARN" />
<logger name="org.springframework" level="WARN" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%-16thread] %-5level %-35.35logger{30} - %msg%n</pattern>
</encoder>
</appender>
<root level="${log.level:-INFO}">
<appender-ref ref="STDOUT" />
</root>
</configuration>

You can set the status listener class via system property:
java -Dlogback.statusListenerClass=ch.qos.logback.core.status.OnConsoleStatusListener ...
See: Logback manual

I could not make it work using the chosen answer. However, the following worked:
java -Dlogback.configurationFile=/path/to/config-debug.xml com.domain.Main
Just add a file (config-debug.xml in this example) somewhere on your server and leave it there when you need to debug. Like the following.
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{dd-MMM-yyyy HH:mm:ss.SSS} %-5level [%thread] %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="STDOUT" />
</root>
</configuration>
Run your application using the afore mentioned -D parameter.
When things are back to normal, remove the -D parameter and restart your application.
Source: Chapter 3: Logback configuration

Well, It's pretty easy. Either you can use
log.level = debug
inside the application.properties of Spring boot.
or you can also set this in the configuration file of logback.xml
<root level="${log.level}">
<appender-ref ref="ANY_APPENDER" />
</root>

In eclipse you can have multiple run configurations. Open your main class. Go to Debug dropdown on eclipse toolbar and select Debug configurations. Click the New launch configuration icon at the top left. Give your launch configuration a better name. Click the Arguments tab under the name and enter -Dlog.level=debug or whatever you want. Click Close or Debug
You can do this again and specify -Dlog.level=warn for example.