Develop Reference

Why is Spring not talking to me? (Or: How do I correctly configure Spring logging?)

My actual problem is with Spring Security and configuring OAuth2 for an authentication server, but to trace my problem there I would like Spring to tell me what actually happens, when a request reaches an endpoint. I am using Spring Boot 2.0.0-SNAPSHOT and the logging starter spring-boot-starter-log4j2and actually it seems to work quite well. My log4j2-configuration is:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configuration>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%-5level%d{MMM dd, yyyy HH:mm:ss.SSS} [%t] %X %logger{36}%n %msg%n"/>
</Console>
<File name="File" fileName="logs/application.log">
<PatternLayout pattern="%-5level%d{MMM dd, yyyy HH:mm:ss.SSS} [%t] %X %logger{36} %msg%n"/>
</File>
</Appenders>
<Loggers>
<Root level="debug">
<AppenderRef ref="Console"/>
<AppenderRef ref="File"/>
</Root>
<logger name="org.springframework" level="debug" additivity="false">
<AppenderRef ref="Console"/>
<AppenderRef ref="File"/>
</logger>
<logger name="org.springframework.security" level="debug" additivity="false">
<AppenderRef ref="Console"/>
<AppenderRef ref="File"/>
</logger>
<logger name="org.hibernate" level="info" additivity="false">
<AppenderRef ref="Console"/>
<AppenderRef ref="File"/>
</logger>
</Loggers>
</Configuration>
When booting up the application (embedded Tomcat) I can see that the configuraiton actually is used and Spring logs a lot of stuff during the process:
...
INFO Feb 02, 2018 11:31:00.295 [restartedMain] {} org.hibernate.type.BasicTypeRegistry
HHH000270: Type registration [java.util.UUID] overrides previous : org.hibernate.type.UUIDBinaryType#41229bd4
INFO Feb 02, 2018 11:31:00.345 [restartedMain] {} org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean
Initialized JPA EntityManagerFactory for persistence unit 'default'
DEBUG Feb 02, 2018 11:31:00.440 [restartedMain] {} org.springframework.data.jpa.repository.query.JpaQueryFactory
Looking up query for method findByEmail
...
But when I send a request to the application to obtain an Oauth2 access token receiving an 4xx-error I would like Spring to tell me what it does so that I can find my configuration error. But it does not log anything. I see Hiberante stuff and my own logging but Spring remains silent. I would expect a lot of messages due to the TRACE-configuration.
Can anybody imagine, what I'm doing wrong? Or is Spring simply not talkative?
UPDATE 02/08/2018:
It seems that my problem is bound to Spring Security. I did some debugging and found different loggers for Spring Security and other parts of the framework.
For example in org.springframework.data.jpa.repository.query.JpaQueryFactorySpring is using the logger org.apache.logging.slf4j.Log4jLogger (which is set to DEBUG in my configuration), but Spring security uses org.apache.commons.logging.impl.Jdk14Logger for example in org.springframework.security.web.FilterChainProxy (which is set to INFO despite my log4j2 configuration).
So I tried to debug into the code where the logger of FilterChainProxy is assigned. But unfortunately the code is either never reached or my breakpoints do not work. I guess it is the second one.

I found the cause of the problem myself:
Spring widely uses slf4J for logging. For that I included a bridge in my project, which works perfectly well and produces the log output in my question.
But Spring Security (and maybe other parts of Spring too) uses Apache Commons Logging which is no problem by itself since Spring 5 comes with spring.jcl which is among other things a bridge from Apache Commons Logging to log4J2.
But here is the devilish thing: I inluded another library that has a transitive dependency to common-logging and so inlucdes the direct implementation of "Apache Commons Logging" into my project. So I had two different org.apache.commons.logging.LogFactory classes in my classpath. Spring security decided to use the implementation in commons-logging instead of the one in spring-jcl and of course therefore my log4j-configuration did not influence the loggers of Spring security.
Actually all I had to do is exclude commons-logging in my build-file and let spring-jcl do its work again.

One option is to extend DefaultAuthenticationEventPublisher which will let you hook into auth success/failure. Of course you would replace the print statements with your own logging.
#Component
public class CustomAuthenticationEventPublisher extends DefaultAuthenticationEventPublisher {
#Override
public void publishAuthenticationSuccess(Authentication authentication) {
super.publishAuthenticationSuccess(authentication);
System.out.println("AUTH SUCCESS " + authentication);
}
#Override
public void publishAuthenticationFailure(AuthenticationException exception, Authentication authentication) {
super.publishAuthenticationFailure(exception, authentication);
System.out.println("AUTH ERROR " + exception.getLocalizedMessage());
}
}

Log4j2 - Why is my "Logger" element hiding content from my "Root" logger?

Before someone closes this question off as a duplicate, just hear me out... I have read through countless blog posts, tutorials, FAQs and SO questions for days now and I'm no closer to understanding why I'm getting this specific behaviour.
Configuration
My log4j2.xml config file contains the following:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="DEBUG">
<Properties>
<Property name="APP_NAME">MyCoolApp</Property>
<Property name="BASE_PACKAGE">my.cool.package</Property>
<Property name="LOG_DIR">${env:LOG_ROOT:-logs}</Property>
<Property name="LOG_PATTERN">%d [%t] %-5level %c{1.}:%L:%M | %m%n</Property>
</Properties>
<Appenders>
<RollingFile name="AppLogFile" fileName="${LOG_DIR}/${APP_NAME}.log" filePattern="${LOG_DIR}/archive/${APP_NAME}.%d{yyyy-MM-dd}.log.gz">
<PatternLayout pattern="${LOG_PATTERN}"/>
<TimeBasedTriggeringPolicy/>
</RollingFile>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${LOG_PATTERN}"/>
</Console>
</Appenders>
<Loggers>
<Logger name="${BASE_PACKAGE}" level="INFO">
<AppenderRef ref="AppLogFile"/>
</Logger>
<Root level="TRACE">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
Then each class initialises a logger with private static final Logger LOGGER = LogManager.getLogger();.
Runtime
From my understanding of log levels, loggers and appenders, this should give me the following:
All INFO and higher level logging output to file
All TRACE and higher level logging output to console
Meanwhile, if I run the app when the AppLogFile appender is enabled, I get less output to the console; counting from only after the Log4j initialisation, I get 255 lines compared to 367.
Looking through the console and file output, when the AppLogFile appender is enabled, I don't get any TRACE or DEBUG output, only INFO and higher. When I comment out just that appender (without changing anything else), then I get everything to console, including TRACE and DEBUG.
I've tried playing around with reordering the "console" and "file" related elements, I've tried explicitly enabling and disabling the logger's additivity property, I've tried using filters in the appenders and loggers, and even multiple appender references inside one logger with explicit level properties.
All I want is to get everything to go to the console and everything INFO level and higher to go to a file. What am I missing here..?

The level attribute and the <AppenderRef> element are entirely independent:
Specifying level changes the logging level of the given logger and all sub-loggers.
Specifying <AppenderRef> adds1 another appender to the given logger and all sub-loggers.
The fact that you do both at the same time, doesn't affect those independent effects.
If you want to limit log entries for the Appender, specify the level attribute on the <AppenderRef> instead.
1) If you wanted the Appender to replace, you need to specify additivity="false"
(OP edit) Just for the sake of clarity, between what I learned from this answer and the Log4j2 FAQ; I streamlined things a bit and ended up with the following <Loggers> configuration:
<Loggers>
<Root level="TRACE">
<AppenderRef ref="AppLogFile" level="INFO"/>
<AppenderRef ref="Console"/>
</Root>
</Loggers>

Log4j2 - Config is found, but not working correctly

Log4j is finding my config, because as soon as I delete it I get an error message saying it couldn't find one, however it's properties are not reflected when logging.
log4j2.properties:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="TRACE">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss} [%t] %-5level %logger{36} - %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
Test.java:
public class Test {
private static Logger logger = LogManager.getLogger(Test.class);
public static void main(String[] args) throws Exception {
logger.info("test");
logger.fatal(logger.getLevel());
}
}
Output:
20:19:31.848 [main] FATAL io.rj93.sarcasm.examples.CnnSentenceClassificationExample - ERROR
As you can see, the logger is returning the level to be ERROR when it is set to INFO, and the time format is including the milliseconds even though it has been removed.
The config file is taken from the log4j website, with only minor changes (the two mentioned, and status="TRACE")
I am using version 2.8.1.

You use a log4j2.properties file with a XML configuration inside it.
It is not consistent.
The log4J initialization doesn't recognize the format used as a properties format. So it uses the default log4J configuration that specifies ERROR level for the root logger.
Simply rename log4j2.properties to log4j2.xml and it should be fine.

Log4j 2 - Deactivate drools logger

I am using Log4j 2 and I am unsuccessfully trying to change the logging level of jBPM/Drools, having it as a reference. The drools class that keeps logging is ExtensibleXmlParser.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
</Console>
</Appenders>
<Loggers>
<Root level="debug">
<AppenderRef ref="Console" />
</Root>
<Logger name="org.drools.core.xml.ExtensibleXmlParser" level="off">
<AppenderRef ref="Console" />
</Logger>
</Loggers>
</Configuration>
If I get it right, ExtensibleXmlParser uses slf4j and Log4j is compatible with sl4j.
Why doesn't this Log4j configuration work? Could someone provide me a working configuration? Should I configure jBPM/Drools logger indendently?

When log4j seems to ignore your tweaks to the log4j.properties/xml file, it most often means you're not fiddling with the right file. It is likely that there is another configuration file in your classpath that gets found earlier and is taken into account.
Search your entire classpath for log4j* pattern and see what comes out (include the contents of .jar files and application server /lib and /ext folders, if any).
If I get it right, ExtensibleXmlParser uses slf4j and Log4j is compatible with sl4j.
"Is compatible" is not equivalent with "is configured to use". SLF4J plugs into appropriate logging framework through the org.slf4j.impl.StaticLoggerBinder class. Search the classpath (best in your IDE) to see what library this class came with. If there is no such class, then SLF4J logs nothing.

Log4J2 and Tomee Plus 7.0.47 not working

According to here, Log4j2 should work with Tomcat7.0.47. I'm using TomEE Plus 7.0.47.
I have a webapplication deployed with a log4j2.xml in my web-inf/classes folder. This is the config:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</Console>
<File name="File" fileName="${sys:catalina.home}/logs/testapp.log">
<PatternLayout>
<pattern>%d %p %C{1.} [%t] %m%n</pattern>
</PatternLayout>
</File>
</Appenders>
<Loggers>
<Logger name="org.alex" level="TRACE" additivity="false">
<AppenderRef ref="File"/>
</Logger>
<Root level="INFO">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
I have a logger declared in a class with name org.alex.util.JSON:
private static final Logger LOG = LoggerFactory.getLogger(JSON.class);
I'm using slf4j-api 1.7.5, and have the following libs added to the tomcat lib:
slf4j-api-1.7.5.jar
log4j-api-2.0-rc1.jar
log4j-core-2.0-rc1.jar
log4j-slf4j-impl-2.0-rc1.jar
If I change the Configuration status to TRACE, I can see my configuration file being picked up and configuration happens as expected. Also I can see the MBeans being added.
However, there's not one logging statement ending up in my logfile. I debugged into the log4j2 Logger, and see that the isEnabled(...) method returns false because the logger (com.alex.util.JSON) has the level "ERROR" set, while the configuration set the package org.alex to TRACE.
Further investigation shows it uses a DefaultConfiguration configured for level=ERROR, and only root is configured. I'm thinking of a classloader issue, but I can't seem to figure out what the cause is and how to solve it.
Does anyone know what I'm doing wrong?

This should work on trunk
Btw saw log4j2 has hacks for tomcat and since tomee wraps classloaders not sure they work as expected...

This is very strange. Please raise a ticket for this in the Log4j2 issue tracker so the Log4j team can take a look.
The problem may go away if you put the jar files you mentioned inside WEB-INF/lib instead of in Tomcat's lib folder.

To be comple log4j2 relies on servletcontainerinitializer which are called after ejb and app scanning so ejbs can be loaded too early. Doing the same with a tomcat context listener would make it working better