Develop Reference

Log4j 2.6.x and log4j 2.15 's xml configuration seems incompatible

To upgrade my log4j2 from 2.6.2 to 2.15.0, I simply uses maven build to change the pom.xml log4j-core and log4j-api 's version. However, once I have deployed the java application to tomcat. Log4j 2.15.0 doesn't log to the specific file but catalina.out only. When I fallback to the previous 2.6.x without changing anything, things worked as expected. The log4j.xml I have is as shown below and The way that I get the logManager in Java is just
private static Logger log = LogManager.getLogger("Logger");
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<RollingFile name="errorLogging" fileName="/opt/app/tomcat8/logs/test.log"
filePattern="/opt/app/tomcat8/logs/test.log.%d{yyyy-MM-dd}">
<PatternLayout>
<Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Root level="info" additivity="false">
</Root>
<Logger name="Logger" level="info">
<AppenderRef ref="errorLogging" level="info"/>
</Logger>
</Loggers>
</Configuration>

Performance of log4j2 compared to log4j1

I'm trying to migrate my app into using log4j2. It is currently using log4j 1.2.16. I also have a performance build for my project, and after upgrading to log4j 2, the performance seemed to have improve a lot.
That is, until I read about bridging. According to the doc, I have to exclude log4j1 JAR from the classpath, and include the bridging JAR - which I assume is named 'org.apache.logging.log4j:log4j-1.2-api'. Once I did this, performance dropped again.
So to summarise:
Performance with log4j2 + bridging jar + log4j-1.2-api + log4j1 : good
Performance with log4j2 + bridging jar + log4j-1.2-api : bad (to the point that it drops back to performance of just log4j1)
I have checked that the log4j-1.2-api is earlier in the classpath. So it should have been loaded first.
Any idea what could cause this problem?
Thank you very much in advance!
Oh my complete classpath for logging are:
org.slf4j:slf4j-api
org.slf4j:log4j-over-slf4j
org.slf4j:jcl-over-slf4j
org.apache.logging.log4j:log4j-slf4j-impl
org.apache.logging.log4j:log4j-core
org.apache.logging.log4j:log4j-api
org.apache.logging.log4j:log4j-1.2-api
log4j:log4j (with & without, as described above)
Versions:
Log4j2 : 2.6.2
slf4j: 1.7.20
log4j1: 1.2.16
My config file looks like:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration
xmlns:xi="http://www.w3.org/2001/XInclude">
<xi:include href="log4j2-xinclude-appenders.xml" />
<Loggers>
<Root level="info">
<AppenderRef ref="rollingFileAppender"/>
<AppenderRef ref="stdOutAppender"/>
</Root>
</Loggers>
</Configuration>
And the log4j2-xinclude-appenders.xml looks like:
<?xml version="1.0" encoding="UTF-8"?>
<appenders>
<RollingRandomAccessFile name="_rollingFileAppender" fileName="./logs/foo-${sys:app.name.suffix}.log"
filePattern="./logs/foo-${sys:foo.app.name.suffix}.log.%i">
<PatternLayout>
<Pattern>%d|%X{active.profiles}| %-5p |%X{fcp.session}|%X{StateMachine.key}|%X{StateMachine.currentState}| %m | %t | %c{1.}%n</Pattern>
</PatternLayout>
<Policies>
<OnStartupTriggeringPolicy minSize="0" />
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
<DefaultRolloverStrategy max="10"/>
</RollingRandomAccessFile>
<Async name="rollingFileAppender" blocking="false" bufferSize="10000">
<AppenderRef ref="_rollingFileAppender"/>
</Async>
<Console name="_stdOutAppender" target="SYSTEM_OUT">
<PatternLayout pattern="%d|%X{active.profiles}| %-5p |%X{fcp.session}|%X{StateMachine.key}|%X{StateMachine.currentState}| %m | %t | %c{1.}%n"/>
</Console>
<Async name="stdOutAppender" blocking="false" bufferSize="10000">
<AppenderRef ref="_stdOutAppender"/>
</Async>
</appenders>
EDIT: This is the log4j 1 xml file that gets included in the classpath
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration debug="false" xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="A1" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d [%t] %-5p %c - %m%n"/>
</layout>
</appender>
<appender name="R" class="com.bar.common.util.RollingFileAppender">
<param name="File" value="./logs/bar.log"/>
<param name="MaxFileSize" value="100MB"/>
<param name="MaxBackupIndex" value="10"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d [%t] %-5p %c - %m%n"/>
</layout>
</appender>
<!-- Performance Appender -->
<appender name="OneSecondStatsAppender"
class="com.foo.perf.AggregatedStatisticsAppender">
<param name="TimeSlice" value="1000"/>
<appender-ref ref="OneSecondStatsLogger"/>
</appender>
<appender name="FiveMinuteStatsAppender"
class="com.bar.perf.DatafabricAggregatedStatisticsAppender">
<param name="TimeSlice" value="300000"/>
<appender-ref ref="FiveMinuteStatsLogger"/>
</appender>
<!-- Aggregated Performance Statistics Appender -->
<appender name="OneSecondStatsLogger" class="org.apache.log4j.FileAppender">
<param name="File" value="./logs/bar-performance.log"/>
<layout class="com.bar.perf.AggregatedStatisticsCsvLayout"/>
<filter class="com.bar.perf.CategorisedStatisticExclusionFilter"/>
</appender>
<appender name="FiveMinuteStatsLogger" class="org.apache.log4j.FileAppender">
<param name="File" value="./logs/bar-minutes.log"/>
<layout class="com.bar.perf.AggregatedStatisticsCsvLayout">
<param name="ShowEmptyStatistics" value="true"/>
</layout>
</appender>
<!-- Loggers -->
<logger name="org.perf4j.TimingLogger" additivity="false">
<level value="INFO"/>
<appender-ref ref="OneSecondStatsAppender"/>
<appender-ref ref="FiveMinuteStatsAppender"/>
</logger>
<logger name="com.bar">
<level value="INFO"/>
</logger>
<logger name="com.gemstone.gemfire">
<level value="INFO"/>
</logger>
<logger name="org.springframework.data">
<level value="INFO"/>
</logger>
<!-- Root logger configuration -->
<root>
<priority value="INFO"/>
<appender-ref ref="R"/>
</root>
</log4j:configuration>
EDIT 2:
Classpath order for poor performance:
log4j-1.2-api-2.6.2.jar
jcl-over-slf4j-1.7.20.jar
slf4j-api-1.7.20.jar
log4j-slf4j-impl-2.6.2.jar
log4j-core-2.6.2.jar
log4j-api-2.6.2.jar
log4j-1.2.16.jar
Classpath order for good performance
log4j-1.2-api-2.6.2.jar
jcl-over-slf4j-1.7.20.jar
slf4j-api-1.7.20.jar
log4j-over-slf4j-1.7.20.jar
log4j-slf4j-impl-2.6.2.jar
log4j-core-2.6.2.jar
log4j-api-2.6.2.jar
log4j-1.2.16.jar

I ran into more strange findings. I enabled '-verbose:class' JVM options to see which classes were loaded, and I can confirm that only classes from the following JARs were loaded (in the order):
slf4j-api-1.7.20.jar
log4j-slf4j-impl-2.6.2.jar
log4j-api-2.6.2.jar
log4j-core-2.6.2.jar
log4j-1.2-api-2.6.2.jar
jcl-over-slf4j-1.7.20.jar
Yet, the following two tests yield different result:
Performance test with including log4j-over-slf4j-1.7.20 & log4j-1.2.16 :
GOOD
Performance test with including log4j-over-slf4j-1.7.20 & but excluding log4j-1.2.16 : BAD
Performance test with excluding log4j-over-slf4j-1.7.20 & including log4j-1.2.16 : BAD
Note that these two JARs were not loaded at all.

I there there are two questions here:
Why does log4j 1 get used in certain classpath configurations
Why is log4j 2 not faster than log4j 1
1. Why is log4j 1 used
I suspect that the following slf4j dependencies caused the old log4j 1.2 to be used:
org.slf4j:log4j-over-slf4j
org.slf4j:jcl-over-slf4j
If you use maven these could bring in the old Log4j 1 as a transitive dependency even if you don't explicitly declare it in your POM.
Please remove these. Log4j 2 has log4j-slf4j-impl and log4j-jcl modules that will accomplish the same but use Log4j 2 instead.
You should not have Log4j 1 in the classpath. If your application (or any of the libraries you use) depend on the Log4j 1 API, then add the log4j-1.2-api module.
2. Why is log4j 2 not faster than log4j 1
The configuration you describe does not take advantage of log4j 2 features. It uses AsyncAppender (which is roughly equivalent in log4j 1 and 2) and ConsoleAppender (which is slightly worse in log4j 2). ConsoleAppender is about 60 times slower than file appender. Be extremely careful when logging to the console in production systems.
Here is what I suggest: remove the following (which now seems to give better performance, but bear with me)
log4j-over-slf4j-1.7.20
log4j-1.2.16
the old lo4j.xml configuration
Add the LMAX Disruptor dependency:
<!-- https://mvnrepository.com/artifact/com.lmax/disruptor -->
<dependency>
<groupId>com.lmax</groupId>
<artifactId>disruptor</artifactId>
<version>3.2.0</version>
</dependency>
Use the following log4j2.xml configuration. Temporarily just make it a simple file without includes, you can put that back later. (Notice I added <Configuration status="trace" to the beginning of the file: that will output internal log4j2 debugging statements so you can confirm that configuration completed without problems.)
Note that I made Console logging WARN level as I suspect it is impacting performance.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="TRACE" xmlns:xi="http://www.w3.org/2001/XInclude">
<appenders>
<RollingRandomAccessFile name="_rollingFileAppender"
fileName="./logs/foo-${sys:app.name.suffix}.log"
filePattern="./logs/foo-${sys:foo.app.name.suffix}.log.%i">
<PatternLayout>
<Pattern>%d|%X{active.profiles}| %-5p |%X{fcp.session}|%X{StateMachine.key}|%X{StateMachine.currentState}| %m | %t | %c{1.}%n</Pattern>
</PatternLayout>
<Policies>
<OnStartupTriggeringPolicy minSize="0" />
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
<DefaultRolloverStrategy max="10"/>
</RollingRandomAccessFile>
<Console name="_stdOutAppender" target="SYSTEM_OUT">
<PatternLayout pattern="%d|%X{active.profiles}| %-5p |%X{fcp.session}|%X{StateMachine.key}|%X{StateMachine.currentState}| %m | %t | %c{1.}%n"/>
</Console>
</appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="_rollingFileAppender"/>
<AppenderRef ref="_stdOutAppender" level="WARN" />
</Root>
</Loggers>
</Configuration>
Now, the final (key) point: enable log4j 2 async loggers by setting system property Log4jContextSelector to org.apache.logging.log4j.core.async.AsyncLoggerContextSelector.
This last bit should make a large performance difference. (Together with disabling Console logging.)

How to configure java application having 2 log4j2.xml files so that both configurations work fine

We are running a Java file named Calc.java and for this application, we are using log4j2.xml file for logging to a file named LogCalc.txt. Our Calc.java is using a .jar file named Addition.Jar, which is composed of Add.java, which also uses log4j2.xml file for its own logging to a file named LogAdd.txt.
The question here is, when we run Calc.java and access a method from Addition.Jar, logging is only happening in LogCalc.txt, the configurations in log4j2.xml of Calc.java is taken into account and log4j2.xml of LogAdd.txt is not taken into account. Due to this I am not able to get logs from Addition.jar, LogAdd.txt is empty.
How can we change our configurations such that we can see both logs from Calc.java in LogCalc.txt as well as from Add.java in LogAdd.txt, i.e., both XML configurations must work fine in our application and both of them logging to different files with their own configurations taken from respective XML files).
Our aim here is to make a project where main app uses log4j2.xml and also the included .jar in this project is using log4j2.xml, but we should get logs from both of them to their respective appenders separately without any problem.
Our Xml files look like this. log4j2.xml used in Addition.jar is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<File name="A1" fileName="A1.log" append="false">
<PatternLayout pattern="%t %-5p %c{2} - %m%n"/>
</File>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%d %-5p [%t] %C{2} (%F:%L) - %m%n"/>
</Console>
</Appenders>
<Loggers>
<Logger name="org.apache.log4j.xml" level="debug">
<AppenderRef ref="A1"/>
</Logger>
<Root level="warn">
<AppenderRef ref="STDOUT"/>
</Root>
</Loggers>
</Configuration>
The log4j2.xml file used with Calc.java is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<File name="A1" fileName="A1.log" append="false">
<PatternLayout pattern="%t %-5p %c{2} - %m%n"/>
</File>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%d %-5p [%t] %C{2} (%F:%L) - %m%n"/>
</Console>
</Appenders>
<Loggers>
<Logger name="org.apache.log4j.xml" level="debug">
<AppenderRef ref="A1"/>
</Logger>
<Root level="info">
<AppenderRef ref="A1"/>
</Root>
</Loggers>
</Configuration>

Log4j will locate the first log4j2.xml that it finds on the classpath and use that for configuration. It does not currently support multiple configurations, although work is in progress to allow it to. Even if it did it would not work the way your configuration won't work because the loggers are the same.
I suggest you review the log4j guide online. Log4j does not log on the basis of jar files but by comparing the names of the loggers you are using against what is configured. If you want everything in one jar to go to one file then use a logger name such as Calc for the first jar and a logger name of LogCalc for the second jar. Then have your configuration route the events for each of the loggers to the appropriate files.

How to write logs to two different loggers?

Im able to create two logs using log4j2 and im able to write the log to a single log file. how to write logs to a two different loggers?
i modified the log4j2.xml file to have two loggers.
Any sample example?

Instead of configuring multiple loggers, you may want to configure multiple appenders. Example:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
<Appenders>
<File name="MyFile" fileName="logs/app.log">
<PatternLayout>
<Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
</PatternLayout>
</File>
<File name="Other" fileName="logs/other.log">
<PatternLayout>
<Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
</PatternLayout>
</File>
</Appenders>
<Loggers>
<Root level="trace">
<AppenderRef ref="MyFile" level="trace"/>
<AppenderRef ref="Other" level="debug"/>
</Root>
</Loggers>
</Configuration>

Point 1
I guess you're asking to write one log into different destinations, ex. different files. As Remko has mentioned, you could configure different appenders to handle each of the log file.
As I don't have enough reputations for commenting Remko's answer, here're some of my additional examples:
If you use the level of DEBUG for your logs, calling logger.debug("your message") and configure your log4j2.xml like the below, you'll see "your message" logged in both file1.log and file2.log.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<File name="File1" fileName="logs/file1.log">
<PatternLayout>
<Pattern>"Add your pattern here"</Pattern>
</PatternLayout>
</File>
<File name="File2" fileName="logs/file2.log">
<PatternLayout>
<Pattern>"Add your pattern here"</Pattern>
</PatternLayout>
</File>
</Appenders>
<Loggers>
<Root level="debug">
<!--This will write the same log into both of the files -->
<AppenderRef ref="File1" level="debug"/>
<AppenderRef ref="File2" level="debug"/>
</Root>
</Loggers>
</Configuration>
Point 2
we should notice that, in Remko's example, all logs go into Other will also be written into MyFile. See more in Log4j2 Documentation, example 6 for explaination. In short, as TRACE is a lower lever than DEBUG, all logs with DEBUG levels could be passed in TRACE and logged.
Point 3
Another possibility of writing into different destinations is through Appender Additivity.
Each enabled logging request for a given logger will be forwarded to
all the apprenders in that Logger's LoggerConfig as well as the
Apprenders of the LoggerConfig's parents.

Log4j2 Silence specific package

I've just updated to Log4. Now i have some small issues configurating my new setup.
The log4j2.xml is quite small and simple, but i'm not sure how i can disable logs (or at least set them to ERROR) for all jersey packages.
Here's my log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="CONSOLE" target="SYSTEM_OUT">
<PatternLayout pattern="%d %p %c{1.} [%t] %m%n" />
</Console>
<File name="FILE" fileName="/logs/m2m/error.log">
<PatternLayout pattern="%d %p %c{1.} [%t] %m%n" />
</File>
<Async name="ASYNC">
<AppenderRef ref="FILE" />
<AppenderRef ref="CONSOLE" />
</Async>
</Appenders>
<Loggers>
<Logger name="org.glassfish.jersey.servlet" level="ERROR" additivity="false">
<AppenderRef ref="ASYNC" />
</Logger>
<Root level="DEBUG">
<AppenderRef ref="ASYNC" />
</Root>
</Loggers>
</Configuration>
And here's the log-output i try to silence:
Jan 24, 2014 8:21:12 AM org.glassfish.jersey.servlet.WebComponent filterFormParameters
WARNING: A servlet request to the URI #### contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using #FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
Thanks

Based on the format of the output I suspect that the log output shown above is actually produced by JUL (java.util.logging) and not by log4j.
It is possible to route calls made to the java.util.logging API to the log4j2 implementation. Your log4j2 configuration (which looks correct) should then filter out WARN-level log events emitted by any class in the org.glassfish.jersey.servlet package.
This involves adding a few more jars to the classpath: see the FAQ.

If you want to do with Logback, here I explain how to do it:
Excessive warning messages from Jersey 2.x