I have a spring application that has configured log4j (via xml) and that runs on Tomcat6 that was working fine until we add a bunch of dependencies via Maven. At some point the whole application just started logging part of what it was supposed to be declared into the log4.xml
"a small rant here" Why logging has to be that hard in java world? why suddenly an application that was just fine start behaving so weird and why it's so freaking hard to debug?
I've been reading and trying to solve this issue for days but so far no luck, hopefully some expert here can give me some insights on this
I've added log4j debug option to check whether log4j is taking reading the config file and its values and this is what part of it shows
log4j: Level value for org.springframework.web is [debug].
log4j: org.springframework.web level set to DEBUG
log4j: Retreiving an instance of org.apache.log4j.Logger.
log4j: Setting [org.compass] additivity to [true].
log4j: Level value for org.compass is [debug].
log4j: org.compass level set to DEBUG
As you can see debug is enabled for compass and spring.web but it only shows "INFO" level for both packages. My log4j config file has nothing out of extraordinary just a plain ConsoleAppender
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<!-- Appenders -->
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<param name="Target" value="System.out" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p: %c - %m%n" />
</layout>
</appender>
What's the trick to make this work? What it's my misunderstanding here?
Can someone point me in the right direction and explain how can I make this logging mess more bullet proof?
It might not be log4j that is doing the logging, and hence your log4j config would be ignored. Spring logs using Commons Logging, which is an api that can delegate to various logging frameworks, including log4j. To decide which implementation to use, commons logging looks into the classpath.
If you have added a dependency that dragged its own logging implementation into the classpath, commons logging might now use the other implementation.
I recommend to set a breakpoint at a call to the logging facility, and trace the execution to see which logging implementation is used.
As it was working until you loaded a number of dependencies via Maven, perhaps there's a Log4j configuration loaded inadvertently via these dependencies ?
Run mvn dependency:tree to see what's being loaded, and then see if any of these dependencies has a Log4j configuration.
I think your problem is that you're not setting the threshold param on your appenders, and (maybe) because you're not assigning those appenders to your loggers.
Try adding param name="threshold" value="debug" to your appenders and then explicitly adding them to the specific (or root) loggers like so:
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<param name="threshold" value="debug" />
<param name="Target" value="System.out" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p: %c - %m%n" />
</layout>
</appender>
<logger name="org.springframework.web">
<level value="debug" />
<appender-ref ref="console" />
</logger>
Also, this page says "This appender will not log any messages with priority lower than the one specified here even if the category's priority is set lower" which is probably the source of your problem.
This seems like a good thread for you to read:
http://forum.springsource.org/showthread.php?t=88250
Cutting to the chase, it seem that the poster had a problem with a Tomcat security setting. An updated Tomcat policy file fixed the issue.
Probably has something do with reading outside of the webapp for your log4j.xml file.
Related
I am currently working on performance of "log4j 1.x" vs "logback" i.e. (slf4j).
I can append async to my logback, but i am not able to find any link which can async my log4j.
Async is only introduced in log4j 2.x? or is there any way to make my log4j 1.x work async.
Please assist me.
Thank you.
Asynchronous logging is one of Log4j 2's strong points. Instead of the AsyncAppender, use Async Loggers: the performance is much better.
The Log4j 2 performance page compares Log4j-1.x, Logback and Log4j 2. Take a look and then decide what logging framework to use.
By the way, Log4j 2 has an adapter called log4j-1.2-api-2.6.jar which lets your application use the old log4j-1.2 API, but uses the new log4j 2 implementation.
(Log4j-1.x also has an AsyncAppender. Just like the Log4j 2 AsyncAppender and the Logback AsyncAppender, it uses a BlockingQueue. Log4j 2's Async Loggers use a non-blocking data structure which is much faster.)
log4j.properties does not handle the advanced configuration features
supported by the DOMConfigurator such as support custom ErrorHandlers,
nested appenders such as the AsyncAppender, etc.
Refer log4j API Documentation
There are two ways to configure AsyncAppender in log4j 1.x
Using log4j.xml
Configure AsyncAppender as below
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="console1" class="org.apache.log4j.ConsoleAppender">
<param name="target" value="System.out" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ABSOLUTE} %-5p [%c{1}] %m%n" />
</layout>
</appender>
<!--wrap ASYNC around other appender if you want -->
<appender name="ASYNC" class="org.apache.log4j.AsyncAppender">
<param name="BufferSize" value="500" />
<!-- LocationInfo is Optional, use only if location info is required. (Default value : false)-->
<!-- <param name="LocationInfo" value="true" /> -->
<appender-ref ref="console1" />
<root>
<priority value="debug" />
<appender-ref ref="ASYNC" />
</root>
</log4j:configuration>
Programmatically
You can configure AsyncAppender programmatically as below
static {
enableAsyncAuditLog(Logger.getRootLogger());
}
private static void enableAsyncAuditLog(Logger targetLogger) {
Enumeration<Appender> appenders = targetLogger.getAllAppenders();
AsyncAppender asyncAppender = new AsyncAppender();
asyncAppender.setBufferSize(500);
asyncAppender.setLocationInfo(true); // Otherwise Class and Line info will not be available to logger
while (appenders.hasMoreElements()) {
Appender appender = appenders.nextElement();
if (!(appender instanceof AsyncAppender)) {
targetLogger.removeAppender(appender);
asyncAppender.addAppender(appender);
}
}
appenders = asyncAppender.getAllAppenders();
if (appenders != null && appenders.hasMoreElements()) {
targetLogger.addAppender(asyncAppender);
}
}
You can refer this project for reference
I know there are lots of Q's regarding this, but I did go through all of them, and kinda confused myself more, I am listing the steps I followed, please let me know where I messed it up.
1) I just want to use Log4j on application level, so need to copy WL_HOME/server/lib/wllog4j.jar, and the log4j.jar in Domail_Home/Lib?
2) I am using Maven, I added the Log4j dependency in my pom.xml [war].I have my WAR wrapped in EAR.
3) Since I want to write the logs in weblogic managed server logs file, I created a custom appender, to use weblogic, NonCatalogAppender as mention in the link - https://blog.desgrange.net/2010/02/15/logging-in-weblogic-console-with-log4j.html
4) I copied the log file in my war/src/main/resources and i see maven added them to classpath i.e war/target/classes, see below for my lo4j xml
http://jakarta.apache.org/log4j/'>
<!-- stdout appender settings -->
<appender name="STDOUT" class="com.xyz.logging.util.WeblogicAppender">
<param name="Threshold" value="DEBUG"/>
<layout class="org.apache.log4j.PatternLayout">
<!-- notes on patterns:
%p - priority (i.e. level) of message
%c - class that threw error
%m - message in logger's method, e.g. "Exiting application HERE..."
%n - carriage return
%d - date information
-->
<param name="ConversionPattern" value="%c{1} %m"/>
</layout>
</appender>
<!-- settings for root debugger -->
<root>
<level value="DEBUG"/>
<appender-ref ref="STDOUT"/>
</root>
5) Now I didn't change anything on the config level, but I don't see anything appending to server logs. When I initialize NonCatlogLogger manually and call the logger it works fine:
NonCatalogLogger logger =new NonCatalogLogger("XYZ");
logger.debug("This is the debug message")
6) When I debug the application in eclipse, it looks like my custom appender is never called.
Got it, just need to put below into arguments under "server start".
-Dweblogic.log.Log4jLoggingEnabled=true
We have a bunch of projects that get deployed in the same Jboss app. server. Each project has its own log4j.properties in its WEB-INF directory.
The idea is for each project to have its own logfile into which it writes its logs.
Any ideas on how this can be achieved.
First of all you don't mention the JBoss version you're using. You've to take in mind that JBoss versions prior to AS7 include its own log4j version (libraries and configuration file: conf/jboss-log4j.xml), so by default JBoss will ignore the log4j.properties file you've in each of your projects.
That said, you have two approaches to configure a log file for each application:
Centralized way: configure it in Jboss, through the
conf/jboss-log4j.xml. This way you won't need to modify your
applications, or
Modify your applications so that they take their own
log4j libraries (you'll have to include them in the ear/war), not
the Jboss ones.
For the first approach you'll have to define an appender for each of your applications, and then categories for the application packages, in the conf/jboss-log4j.xml file. For example, to configure a log file for an application called app1.war which classes are in a package called com.foo.app1, you'd have to add:
<appender name="APP1_APPENDER" class="org.jboss.logging.appender.DailyRollingFileAppender">
<errorHandler class="org.jboss.logging.util.OnlyOnceErrorHandler"/>
<param name="File" value="${jboss.server.log.dir}/app1.log"/>
<param name="Append" value="true"/>
<!-- Rollover at midnight each day -->
<param name="DatePattern" value="'.'yyyy-MM-dd"/>
<layout class="org.apache.log4j.PatternLayout">
<!-- The default pattern: Date Priority [Category] (Thread) Message\n -->
<param name="ConversionPattern" value="%d %-5p [%c] (%t) %m%n"/>
</layout>
</appender>
...
...
<category name="com.foo.app1" additivity="true">
<priority value="INFO"/>
<appender-ref ref="APP1_APPENDER"/>
</category>
You'll have to add a customized set of blocks like this one for each project you want to define a log file for, respectively.
If you prefer the second approach, as said you'll have to include the log4j libraries in each project and isolate each application so that it takes the libraries included in the application not the ones from JBoss. Look at section 10.3.7 of the Jboss logging file for more information about this approach.
I'm working on a set of web services and we'd like to have a daily rotated log.
I'm trying to get org.apache.log4j.rolling.RollingFileAppender from the log4j extras companion working, since the documentation suggests this is best for production environments.
I have both the main log4J library (log4j-1.2.15.jar) and the log4j extras library (apache-log4j-extras-1.1.jar) on the classpath.
I have the following configuration for the appender in the log4j.properties file:
### SOAP Request Appender
log4j.appender.request=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.request.File=SOAPmessages.log
log4j.appender.request.RollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.request.RollingPolicy.ActiveFileName =SOAPmessages-%d.log
log4j.appender.request.RollingPolicy.FileNamePattern=SOAPmessages-%d.log.zip
log4j.appender.request.layout = org.apache.log4j.PatternLayout
log4j.appender.request.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n
However, when I start the web service with log4j in debug mode I get these error messages:
log4j: Parsing appender named "request".
log4j: Parsing layout options for "request".
log4j: Setting property [conversionPattern] to [%d{ABSOLUTE} %5p %c{1}:%L - %m%n].
log4j: End of parsing for "request".
log4j: Setting property [file] to [/logs/SOAPmessages.log].
log4j:WARN Failed to set property [rollingPolicy] to value "org.apache.log4j.rolling.TimeBasedRollingPolicy".
log4j:WARN Please set a rolling policy for the RollingFileAppender named 'request'
log4j: Parsed "request" options.
I've found documentation about how to configure this appender a little sparse, so can anyone help me fix my configuration?
EDIT0: Added debug mode output, rather than just the standard warnings
I had a similar problem and just found a way to solve it (by single-stepping through log4j-extras source, no less...)
The good news is that, unlike what's written everywhere, it turns out that you actually CAN configure TimeBasedRollingPolicy using log4j.properties (XML config not needed! At least in versions of log4j >1.2.16 see this bug report)
Here is an example:
log4j.appender.File = org.apache.log4j.rolling.RollingFileAppender
log4j.appender.File.rollingPolicy = org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.File.rollingPolicy.FileNamePattern = logs/worker-${instanceId}.%d{yyyyMMdd-HHmm}.log
BTW the ${instanceId} bit is something I am using on Amazon's EC2 to distinguish the logs from all my workers -- I just need to set that property before calling PropertyConfigurator.configure(), as follow:
p.setProperty("instanceId", EC2Util.getMyInstanceId());
PropertyConfigurator.configure(p);
Faced more issues while making this work. Here are the details:
To download and add apache-log4j-extras-1.1.jar in the classpath, didn't notice this at first.
The RollingFileAppender should be org.apache.log4j.rolling.RollingFileAppender instead of org.apache.log4j.RollingFileAppender. This can give the error: log4j:ERROR No output stream or file set for the appender named [file].
We had to upgrade the log4j library from log4j-1.2.14.jar to log4j-1.2.16.jar.
Below is the appender configuration which worked for me:
<appender name="file" class="org.apache.log4j.rolling.RollingFileAppender">
<param name="threshold" value="debug" />
<rollingPolicy name="file"
class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
<param name="FileNamePattern" value="logs/MyLog-%d{yyyy-MM-dd-HH-mm}.log.gz" />
<!-- The below param will keep the live update file in a different location-->
<!-- param name="ActiveFileName" value="current/MyLog.log" /-->
</rollingPolicy>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%5p %d{ISO8601} [%t][%x] %c - %m%n" />
</layout>
</appender>
Update: at least as early as 2013 (see Mubashar's comment) this started working.
According to Log4jXmlFormat you cannot configure it with log4j.properties, but only using the XML config format:
Note that TimeBasedRollingPolicy can only be configured with xml, not log4j.properties
Unfortunately, the example log4j.xml they provide doesn't work either:
log4j:ERROR Parsing error on line 14 and column 76
log4j:ERROR Element type "rollingPolicy" must be declared.
...
log4j:WARN Please set a rolling policy for the RollingFileAppender named 'FILE'
Toolbear74 is right log4j.XML is required.
In order to get the XML to validate the <param> tags need to be BEFORE the <rollingPolicy>
I suggest setting a logging threshold <param name="threshold" value="info"/>
When Creating a Log4j.xml file don't forget to to copy the log4j.dtd into the same location.
Here is an example:
<?xml version="1.0" encoding="windows-1252"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd" >
<log4j:configuration>
<!-- Daily Rolling File Appender that compresses old files -->
<appender name="file" class="org.apache.log4j.rolling.RollingFileAppender" >
<param name="threshold" value="info"/>
<rollingPolicy name="file"
class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
<param name="FileNamePattern"
value="${catalina.base}/logs/myapp.log.%d{yyyy-MM-dd}.gz"/>
<param name="ActiveFileName" value="${catalina.base}/logs/myapp.log"/>
</rollingPolicy>
<layout class="org.apache.log4j.EnhancedPatternLayout" >
<param name="ConversionPattern"
value="%d{ISO8601} %-5p - %-26.26c{1} - %m%n" />
</layout>
</appender>
<root>
<priority value="debug"></priority>
<appender-ref ref="file" />
</root>
</log4j:configuration>
Considering that your setting a FileNamePattern and an ActiveFileName I think that setting a File property is redundant and possibly even erroneous
Try renaming your log4j.properties and dropping in a log4j.xml similar to my example and see what happens.
Regarding error: log4j:ERROR Element type "rollingPolicy" must be declared
Use a log4j.jar version newer than log4j-1.2.14.jar, which has a log4j.dtd defining rollingPolicy.
of course you also need apache-log4j-extras-1.1.jar
Check if any other third party jars you are using perhaps have an older version of log4j.jar packed inside.
If so, make sure your log4j.jar comes first in the order before the third party containing the older log4j.jar.
In Log4j2, the "extras" lib is not mandatory any more. Also the configuration format has changed.
An example is provided in the Apache documentation
property.filename = /foo/bar/test.log
appender.rolling.type = RollingFile
appender.rolling.name = RollingFile
appender.rolling.fileName = ${filename}
appender.rolling.filePattern = /foo/bar/rolling/test1-%d{MM-dd-yy-HH-mm-ss}-%i.log.gz
appender.rolling.layout.type = PatternLayout
appender.rolling.layout.pattern = %d %p %C{1.} [%t] %m%n
appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 2
appender.rolling.policies.time.modulate = true
appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size=100MB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.max = 5
logger.rolling.name = com.example.my.class
logger.rolling.level = debug
logger.rolling.additivity = false
logger.rolling.appenderRef.rolling.ref = RollingFile
I'm suspicious of the ActiveFileName property. According to the log4j javadoc, there isn't such a property on the TimeBasedRollingPolicy class. (No setActiveFileName or getActiveFileName methods.)
I don't know what specifying an unknown property would do, but it is possible that it will cause the containing object to be abandoned, and that could lead to the warnings that you saw.
Try leaving it out and seeing what happens.
You have a bad package name
org.apache.log4j.rolling.RollingFileAppender
The correct one is:
org.apache.log4j.RollingFileAppender
Is there a way to configure log4j so that it outputs different levels of logging to different appenders?
I'm trying to set up multiple log files. The main log file would catch all INFO and above messages for all classes. (In development, it would catch all DEBUG and above messages, and TRACE for specific classes.)
Then, I would like to have a separate log file. That log file would catch all DEBUG messages for a specific subset of classes, and ignore all messages for any other class.
Is there a way to get what I'm after?
This should get you started:
log4j.rootLogger=QuietAppender, LoudAppender, TRACE
# setup A1
log4j.appender.QuietAppender=org.apache.log4j.RollingFileAppender
log4j.appender.QuietAppender.Threshold=INFO
log4j.appender.QuietAppender.File=quiet.log
...
# setup A2
log4j.appender.LoudAppender=org.apache.log4j.RollingFileAppender
log4j.appender.LoudAppender.Threshold=DEBUG
log4j.appender.LoudAppender.File=loud.log
...
log4j.logger.com.yourpackage.yourclazz=TRACE
Perhaps something like this?
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<!-- general application log -->
<appender name="MainLogFile" class="org.apache.log4j.FileAppender">
<param name="File" value="server.log" />
<param name="Threshold" value="INFO" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p %t [%-40.40c] %x - %m%n"/>
</layout>
</appender>
<!-- additional fooSystem logging -->
<appender name="FooLogFile" class="org.apache.log4j.FileAppender">
<param name="File" value="foo.log" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p %t [%-40.40c] %x - %m%n"/>
</layout>
</appender>
<!-- foo logging -->
<logger name="com.example.foo">
<level value="DEBUG"/>
<appender-ref ref="FooLogFile"/>
</logger>
<!-- default logging -->
<root>
<level value="INFO"/>
<appender-ref ref="MainLogFile"/>
</root>
</log4j:configuration>
Thus, all info messages are written to server.log; by contrast, foo.log contains only com.example.foo messages, including debug-level messages.
I had this question, but with a twist - I was trying to log different content to different files. I had information for a LowLevel debug log, and a HighLevel user log. I wanted the LowLevel to go to only one file, and the HighLevel to go to both a file, and a syslogd.
My solution was to configure the 3 appenders, and then setup the logging like this:
log4j.threshold=ALL
log4j.rootLogger=,LowLogger
log4j.logger.HighLevel=ALL,Syslog,HighLogger
log4j.additivity.HighLevel=false
The part that was difficult for me to figure out was that the 'log4j.logger' could have multiple appenders listed. I was trying to do it one line at a time.
Hope this helps someone at some point!
For the main logfile/appender, set up a .Threshold = INFO to limit what is actually logged in the appender to INFO and above, regardless of whether or not the loggers have DEBUG, TRACE, etc, enabled.
As for catching DEBUG and nothing above that... you'd probably have to write a custom appender.
However I'd recommend not doing this, as it sounds like it would make troubleshooting and analysis pretty hard:
If your goal is to have a single file where you can look to troubleshoot something, then spanning your log data across different files will be annoying - unless you have a very regimented logging policy, you'll likely need content from both DEBUG and INFO to be able to trace execution of the problematic code effectively.
By still logging all of your debug messages, you are losing any performance gains you usually get in a production system by turning the logging (way) down.
Demo link: https://github.com/RazvanSebastian/spring_multiple_log_files_demo.git
My solution is based on XML configuration using spring-boot-starter-log4j. The example is a basic example using spring-boot-starter and the two Loggers writes into different log files.