Develop Reference

How to create different files for logger hierarchy in tomcat?

In my Tomcat application, I want to use two loggers to log general event informations and errors to two different files.
Logger eventLogger = Logger.getLogger("event");
Logger errorLogger = Logger.getLogger("error");
Now I want to do the following: the logs of the eventLogger should be written to a "events.log" file, and the errorLogger logs should be written in a "errors.log" file. All logs from any other logger (e.g. Tomcat logs) should be written to catalina.out (or any other default file).
How can I achive this with JULI? Or do I need a third party lib?

This can't be achieved with Juli. You might use log4j.
Basically you need appenders. They will manage the logs to go to separate files. Example configuration in xml:
<?xml version="1.0"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.EnhancedPatternLayout">
<param name="ConversionPattern" value="%d{ABSOLUTE} %-5p [%t][%c{1}:%L] %m%n" />
</layout>
</appender>
<appender name="eventFILE" class="org.apache.log4j.DailyRollingFileAppender">
<param name="File" value="#tomcat.home#/logs/event.log" />
<param name="DatePattern" value="'.'yyyy-MM-dd" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n"/>
</layout>
</appender>
<appender name="errorFILE" class="org.apache.log4j.DailyRollingFileAppender">
<param name="File" value="#tomcat.home#/logs/error.log" />
<param name="DatePattern" value="'.'yyyy-MM-dd" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n"/>
</layout>
</appender>
<logger name="event" additivity="false">
<level value="INFO" />
<appender-ref ref="eventFILE" />
</logger>
<logger name="error" additivity="false">
<level value="INFO" />
<appender-ref ref="errorFILE" />
</logger>
<root>
<priority value="INFO" />
<appender-ref ref="FILE" />
</root>
</log4j:configuration>

you have to define an appender for each file in your log4j.xml and connect them to your loggers:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<!-- Appenders -->
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<param name="Target" value="System.out" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %5p [%c.%M:%L] - %m%n" />
</layout>
</appender>
<appender name="FILE-error" class="org.apache.log4j.DailyRollingFileAppender">
<param name="File" value="${catalina.base}/logs/error.log" />
<param name="DatePattern" value="'.'yyyy-MM-dd" />
<param name="ImmediateFlush" value="true" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %5p [%c.%M:%L] - %m%n" />
</layout>
</appender>
<appender name="FILE-event" class="org.apache.log4j.DailyRollingFileAppender">
<param name="File" value="${catalina.base}/logs/event.log" />
<param name="DatePattern" value="'.'yyyy-MM-dd" />
<param name="ImmediateFlush" value="true" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss};%m%n" />
</layout>
</appender>
<!-- Loggers -->
<logger name="error" additivity="false">
<level value="warn" />
<appender-ref ref="FILE-error" />
</logger>
<logger name="event" additivity="false">
<level value="info" />
<appender-ref ref="FILE-event" />
</logger>
<!-- Root Logger -->
<root>
<priority value="warn" />
<appender-ref ref="FILE-error" />
</root>
</log4j:configuration>

Tomcat documentation does not provide any way to define more than one file per application when using JULI. Actually it provides the steps to use log4j instead of it because JULI configuration can be too basic.
But again this configuration is at container level:
Note: The steps described in this section are needed when you want to
reconfigure Tomcat to use Apache log4j for its own logging. These
steps are not needed if you just want to use log4j in your own web
application. — In that case, just put log4j.jar and log4j.properties
into WEB-INF/lib and WEB-INF/classes of your web application.
So using a third party library is the logical choice and you can find how to configure it in this SO post

Tomcat JULI logging supports this only via the logging.properties located in the ${catalina.base}/config folder.
#Declares the handlers allowed for use.
handlers = 100catalina.org.apache.juli.FileHandler, 200catalina.org.apache.juli.FileHandler, 300catalina.org.apache.juli.FileHandler
#Install the 'all' hander on the root logger.
.handlers=300catalina.org.apache.juli.FileHandler
#Install the 'event' handler on the 'event' logger and don't report to root.
event.handlers=100catalina.org.apache.juli.FileHandler
event.useParentHandlers=false
#Install the 'error' handler on the 'error' logger and don't report to root.
error.handlers=200catalina.org.apache.juli.FileHandler
error.useParentHandlers=false
#Event handler settings.
100catalina.org.apache.juli.FileHandler.level = ALL
100catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
100catalina.org.apache.juli.FileHandler.prefix = event.
#Error handler settings.
200catalina.org.apache.juli.FileHandler.level = ALL
200catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
200catalina.org.apache.juli.FileHandler.prefix = error.
#Root handler settings.
300catalina.org.apache.juli.FileHandler.level = ALL
300catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
300catalina.org.apache.juli.FileHandler.prefix = all.
This config will not work if you try to install this file in the WEB-INF/classes/logging.properties. This is because the ClassLoaderLogManager doesn't allow you to install handlers on named loggers.

can't log Hibernate queries log4j

I am trying to log hibernate queries into a webapp specific log file.
I can log any message but I am still unable to log hibernate queries.
Them are being logged inside the server.log, even in a different file, but I need an app-specific log for each app running on the same server.
I am using log4j 1.2, I am able to create the log files, to log every information passed to console but hibernate queries.
Inside a database.properties file I found the hibernate.show_sql=true that is responsible for the logging output in console.
This is the kind of console's logging output I refer:
16:10:35,827 INFO [stdout] (http-localhost-127.0.0.1-8080-1) Hibernate: select [...](here is outputted the correct query passed, with the ? of the prepared statement)
Anyway, what I need is to log that output.
I tried different log4j.xml settings, but still with no success, it creates log files but does not log those outputs.
Here is the current setting, that is logging everything inside the same file, when I will be able to log even those information I will go on separating the logs.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd" >
<log4j:configuration>
<appender name="mainFileAppender2" class="org.apache.log4j.RollingFileAppender">
<param name="append" value="false" />
<param name="maxFileSize" value="100MB" />
<param name="maxBackupIndex" value="50" />
<param name="File" value="${webapp.root}/WEB-INF/logs/mainCORE.log"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %-5p %c{1}:%L %m %n" />
</layout>
</appender>
<appender name="journaldev-hibernate2" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="${webapp.root}/WEB-INF/logs/hib-queriesCORE.log" />
<param name="Append" value="false"/><!--value="true" /-->
<param name="ImmediateFlush" value="true" />
<param name="MaxFileSize" value="200MB" />
<param name="MaxBackupIndex" value="50" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %d{Z} [%t] %-5p (%F:%L) - %m%n" />
</layout>
</appender>
<root>
<priority value="debug"></priority>
<!-- <appender-ref ref="stdout"/> -->
<appender-ref ref="mainFileAppender2"/>
</root>
This configuration logs everything but those outputs. In the server.log are logged as follows:
16:10:35,827 INFO [stdout] (http-localhost-127.0.0.1-8080-1) Hibernate: select [...](rest of the query)

Configure the logger for the following categories:
log4j.logger.org.hibernate.SQL=DEBUG
log4j.logger.org.hibernate.type=TRACE
The first one will log the SQL-Statement (with ? for parameter values), the second one will print those parameter values if you need them.
So something like this should do the trick in your case:
<category name="org.hibernate.SQL">
<appender-ref ref="journaldev-hibernate2"/>
<priority value="DEBUG"/>
</category>
<category name="org.hibernate.type">
<appender-ref ref="journaldev-hibernate2"/>
<priority value="TRACE"/>
</category>

Solved this way; Anyway it also extracts the values extracted from the query
log = ${jboss.server.log.dir}/log/
#Loggers
log4j.rootLogger = INFO, FILE
log4j.logger.org.hibernate.SQL = DEBUG, HIBERNATE
log4j.logger.org.hibernate.type = TRACE, HIBERNATE
#Appenders
# Main appender
log4j.appender.FILE=org.apache.log4j.DailyRollingFileAppender
log4j.appender.FILE.File=${log}/main.log
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.ConversionPattern=%d %-5p %c{1}:%L %m %n
log4j.appender.FILE.ImmediateFlush=true
log4j.appender.FILE.Threshold=debug
log4j.appender.FILE.Append=true
log4j.appender.FILE.DatePattern=dd-MM-yyyy
# Hibernate queries appender
log4j.appender.HIBERNATE=org.apache.log4j.DailyRollingFileAppender
log4j.appender.HIBERNATE.File=${log}/hib-queries.log
log4j.appender.HIBERNATE.layout=org.apache.log4j.PatternLayout
log4j.appender.HIBERNATE.layout.ConversionPattern=%d %d{Z} [%t] %-5p (%F:%L) - %m%n
log4j.appender.HIBERNATE.ImmediateFlush=true
log4j.appender.HIBERNATE.Threshold=debug
log4j.appender.HIBERNATE.Append=true
log4j.appender.HIBERNATE.DatePattern=dd-MM-yyyy

Logging different levels using same object to different files

My simple question is : If I have two statements like logger.info() and logger.error() in Java (using log4j), I want the results of these two lines to be printed in two separate files. That is, logger.info(...) should print to a file say myLog.info and logger.error(...) should print to myLog.error file. I am using rolling file appender for this task. Also I want just one logger object to do the task. Someone might suggest two or more different loggers one for each file, but that's not the case.
I tried searching a solution for the problem. One of the links Creating multiple log files of different content with log4j says about "threshold" and I even tried to add threshold in my xml configuration file. But what it actually is doing that : info log is being printed in myLog.info file but error log gets printed in both the files. Can it be done through xml configuration file alone or a separate properties file is needed? If xml file is sufficient, then what needs to be done?
I am preferring xml file over properties file. If there is a working solution using only xml configuration file, that would be sufficient. Thanks in advance.

You could use filters to deny any messages except those of the level you want. Here is an example of how to do this:
First a class to test our efforts:
package test;
import org.apache.log4j.Logger;
public class Main {
private static final Logger logger = Logger.getLogger(Main.class);
public static void main(String[] args) {
logger.debug("here's some debug");
logger.info("here's some info");
logger.warn("here's some warn");
logger.error("here's some error");
logger.fatal("here's some fatal");
}
}
Next a log4j.xml config file to set up the appenders and loggers:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="consoleAppender" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %-5p [%c{1}] %m %n" />
</layout>
</appender>
<appender name="debugLog" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="logs/debug.log" />
<param name="Append" value="true" />
<param name="MaxFileSize" value="5000KB" />
<param name="maxBackupIndex" value="5" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} [%t] %-5p - %m%n" />
</layout>
<filter class="org.apache.log4j.varia.LevelMatchFilter">
<param name="LevelToMatch" value="DEBUG" />
<param name="AcceptOnMatch" value="true" />
</filter>
<filter class="org.apache.log4j.varia.DenyAllFilter" />
</appender>
<appender name="infoLog" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="logs/info.log" />
<param name="Append" value="true" />
<param name="MaxFileSize" value="5000KB" />
<param name="maxBackupIndex" value="5" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} [%t] %-5p - %m%n" />
</layout>
<filter class="org.apache.log4j.varia.LevelMatchFilter">
<param name="LevelToMatch" value="INFO" />
<param name="AcceptOnMatch" value="true" />
</filter>
<filter class="org.apache.log4j.varia.DenyAllFilter" />
</appender>
<logger name="test" additivity="false">
<level value="DEBUG" />
<appender-ref ref="consoleAppender" />
<appender-ref ref="debugLog" />
<appender-ref ref="infoLog" />
</logger>
</log4j:configuration>
This pattern will allow you to generate a separate log for each log level, just repeat the configuration that I have provided for either debug or info logging. Note that the console appender will accept all levels.
I was able to gain some insight from this post so I thought I should give credit.

Why is log4j logging to the console?

This is a standalone java application.
I am using the configuration file below and having two problems.
1) I'm getting logs to stdout and I don't know why.
2) I'm getting ALL log messages in my error log even though I have tried to direct only error and higher to the error log.
I am using the BasicConfigurator without specifying an explicit path to the log4j.xml file. The xml file is in the same jar as my classes. It is creating and writing to the appropriate logs in addition to these problems so the configuration is being applied.
3) In addition, I have had no luck having the log4j.xml file outside of the jar so I can change it at runtime. How do I do that?
<!--appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
<param name="Threshold" value="DEBUG"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p [%F:%L] - %m%n"/>
</layout>
</appender-->
<!-- working dir is $CATALINA_TMPDIR. send logs to log dir. -->
<appender name="ROLL" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="/var/log/company/application.log"/>
<param name="MaxFileSize" value="5MB"/>
<param name="MaxBackupIndex" value="9"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d [%t] %-5p %c %x - %m%n"/>
</layout>
</appender>
<appender name="ERRORLOG" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="/var/log/rocketvox/company/error.log"/>
<param name="MaxFileSize" value="5MB"/>
<param name="MaxBackupIndex" value="9"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d [%t] %-5p %c %x - %m%n"/>
</layout>
</appender>
<category name="com.company">
<priority value="ALL"/>
<appender-ref ref="ROLL"/>
</category>
<category name="com.mattx">
<priority value="ALL"/>
<appender-ref ref="ROLL"/>
</category>
<root>
<priority value="error"/>
<appender-ref ref="ERRORLOG"/>
</root>

Add -Dlog4j.debug to the application's JVM args to see exactly what log4j is doing and which configuration file it uses.
Your problem is using BasicConfigurer - for configuring using a file named log4j.xml, you don't need to use any explicit log4j Configurer in your application code, as the default log4j initialization logic will kick in to pick log4j.xml (or log4j.properties, if no xml is found) from the classpath.
See Default Initialization Procedure from the log4j manual.

Why is the root logger collecting all log types regardless the configuration?

I am having problem that even though I specify the level to ERROR in the root tag, the specified appender logs all levels (debug, info, warn) to the file regardless the settings. I am not a Log4j expert so any help is appreciated.
I have checked the classpath for log4j.properties (there is none) except the log4j.xml.
Here is the log4j.xml file:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
<!-- ============================== -->
<!-- Append messages to the console -->
<!-- ============================== -->
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<param name="Target" value="System.out" />
<layout class="org.apache.log4j.PatternLayout">
<!-- The default pattern: Date Priority [Category] Message\n -->
<param name="ConversionPattern" value="[AC - %5p] [%d{ISO8601}] [%t] [%c{1} - %L] %m%n" />
</layout>
</appender>
<appender name="logfile" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="./logs/server.log" />
<param name="MaxFileSize" value="1000KB" />
<param name="MaxBackupIndex" value="2" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="[AC - %-5p] {%d{dd.MM.yyyy - HH.mm.ss}} %m%n" />
</layout>
</appender>
<appender name="payloadAppender" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="./logs/payload.log" />
<param name="MaxFileSize" value="1000KB" />
<param name="MaxBackupIndex" value="10" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="[AC - %-5p] {%d{dd.MM.yyyy - HH.mm.ss}} %m%n" />
</layout>
</appender>
<appender name="errorLog" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="./logs/error.log" />
<param name="MaxFileSize" value="1000KB" />
<param name="MaxBackupIndex" value="10" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="[AC - %-5p] {%d{dd.MM.yyyy - HH.mm.ss}} %m%n" />
</layout>
</appender>
<appender name="traceLog"
class="org.apache.log4j.RollingFileAppender">
<param name="File" value="./logs/trace.log" />
<param name="MaxFileSize" value="1000KB" />
<param name="MaxBackupIndex" value="20" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern"
value="[AccessControl - %-5p] {%t: %d{dd.MM.yyyy - HH.mm.ss,SSS}} %m%n" />
</layout>
</appender>
<appender name="traceSocketAppender" class="org.apache.log4j.net.SocketAppender">
<param name="remoteHost" value="localhost" />
<param name="port" value="4445" />
<param name="locationInfo" value="true" />
</appender>
<logger name="TraceLogger">
<level value="trace" /> <!-- Set level to trace to activate tracing -->
<appender-ref ref="traceLog" />
</logger>
<logger name="org.springframework.ws.server.endpoint.interceptor">
<level value="DEBUG" />
<appender-ref ref="payloadAppender" />
</logger>
<root>
<level value="error" />
<appender-ref ref="errorLog" />
</root>
</log4j:configuration>
If I replace the root with another logger, then nothing gets logged at all to the specified appender.
<logger name="com.mydomain.logic">
<level value="error" />
<appender-ref ref="errorLog" />
</logger>

The root logger resides at the top of the logger hierarchy. It is exceptional in three ways:
it always exists,
its level cannot be set to null
it cannot be retrieved by name.
The rootLogger is the father of all appenders. Each enabled logging request for a given logger will be forwarded to all the appenders in that logger as well as the appenders higher in the hierarchy (including rootLogger)
For example, if the console appender is added to the root logger, then all enabled logging requests will at least print on the console. If in addition a file appender is added to a logger, say L, then enabled logging requests for L and L's children will print on a file and on the console. It is possible to override this default behavior so that appender accumulation is no longer additive by setting the additivity flag to false.
From the log4j manual
To sum up:
If you want not to propagate a logging event to the parents loggers (say rootLogger) then add the additivity flag to false in those loggers. In your case:
<logger name="org.springframework.ws.server.endpoint.interceptor"
additivity="false">
<level value="DEBUG" />
<appender-ref ref="payloadAppender" />
</logger>
In standard log4j config style (which I prefer to XML):
log4j.logger.org.springframework.ws.server.endpoint.interceptor = INFO, payloadAppender
log4j.additivity.org.springframework.ws.server.endpoint.interceptor = false
Hope this helps.

Run your program with -Dlog4j.debug so that standard out gets info about how log4j is configured -- I suspected that it isn't configured the way that you think it is.

To add on to what James A. N. Stauffer and cynicalman said - I would bet that there is another log4j.xml / log4j.properties on your classpath other than the one you wish to be used that is causing log4j to configure itself the way it is.
-Dlog4j.debug is an absolute killer way to troubleshoot any log4j issues.

Two things: Check additivity and decide whether you want log events captured by more detailed levels of logging to propagate to the root logger.
Secondly, check the level for the root logger. In addition you can also add filtering on the appender itself, but this should normally not be necessary.

If you are using a log4j.properties file, this file is typically expected to be in the root of your classpath, so make sure it's there.

This is correct behavior. The root logger is like the default behavior. So if you don't specify any logger it will take root logger level as the default level but this does not mean that root logger level is the level for all your logs.
Any of your code which logs using 'TraceLogger'logger or 'org.springframework.ws.server.endpoint.interceptor' logger will log messages using TRACE and DEBUG level respectively any other code will use root logger to log message using level, which is in your case ERROR.
So if you use logger other than root, root log level will be overridden by that logger's log level. To get the desired output change the other two log level to ERROR.
I hope this is helpful.