Develop Reference

log4j new log file is not creating when tomcat 7 starting in AWS server instance

I have implemented a log4j XML file to create a new log file.in my local machine, when the tomcat starts the java web application, log file is being created without any issue.but i have hosted my java web application in aws instance configured with tomcat 7.but when i logged to aws instance and check tomcat log folder grid-services.log file cannot be located.can anyone tell why this happened?
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd" >
<log4j:configuration debug="false">
<appender name="default.console" class="org.apache.log4j.ConsoleAppender">
<param name="target" value="System.out" />
<param name="threshold" value="debug" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %-5p [%c{1}] - %m%n" />
</layout>
</appender>
<appender name="default.file" class="org.apache.log4j.FileAppender">
<param name="file" value="${catalina.home}/logs/grid-services.log" />
<param name="append" value="false" />
<param name="threshold" value="debug" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %-5p [%c{1}] - %m%n" />
</layout>
</appender>
<logger name="com.customer"
additivity="false">
<level value="info" />
<appender-ref ref="default.file" />
</logger>
<root>
<priority value="info" />
<appender-ref ref="default.console" />
<appender-ref ref="default.file" />
</root>
</log4j:configuration>

Look in your ${TOMCAT_HOME}/bin/catalina.sh. Check if the
-Dcatalina.home=
variable is set on your aws instance.

How to create different files for logger hierarchy in tomcat?

In my Tomcat application, I want to use two loggers to log general event informations and errors to two different files.
Logger eventLogger = Logger.getLogger("event");
Logger errorLogger = Logger.getLogger("error");
Now I want to do the following: the logs of the eventLogger should be written to a "events.log" file, and the errorLogger logs should be written in a "errors.log" file. All logs from any other logger (e.g. Tomcat logs) should be written to catalina.out (or any other default file).
How can I achive this with JULI? Or do I need a third party lib?

This can't be achieved with Juli. You might use log4j.
Basically you need appenders. They will manage the logs to go to separate files. Example configuration in xml:
<?xml version="1.0"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.EnhancedPatternLayout">
<param name="ConversionPattern" value="%d{ABSOLUTE} %-5p [%t][%c{1}:%L] %m%n" />
</layout>
</appender>
<appender name="eventFILE" class="org.apache.log4j.DailyRollingFileAppender">
<param name="File" value="#tomcat.home#/logs/event.log" />
<param name="DatePattern" value="'.'yyyy-MM-dd" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n"/>
</layout>
</appender>
<appender name="errorFILE" class="org.apache.log4j.DailyRollingFileAppender">
<param name="File" value="#tomcat.home#/logs/error.log" />
<param name="DatePattern" value="'.'yyyy-MM-dd" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n"/>
</layout>
</appender>
<logger name="event" additivity="false">
<level value="INFO" />
<appender-ref ref="eventFILE" />
</logger>
<logger name="error" additivity="false">
<level value="INFO" />
<appender-ref ref="errorFILE" />
</logger>
<root>
<priority value="INFO" />
<appender-ref ref="FILE" />
</root>
</log4j:configuration>

you have to define an appender for each file in your log4j.xml and connect them to your loggers:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<!-- Appenders -->
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<param name="Target" value="System.out" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %5p [%c.%M:%L] - %m%n" />
</layout>
</appender>
<appender name="FILE-error" class="org.apache.log4j.DailyRollingFileAppender">
<param name="File" value="${catalina.base}/logs/error.log" />
<param name="DatePattern" value="'.'yyyy-MM-dd" />
<param name="ImmediateFlush" value="true" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %5p [%c.%M:%L] - %m%n" />
</layout>
</appender>
<appender name="FILE-event" class="org.apache.log4j.DailyRollingFileAppender">
<param name="File" value="${catalina.base}/logs/event.log" />
<param name="DatePattern" value="'.'yyyy-MM-dd" />
<param name="ImmediateFlush" value="true" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss};%m%n" />
</layout>
</appender>
<!-- Loggers -->
<logger name="error" additivity="false">
<level value="warn" />
<appender-ref ref="FILE-error" />
</logger>
<logger name="event" additivity="false">
<level value="info" />
<appender-ref ref="FILE-event" />
</logger>
<!-- Root Logger -->
<root>
<priority value="warn" />
<appender-ref ref="FILE-error" />
</root>
</log4j:configuration>

Tomcat documentation does not provide any way to define more than one file per application when using JULI. Actually it provides the steps to use log4j instead of it because JULI configuration can be too basic.
But again this configuration is at container level:
Note: The steps described in this section are needed when you want to
reconfigure Tomcat to use Apache log4j for its own logging. These
steps are not needed if you just want to use log4j in your own web
application. — In that case, just put log4j.jar and log4j.properties
into WEB-INF/lib and WEB-INF/classes of your web application.
So using a third party library is the logical choice and you can find how to configure it in this SO post

Tomcat JULI logging supports this only via the logging.properties located in the ${catalina.base}/config folder.
#Declares the handlers allowed for use.
handlers = 100catalina.org.apache.juli.FileHandler, 200catalina.org.apache.juli.FileHandler, 300catalina.org.apache.juli.FileHandler
#Install the 'all' hander on the root logger.
.handlers=300catalina.org.apache.juli.FileHandler
#Install the 'event' handler on the 'event' logger and don't report to root.
event.handlers=100catalina.org.apache.juli.FileHandler
event.useParentHandlers=false
#Install the 'error' handler on the 'error' logger and don't report to root.
error.handlers=200catalina.org.apache.juli.FileHandler
error.useParentHandlers=false
#Event handler settings.
100catalina.org.apache.juli.FileHandler.level = ALL
100catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
100catalina.org.apache.juli.FileHandler.prefix = event.
#Error handler settings.
200catalina.org.apache.juli.FileHandler.level = ALL
200catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
200catalina.org.apache.juli.FileHandler.prefix = error.
#Root handler settings.
300catalina.org.apache.juli.FileHandler.level = ALL
300catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
300catalina.org.apache.juli.FileHandler.prefix = all.
This config will not work if you try to install this file in the WEB-INF/classes/logging.properties. This is because the ClassLoaderLogManager doesn't allow you to install handlers on named loggers.

Logging different levels using same object to different files

My simple question is : If I have two statements like logger.info() and logger.error() in Java (using log4j), I want the results of these two lines to be printed in two separate files. That is, logger.info(...) should print to a file say myLog.info and logger.error(...) should print to myLog.error file. I am using rolling file appender for this task. Also I want just one logger object to do the task. Someone might suggest two or more different loggers one for each file, but that's not the case.
I tried searching a solution for the problem. One of the links Creating multiple log files of different content with log4j says about "threshold" and I even tried to add threshold in my xml configuration file. But what it actually is doing that : info log is being printed in myLog.info file but error log gets printed in both the files. Can it be done through xml configuration file alone or a separate properties file is needed? If xml file is sufficient, then what needs to be done?
I am preferring xml file over properties file. If there is a working solution using only xml configuration file, that would be sufficient. Thanks in advance.

You could use filters to deny any messages except those of the level you want. Here is an example of how to do this:
First a class to test our efforts:
package test;
import org.apache.log4j.Logger;
public class Main {
private static final Logger logger = Logger.getLogger(Main.class);
public static void main(String[] args) {
logger.debug("here's some debug");
logger.info("here's some info");
logger.warn("here's some warn");
logger.error("here's some error");
logger.fatal("here's some fatal");
}
}
Next a log4j.xml config file to set up the appenders and loggers:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="consoleAppender" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %-5p [%c{1}] %m %n" />
</layout>
</appender>
<appender name="debugLog" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="logs/debug.log" />
<param name="Append" value="true" />
<param name="MaxFileSize" value="5000KB" />
<param name="maxBackupIndex" value="5" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} [%t] %-5p - %m%n" />
</layout>
<filter class="org.apache.log4j.varia.LevelMatchFilter">
<param name="LevelToMatch" value="DEBUG" />
<param name="AcceptOnMatch" value="true" />
</filter>
<filter class="org.apache.log4j.varia.DenyAllFilter" />
</appender>
<appender name="infoLog" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="logs/info.log" />
<param name="Append" value="true" />
<param name="MaxFileSize" value="5000KB" />
<param name="maxBackupIndex" value="5" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} [%t] %-5p - %m%n" />
</layout>
<filter class="org.apache.log4j.varia.LevelMatchFilter">
<param name="LevelToMatch" value="INFO" />
<param name="AcceptOnMatch" value="true" />
</filter>
<filter class="org.apache.log4j.varia.DenyAllFilter" />
</appender>
<logger name="test" additivity="false">
<level value="DEBUG" />
<appender-ref ref="consoleAppender" />
<appender-ref ref="debugLog" />
<appender-ref ref="infoLog" />
</logger>
</log4j:configuration>
This pattern will allow you to generate a separate log for each log level, just repeat the configuration that I have provided for either debug or info logging. Note that the console appender will accept all levels.
I was able to gain some insight from this post so I thought I should give credit.

log4j smtp appender overriding gives unexpected behavior?

I have overridden log4j smtp appender for some purpose. The problem is if i keep <param name="Threshold" value="INFO"/> Then it would send mails properly. But if i keep <param name="Threshold" value="ERROR"/> Then no mails are sent. What is the problem? Please find my code.
public class MyAppender extends SMTPAppender{
#Override
protected void sendBuffer() {
//Some code to format email body
}
}
log4j.xml
<!-- Appenders -->
<appender name="mail" class="com.service.MyAppender">
<param name="BufferSize" value="2"/>
<param name="BufferedIO" value="true"/>
<param name="Threshold" value="ERROR"/>
<param name="SMTPHost" value="myhost"/>
<param name="From" value="abc#co.com"/>
<param name="To" value="def#ld.com"/>
<param name="Subject" value="Testing Testing"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern"
value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{2}:%L - %m%n"/>
</layout>
</appender>
<appender name="ASYNC" class="org.apache.log4j.AsyncAppender">
<param name="BufferSize" value="500"/>
<appender-ref ref="mail"/>
</appender>
<root>
<level value="ERROR" />
<appender-ref ref="ASYNC" />
</root>
I looked into SMTPAppender source code and i believe below lines are the problem.
if(evaluator.isTriggeringEvent(event)) {
sendBuffer();
}

According to log4j documentation, all log events with value lower than Treshold will be filtered out (i.e. not sent).
It's possible that changing Treshold to more restrictive filters out all messages that are logged in your case.
Make sure that you actually have events with log level ERROR (or higher - FATAL).

Log4j Maxfilesize

Problem i have with log4j is that when i specify maxfilesize it works properly for the files generated in the src folder of project, but the log files generated in tomcat bin, are not splited into seperate files and then i have a large log file which is not desirable.
The following is log4j.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd" >
<log4j:configuration>
<appender name="fatalFile" class="org.apache.log4j.RollingFileAppender">
<param name="MaxFileSize" value="100KB" />
<param name="MaxBackupIndex" value="5" />
<param name="File" value="logs/pnusn/fatal.log" />
<param name="threshold" value="fatal" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%5p in Class:%C{2}, Thread:%t, at:%d{ABSOLUTE}:: '%m'%n" />
</layout>
</appender>
<appender name="othersFile" class="org.apache.log4j.RollingFileAppender">
<param name="MaxFileSize" value="100KB" />
<param name="MaxBackupIndex" value="5" />
<param name="File" value="logs/pnusn/others.log" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%5p in Class:%C{2}, Thread:%t, at:%d{ABSOLUTE}:: '%m'%n" />
</layout>
<filter class="org.apache.log4j.varia.LevelRangeFilter">
<param name="LevelMin" value="debug" />
<param name="LevelMax" value="error" />
</filter>
</appender>
<appender name="traceFile" class="org.apache.log4j.RollingFileAppender">
<param name="MaxFileSize" value="200KB" />
<param name="MaxBackupIndex" value="20" />
<param name="File" value="logs/pnusn/trace.log" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%5p in Class:%C{2}, Thread:%t, at:%d{ABSOLUTE}:: '%m'%n" />
</layout>
<filter class="org.apache.log4j.varia.LevelRangeFilter">
<param name="LevelMin" value="trace" />
<param name="LevelMax" value="trace" />
</filter>
</appender>
<root>
<priority value="trace"></priority>
<appender-ref ref="fatalFile" />
<appender-ref ref="othersFile" />
<appender-ref ref="traceFile" />
</root>
</log4j:configuration>
Does anybody know how can i fix it?
Thanks for your consideration.

The log4j log files are appearing in your tomcat's bin directory because you haven't told it to do otherwise.
The tomcat docs here describe how to integrate log4j, including the use of the ${catalina.out} substitution to refer to the tomcat root directory. The instructions refer to the log4j properties file format, too, it's not clear if it'll work for log4j.xml
As for the file size problem, I'm not sure why that's happening. Try using the DailyRollingFileAppender instead, and see if that works.