I have a Java application that runs for a few minutes each hour via a cron job (java -jar...). It runs in its own JVM, not in a continuously running environment like Tomcat. I am using Log4j 2.11 to do logging. Within the application, I have a particular logger with specific rollover requirements:
Log to "rolling.log"
At the end of each day (or on the first logging event of a new day), rolling.log should be rolled over to rolling-yyyy-MM-dd.log.gz and new logging events added to a fresh rolling.log.
I have not been able to get the rollover to work. All log messages are to "rolling.log" only and no rolling-yyyy-MM-dd.log.gz is ever created. To test this in the most simple way possible, I created a simple Java console application with the following two files. I would expect the log file to roll over on every execution as long as the system clock is showing a different minute, but this does not happen.
LoggingTest.java:
package log4jtest;
import java.time.Instant;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
public class LoggingTest {
private static final Logger logger = LogManager.getLogger();
public static void main(String[] args) {
System.out.println(Instant.now() + " - BEGIN: Logging to log4j");
logger.error("Test log message");
System.out.println(Instant.now() + " - DONE");
}
}
log4j2.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configuration>
<Configuration status="WARN">
<Appenders>
<RollingFile name="RollingLogFile"
fileName="logs/rolling.log"
filePattern="logs/rolling-%d{yyyy-MM-dd-HH-mm}.log.gz"
createOnDemand="true">
<PatternLayout
pattern="%d{HH:mm:ss.SSS} %-5level - %msg%n" />
<Policies>
<TimeBasedTriggeringPolicy modulate="true"
interval="1" />
<OnStartupTriggeringPolicy />
</Policies>
<DefaultRolloverStrategy />
</RollingFile>
</Appenders>
<Loggers>
<Root level="error">
<AppenderRef ref="RollingLogFile" />
</Root>
</Loggers>
</Configuration>
My guess is that since the JVM hosting the application and the Log4j instance is not running at the time the rollover would happen, then the rollover does not get triggered.
Ultimately, I abandoned the use of the RollingFileAppender and went with a straight-up FileAppender:
<File name="RollingLogFile"
fileName=logs/rolling-${date:yyyy-MM-dd}.log"
createOnDemand="true">
<PatternLayout
pattern="%d{HH:mm:ss.SSS} %-5level - %msg%n" />
</File>
This works, but it has a few disadvantages:
I cannot monitor the simply named "rolling.log" (since it doesn't exist), but have to use the date-specific version of the filename.
I cannot make use of log4j's compression on rollover features.
I cannot make use of log4j's delete on rollover retention policies.
So, the question, restated, is: Using log4j, is it possible for an application executed in brief intervals to use a time-based log file rollover strategy in the same way that continuously running applications can?
Please, try the following configuration:
<RollingFile name="LogSpecial"
fileName="${sys:special.directory}/special.csv"
filePattern="${sys:special.directory}/special-%d{yyyy-MM-dd}.csv.gz"
createOnDemand="true">
<CsvParameterLayout />
<Policies>
<TimeBasedTriggeringPolicy modulate="true" interval="1" />
<OnStartupTriggeringPolicy />
</Policies>
<DefaultRolloverStrategy>
<Delete basePath="{sys:special.directory}">
<IfFileName glob="special-*.csv.gz" />
<IfLastModified age="60d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
You can also switch on log4j2.debug system property to debug this configuration and investigate what's wrong.
It turns out that log4j2 is capable of doing exactly what I was aiming for but part of my configuration, for unknown reasons, prevented it from working properly. The solution was to remove the createOnDemand="true" attribute from the RollingFile element.
After excluding that attribute, the following configuration works exactly as I would expect (rolling over at startup, at most every minute):
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configuration>
<Configuration status="WARN">
<Appenders>
<RollingFile name="RollingLogFile"
fileName="logs/rolling.log"
filePattern="logs/rolling-%d{yyyy-MM-dd-HH-mm}.log.gz">
<PatternLayout
pattern="%d{HH:mm:ss.SSS} %-5level - %msg%n" />
<TimeBasedTriggeringPolicy />
</RollingFile>
</Appenders>
<Loggers>
<Root level="error">
<AppenderRef ref="RollingLogFile" />
</Root>
</Loggers>
</Configuration>
Related
I am new to log4j2. Previously I am using log4j. The reason I am migrating into part 2 is for Asynchronous logging. After searching Internet I am able to write a configuration file that actually creates two log files "Errors.log" and "Messages.log". Now the problem is : I would be communicating with Servers that are kept far away from me. I wrote a client that communicates with the server and sends a request and in back the Server sends me a response. In any situation it takes at least 10 milli-seconds for the request to reach the Server and get back the response from it. But in my log files it is showing that the request sent to the Server and receive from the Server is at same time (Same milli-second). I am using the Asynchronous logging. Is this causing the wrong timestamp? or else the policies which I have used are creating these issues?
Below is my Log4j2 XML CONFIG file:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
<Appenders>
<File name="my_file_appender" fileName="LOG4j_LOGS/Errors.log" immediateFlush="false" append="false">
<PatternLayout>
<Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} - %msg%n</Pattern>
</PatternLayout>
</File>
<Async name="async_appender">
<AppenderRef ref="my_file_appender" />
</Async>
<!-- file appender -->
<RollingFile name="Error-log" fileName="LOG4j_LOGS/Messages.log"
filePattern="LOG4j_LOGS/Messages-%d{yyyy-MM-dd}.log">
<!-- log pattern -->
<PatternLayout>
<Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} - %msg%n</Pattern>
</PatternLayout>
<!-- set file size policy -->
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
<DefaultRolloverStrategy max="25"/>
</RollingFile>
</Appenders>
<Loggers>
<Logger name="Error-log" level="info" additivity="false">
<appender-ref ref="Error-log" level="debug"/>
</Logger>
<Root level="info" includeLocation="false">
<AppenderRef ref="async_appender"/>
</Root>
</Loggers>
</Configuration>
Can anyone please check my CONFIG file. All I want is to create two separate log files, one for storing info messages and other for storing errors. And this should create a new file every time I run my application and it should not delete the previous logs. The size of the logs can be anything. If the size has exceeded it should create a new file and write the data into it. No matter how many days I run the application the daily logs needs to be stored and the entire process has to be done in Asynchronously.
I am also using the below VM options for logging asynchronously :
-DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector
When logging is done asynchronously you log messages got onto a separate queue. They are timestamped at the moment of processing by background thread that writes logs to disk (since timestamp is part of you Appender pattern). So only the order is preserved. Timestamps may be slightly different.
See https://logging.apache.org/log4j/2.x/manual/async.html for more info.
We have WAR deployments running on Tomcat containers which are using log4j 2.5 for logging events. We have now amended the deployments' log4j2.xml configuration to have the log files roll over every 24 hours but, with this new configuration, the rollover of files are not taking place as we would expect.
Sample configuration:
<RollingFile name="file"
fileName="${sys:catalina.base}/logs/${web:contextPath}.log"
filePattern="${sys:catalina.base}/logs/${web:contextPath}-%d{dd-MMM-yyyy}.log"
append="true">
<PatternLayout pattern="%d{dd-MMM-yyyy HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" >
<header>LOG START DATE=${date:dd-MMM-yyyy HH:mm:ss.SSS} APP=${web:contextPath} TOMCAT=${env:HOSTNAME}:${env:CONNECTOR_PORT}${sys:line.separator}</header>
<footer>LOG END DATE=${date:dd-MMM-yyyy HH:mm:ss.SSS} APP=${web:contextPath} TOMCAT=${env:HOSTNAME}:${env:CONNECTOR_PORT}${sys:line.separator}</footer>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy/>
</Policies>
</RollingFile>
Any ideas why the rollover is not taking place?
NOTE: The same configuration but with a <CronTriggeringPolicy schedule="0 0 0 * * ?" /> instead of TimeBasedTriggeringPolicy does rollover but, in this case, the rolled over files get created with today's date in the filename and NOT yesterday's date.
NOTE2: We have other deployments with similar configuration that do rollover every 24 hours but those configurations have the filename hardcoded instead of using ${web:contextPath}. Could this lookup have something to do with why RollingFile might not work?
--- EDIT ---
UPDATE: We are able to get TimeBasedTriggeringPolicy to rollover files using above configuration when the Tomcat instance is running on Windows but NOT when the Tomcat instance is running on Linux.
There is nothing wrong with your configuration snippet as I get the desired behaviour of time based rolling.
To test, I changed the dd-MMM-yyyy to dd-MMM-yyyy-HH-mm and my log file rolls every minute.
It must be something else that is preventing you from achieving the desired behaviour.
My setup #1:
Log4j2 v2.8.2
Apache Tomcat 8.5.13
Windows 7 Enterprise SP1
My setup #2:
Log4j2 v2.5
Apache Tomcat 7.0.77
CentOS 7 (1611)
I have the following 3 JARs in WEB-INF/lib:
log4j-api-2.5.jar
log4j-core-2.5.jar
log4j-web-2.5.jar
Here is my complete log4j2.xml for your reference:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="DEBUG">
<Appenders>
<RollingFile name="RollingFileAppender"
fileName="${sys:catalina.base}/logs/${web:contextPath}.log"
filePattern="${sys:catalina.base}/logs/${web:contextPath}-%d{dd-MMM-yyyy-HH-mm}.log"
append="true">
<PatternLayout pattern="%d{dd-MMM-yyyy HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" >
<header>LOG START DATE=${date:dd-MMM-yyyy HH:mm:ss.SSS} APP=${web:contextPath} TOMCAT=${env:HOSTNAME}:${env:CONNECTOR_PORT}${sys:line.separator}</header>
<footer>LOG END DATE=${date:dd-MMM-yyyy HH:mm:ss.SSS} APP=${web:contextPath} TOMCAT=${env:HOSTNAME}:${env:CONNECTOR_PORT}${sys:line.separator}</footer>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy/>
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Logger name="root" level="debug" additivity="false">
<appender-ref ref="RollingFileAppender" level="debug"/>
</Logger>
<Root level="debug" additivity="false">
<AppenderRef ref="RollingFileAppender"/>
</Root>
</Loggers>
</Configuration>
I am working with Java Web App {Servlet + JSP + Tomcat} and having one log management for the info and error logs.
Now I want to add some actual data log functionality with different path name and property values. How can I achieve this in Java ?
Existing Log Property
<Configuration status="ALL" name="Logger" packages="">
<Properties>
<Property name="baseDir">D:\\Log\\</Property>
</Properties>
<Appenders>
<RollingFile name="RollingFile" append="true" fileName="${baseDir}/Log.log" filePattern="${baseDir}/$${date:yyyy-MM}/app-%d{yyyy-MM-dd-HH}-%i.log.gz">
<PatternLayout pattern="%d{dd/MM/YYYY HH:mm:ss.SSS} [%t] %-5level %logger{36} %msg %n"/>
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="30 MB" />
</Policies>
<!-- Create 5 archives for the same date if the log file size exceeds 30MB -->
<!-- Deletes the logs which are older than 60 days from the current date -->
<DefaultRolloverStrategy max="5">
<Delete basePath="${baseDir}" maxDepth="2">
<IfFileName glob="*/app-*.log.gz">
<IfLastModified age="60d" />
</IfFileName>
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
</Appenders>
<Loggers>
<Root level="ALL">
<AppenderRef ref="RollingFile"/>
</Root>
</Loggers>
</Configuration>
Java
public class Logj {
static Logger objLogger = LogManager.getLogger(Logj.class);
public static void doLog(Exception objException){
objLogger.error("StackTrace - "+objException.fillInStackTrace());
}
public static void doActivityLog(String strActivityMessage){
objLogger.info(" Activity - "+strActivityMessage);
}
}
I need same functionality to be execute for the new requirement
Will it support to add multiple Properties to add different directory path and adding multiple Appenders with different names ?
Thank in advance!!
You need to add one more property with different path in properties tag,
along with add one more rolling file appender and refer that appender in the root level, that will create two logs.
I am trying to configure a log4j2 config to route messages to different logfiles for a multithreaded program via threadname.
Here is what I have, so far (relevant to log4j2 config):
|-/src/main/java/log4j2/plugins
|-- ThreadLookup.java
|-/src/main/resources
|-- log4j2.xml
ThreadLookup.java:
package log4j2.plugins;
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.lookup.StrLookup;
#Plugin(name="threadLookup", category=StrLookup.CATEGORY)
public class ThreadLookup implements StrLookup {
#Override
public String lookup(String key) {
return Thread.currentThread().getName();
}
#Override
public String lookup(LogEvent event, String key) {
// Check event first:
if (event.getThreadName() != null) {
return event.getThreadName();
}
// Fallback to key if event doesn't define a threadName:
return this.lookup(key);
}
}
Log4j2.xml (It is my understanding that the packages attribute of Configuration should read in ThreadLookup.java and based on the annotation create a new threadLookup prefix to let me call lookup(String key) with whatever value I want to -- in this case I am not using a specific value because this class will only do a threadName lookup):
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="error" strict="true" schema="Log4J-V2.0.xsd"
packages="log4j2.plugins">
<Properties>
<Property name="logMsgPattern">%date{yyyy/MM/dd HH:mm:ss.SSS} %-5level ${sys:pid}[%thread] %class %method:%line - %message%n</Property>
</Properties>
<Appenders>
<Console name="console" target="SYSTEM_OUT" >
<PatternLayout pattern="${logMsgPattern}" />
</Console>
<Routing name="routing">
<Routes pattern="$${threadLookup:threadName}">
<Route>
<RollingFile name="RollingFile-${threadLookup:threadName}"
fileName="${sys:log4j.dir}/thread-${threadLookup:threadName}.log"
filePattern="${sys:log4j.dir}/thread-${threadLookup:threadName}-%i.log.gz">
<PatternLayout pattern="${logMsgPattern}"/>
<Policies>
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
<!-- Config for other appenders snipped -->
</Appenders>
<Loggers>
<!-- Config for other loggers snipped -->
<Root level="${sys:log4j.console.threshold}">
<AppenderRef ref="rootOut" level="trace" />
<AppenderRef ref="rootErr" level="error" />
<AppenderRef ref="console" level="${sys:log4j.console.threshold}" />
<AppenderRef ref="routing" level="trace" />
</Root>
</Loggers>
</Configuration>
However, when I launch my app, it just creates an additional file called thread-${threadLookup (no extension) in my log directory. It also never hits any breakpoints in ThreadLookup.java.
How can I register the plugin with log4j2 (I was using version 2.2, I also tried 2.3)? Note, I am using a spring-framework 4.1.7 project, if that helps at all; I use maven for the project as well, but I am only using it to resolve dependencies, I build the project via ant script.
UPDATE
When I build the script via ant, I do actually get a Log4j2Plugins.dat that shows up in my classpath (-cp resources:bin), but it doesn't seem to effect the outcome of the logs that are generated on the server:
$ find bin/META-INF/ -type f
bin/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
$ cat bin/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
lookup
threadlookupog4j2.plugins.ThreadLookup
threadLookup
$ vi bin/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
^A^Flookup^A^Lthreadlookup^[log4j2.plugins.ThreadLookup^LthreadLookup
$ find logs -type f -name "thread-*"
logs/thread-${threadLookup:threadName}.log
logs/thread-${threadLookup:threadName}-1.log.gz</pre>
Thanks
I ended up finding out that the issue was that my Plugin name cannot be camelCased.
I was debugging through PluginManager.java (Log4j2 - 2.3), on line 169 in private static void mergeByName(final Map<String, PluginType<?>> newPlugins, final List<PluginType<?>> plugins), and I saw that I had the following properties to go into newPlugins.put(key, pluginType);
key: threadlookup,
pluginType: PluginType [pluginClass=class log4j2.plugins.ThreadLookup, key=threadlookup, elementName=threadLookup, isObjectPrintable=false, isDeferChildren==false, category=Lookup]
After seeing that, I modified my Routing appender in my log4j2.xml config to the following (without needing to change the annotation in my Plugin class that implemented StrLookup) and it worked:
<Routing name="routing">
<Routes pattern="$${threadlookup:threadName}">
<Route>
<RollingFile name="RollingFile-${threadlookup:threadName}"
fileName="${sys:log4j.dir}/thread-${threadlookup:threadName}.log"
filePattern="${sys:log4j.dir}/thread-${threadlookup:threadName}-%i.log.gz">
<PatternLayout pattern="${logMsgPattern}"/>
<Policies>
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
Hopefully this can help others out, as I had to spend a few days to figure this out and I didn't find this in any of the documentation or questions I was reviewing for Log4j2.
Thanks!
I'm using Log4J 2.0 to create logs for a project that I'm doing. The logs are small and I have a requirement to maintain them for 3 months. I'd like to have the current month's log with 3 archives (each containing a month's worth of logs).
The problem that I need help with is configuring log4j to rotate the logs at the beginning of the month (or the end of the month).
Pretty much every thing that I've found researching this problem is for log4j 1.x and talks about a datePattern parameter that doesn't appear to exist in 2.0.
Here's my log4j2.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<configuration status="warn" name="NKMS" packages="">
<appenders>
<FastRollingFile name="LogFile" fileName="logs/Tier2HttpServer.log" filePattern="logs/app-%d{yyyy-MM-dd}.log.gz">
<ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
<PatternLayout pattern="%d %p %c{1.} [%t] %m%n"/>
<Policies>
<TimeBasedTriggeringPolicy/>
<SizeBasedTriggeringPolicy size="250 MB"/>
</Policies>
<DefaultRolloverStrategy max="4"/>
</FastRollingFile>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%d %-5p [%t] %C{2} (%F:%L) - %m%n"/>
</Console>
</appenders>
<loggers>
<logger name="mil.navy.nrl.itd.xml_filter" level="trace"/>
<root level="trace">
<appender-ref ref="STDOUT"/>
<appender-ref ref="LogFile"/>
</root>
</loggers>
</configuration>
I'm writing INFO and above to the log file and debug to the console (for now). The files are written to just fine, but they appear to rollover daily (which appears to be the default).
I've tried changing the FastRollingFile:filePattern to "'.'yyyy-MM" but that causes weird things to happen (only a single entry is written to file and an archive is immediately created).
I downloaded the source for log4j-2.0-beta8 and the PatternProcessor parses a RolloverFrequency that contains the enum RolloverFrequency.MONTHLY, but there again, I can't figure out how to implement / use it.
As always, any assistance or advice that you can provide would be GREATLY APPRECIATED!
-Ace
You may have found a bug. I would expect the filePattern of "logs/app-%d{yyyy-MM}.log.gz" to give you what you're looking for.
To clarify my understanding of the problem: The initial log event immediately triggers a rollover (creating an archive file). Instead, it should collect log events into the log file and not roll over until the end/beginning of the month. Is that description correct? Is there any other issue in addition to this initial unnecessary rollover?
Could I ask you to raise a JIRA ticket for this? https://issues.apache.org/jira/browse/LOG4J2