java 1.8
maven 3.5
In pom.xml
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
</dependency>
In my Maven java project I want to generate new log file every new day (midnight).
So I use RollingFile and cron 0 0 0 * * ?. Here my log4j2.xml
<Properties>
<Property name="baseDir">logs</Property>
<Property name="patterLayout">%d{[dd.MM.yyyy HH:mm:ss.SSS]} %l %p:%n %m%n</Property>
</Properties>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${patterLayout}"/>-
</Console>
<RollingFile name="RollingFile" fileName="${baseDir}/app.log"
filePattern="${baseDir}/$${date:yyyy-MM}/app-%d{yyyy-MM-dd}.log.gz">
<PatternLayout pattern="${patterLayout}"/>-
<CronTriggeringPolicy schedule="0 0 0 * * ?"/>
<DefaultRolloverStrategy>
<Delete basePath="${baseDir}" maxDepth="2">
<IfFileName glob="*/app-*.log.gz"/>
<IfLastModified age="60d"/>
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
</Appenders>
<Loggers>
<Root level="trace">
<AppenderRef ref="Console"/>
<AppenderRef ref="RollingFile"/>
</Root>
</Loggers>
But all logs print in ONE file app.log.
Here result:
[27.11.2021 23:59:05.235] com.myproject.xls2metro2.App.main(App.java:18) TRACE:
JDK: 1.8.0_202
[27.11.2021 23:59:05.238] com.myproject.xls2metro2.App.main(App.java:33) WARN:
File path is mandatory
[28.11.2021 00:01:13.347] com.myproject.xls2metro2.App.main(App.java:18) TRACE:
JDK: 1.8.0_202
[28.11.2021 00:01:13.351] com.myproject.xls2metro2.App.main(App.java:33) WARN:
File path is mandatory
At first, verify that your original configuration doesn't have this hyphen character (probably you have added it accidentally while writing a question)
<PatternLayout pattern="${patterLayout}"/>-
Also, make sure that your code is wrapped inside the Configuration tag (I assume that you have omitted it for the sake of brevity):
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" name="MyApp" packages="">
... your code here
</Configuration>
At second, your config looks exactly the same as in the official documentation examples, so it should be fine.
But in the documentation stated that:
The CronTriggeringPolicy triggers rollover based on a cron expression.
This policy is controlled by a timer and is asynchronous to processing
log events, so it is possible that log events from the previous or
next time period may appear at the beginning or end of the log file.
The filePattern attribute of the Appender should contain a timestamp
otherwise the target file will be overwritten on each rollover.
So probably, this is your case. Just wait a little longer to verify, that the next logs will be written to a separate file.
Related
I have a spring boot application and using log4j2 to generate console and persists logs in a centos linux.
I wanted to maintain only 5mb of log files in archive.
But the problem is, my archived log files are 5mb in total. but my main console log which is saving in the main log file i.e wc-notification.out is going beyond 1mb.
so my disk gets full and it causes an issue.
The brute force method solution is:
whenever restarting(hard stop and start) my spring boot application, the log is cleared in from wc-notification.out.
my log4j2 configuration xml file:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO" monitorInterval="30">
<Properties>
<Property name="LOG_PATTERN">
[ %d{yyyy-MMM-dd HH:mm:ss a} ] - [%t] %-5level %logger{36} - %msg%n
</Property>
</Properties>
<Appenders>
<Console name="ConsoleAppender" target="SYSTEM_OUT" follow="true">
<PatternLayout pattern="${LOG_PATTERN}"/>
</Console>
<RollingFile name="FileAppender" fileName="/home/ec2-user/apps/wc-notification-service/wc-notification.out"
filePattern="/home/ec2-user/apps/wc-notification-service/archives_test/wc-notification.out-%d{yyyy-MM-dd}-%i">
<PatternLayout>
<Pattern>${LOG_PATTERN}</Pattern>
</PatternLayout>
<Policies>
<SizeBasedTriggeringPolicy size="1MB" />
</Policies>
<DefaultRolloverStrategy>
<Delete basePath="logs" maxDepth="1">
<IfFileName glob="wc-notification.out-*.log" />
<IfLastModified age="1m" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
</Appenders>
<Loggers>
<Root level="info">
<!--<AppenderRef ref="ConsoleAppender" /> -->
<AppenderRef ref="FileAppender" />
</Root>
</Loggers>
</Configuration>
somehow, the files are in range of 1mb, and the roll strategy is working, it is deleting the file
but, my disk space is still occupied with the space. what can be the reason?
As you are providing your own log4j2.xml configuration file, you are overwriting the Spring Boot default logging configuration, and it is safe to assume that it will be configuration used by Log4j2.
Your configuration is almost correct. If you want to achieve the desired behavior, I would suggest you the following changes:
Be aware that you are pointing your Delete action basePath to the wrong location, it should point to the directory in which your logs are stored.
The IfFileName glob pattern is wrong as well, it should match your logs filenames.
Finally, your are using the IfLastModified condition. As its name implies, this condition has to do with the last modification date of the logs files, not with their size. Please, consider to use instead IfAccumulatedFileSize (or maybe IfAccumulatedFileCount). See the relevant documentation.
For these reasons, your logs are not being deleted correctly and the disk space occupied is greater than the desired amount. Without deletion, your log files are being rotated every 1 MB as configured by your SizeBasedTriggeringPolicy, and will keep until the value of the max attribute of DefaultRolloverStrategy, 7 by default, is reached, and always, plus the amount of your current log file.
In summary, please, try a configuration like this:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO" monitorInterval="30">
<Properties>
<Property name="LOG_PATTERN">
[ %d{yyyy-MMM-dd HH:mm:ss a} ] - [%t] %-5level %logger{36} - %msg%n
</Property>
</Properties>
<Appenders>
<Console name="ConsoleAppender" target="SYSTEM_OUT" follow="true">
<PatternLayout pattern="${LOG_PATTERN}"/>
</Console>
<RollingFile name="FileAppender" fileName="/home/ec2-user/apps/wc-notification-service/wc-notification.out"
filePattern="/home/ec2-user/apps/wc-notification-service/wc-notification.out-%d{yyyy-MM-dd}-%i">
<PatternLayout>
<Pattern>${LOG_PATTERN}</Pattern>
</PatternLayout>
<Policies>
<SizeBasedTriggeringPolicy size="1MB" />
</Policies>
<DefaultRolloverStrategy>
<Delete basePath="/home/ec2-user/apps/wc-notification-service">
<IfFileName glob="wc-notification.out-*" />
<IfAccumulatedFileSize exceeds="5 MB" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
</Appenders>
<Loggers>
<Root level="info">
<!--<AppenderRef ref="ConsoleAppender" /> -->
<AppenderRef ref="FileAppender" />
</Root>
</Loggers>
</Configuration>
The solution relies in storing all your logs in the same location. It will affect your RollingFile filePattern attribute.
Please, be careful with the delete action, it can delete not only your log files, but all that matches your glob pattern.
In addition, although maybe irrelevant fo your use case, be aware that if the filePattern of your archive files ends with ".gz", ".zip", ".bz2", among others, the resulting archive will be compressed using the compression scheme that matches the suffix, which can allow you to store more archives for the same space if required.
For your comments, it seems you encountered a problem when using large file sizes: please, see this bug, I think that clearly describes your problem.
My best advice will be to reduce the size of the logs files to one that it is properly working, or try a more recent version of the library.
I am aware that you are using Spring Boot to manage your dependencies:
please, verify your maven tree and see the actual version library you are using and change it if necessary.
You need to leverage the DeleteAction of the RollingFileAppender. I would recommend taking a look at the documentation as there are a lot of good examples.
The reason for your behavior is as follows: look at your parameters
appender.gateway.policies.size.size=1MB
appender.gateway.strategy.type = DefaultRolloverStrategy
appender.gateway.strategy.max = 5
What you are asking here is log file should be allowed to grow up to 1M. Once it reaches 1M size it is copied to file logfile.log-1 and a new file logfile.log is created. As the files get produced you required that you want to keep only 5 last files. The older files get deleted automatically. So it looks like the behavior is exactly as you configured. What you can do is to configure to keep more than 5 files back and possibly in a different folder where you have sufficient space.
Just for information to people who are facing similar issue, i just changing my logging mechanism to logback. it works like an charm and have no issue.
the reference link which i used: spring boot with logback
This question already has an answer here:
ParserConfigurationException while using log4j2 with junit
(1 answer)
Closed 10 months ago.
Switched from log4j1 to log4j2 (1.2.17 to 2.12).
I have 5 projects that use the new log4j2. I removed the old jar for log4j1, and all the references and paths now refer to the new added jar for log4j2.
I am using log4j2.xml to set the configuration for log4j2.
4 out of 5 other projects are running perfectly with the new log4j2, logging to the right file, with the correct pattern described in the log4j2.xml file.
One project - let's call it "ProjectX" - is returning an error when trying to access the same logger:
ERROR StatusLogger Caught javax.xml.parsers.ParserConfigurationException
The following are used when running the project:
jdk1.7.0_79
oracle.jdbc_11.1.1\ojdbc6dms.jar;
oracle.nlsrtl_11.1.0\orai18n.jar;
oracle.odl_11.1.1\ojdl.jar;
oracle.dms_11.1.1\dms.jar;
oracle.xdk_11.1.0\xmlparserv2.jar;
oracle.xdk_11.1.0\xml.jar;
com.oracle.toplink_1.0.0.0_11-1-1-5-0.jar;
org.eclipse.persistence_1.1.0.0_2-1.jar;
com.bea.core.antlr.runtime_2.7.7.jar;
javax.persistence_1.0.0.0_2-0-0.jar;
wlserver_10.3\server\lib\weblogic.jar;
commons-io-2.4.jar;
log4j-api-2.12.0.jar;
log4j-core-2.12.0.jar;
wlserver_10.3\server\lib\weblogic.jar
The log4j2.xml I'm using:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Properties>
<!-- Path to the log files -->
<Property name="log-path">C:/logs</Property>
</Properties>
<Appenders>
<!-- Logger that prints to the Console window -->
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout>
<pattern>%-6p%d{DATE} - %C{1}.%M:%L - %m%n</pattern>
</PatternLayout>
</Console>
<!-- Logger that prints to the log file specified in the fileName -->
<RollingFile name="LoggerFile" fileName="${log-path}/App.log" filePattern="${log-path}/App-%d{yyyy-MM-dd}-%i.log" immediateFlush="true" append="true">
<PatternLayout>
<pattern>%-6p%d{DATE} - %C{1}.%M:%L - %m%n</pattern>
</PatternLayout>
<!-- The maximum size of a log file -->
<Policies>
<SizeBasedTriggeringPolicy size="1 MB" />
</Policies>
<!-- Number of log files before starting to roll over -->
<DefaultRolloverStrategy max="10"/>
</RollingFile>
</Appenders>
<Loggers>
<Logger name="RollingFileLogger" level="DEBUG">
<AppenderRef ref="LoggerFile"/>
</Logger>
<Root level="DEBUG">
<AppenderRef ref="LoggerFile"/>
</Root>
</Loggers>
</Configuration>
Any idea what causes the error ERROR StatusLogger Caught javax.xml.parsers.ParserConfigurationException?
Could it be oracle.xdk_11.1.0\xmlparserv2.jar or xml.jar?
I tried xmlparserv2.jar version 12.2, and it fixed the error.
To download version 12.2 of xmpparserv2.jar:
https://maven.oracle.com/com/oracle/jdbc/xmlparserv2/12.2.0.1/xmlparserv2-12.2.0.1.jar
I upgraded the xmlparser jar from version 11.1 to version 12.2: xmlparserv2-12.2.0.1.jar.
I added it to the project and it fixed the problem.
I am working with Java Web App {Servlet + JSP + Tomcat} and having one log management for the info and error logs.
Now I want to add some actual data log functionality with different path name and property values. How can I achieve this in Java ?
Existing Log Property
<Configuration status="ALL" name="Logger" packages="">
<Properties>
<Property name="baseDir">D:\\Log\\</Property>
</Properties>
<Appenders>
<RollingFile name="RollingFile" append="true" fileName="${baseDir}/Log.log" filePattern="${baseDir}/$${date:yyyy-MM}/app-%d{yyyy-MM-dd-HH}-%i.log.gz">
<PatternLayout pattern="%d{dd/MM/YYYY HH:mm:ss.SSS} [%t] %-5level %logger{36} %msg %n"/>
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="30 MB" />
</Policies>
<!-- Create 5 archives for the same date if the log file size exceeds 30MB -->
<!-- Deletes the logs which are older than 60 days from the current date -->
<DefaultRolloverStrategy max="5">
<Delete basePath="${baseDir}" maxDepth="2">
<IfFileName glob="*/app-*.log.gz">
<IfLastModified age="60d" />
</IfFileName>
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
</Appenders>
<Loggers>
<Root level="ALL">
<AppenderRef ref="RollingFile"/>
</Root>
</Loggers>
</Configuration>
Java
public class Logj {
static Logger objLogger = LogManager.getLogger(Logj.class);
public static void doLog(Exception objException){
objLogger.error("StackTrace - "+objException.fillInStackTrace());
}
public static void doActivityLog(String strActivityMessage){
objLogger.info(" Activity - "+strActivityMessage);
}
}
I need same functionality to be execute for the new requirement
Will it support to add multiple Properties to add different directory path and adding multiple Appenders with different names ?
Thank in advance!!
You need to add one more property with different path in properties tag,
along with add one more rolling file appender and refer that appender in the root level, that will create two logs.
I am trying to configure a log4j2 config to route messages to different logfiles for a multithreaded program via threadname.
Here is what I have, so far (relevant to log4j2 config):
|-/src/main/java/log4j2/plugins
|-- ThreadLookup.java
|-/src/main/resources
|-- log4j2.xml
ThreadLookup.java:
package log4j2.plugins;
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.lookup.StrLookup;
#Plugin(name="threadLookup", category=StrLookup.CATEGORY)
public class ThreadLookup implements StrLookup {
#Override
public String lookup(String key) {
return Thread.currentThread().getName();
}
#Override
public String lookup(LogEvent event, String key) {
// Check event first:
if (event.getThreadName() != null) {
return event.getThreadName();
}
// Fallback to key if event doesn't define a threadName:
return this.lookup(key);
}
}
Log4j2.xml (It is my understanding that the packages attribute of Configuration should read in ThreadLookup.java and based on the annotation create a new threadLookup prefix to let me call lookup(String key) with whatever value I want to -- in this case I am not using a specific value because this class will only do a threadName lookup):
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="error" strict="true" schema="Log4J-V2.0.xsd"
packages="log4j2.plugins">
<Properties>
<Property name="logMsgPattern">%date{yyyy/MM/dd HH:mm:ss.SSS} %-5level ${sys:pid}[%thread] %class %method:%line - %message%n</Property>
</Properties>
<Appenders>
<Console name="console" target="SYSTEM_OUT" >
<PatternLayout pattern="${logMsgPattern}" />
</Console>
<Routing name="routing">
<Routes pattern="$${threadLookup:threadName}">
<Route>
<RollingFile name="RollingFile-${threadLookup:threadName}"
fileName="${sys:log4j.dir}/thread-${threadLookup:threadName}.log"
filePattern="${sys:log4j.dir}/thread-${threadLookup:threadName}-%i.log.gz">
<PatternLayout pattern="${logMsgPattern}"/>
<Policies>
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
<!-- Config for other appenders snipped -->
</Appenders>
<Loggers>
<!-- Config for other loggers snipped -->
<Root level="${sys:log4j.console.threshold}">
<AppenderRef ref="rootOut" level="trace" />
<AppenderRef ref="rootErr" level="error" />
<AppenderRef ref="console" level="${sys:log4j.console.threshold}" />
<AppenderRef ref="routing" level="trace" />
</Root>
</Loggers>
</Configuration>
However, when I launch my app, it just creates an additional file called thread-${threadLookup (no extension) in my log directory. It also never hits any breakpoints in ThreadLookup.java.
How can I register the plugin with log4j2 (I was using version 2.2, I also tried 2.3)? Note, I am using a spring-framework 4.1.7 project, if that helps at all; I use maven for the project as well, but I am only using it to resolve dependencies, I build the project via ant script.
UPDATE
When I build the script via ant, I do actually get a Log4j2Plugins.dat that shows up in my classpath (-cp resources:bin), but it doesn't seem to effect the outcome of the logs that are generated on the server:
$ find bin/META-INF/ -type f
bin/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
$ cat bin/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
lookup
threadlookupog4j2.plugins.ThreadLookup
threadLookup
$ vi bin/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
^A^Flookup^A^Lthreadlookup^[log4j2.plugins.ThreadLookup^LthreadLookup
$ find logs -type f -name "thread-*"
logs/thread-${threadLookup:threadName}.log
logs/thread-${threadLookup:threadName}-1.log.gz</pre>
Thanks
I ended up finding out that the issue was that my Plugin name cannot be camelCased.
I was debugging through PluginManager.java (Log4j2 - 2.3), on line 169 in private static void mergeByName(final Map<String, PluginType<?>> newPlugins, final List<PluginType<?>> plugins), and I saw that I had the following properties to go into newPlugins.put(key, pluginType);
key: threadlookup,
pluginType: PluginType [pluginClass=class log4j2.plugins.ThreadLookup, key=threadlookup, elementName=threadLookup, isObjectPrintable=false, isDeferChildren==false, category=Lookup]
After seeing that, I modified my Routing appender in my log4j2.xml config to the following (without needing to change the annotation in my Plugin class that implemented StrLookup) and it worked:
<Routing name="routing">
<Routes pattern="$${threadlookup:threadName}">
<Route>
<RollingFile name="RollingFile-${threadlookup:threadName}"
fileName="${sys:log4j.dir}/thread-${threadlookup:threadName}.log"
filePattern="${sys:log4j.dir}/thread-${threadlookup:threadName}-%i.log.gz">
<PatternLayout pattern="${logMsgPattern}"/>
<Policies>
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
Hopefully this can help others out, as I had to spend a few days to figure this out and I didn't find this in any of the documentation or questions I was reviewing for Log4j2.
Thanks!
1. I am trying to add log message at beginning of file so that I can see latest message first on log file.
2. I want to add month and year after log file but I am not getting that in current active file. eg - test_2015-04-22.log
property file is given below -
log4j.appender.APP=org.apache.log4j.DailyRollingFileAppender
log4j.appender.APP.File=${catalina.base}/logs/test.log
log4j.appender.APP.Append=true
log4j.appender.APP.Encoding=UTF-8
log4j.appender.APP.DatePattern='.'yyyy-MM
log4j.appender.APP.layout = org.apache.log4j.PatternLayout
log4j.appender.APP.layout.ConversionPattern =%d{yyyy-MM-dd HH:mm} - %m%n
log4j.appender.APP.filePattern =Test_%d{yyyy-MM-dd}.log
Your question shows a log4j-1.2 configuration, but since the question also has a log4j2 tag, I feel free to answer this by showing you how to accomplish this with log4j2.
In log4j2, you can declare a property that formats the date, then use this property to configure your appender file name.
Also, you can use the header attribute of the pattern layout to set a header that is output at the beginning of a file. For RollingFileAppender this header will be output on every rollover.
Use one of the lookups built-in to log4j2 to dynamically change the output of your header at rollover time.
Example:
<Configuration status="WARN"><!-- use TRACE to troubleshoot your config if needed-->
<Properties>
<property name="yyyyMMdd">${date:yyyyMMdd}</property>
</Properties>
<Appenders>
<RollingFile name="Application"
fileName="${sys:catalina.base}/logs/test${sys:yyyyMMdd}.log"
filePattern="logs/$${date:yyyy-MM}/app-%d{MM-dd-yyyy}-%i.log.gz">
<PatternLayout header="File: ${main:--file}">
<Pattern>%d{yyyy-MM-dd HH:mm} - %m%n</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy />
</Policies>
</RollingFile>
<Appenders>
<Loggers>
<root level="trace">
<AppenderRef ref="Application" />
</root>
</Loggers>
</Configuration>
it's not possible , think of the file as a stack which respects LAST In FIRST Out which makes perfect sense when debugging to see the last message at the end.
The log for today will have the name test.log but tomorrow it will be renamed to Test_2015-04-22.log