RollingFileAppender that also deletes files older than some date - java

Do any of the popular Java logging frameworks support a rolling file appender, that I can configure to rollover daily, and also delete any log file that is over some number of days old? I know I could use a rolling file appender and a cron, but was wondering if anyone knew of an appender that can do both.

Logback's classic RollingFileAppender provides this and more. An example configuration from the manual (http://logback.qos.ch/manual/appenders.html#onRollingPolicies)
<configuration>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>logFile.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>logFile.%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- keep 30 days' worth of history -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="FILE" />
</root>
</configuration>
This provides daily rollover and 30 days of history. Place this in a file called logback.xml, or logback-test.xml for test trees, and place it in the classpath.

After working a while on Log4j, I read that Logback was developed and designed to be the successor of Log4j, and I have to say, Logback is wide stepper!
For example, to configure, this rolling file Appender, and zip the older logs, and have a maximum of 30, on Log4J you have to perform some code changes (please review my previews answer for more details), but on logback, everything is done on the config file like this:
<configuration>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>log/logFile.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>log/logFile.%d{yyyy-MM-dd}.log.zip</fileNamePattern>
<!-- keep 30 days' worth of history -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>[%-5level]: [%logger{35}] - %msg%n</pattern>
</encoder>
</appender>
<root level="trace">
<appender-ref ref="FILE" />
</root>
</configuration>
And that's it, you will have the current log unzipped, and the older days, zip, one zip file by each day.
Wide more easier than log4j!
If you want to play with more config options, just check this link.

Indeed, if you are using Log4J you can use this appender:
http://blog.kimb3r.com/2008/07/improving-log4j-dailyrollingfileappende.html
It's very, very, VERY easy to use, just download the whole class(yes, the code of the class in the above link), include it in your project (wherever you want, you can change the package to match inside your project), and then configure the log4j.properties file like this(this file has to be in your classpath, for example in the src/main/resources folder):
# Define the root logger with appender file
log = log
log4j.rootLogger = TRACE, FILE
# Define the logical path where you put the class you downloaded from "blog.kimb3r.com" link (above)
log4j.appender.FILE=com.yourapp.yourpackage.log.CustodianDailyRollingFileAppender
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.conversionPattern=%m%n
log4j.appender.FILE.File=${log}/yourlogfile.log
log4j.appender.FILE.DatePattern='.'yyyy-MM-dd
# How many files you want to keep?, in this case I'm having just 15 days of files, one file per day:
log4j.appender.FILE.MaxNumberOfDays=15
# If True, the older files will be compressed into a zip file (*which will save you a lot of space on the server*)
log4j.appender.FILE.CompressBackups=true
and besides this just add the dependency to log4j into your pom like this:
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
And that's it!

Why not look at this might help?
You can probably set maximum nuber of days to retain the logs.
log4j.rootLogger=INFO, FILE
log4j.appender.FILE=ca.justtechnologies.utils.logging.CustodianDailyRollingFileAppender
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.ConversionPattern=%d{MMM dd yyyy
HH:mm:ss,SSS} [%t] %-5p %l - %m%n
log4j.appender.FILE.File=/var/log/web-apps/Dashboard.log
log4j.appender.FILE.DatePattern='.'yyyy-MM-dd
log4j.appender.FILE.MaxNumberOfDays=14
log4j.appender.FILE.CompressBackups=true

Related

how to use slf4j with multiple class in java spring?

I want to write log for my process from multi file java in 1 file log in java spring.
This is my logback.xml file:
<configuration>
<property name="HOME_LOG" value="logs"/>
<appender name="001" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${HOME_LOG}/001/application.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${HOME_LOG}/001.%d{yyyy-MM-dd ss}.%i.log.gz</fileNamePattern>
<!-- each archived file, size max 10MB -->
<maxFileSize>10MB</maxFileSize>
<!-- total size of all archive files, if total size > 20GB, it will delete old archived file -->
<totalSizeCap>10GB</totalSizeCap>
<!-- 60 days to keep -->
<maxHistory>60</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%d %p %c{1.} [%t] %m%n</pattern>
</encoder>
</appender>
<logger name="001" level="INFO">
<appender-ref ref="001"/>
</logger>
Now I want to write log from 2 file java is : demo1.java and demo2.java. This is code java :
file demo1.java
private final static Logger log001 = LoggerFactory.getLogger("001");
log001.info("I am java 1");
file demo2.java
private final static Logger log001 = LoggerFactory.getLogger("001");
log001.info("I am java 2");
I want to my log file can show 2 text : 'I am java 1' and 'I am java 2'. Now it is writing override.
The reason your log file is overwritten on each run, is due to the way you configured the fileNamePattern in the rollingPolicy, i.e your application.log is created anew on each run since the file pattern granularity is in the seconds {yyyy-MM-dd ss}
Quick fix: change
<fileNamePattern>${HOME_LOG}/001.%d{yyyy-MM-dd ss}.%i.log.gz</fileNamePattern>
to
<fileNamePattern>${HOME_LOG}/001.%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>

Logback TimeBasedRollingPolicy with spring boot

I wrote a TimeBasedRollingPolicy logback.xml, and while the log file is created succesfully, it seems to be reading it from the application.properties instead. Here is my setup:
logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
<appender name="ROLLING-FILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<encoder>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
<file>${LOG_FILE}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>${LOG_FILE}-%d{yyyy-MM-dd}.log</fileNamePattern>
</rollingPolicy>
</appender>
<root level="INFO">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="ROLLING-FILE"/>
</root>
</configuration>
application.properties
logging.path=/path/to/log/folder/
logging.file=${logging.path}myLog
logging.pattern.file=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
When I run my application, the log is successfully saved in the correct path, but only as myLog. I would want it to be appended with the date (as in the logback.xml). Note I do want to keep taking the logging.path and logging.file from the application.properties because I have multiple depending on the environment.
Thanks
1) You need to set a logging.file or logging.path property, not both.
So logging.file=/path/to/log/folder/myLog should be enough to get a myLog log file in the specified path.
The spring boot documentation mentions that.
2) This is the format pattern for the log rolling, not for the current log :
<fileNamePattern>${LOG_FILE}-%d{yyyy-MM-dd}.log</fileNamePattern>
You get this format automatically for the current log file when it is rolled/archived because the time limit defined by the pattern is reached. In your case, it means everyday.

Logback programmatically set to log to a gzip file

I'm using Logback and I want to log some strings to a log file and let it gzip the file when I'm done.
I was following the example of:
Programmatically configure LogBack appender
But haven't figured out how to, when I finish logging, tell it to gzip the file.
All the examples show to use fileNamePattern. The examples I've seen show to to define this in logback.xml, but I'm trying to do this by code.
Would appreciate some pointers / examples for this :)
<!-- Time and Size based: Roll every day and split big file in smaller peaces -->
<appender name="ROOT" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_HOME}/root.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_HOME}/root-%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>10</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%date %-5level [%thread] - %mdc{loginName} - [%logger]- %msg%n</pattern>
</encoder>
</appender>
Notice the ".gz": this indicates that the logfile will be compressed. Replace this with .zip to use a zip-file.
There are some limitations, but is basically the easiest flow. The docs state.
Just like FixedWindowRollingPolicy, TimeBasedRollingPolicy supports automatic file compression. This feature is enabled if the value of the fileNamePattern option ends with .gz or .zip.

Logback-classic in Jboss and Spring

I'm working on a project where I thought I would try using logback-classic for debugging and log rotation. I'm using this in a Maven context for building and creating a .war file to be deployed in JBoss 7.1 Application Server.
I've placed a logback.xml file in the resources folder in the code and a logback-test.xml in test/resources.
The active jar I'm using is the SLF4J to print the actual debugging.
public class MyClass extends MultiActionController {
private Logger logger = LoggerFactory.getLogger(MyClass.class);
public MyClass() {
logger.debug("hello");
}
}
When I run a JUnit test on the code in Maven itself it works, but after building a .war file i don't get any debugging in STDOUT nor can I find a file created.
I know that I've removed the actual logging from STDOUT in the config file, but where is the logging going...
the logback.xml:
<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>DEBUG</level>
</filter>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<FileNamePattern>logs/myproject.%d{yyyy-MM-dd}.log</FileNamePattern>
<MaxHistory>30</MaxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%d{yyMMdd HH:mm:ss.SSS} [%-5.5level] [%-25.25logger] %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="FILE" />
</root>
</configuration>
The folder "logs" do I need to explicit create it in the JBoss instance or need to refer relative to it for this to work? Or is there something I missed? Do I need to put the logback.xml file in the JBoss instance?
best,
Henrik
Firstly you should ensure that the logback.xml is at the classpath, e.g. WEB-INF/classes/logback.xml as the mentioning at Chapter 3: Logback configuration.
The Chapter 4: Appenders mentions about FileAppender and RollingFileAppender properties as
file *String* The name of the file to write to. If the file does not exist, it is created. On the MS Windows platform users frequently forget to escape back slashes. For example, the value c:\temp\test.log is not likely to be interpreted properly as '\t' is an escape sequence interpreted as a single tab character (\u0009). Correct values can be specified as c:/temp/test.log or alternatively as c:\temp\test.log. The File option has no default value.
If the parent directory of the file does not exist, FileAppender will automatically create it, including any necessary but nonexistent parent directories.
The example is for logback.xml is as the following:-
<configuration>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>logFile.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>logFile.%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- keep 30 days' worth of history -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="FILE" />
</root>
</configuration>
I hope this may help.
You need to define the file-tag (as the previous answers already said). But you also need to know how it's interpreted. Logback sees it as relative to the application it's used in. So if you only use
<file>logs/<myfile>.log</file>
logback should create a "logs"-folder in the root of your JBoss (more specific: the folder where the .sh-script you start it with is located). Check from the root of your JBoss to see if you can find your logs-folder and file from there.

Logback is not generating log files on Ubuntu

We have configured logback for our logging needs in our application. When we run our application on Windows machine, it works as expected and generates the log files with proper logs. However when we deployed the same runnable jar file on Ubuntu machine, it is not generating log files. Following is the code in logback.xml
<configuration>
<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
<!-- in the absence of the class attribute, it is assumed that the
desired discriminator type is
ch.qos.logback.classic.sift.MDCBasedDiscriminator -->
<discriminator>
<key>uniqueNumber</key>
<defaultValue>unknown</defaultValue>
</discriminator>
<sift>
<appender name="FILE-${uniqueNumber}" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${uniqueNumber}.log</file>
<append>true</append>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<FileNamePattern>${uniqueNumber}_%i.log</FileNamePattern>
<MinIndex>1</MinIndex>
<MaxIndex>10000</MaxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<MaxFileSize>5MB</MaxFileSize>
</triggeringPolicy>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%d [%thread] %level %mdc %logger{35} - %msg%n</pattern>
</layout>
</appender>
</sift>
</appender>
<root level="DEBUG">
<appender-ref ref="SIFT" />
</root>
</configuration>
If anything goes wrong with the configuration logback prints out a lot of debug messages to System.out. If you do not see those, then perhaps the files ARE generated but you just don't know where.
You can explicitly enable logback printing with <configuration debug="true"> which should give you much more to work on.
If you just cannot locate the files consider using lsof to locate the full path of the open files of your application.
I had a similar problem to yours, even with a simpler logback configuration.
In my logback.xml file I used absolute paths instead of relative ones for the appender, but my configuration is Linux-only, and the machines where we deploy our Java application all share the same configuration/partitioning-scheme.
The files, at least this is what was happening to me, should be one directory up of your application directory. Lets say you are executing your JAR from PATH/my.jar, the logs should be in ../PATH.

Categories

Resources