Develop Reference

Logback log to main file and clients in a separate directory

I have a Java server application for which I log all the general data to server.log and each individual client to its own hostname.log file. I want to put the hostname.log files in a different directory that server.log ideally in some organized fashion since there are thousands of hostname.log files.
This is the config I'm using now:
<appender name="SiftAppender" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator>
<key>descriminatorid</key>
<defaultValue>server</defaultValue>
</discriminator>
<sift>
<appender name="FILE-${descriminatorid}" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${cmb.log.dir}/${descriminatorid}.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>${cmb.log.dir}/archive/${descriminatorid}.%d{yyyy-MM-dd}.log.gz</fileNamePattern>
<!-- keep ${maxbackupindex} days' worth of history capped at ${maxfilesize} total size -->
<maxHistory>${cmb.log.maxbackupindex}</maxHistory>
<totalSizeCap>${cmb.log.maxfilesize}</totalSizeCap>
</rollingPolicy>
<encoder>
<pattern>%date{HH:mm:ss.SSS} [%thread] %-5level %msg%n</pattern>
</encoder>
</appender>
</sift>
</appender>
This config logs everything to $cmb.log.dir. What I'd like is something like this:
Server logs: $cmb.log.dir/server.log
Client logs: $cmb.log.dir/client/${descriminatorid}.log
Or even better would be:
Client logs: $cmb.log.dir/client/${firstLetter}/${descriminatorid}.log
where ${firstLetter} is the first letter of $descriminitorid. That way the logs would be distributed in a more scalable hierarchy.

The solution I found was very simple. Since I am generating ${descriminatorid} in my java code I simple included the directory structure i wanted in that value. So instead of "descriminatorid" being "hostname" it's now "client/firstletter/hostname".
I was definitely trying to overthink this one.

Generic name for the active log file

I have the following logback.xml file for my spring boot application where I intend to create daily log files:
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true">
<appender name="rollingFile" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>./logs/my-log-file.%d{yyyy-MM-dd}.log</fileNamePattern>
<maxHistory>15</maxHistory>
<totalSizeCap>15MB</totalSizeCap>
</rollingPolicy>
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>%d [%-5thread] [%-1p] [%logger{35}] - %msg%n</Pattern>
</layout>
</appender>
<root level="WARN">
<appender-ref ref="rollingFile"/>
</root>
<root level="INFO">
<appender-ref ref="rollingFile"/>
</root>
</configuration>
The files are created like this:
my-log-file.2018-11-10.log
my-log-file.2018-11-11.log
my-log-file.2018-11-12.log
I am looking to have the current active log file without the date so it is generic name like my-log-file.log and only when the date rolls to a new day, the file is renamed with date.
What change is required on the logback.xml to enable this configuration?

Add a <file> to the <appender>, like so:
<appender name="rollingFile" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>./logs/my-log-file.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>./logs/my-log-file.%d{yyyy-MM-dd}.log</fileNamePattern>
<maxHistory>15</maxHistory>
<totalSizeCap>15MB</totalSizeCap>
</rollingPolicy>
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>%d [%-5thread] [%-1p] [%logger{35}] - %msg%n</Pattern>
</layout>
</appender>
From the Logback documentation:
Note that the file property in RollingFileAppender (the parent of
TimeBasedRollingPolicy) can be either set or omitted. By setting the
file property of the containing FileAppender, you can decouple the
location of the active log file and the location of the archived log
files. The current logs will be always targeted at the file specified
by the file property. It follows that the name of the currently active
log file will not change over time. However, if you choose to omit the
file property, then the active file will be computed anew for each
period based on the value of fileNamePattern.

RollingFileAppender that also deletes files older than some date

Do any of the popular Java logging frameworks support a rolling file appender, that I can configure to rollover daily, and also delete any log file that is over some number of days old? I know I could use a rolling file appender and a cron, but was wondering if anyone knew of an appender that can do both.

Logback's classic RollingFileAppender provides this and more. An example configuration from the manual (http://logback.qos.ch/manual/appenders.html#onRollingPolicies)
<configuration>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>logFile.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>logFile.%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- keep 30 days' worth of history -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="FILE" />
</root>
</configuration>
This provides daily rollover and 30 days of history. Place this in a file called logback.xml, or logback-test.xml for test trees, and place it in the classpath.

After working a while on Log4j, I read that Logback was developed and designed to be the successor of Log4j, and I have to say, Logback is wide stepper!
For example, to configure, this rolling file Appender, and zip the older logs, and have a maximum of 30, on Log4J you have to perform some code changes (please review my previews answer for more details), but on logback, everything is done on the config file like this:
<configuration>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>log/logFile.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>log/logFile.%d{yyyy-MM-dd}.log.zip</fileNamePattern>
<!-- keep 30 days' worth of history -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>[%-5level]: [%logger{35}] - %msg%n</pattern>
</encoder>
</appender>
<root level="trace">
<appender-ref ref="FILE" />
</root>
</configuration>
And that's it, you will have the current log unzipped, and the older days, zip, one zip file by each day.
Wide more easier than log4j!
If you want to play with more config options, just check this link.

Indeed, if you are using Log4J you can use this appender:
http://blog.kimb3r.com/2008/07/improving-log4j-dailyrollingfileappende.html
It's very, very, VERY easy to use, just download the whole class(yes, the code of the class in the above link), include it in your project (wherever you want, you can change the package to match inside your project), and then configure the log4j.properties file like this(this file has to be in your classpath, for example in the src/main/resources folder):
# Define the root logger with appender file
log = log
log4j.rootLogger = TRACE, FILE
# Define the logical path where you put the class you downloaded from "blog.kimb3r.com" link (above)
log4j.appender.FILE=com.yourapp.yourpackage.log.CustodianDailyRollingFileAppender
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.conversionPattern=%m%n
log4j.appender.FILE.File=${log}/yourlogfile.log
log4j.appender.FILE.DatePattern='.'yyyy-MM-dd
# How many files you want to keep?, in this case I'm having just 15 days of files, one file per day:
log4j.appender.FILE.MaxNumberOfDays=15
# If True, the older files will be compressed into a zip file (*which will save you a lot of space on the server*)
log4j.appender.FILE.CompressBackups=true
and besides this just add the dependency to log4j into your pom like this:
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
And that's it!

Why not look at this might help?
You can probably set maximum nuber of days to retain the logs.
log4j.rootLogger=INFO, FILE
log4j.appender.FILE=ca.justtechnologies.utils.logging.CustodianDailyRollingFileAppender
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.ConversionPattern=%d{MMM dd yyyy
HH:mm:ss,SSS} [%t] %-5p %l - %m%n
log4j.appender.FILE.File=/var/log/web-apps/Dashboard.log
log4j.appender.FILE.DatePattern='.'yyyy-MM-dd
log4j.appender.FILE.MaxNumberOfDays=14
log4j.appender.FILE.CompressBackups=true

Logback-classic in Jboss and Spring

I'm working on a project where I thought I would try using logback-classic for debugging and log rotation. I'm using this in a Maven context for building and creating a .war file to be deployed in JBoss 7.1 Application Server.
I've placed a logback.xml file in the resources folder in the code and a logback-test.xml in test/resources.
The active jar I'm using is the SLF4J to print the actual debugging.
public class MyClass extends MultiActionController {
private Logger logger = LoggerFactory.getLogger(MyClass.class);
public MyClass() {
logger.debug("hello");
}
}
When I run a JUnit test on the code in Maven itself it works, but after building a .war file i don't get any debugging in STDOUT nor can I find a file created.
I know that I've removed the actual logging from STDOUT in the config file, but where is the logging going...
the logback.xml:
<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>DEBUG</level>
</filter>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<FileNamePattern>logs/myproject.%d{yyyy-MM-dd}.log</FileNamePattern>
<MaxHistory>30</MaxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%d{yyMMdd HH:mm:ss.SSS} [%-5.5level] [%-25.25logger] %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="FILE" />
</root>
</configuration>
The folder "logs" do I need to explicit create it in the JBoss instance or need to refer relative to it for this to work? Or is there something I missed? Do I need to put the logback.xml file in the JBoss instance?
best,
Henrik

Firstly you should ensure that the logback.xml is at the classpath, e.g. WEB-INF/classes/logback.xml as the mentioning at Chapter 3: Logback configuration.
The Chapter 4: Appenders mentions about FileAppender and RollingFileAppender properties as
file *String* The name of the file to write to. If the file does not exist, it is created. On the MS Windows platform users frequently forget to escape back slashes. For example, the value c:\temp\test.log is not likely to be interpreted properly as '\t' is an escape sequence interpreted as a single tab character (\u0009). Correct values can be specified as c:/temp/test.log or alternatively as c:\temp\test.log. The File option has no default value.
If the parent directory of the file does not exist, FileAppender will automatically create it, including any necessary but nonexistent parent directories.
The example is for logback.xml is as the following:-
<configuration>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>logFile.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>logFile.%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- keep 30 days' worth of history -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="FILE" />
</root>
</configuration>
I hope this may help.

You need to define the file-tag (as the previous answers already said). But you also need to know how it's interpreted. Logback sees it as relative to the application it's used in. So if you only use
<file>logs/<myfile>.log</file>
logback should create a "logs"-folder in the root of your JBoss (more specific: the folder where the .sh-script you start it with is located). Check from the root of your JBoss to see if you can find your logs-folder and file from there.

Logback is not generating log files on Ubuntu

We have configured logback for our logging needs in our application. When we run our application on Windows machine, it works as expected and generates the log files with proper logs. However when we deployed the same runnable jar file on Ubuntu machine, it is not generating log files. Following is the code in logback.xml
<configuration>
<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
<!-- in the absence of the class attribute, it is assumed that the
desired discriminator type is
ch.qos.logback.classic.sift.MDCBasedDiscriminator -->
<discriminator>
<key>uniqueNumber</key>
<defaultValue>unknown</defaultValue>
</discriminator>
<sift>
<appender name="FILE-${uniqueNumber}" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${uniqueNumber}.log</file>
<append>true</append>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<FileNamePattern>${uniqueNumber}_%i.log</FileNamePattern>
<MinIndex>1</MinIndex>
<MaxIndex>10000</MaxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<MaxFileSize>5MB</MaxFileSize>
</triggeringPolicy>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%d [%thread] %level %mdc %logger{35} - %msg%n</pattern>
</layout>
</appender>
</sift>
</appender>
<root level="DEBUG">
<appender-ref ref="SIFT" />
</root>
</configuration>

If anything goes wrong with the configuration logback prints out a lot of debug messages to System.out. If you do not see those, then perhaps the files ARE generated but you just don't know where.
You can explicitly enable logback printing with <configuration debug="true"> which should give you much more to work on.
If you just cannot locate the files consider using lsof to locate the full path of the open files of your application.

I had a similar problem to yours, even with a simpler logback configuration.
In my logback.xml file I used absolute paths instead of relative ones for the appender, but my configuration is Linux-only, and the machines where we deploy our Java application all share the same configuration/partitioning-scheme.
The files, at least this is what was happening to me, should be one directory up of your application directory. Lets say you are executing your JAR from PATH/my.jar, the logs should be in ../PATH.