Develop Reference

How to retain log4j2's plugin configs after server restart or altering xml config file

I have developed a custom rolling file appender and it works fine. the only problem is that it's getting re-initialized each time I restart the server or change the log4j2.xml file(any part of the file) and suddenly all the previous logs would get wiped off. I haven't observed such behavior with the shipped default appenders so I wonder what can I do to retain my configurations.
<CustomAppender name="CustomAppender"
fileName="${log.file.directory}/file.log"
filePattern="${log.file.directory}/file.log.%d{yyyy-MM-dd}-%i.gz"
immediateflush="true"
append="true">
<CustomLayout/>
<Policies>
<SizeBasedTriggeringPolicy size="3 KB"/>
<TimeBasedTriggeringPolicy interval="1"/>
</Policies>
</CustomAppender>
P.S. I have tried to make it a singleton but aside from that it didn't work I really don't want to keep it away from being reconfigured, I just want to retain my previously generated logs.
update
Apparently everytime the server shutdown or log4j2.xml file is being altered the manager rebuilds the appender from scratch, though built-in appenders retain their state even after restart or reconfiguration. They do this by overriding "stop" method from AbstractOutputStreamAppender. I did the same for my appender but it still doesn't behave as I expect.
#Override
public boolean stop(long timeout, TimeUnit timeUnit) {
setStopping();
final boolean stopped = super.stop(timeout, timeUnit, false);
setStopped();
return stopped;
}
And this is the manager I used in appender builder:
final RollingFileManager manager = RollingFileManager.getFileManager(fileName, filePattern, append,
isBufferedIo, policy, strategy, advertiseUri, layout, bufferSize, isImmediateFlush(),
createOnDemand, filePermissions, fileOwner, fileGroup, getConfiguration());
if (manager == null) {
return null;
}
manager.initialize();

When a reconfiguration takes place the following happens:
The new configuration file is read and converted to a Node tree.
The Plugins associated with the Nodes are accessed and their corresponding classes are created.
If the configuration was created successfully it is started.
The Loggers are updated to reference the new configuration.
The old configuration is stopped.
As you might imagine, since there are two configuration running at the same time this can lead to problems. For example, if both components try to use the same port then the second configuration will likely fail. To avoid these problems Appenders in Log4j 2 use Managers. The Manager will typically have a "name" that includes any items that, if they have the same value in the new configuration, indicate that the Manager should be reused. This allows things like OutputStreams to remain open during the reconfiguration. However, the side effect of this is that if a parameter is changed on an appender and is not part of the "name" then the change in value may be ignored since the Manager was not modified.
So when creating a new Appender, care must be given to the name passed to the Manager and the Appender should try to update the relevant values in the Manager when they are not part of the name. Also, the Manager must account for it being started and stopped multiple times. AbstractManager will actually handle this for you.

Logback: purpose and scope of StatusManager for Logstash-using microservice

According to the manual:
Logback collects its internal status data in a StatusManager object,
accessible via the LoggerContext.
Given a StatusManager you can access all the status data associated
with a logback context. To keep memory usage at reasonable levels, the
default StatusManager implementation stores the status messages in two
separate parts: the header part and the tail part. The header part stores
the first H status messages whereas the tail part stores the last T messages. At present time H=T=150, although these values may change in future releases.
The purpose seems to be to allow incoming log messages to be listened-to, also viewed via a servlet.
In our case we use the logstash-logback-encoder and emit our Logback messages to the console so fluentd / Filebeat etc. can ship to a remote Logstash - and so our microservices have no need to inspect or listen to them.
So the question is: can / should we override the default BasicStatusManager (with a NOOP implementation) to suppress the internal storage of these messages, or does anyone know if the Logstash encoder depends upon, or hooks into this status mechanism itself?
Our logback.xmls are pretty simple:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="lsAppender" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>
<logger name="a.b.c" level="debug"/>
<root level="debug">
<appender-ref ref="lsAppender"/>
</root>
</configuration>
My second reason for asking is that we recently found ourselves getting OOMEs from microservices with a 120 MB heap. Sure, those individual log messages are too big, but that's only a problem if they're being held-onto by Logback itself rather than sent off and forgotten about. It would be one less scalability concern.
Screenshot of YourKit output:

I noticed this again recently. The problem was actually nothing related to log size, or logging throughput, and there is no need to control the size of StatusManager in any way.
The actual cause was that the Logstash Logback Encoder (version 6.4) was failing "silently" due to unencodable Markers:
import net.logstash.logback.marker.Markers;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.context.request.WebRequest;
import java.util.HashMap;
#ControllerAdvice
public class MyExceptionHandler {
private static final Logger LOG = LoggerFactory.getLogger(MyExceptionHandler.class);
public void logException(Throwable e, WebRequest request) {
var fields = new HashMap<>();
fields.put("Request", request);
LOG.error(Markers.appendEntries(fields), "Error occurred", e);
}
}
The WebRequest is an instance of org.springframework.web.context.request.ServletWebRequest which, when serialized to JSON by the encoder using jackson-databind, throws a StackOverflowError, which is caught and ignored by said encoder. The original log request is not flushed from the internal Logback cache.
Given enough errors of this type, we eventually ran out of memory.

SLF4J - Logging regardless of the log level

In my Java application, I use SLF4J + Logback for logging. I use DEBUG level logging on development and ERROR level logging on the production environment. But there are some messages that I want to log in any case, regardless of the log level (similar to System.out.println("Some Message") but using logger).
Is there any practical way to achieve this? I can use error or lower levels, but what I want to do is to give some information, so logger.error("Some message"); is semantically wrong, it is not an error.
I can define another logger in my logback.xml for my class/classes, but it is not convenient.
What is the right way to achieve this?

This is where the wonderful "Marker" feature of SLF4J/Logback comes in very handy. Say you want to log to the console all warnings/errors, as well as anything that's a special "Status" message.
You can have a class do logging like this:
public class MyClass {
private static final Marker status = MarkerFactory.getMarker("STATUS");
private static final Logger logger = LoggerFactory.getLogger(MyClass.class);
public void doSomething() {
logger.info(status, "Beginning");
logger.info("Regular info log");
logger.warn("Warning");
logger.info(status, "Done");
}
}
Then, in logback.xml, you filter to show all warnings and messages marked "STATUS":
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator class="ch.qos.logback.classic.boolex.OnMarkerEvaluator">
<marker>STATUS</marker>
</evaluator>
<OnMatch>ACCEPT</OnMatch>
<OnMismatch>NEUTRAL</OnMismatch>
</filter>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>WARN</level>
</filter>
<encoder>
<pattern>${pattern}</pattern>
</encoder>
</appender>
Basically, you have the logger's level be DEBUG so that everything goes to the appender, and then in the appender do further filtering to get the exact lines you're looking for. If you are trying to have more different logging levels for different classes it can get a bit trickier, but Logback gives a lot of flexibility with filters (and you can even make your own filters if needed) which lets one handle pretty much anything needed.

Is there a way to dyamic name log files in java?

I'm using logback as my logging framework and have a couple of jobs that run the same main function with different parameters and would like to create a log file for each job and name the log file with the job's name.
For example, if I had jobs a,b,c that all run MyClass.main() but with different parameters, then I'd like to see a-{date}.log, b-{date}.log, c-{date}.log.
I can achieve the {date} part by specifying a <fileNamePattern>myjob-%d{yyyy-MM-dd}.log</fileNamePattern> in my logback.xml, but I'm not sure how to (or if it is even possible) create the prefix of the file names dynamically (to be the job's name).
Is there a way to dynamically name logfiles in logback? Is there another logging framework that makes this possible?
As a follow up question, am I just taking a bad approach for having multiple jobs that call the same main function with different parameters and wanting a log file named after each job? If so is there a standard/best practice solution for this case?
EDIT: The reason why I want to name each log file after the name of the job is that each job naturally defines a "unit of work" and it is easier for me to find the appropriate log file in case one of the job fails. I could simply use a rolling log file for jobs a,b,c but I found it harder for me to look through the logs and pinpoint where each job started and ended.

I would use you own logging.
public static PrintWriter getLogerFor(String prefix) {
SimpleDatFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
String filename= prefix + sdf.format(new Date());
return new PrintWriter(filename, true); // auto flush.
}
You can write a simple LRU cache e.g. with LinkedHashMap to reuse the PrintWriters.

Is there a way to dynamically name logfiles in logback? Is there another logging framework that makes this possible?
I don't believe this is possible using the out of the box appenders (File, RollingFile etc) configured by a standard logback.xml file. To do what you want, you would need to dynamically create appenders on the fly and assign loggers to different appenders. Or you would need to invent a new appender that was smart enough to write to multiple files at the same time, based on the logger name.
am I just taking a bad approach for having multiple jobs that call the same main function with different parameters and wanting a log file named after each job?
The authors of logback address this issue and slightly discourage it in the section on Mapped Diagnostic Context
A possible but slightly discouraged approach to differentiate the logging output of one client from another consists of instantiating a new and separate logger for each client. This technique promotes the proliferation of loggers and may increase their management overhead. ... A lighter technique consists of uniquely stamping each log request servicing a given client.
Then they go on to discuss mapped diagnostic contexts as a solution to this problem. They give an example of a NumberCruncherServer which is crunching numbers, for various clients in various threads simultaneously. By setting the mapped diagnostic context and an appropriate logging pattern it becomes easy to determine which log events originated from which client. Then you could simply use a grep tool to separate logging events of interest into a separate file for detailed analysis.

Yes you can.
First you have to familiarize your self with these 2 concepts: Logger and Appender. Generally speaking, your code obtains a Logger, and invoke logging method such as debug(), warn(), info() etc. Logger has Appender attached to it, and Appender presents the logging information to the user according to the configuration set to it.
Once you're familiar, what you need to do is to dynamically create a FileAppender with a different file name for each different job type, and attach it to your Logger.
I suggest you spend some time with logback manual if none of above make sense.

You can make use of the logback discriminators, as discriminators' keys can be used in the <FileNamePattern> tag. I can think of two options:
Option One:
You can use the Mapped Diagnostic Context discriminator to implement your logging separation, you'll need to set a distinct value from each job using MDC.put();
Once you've done that your appender on logback configuration would look something like:
<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator class="ch.qos.logback.classic.sift.MDCBasedDiscriminator">
<key>jobName</key> <!-- the key you used with MDC.put() -->
<defaultValue>none</defaultValue>
</discriminator>
<sift>
<appender name="jobsLogs-${jobName}" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<FileNamePattern>${jobName}.%d{dd-MM-yyyy}.log.zip</FileNamePattern>
.
.
.
</rollingPolicy>
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>...</Pattern>
</layout>
</appender>
</sift>
</appender>
Option Two:
Implement your own discriminator -implementing ch.qos.logback.core.sift.Discriminator-, to discriminate based on the thread name. Would look something like this:
public class ThreadNameDiscriminator implements Discriminator<ILoggingEvent> {
private final String THREAD_NAME_KEY = "threadName";
#Override
public String getDiscriminatingValue(ILoggingEvent event) {
return Thread.currentThread().getName();
}
#Override
public String getKey() {
return THREAD_NAME_KEY;
}
// implementation for more methods
.
.
.
}
The logging appender would look like option one with the discriminator class being ThreadNameDiscriminator and the key being threadName. In this option there is no need to set a value to the MDC from your jobs, hence, no modification on them is required.

Log4j multiple files

I have 2 logging files.
I have defined two appenders for the 2 files.
In a class, if i need to output few logs to one file, and few to the other, do i need to have 2 logger instances.
Is there a cleaner way of achieving this requirement?
Or is there some log4j configuration that will help me?

You can create two named loggers like the following:
log4j.logger.system=debug, sys
log4j.appender.sys=org.apache.log4j.RollingFileAppender
log4j.appender.sys.file=/logs/system.log
log4j.appender.sys.maxFileSize=1MB
log4j.appender.sys.maxBackupIndex=25
log4j.appender.sys.layout=org.apache.log4j.PatternLayout
log4j.appender.sys.layout.conversionPattern=%d{MMM dd HH:mm:ss} %-5p (%F:%L) - %m%n
which you can locate and use from code:
Logger.getLogger("system").debug("...");

If this log filtering is based on severity, you can define a different log level for each appender (DEBUG, INFO, WARNING, etc.).
If you want a different filtering, you can create a personnal log filter, and apply this filter on your appender. Look at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/spi/Filter.html