I'm trying to figure out how I can add an appender to a logger dependent on whether a java system property is given / set.
So let's say I have a basic configuration like this:
<Logger name="myLogger" level="info" additivity="false">
<AppenderRef ref="myAppender1" />
<AppenderRef ref="myAppender2" />
</Logger>
So now I'd like to figure out a way to conditionally only add the 2nd appender if I provide a parameter -PaddAppender2. Something like this:
<Logger name="myLogger" level="info" additivity="false">
<AppenderRef ref="myAppender1" />
<?if (${sys:enableAppender2:-false) == "true"}>
<AppenderRef ref="myAppender2" />
</?if>
</Logger>
How do I do that?
I know I can for example make the level dynamic on a given property ("logLevel") like that (where "info" is the default if the property is not given):
<Logger name="test" level="${sys:logLevel:-info}" additivity="false">
I looked at the documentation for filters, and I can't figure it out. That is of course if filters are even the right way to go here.
Solution without any scripting:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="error" strict="true">
<Properties>
<Property name="appenderToUse">stdout_${sys:LOG4J_LAYOUT:-plain}</Property>
</Properties>
<Appenders>
<Appender type="Console" name="stdout_plain">
<Layout type="PatternLayout" pattern="%d [%t] %-5p %c - %m%n"/>
</Appender>
<Appender type="Console" name="stdout_json">
<Layout type="JSONLayout" compact="true" eventEol="true" stacktraceAsString="true" properties="true"/>
</Appender>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="${appenderToUse}"/>
</Root>
</Loggers>
</Configuration>
The solution provided by Robert works, but it is not efficient as the script will be evaluated once per log record.
A more efficient solution that evaluates the script only once is to use ScriptAppenderSelector together with the NullAppender:
According to the docs:
ScriptAppenderSelector
When the configuration is built, the ScriptAppenderSelector appender calls a Script to compute an appender name. Log4j then creates one of the appender named listed under AppenderSet using the name of the ScriptAppenderSelector. After configuration, Log4j ignores the ScriptAppenderSelector.
NullAppender
An Appender that ignores log events. Use for compatibility with version 1.2 and handy for composing a ScriptAppenderSelector.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" name="ScriptAppenderSelectorExample">
<Appenders>
<ScriptAppenderSelector name="SelectConsole">
<Script language="groovy"><![CDATA[
if (System.getProperty("CONSOLE_APPENDER_ENABLED", 'true').equalsIgnoreCase('true')) {
return "Console"
} else {
return "Null"
}
]]></Script>
<AppenderSet>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</Console>
<Null name="Null" />
</AppenderSet>
</ScriptAppenderSelector>
<ScriptAppenderSelector name="SelectFile">
<Script language="groovy"><![CDATA[
if (System.getProperty("FILE_APPENDER_ENABLED", 'true').equalsIgnoreCase('true')) {
return "File"
} else {
return "Null"
}
]]></Script>
<AppenderSet>
<File name="File" fileName="application.log">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
</File>
<Null name="Null" />
</AppenderSet>
</ScriptAppenderSelector>
<ScriptAppenderSelector name="SelectSMTP">
<Script language="groovy"><![CDATA[
if (System.getProperty("SMTP_APPENDER_ENABLED", 'true').equalsIgnoreCase('true')) {
return "SMTP"
} else {
return "Null"
}
]]></Script>
<AppenderSet>
<SMTP name="SMTP"
subject="App: Error"
from="log4j#example.com"
to="support#example.com"
smtpHost="smtp.example.com"
smtpPort="25"
bufferSize="5">
</SMTP>
<Null name="Null" />
</AppenderSet>
</ScriptAppenderSelector>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="SelectConsole"/>
<AppenderRef ref="SelectFile"/>
<AppenderRef ref="SelectSMTP"/>
</Root>
</Loggers>
</Configuration>
References
Is there a low overhead way to enable/disable a given appender based on the value of an environment variable or system property?
Similar to rgoers solution but using nashorn instead of groovy. This solution benefits from fact that Nashorn engine is part of
Java 8 so there is no additional dependencies needed.
<Scripts>
<Script name="isAppender2Enabled" language="nashorn"><![CDATA[
var System = Java.type('java.lang.System'),
Boolean = Java.type('java.lang.Boolean');
Boolean.parseBoolean(System.getProperty('enableAppender2', 'false'));
]]></Script>
</Scripts>
<Loggers>
<Logger name="myLogger" level="info" additivity="false">
<AppenderRef ref="myAppender1" />
<AppenderRef ref="myAppender2">
<ScriptFilter onMatch="ACCEPT" onMisMatch="DENY">
<ScriptRef ref="isAppender2Enabled" />
</ScriptFilter>
</AppenderRef>
</Logger>
</Loggers>
Note that ScriptFilter is evaluating the script every time when Log4j event occurs. Therefore it is possible to enable/disable the appender on the run time (by changing the value of the system property) with immediate effect. On the other hand, script evaluation can have negative impact on logging performance.
I wasn't able to figure out a solution via config file alone, but I found one that solves the problem programmatically.
Note that in our specific case, we always log to a "local log" ("splunk local"), but in given cases (controlled by the property), we also want to log the same information to another location (that is not relative) and is periodically read and forwarded to a splunk server ("splunk forwarder").
And that's why we can copy most of the properties from one logger to the other.
private static final Logger SPLUNK_LOG = getLogger();
private static Logger getLogger() {
if (!BooleanUtils.toBoolean(SystemUtils.getJavaPropertyValue(ENABLE_PROPERTY_NAME, "false"))) {
return LoggerFactory.getLogger(SPLUNK_LOG_NAME);
} else {
LOG.info("Dynamically adding splunk forwarder appender");
try {
final LoggerContext loggerContext = (LoggerContext) LogManager.getContext();
final Configuration configuration = loggerContext.getConfiguration();
// configure appender based on local splunk appender
final RollingFileAppender splunkLocal = (RollingFileAppender) configuration.getAppender(LOCAL_LOG_NAME);
final RollingFileAppender splunkForwarder = RollingFileAppender.createAppender(FORWARDER_FILE_NAME,
FORWARDER_FILE_PATTERN, FORWARDER_APPEND, FORWARDER_NAME, null, null, null,
splunkLocal.getManager().getTriggeringPolicy(), splunkLocal.getManager().getRolloverStrategy(),
splunkLocal.getLayout(), splunkLocal.getFilter(), null, FORWARDER_ADVERTISE, null, null);
splunkForwarder.start();
// add splunk forwarder appender to splunk logger
final LoggerConfig loggerConfig = configuration.getLoggerConfig(SPLUNK_LOG_NAME);
loggerConfig.addAppender(splunkForwarder, Level.INFO, null);
LOG.info("Successfully added splunk forwarder appender");
return loggerContext.getLogger(SPLUNK_LOG_NAME);
} catch (Exception ex) {
throw new IllegalStateException("Failed to dynamically add splunk forwarder appender", ex);
}
}
}
If anyone knows how to do this via config file alone, that would be great.
The way this is intended to be handled is by using a filter. In this case you can use a Script filter.
<Logger name="myLogger" level="info" additivity="false">
<AppenderRef ref="myAppender1" />
<AppenderRef ref="myAppender2">
<ScriptFilter onMatch="ACCEPT" onMisMatch="DENY">
<Script language="groovy"><![CDATA[
return System.getProperty("enableAppender2", "false").equalsIgnoreCase("true");
]]></Script>
</ScriptFilter>
</AppenderRef>
</Logger>
Building on some of the ideas in this thread, here's what I did to conditionally log to console.
Example use-case
Always log to a file appender.
Log to Console only in some environments.
Solution
For Console logging, set system property additional.log.appender=console
Or, disable Console logging by omitting this property.
In the Logger AppenderRef, use ${sys:additional.log.appender:-null}.
Sends logs to the console appender if the system property was set, or defaults to null appender if not set. (null appender ignores logs)
System Property
# set for console logging
additional.log.appender=console
log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<RollingFile name="file"
fileName="my-file.log"
filePattern="my-file%i.log">
<PatternLayout pattern="%d %5p [%t] %c - %m%n" />
<Policies>
<SizeBasedTriggeringPolicy size="10 MB" />
</Policies>
<DefaultRolloverStrategy max="10" />
</RollingFile>
<Console name="console" target="SYSTEM_OUT">
<PatternLayout pattern="%d %5p [%t] %c - %m%n" />
</Console>
<Null name="null" />
</Appenders>
<Loggers>
<Logger name="com.acme" level="DEBUG">
</Logger>
<Root level="INFO">
<AppenderRef ref="file" />
<AppenderRef ref="${sys:additional.log.appender:-null}" />
</Root>
</Loggers>
</Configuration>
Related
I'm having a global logger config that I want to inherit throughout my projects:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<Console name="CONSOLE" target="SYSTEM_OUT">
<PatternLayout pattern="%d %p %c{1.}: %m%n"/>
<ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY" />
</Console>
<RollingRandomAccessFile name="APP" fileName="/logs/application.logs" filePattern="/logs/application-%d{yyyy-MM-dd}.log.gz">
<PatternLayout pattern="%d %p %c{1.}: %m%n"/>
<Filters>
<ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
</Filters>
</RollingRandomAccessFile>
</Appenders>
<Loggers>
<Root level="INFO">
<AppenderRef ref="APP" />
<AppenderRef ref="CONSOLE" />
</Root>
</Loggers>
</Configuration>
Problem: only in production, I want to remove one of the loggers, the CONSOLE appender.
As log4j2.xml configuration files do not support conditionals, I thought about removing the console appender programmatically:
final LoggerContext context = (LoggerContext) LogManager.getContext(false);
final Configuration config = context.getConfiguration();
System.out.println(config.getAppenders());
Problem: this prints only {DefaultConsole-2=DefaultConsole-2}
Question: why can't I see the APP or CONSOLE appender here? And moreover, how can I remove the console appender then?
Maybe it is possible to intercept the log4j context loading somehow, so that I could skip the CONSOLE appender programmatically?
Sidenote: I'm logging as follows, which should in production only go to the APP appender, not to console.
private final Logger LOGGER = LoggerFactory.getLogger(this.getClass());
LOGGER.info("test");
For <ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY" /> in Console Appender you can add environment variable <ThresholdFilter level="${CONSOLE_LOG_LEVEL}" onMatch="ACCEPT" onMismatch="DENY" /> or something similar and set this variable to OFF. I faces with this problem too and it works for me.
This also works with vm args as follows:
<ThresholdFilter level="${sys:console.log.level}" ... />
When starting the app: java -jar -Dconsole.log.level=ERROR
Sidenote: programatically, removing the console appender would also work:
public static void main(String[] args) {
ctx = SpringApplication.run(MyApp.class, args);
final LoggerContext context = (LoggerContext) LogManager.getContext(false);
final Configuration config = context.getConfiguration();
LoggerConfig loggerConfig = config.getLoggerConfig("loggerName");
loggerConfig.removeAppender("CONSOLE");
context.updateLoggers();
}
Most important for the programmatic approach is that the SpringApplication context must be initialized before! Otherwise the configured loggers are not visible!
I'm having some trouble making log4j2 working with ActiveMQ.
Here is my log4j2.xml :
<Configuration>
<ThresholdFilter level="all"/>
<Appenders>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%m%n"/>
<Filters>
<ThresholdFilter level="info" />
</Filters>
</Console>
<File name="baseLog" filename="\\\\p02630\\c$\\tmp\\logs\\logws-gendb.log">
<PatternLayout>
<Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
</PatternLayout>
<Filters>
<ThresholdFilter level="error"/>
</Filters>
</File>
<JMS name="AMQError" providerurl="tcp://169.3.200.150:61616" password="admin" userName="admin">
<factoryName>org.apache.activemq.jndi.ActiveMQInitialContextFactory</factoryName>
<factoryBindingName>ConnectionFactory</factoryBindingName>
<TopicBindingName>logError</TopicBindingName>
</JMS>
<Failover name="FailOverAMQ" primary="AMQError">
<Failovers>
<appender-ref ref="baseLog"/>
</Failovers>
</Failover>
</Appenders>
<Loggers>
<root>
<AppenderRef ref="STDOUT" />
<AppenderRef ref="baseLog" />
<AppenderRef ref="FailOverAMQ" />
</root>
</Loggers>
</Configuration>
Goal is to be able to log errors in ActiveMQ. But if/when the AMQ server's down, I want the logger to be able to reconnect automatically and to still log errors inside a file Appender.
While the server is running, everything is working fine. But when I shutdown the server, nor the ActiveMQ (normal), nor the file Appender are working and when the server comes back up, log4j is not autoreconnecting after the 60s auto reconnect time(default). More troubling, my file appender is not working anymore after server shutdown.
I first had a problem with the "no parameter that matches element Failovers" detailed here and tried to add the FallBack class registering the "Failovers" element. It did remove the error message but the failover appender is not working any better. I have the impression all this class did was obfuscating the error.
Have you had any luck working with Failover appenders ?
Thanks for the help.
Just a quick follow up.
I could not found any solution to this problem. Seems to be a bug in log4j2.
In the end, I developped a short class using javax.jms package to manage my connection to ActiveMQ.
Not perfect but at least, it works.
Expectation is to have the current log with the same name (i.e test.log)
and the archival files name should be test_(CurrentDate).log.1.
/logs/projectlogs/test.log
/logs/projectlogs/ test_20160430.log.1
/logs/projectlogs/ test_20160430.log.2
/logs/projectlogs/ test_20160430.log.3
Using below properties file, current date is getting appended with all the files.
log4j.properties::
log4j.rootLogger= ALL, A1, file, rollingAppender
#log4j.date=contains current date
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.File=C:/logs/projectlogs/test_${log4j.date}.log
log4j.appender.file.MaxFileSize=100KB
log4j.appender.file.MaxBackupIndex=5
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss:SSS zzz} %-5p[%t] %m%n
The log4j 2 manual has many example configurations, and the section on RollingFileAppender has an example that matches your requirements:
http://logging.apache.org/log4j/2.x/manual/appenders.html#RollingFileAppender
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" name="MyApp" packages="">
<Appenders>
<RollingFile name="RollingFile" fileName="logs/projectlogs/test.log"
filePattern="logs/projectlogs/$${date:yyyy-MM}/test-%d{yyyy-MM-dd-HH}-%i.log.gz">
<PatternLayout>
<Pattern>%d %-5p[%t] %c{1.} %m%n</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="5" modulate="true"/>
<SizeBasedTriggeringPolicy size="100 MB"/>
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Root level="trace">
<AppenderRef ref="RollingFile"/>
</Root>
</Loggers>
</Configuration>
What may also be of interest to you is that Log4j-2.5 introduced a Delete action that gives users more control over what files are deleted at rollover time. See http://logging.apache.org/log4j/2.x/manual/appenders.html#CustomDeleteOnRollover
What you want to achieve is called "Size And Time Based File Naming And Triggering Policy" (SizeAndTimeBasedFNATP) and can be found in the latest release of logback package, which is at the moment 1.1.7, however, it can be found in 1.1.2 as well, which I use since it produces more predictable results without too much of asynchronous processes (even though 1.1.2 is slower).
The logback package works in similar way as log4j, but uses XML based configuration file (logback.xml) which in your case may look like one below (note test_%d{yyyyMMdd}.log.%i -- this is your pattern).
Please note, that unfortunately, it is not (yet) possible to limit indexes within dates. So you can't have only 5 chunks of logs for each date -- each date will be logged entirely with log chunks indexes independent for each date. You can, however, limit total number of dates and (in 1.1.7) total size of log folder (using ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy).
Also I suggest you use chunk size of 10Mb at least (100Kb is tiny), as bigger chunks, generally speaking, easier to maintain by log lib (less CPU consumption).
<?xml version="1.0" ?>
<configuration>
<property name="log.folder" value="C:/logs/projectlogs"/>
<!-- UNCOMMENT BELOW SECTION IF CONSOLE IS REQUIRED -->
<!--
<appender class="ch.qos.logback.core.ConsoleAppender" name="CONSOLE">
<encoder>
<pattern>[%p] [%thread] %logger - %msg%n</pattern>
</encoder>
</appender>
-->
<appender class="ch.qos.logback.core.rolling.RollingFileAppender" name="FILE">
<File>${log.folder}/test.log</File>
<Append>true</Append>
<encoder>
<pattern>%d{ISO8601} [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.folder}/test_%d{yyyyMMdd}.log.%i</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100KB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!-- keep 30 days' worth of history -->
<maxHistory>30</maxHistory>
<!-- up to 10 GB max -->
<totalSizeCap>10GB</totalSizeCap>
<MaxFileSize>100KB</MaxFileSize>
</rollingPolicy>
</appender>
<root>
<level value="INFO"/>
<appender-ref ref="FILE"/>
</root>
<logger name="Main">
<level value="DEBUG" />
</logger>
<logger name="ch.qos">
<level value="WARN"/>
</logger>
</configuration>
Test Java App
package logtester;
import org.apache.log4j.Logger;
public class LogTester {
public static void main(String[] args) {
Logger logger = Logger.getLogger("Main");
for(int i = 1; i<=20000; i++)
logger.info("Log message "+i);
}
}
Folder structure after run:
13,230 test.log
102,929 test_20160430.log.0
103,168 test_20160430.log.1
102,816 test_20160430.log.10
102,816 test_20160430.log.11
103,168 test_20160430.log.2
103,168 test_20160430.log.3
103,168 test_20160430.log.4
103,168 test_20160430.log.5
102,815 test_20160430.log.6
102,816 test_20160430.log.7
102,816 test_20160430.log.8
102,816 test_20160430.log.9
Hope it helps.
My Java project uses Log4J2. Currently, it is successfully writing logs to a file. Now, I'm trying to push the logs to LogStash via a Socket Appender. Unfortunately, I am not having any success with these efforts. At this time, I'm looking at two pieces: my log4j2.xml file and my logstash config file. I've provided both here in hopes that someone can help me identify my problem.
log4j2.xml
<Configuration status="WARN" monitorInterval="30">
<Appenders>
<Socket name="A1" host="0.0.0.0" port="4560">
<SerializedLayout/>
</Socket>
<RollingRandomAccessFile name="RollingFile" fileName="/logs/server.log"
filePattern="/logs/$${date:yyyy-MM}/server-%d{yyyy-MM-dd-HH}-%i.log.gz">
<JSONLayout complete="false"></JSONLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="6" modulate="true"/>
<SizeBasedTriggeringPolicy size="100 MB"/>
</Policies>
</RollingRandomAccessFile>
<Async name="AsyncFile">
<AppenderRef ref="RollingFile"/>
</Async>
</Appenders>
<Loggers>
<Logger name="com.company" level="trace" additivity="false">
<AppenderRef ref="AsyncFile"/>
</Logger>
<Root level="trace">
<AppenderRef ref="AsyncFile"/>
</Root>
</Loggers>
</Configuration>
That was my log4j2 configuration. Here is my logstash configuration:
logstash.conf
input {
log4j {
mode => "server"
type => "log4j"
host => "0.0.0.0"
port => "4560"
}
}
output {
stdout {
codec => "json"
}
}
I start logstash from the command-line using logstash-1.4.0/bin/logstash --config /etc/logstash/logstash.conf --debug. I do not see any errors. At the same time, I do not see any logs written to the console window. I know that logs should appear because they are appearing in my rolling server.log file that was configured as the second appender in log4j.
What am I doing wrong? I've been tinkering with this for 3 days.
The log4j input in Logstash is not compatible with Log4j2. Log4j2 uses a new format to serialize the log data.
There is however a plugin which enables parsing of Log4j2 sockets:
https://github.com/jurmous/logstash-log4j2
In logstash 1.5+ it can be installed from:
https://rubygems.org/gems/logstash-input-log4j2
I think you should replace 0.0.0.0 with your IP on the following line:
<Socket name="A1" host="0.0.0.0" port="4560">
And add the following lines in elements <Root ...> and <Logger ...> (like you did for AsyncFile appender):
<AppenderRef ref="A1" />
<AppenderRef ref="RollingFile" />
I'm using log4j 2.0-beta9. I have a question about the SMTP appender. I need to configure the subject, from and to values from properties. I'm logging a MapMessage and my config is as below -
<?xml version="1.0" encoding="UTF-8"?>
<configuration status="DEBUG">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d [%t] %-5p %c - %m%n"/>
</Console>
<SMTP name="Mail" subject="Error Log for ${env:HOSTNAME}" to="${sys:mail.to}" from="${sys:mail.from}"
smtpHost="${sys:mail.host}" smtpPort="${sys:mail.port}" smtpDebug="true" bufferSize="1">
<PatternLayout>
<pattern>%d [%t] %-5p %c - %m%n</pattern>
</PatternLayout>
</SMTP>
<Async name="AsyncMail">
<appender-ref ref="Mail" />
</Async>
</appenders>
<loggers>
<root level="info">
<appender-ref ref="Console"/>
<appender-ref ref="AsyncMail">
<MapFilter onMatch="ACCEPT" onMismatch="DENY">
<KeyValuePair key="throwable.class" value="java.lang.RuntimeException" />
</MapFilter>
</appender-ref>
</root>
</loggers>
</configuration>
// Java Code to log the msg
Throwable throwable; // this is the exception that is thrown by the app.
MapMessage message = new MapMessage();
message.put("throwable.message", throwable.getMessage());
message.put("throwable.class", throwable.getClass().getName());
message.put("throwable.stacktrace", ExceptionUtils.getStackTrace(throwable)); // ExceptionUtils from apache-commons
LOGGER.error(message, throwable); // org.apache.logging.log4j.Logger
The problem is that none of these values are replaced dynamically. Is there any way to do this?
Thanks in advance.
You need to set the mail.to and mail.from System properties. The problem you're having might be that you're running in a Servlet 3.0 environment, in which case the Log4j2.xml file is being processed before your code that sets the properties is executed.
If that is the case, you can create a servlet container initializer that you configure in your web.xml file to load before Log4j2's servlet container initializer is loaded.