I am running a Java application in which I am invoking multiple threads, each with some unique names. Now I want to create multiple log files for each of them and the name of the log files should be as the thread names. Is this possible using log4j2. Please help me write log4j2 configuration files.
Thank you in advance.
I agree a RoutingAppender is the way to go. I initially used the routing appender in conjunction with the ${ctx:threadName} lookup where the 'ctx' uses the ThreadContext. I found that I would have to sprinkle in the code a line like this:
ThreadContext.put("threadName", Thread.currentThread().getName());
While that code works it's not extensible in the design of the code. If I were to add a new java.lang.Runnable to the code base, I would have to also include that line.
Rather, the solution seems to be to implement the 'org.apache.logging.log4j.core.lookup.StrLookup' and register the #Plugin with the PluginManager Like this:
Class: ThreadLookup
package my.logging.package
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.lookup.StrLookup;
#Plugin(name = "thread", category = StrLookup.CATEGORY)
public class ThreadLookup implements StrLookup {
#Override
public String lookup(String key) {
return Thread.currentThread().getName();
}
#Override
public String lookup(LogEvent event, String key) {
return event.getThreadName() == null ? Thread.currentThread().getName()
: event.getThreadName();
}
}
Configuration: log4j2.xml (packages attribute of the Configuration registers the #Plugin with the PluginManager)
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" packages="my.logging.package">
<Appenders>
<Routing name="Routing">
<Routes pattern="$${thread:threadName}">
<Route>
<RollingFile name="logFile-${thread:threadName}"
fileName="logs/concurrent-${thread:threadName}.log" filePattern="logs/concurrent-${thread:threadName}-%d{MM-dd-yyyy}-%i.log">
<PatternLayout pattern="%d %-5p [%t] %C{2} - %m%n" />
<Policies>
<SizeBasedTriggeringPolicy size="50 MB" />
</Policies>
<DefaultRolloverStrategy max="100" />
</RollingFile>
</Route>
</Routes>
</Routing>
<Async name="async" bufferSize="1000" includeLocation="true">
<AppenderRef ref="Routing" />
</Async>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="async" />
</Root>
</Loggers>
This can be done with the RoutingAppender. The FAQ page has a good example config.
This question and the answers was a good starting point to me, however if you are using slf4j inside your implementation and log4j only for the logging side, this could be a bit trickier, which I wanted to share here.
First of all the previously mentioned log4j FAQ with the example could be found for example here: https://logging.apache.org/log4j/2.x/faq.html#separate_log_files
Which was unfortunately not useful to me as we are using MDC inside slf4j instead of ThreadContext. However it was not clear to me at first sight, but you can use ThreadContext and MDC in log4j xml the same. For example:
Routes pattern="$${ctx:macska}">
could refer an
MDC.put("macska", "cica" + Thread.currentThread().getId());
in the code. So as the same as ThreadContext.
But finally my solution was not using the MDC, as I found out log4j has EventLookup: https://logging.apache.org/log4j/2.x/manual/lookups.html#EventLookup
and I modified only my log4j xml with this:
<Routes pattern="$${event:ThreadName}">
Related
I am planning to use log4j2's burstFilter in my application for burst logging management. The idea is to have it disabled until the administrator really wants to use it (I am planning to give an option in the application GUI to take parameters from the user and activate burstFilter accordingly).
I studied its documentation and realized that it's just a config change inside the log4j2.xml file. This xml config will be bundled along with the application anyway and I will include the filter like this..
<Console name="console">
<PatternLayout pattern="%-5p %d{dd-MMM-yyyy HH:mm:ss} %x %t %m%n"/>
<filters>
<Burst level="INFO" rate="16" maxBurst="100"/>
</filters>
</Console>
Now, here rate and maxBurst fields are set to some values which is not what I expect by default. One solution I thought of to just not use <filters> tag by default and explicitly write in the log4j2.xml once user sets these paramters in the GUI like below.
<Burst level="INFO" rate="16" maxBurst="100"/>
This feels like the rookiest solution, so I was wondering if there is any default attribute which I can toggle to switch the filter ON or OFF.
Expectation:
Default log4j2.xml:
<filters>
<Burst Activated="False" rate="16" maxBurst="100"/>
</filters>
If user wants to activate it:
<filters>
<Burst Activated="True" rate="16" maxBurst="100"/>
</filters>
Any help would be appreciated. Thank you
Unless the filter provides a parameter to enable/disable it I don't know of any way to directly toggle it. However, you can implement a toggle if you use the RoutingAppender to switch from an appender with the filter to one without the filter. The disadvantage is, obviously, that you need to have 2 appenders configured which could lead to some duplication of the configuration data. (You could extract some things into common properties which are shared between the two)
Here is a simple example using the ThresholdFilter which demonstrates the above strategy:
First, the log4j2.xml configuration file:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<!--
See how I defined a common property to hold the pattern since it is used
in both appenders below.
-->
<Properties>
<Property name="pattern">%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n</Property>
</Properties>
<Appenders>
<Console name="ConsoleWithFilter" target="SYSTEM_OUT">
<PatternLayout pattern="${pattern}" />
<ThresholdFilter level="ERROR" onMatch="ACCEPT" onMismatch="DENY"/>
</Console>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${pattern}" />
</Console>
<Routing name="Routing">
<Routes pattern="$${ctx:filterToggle}">
<!-- This route is chosen if ThreadContext has value "ENABLE" for key filterToggle. -->
<Route key="ENABLE" ref="ConsoleWithFilter" />
<!-- This is the default route when no others match -->
<Route ref = "Console"/>
</Routes>
</Routing>
</Appenders>
<Loggers>
<Root level="ALL">
<AppenderRef ref="Routing" />
</Root>
</Loggers>
</Configuration>
Next, some Java code to generate logs:
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;
public class SomeClass {
private static final Logger log = LogManager.getLogger();
public static void main(String[] args){
//We start with no value in the ThreadContext for filterToggle
// this should cause all logs to appear in the console.
if(log.isDebugEnabled())
log.debug("This is some debug! (This should appear in console)");
log.info("Here's some info! (This should appear in console)");
log.error("Some error happened! (We will also see this in the console)");
//Now we enable the filter by switching appenders
ThreadContext.put("filterToggle", "ENABLE");
log.info("This should not appear in the console");
log.debug("This also should --not-- appear");
log.fatal("This will appear since it's more specific than ERROR level.");
}
}
Finally, here is the output of the above:
12:22:35.922 [main] DEBUG example.SomeClass - This is some debug! (This should appear in console)
12:22:35.923 [main] INFO example.SomeClass - Here's some info! (This should appear in console)
12:22:35.923 [main] ERROR example.SomeClass - Some error happened! (We will also see this in the console)
12:22:35.924 [main] FATAL example.SomeClass - This will appear since it's more specific than ERROR level.
I hope this helps you!
I have a single java class (a device controller) that is being used to create 5 separate processes. Each of the processes is assigned an identifier. I would like each of the processes to write to its own log file based on its assigned identifier. I have all of the appenders and loggers defined in a shared log4j2.xml config file.
Issue: When I start the first device controller, it successfully writes to the correct log file. However, when I start the second device controller, log4j will roll-over all of the loggers in the log4j2.xml config file and will only write to the log file assigned to the new process. All of the log messages for the first process will go to the rolled-over log file, but new messages are no longer written to its newly rolled-over log file.
(OS: Linux, log4j version: 2.8.2)
Below is an abbreviated version of the log4j2.xml config file that I used.
...
<Appenders>
...
<RollingFile name="RollingFile-1" fileName="/logs/EPDU/Device-1.log" filePattern="/logs/EPDU/Device-1_%d{dd-MMM-yyyy::HH:mm:ss}.log">
<PatternLayout>
...
</PatternLayout>
<Policy>
<OnStartUpTriggeringPolicy minSize="1"/>
<SizeBasedTriggeringPolicy size="20 MB"/>
</Policy>
<DefaultRolloverStrategy fileIndex="nomax"/>
</RollingFile>
...
<RollingFile name="RollingFile-5" fileName="/logs/EPDU/Device-5.log" filePattern="/logs/EPDU/Device-5_%d{dd-MMM-yyyy::HH:mm:ss}.log">
<PatternLayout>
...
</PatternLayout>
<Policy>
<OnStartUpTriggeringPolicy minSize="1"/>
<SizeBasedTriggeringPolicy size="20 MB"/>
</Policy>
<DefaultRolloverStrategy fileIndex="nomax"/>
</RollingFile>
</Appenders>
<Loggers>
<Logger name="device-1" level="trace" additivity="false">
<AppenderRef ref="RollingFile-1" level="debug"/>
</Logger>
...
<Logger name="device-5" level="trace" additivity="false">
<AppenderRef ref="RollingFile-5" level="debug"/>
</Logger>
</Loggers>
The Logger variable is initialized and assigned in the main method after the device identifier is determined similar to the code below:
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
public class DeviceController {
private satic Logger deviceLogger;
public DeviceController(Param param1, Param param2){
...
}
...
public static void main(String[] args) {
/**
* Fancy code to find device identifier...
* String loggerName = (results of fancy code is "device-[1..5]");
*/
deviceLogger = LogManager.getLogger(loggerName);
deviceLogger.info("Start logging stuff in device log.");
new DeviceController(param1, param2);
}
...
}
How can I prevent all of the loggers from rolling over, but instead leave the currently running processes/logs alone as the next process and log is started?
Note: I tried to provide a "Goldilocks" amount of detail to explain the problem. Sorry if I provided too much or not enough information.
Could you show a little bit more of your code? I think your issue comes from the fact you have a static logger. So from your above snippet, I believe you overwrite for each new DeviceController the deviceLogger with a new Logger with the next identifier you fetch. I would guess that at the end, all your logs are being appended to your device-5 log file, aren't they?
Side note, I think it's good practice to use the sl4f interface to declare your logger but then assigned the log4j implementation to your logger.
I'm trying to use Log4j2 in an OSGi environment. I've got it to work so far, but while inspecting the logs from the console and from the file, I noticed that some of them were missing, specifically logs that were called from a static method.
The Log class in the example below is just a convenience class to let my colleagues call the logging functionality more easily (in the example for just a String it seems like overkill of course) through a create method. It does nothing more than create an instance of the Log class that has a Logger internally that calls the respective method from the Log4j2 logger.
My question is: Do I just have a simple error in my project or is Log4j2 not able to log to files from static methods?
Here's a code example to make it a bit more clear:
Log log = Log.testLog();
log.info("non static log" );
That's the code I call from a non-static method.
And here's the testLog()-method:
public static Log testLog() {
Log.create( Log.class ).info( "static log" );
return Log.create( Log.class );
}
Results:
Both #info() calls write to the Console Appender, but only the "non static log" message is written to the file.
Here's my log4j2.xml:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<Console name="Console">
<PatternLayout pattern="!ENTRY %logger{1.} %level %d{DEFAULT} [%t]%n!MESSAGE %msg%n%n"/>
</Console>
<RollingFile name="RollingFile" fileName="${sys:osgi.logfile}.log4j.log"
filePattern="${sys:osgi.logfile}.log4j_bak_%i.log">
<PatternLayout>
<pattern>!ENTRY %logger{1.} %level %d{DEFAULT} [%t]\n!MESSAGE %msg%n%n</pattern>
</PatternLayout>
<Policies>
<SizeBasedTriggeringPolicy size="1 MB"/>
</Policies>
<DefaultRolloverStrategy max="10"
fileIndex="min"/>
</RollingFile>
</Appenders>
<Loggers>
<Root level="TRACE" additivity="false">
<AppenderRef ref="RollingFile"/>
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
Finally found the source of my particular problem, which is OSGi (in this case the Equinox Framework). My application uses the osgi.logfile system property to point to the location where the logs should be saved at.
Unfortunately Equinox not only creates that property, but also changes it at startup to a different location. For Log4j2 I used ${sys:osgi.logfile} to get this system property, but because a few particular plugins started so early, Log4j2 still had the wrong (aka. old) location configured for these plugins (more specifically: their LoggerContext).
What helped me in this case was a simple LoggerContext.reconfigure() on the LoggerContext that still had the old location.
I am trying to configure a log4j2 config to route messages to different logfiles for a multithreaded program via threadname.
Here is what I have, so far (relevant to log4j2 config):
|-/src/main/java/log4j2/plugins
|-- ThreadLookup.java
|-/src/main/resources
|-- log4j2.xml
ThreadLookup.java:
package log4j2.plugins;
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.lookup.StrLookup;
#Plugin(name="threadLookup", category=StrLookup.CATEGORY)
public class ThreadLookup implements StrLookup {
#Override
public String lookup(String key) {
return Thread.currentThread().getName();
}
#Override
public String lookup(LogEvent event, String key) {
// Check event first:
if (event.getThreadName() != null) {
return event.getThreadName();
}
// Fallback to key if event doesn't define a threadName:
return this.lookup(key);
}
}
Log4j2.xml (It is my understanding that the packages attribute of Configuration should read in ThreadLookup.java and based on the annotation create a new threadLookup prefix to let me call lookup(String key) with whatever value I want to -- in this case I am not using a specific value because this class will only do a threadName lookup):
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="error" strict="true" schema="Log4J-V2.0.xsd"
packages="log4j2.plugins">
<Properties>
<Property name="logMsgPattern">%date{yyyy/MM/dd HH:mm:ss.SSS} %-5level ${sys:pid}[%thread] %class %method:%line - %message%n</Property>
</Properties>
<Appenders>
<Console name="console" target="SYSTEM_OUT" >
<PatternLayout pattern="${logMsgPattern}" />
</Console>
<Routing name="routing">
<Routes pattern="$${threadLookup:threadName}">
<Route>
<RollingFile name="RollingFile-${threadLookup:threadName}"
fileName="${sys:log4j.dir}/thread-${threadLookup:threadName}.log"
filePattern="${sys:log4j.dir}/thread-${threadLookup:threadName}-%i.log.gz">
<PatternLayout pattern="${logMsgPattern}"/>
<Policies>
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
<!-- Config for other appenders snipped -->
</Appenders>
<Loggers>
<!-- Config for other loggers snipped -->
<Root level="${sys:log4j.console.threshold}">
<AppenderRef ref="rootOut" level="trace" />
<AppenderRef ref="rootErr" level="error" />
<AppenderRef ref="console" level="${sys:log4j.console.threshold}" />
<AppenderRef ref="routing" level="trace" />
</Root>
</Loggers>
</Configuration>
However, when I launch my app, it just creates an additional file called thread-${threadLookup (no extension) in my log directory. It also never hits any breakpoints in ThreadLookup.java.
How can I register the plugin with log4j2 (I was using version 2.2, I also tried 2.3)? Note, I am using a spring-framework 4.1.7 project, if that helps at all; I use maven for the project as well, but I am only using it to resolve dependencies, I build the project via ant script.
UPDATE
When I build the script via ant, I do actually get a Log4j2Plugins.dat that shows up in my classpath (-cp resources:bin), but it doesn't seem to effect the outcome of the logs that are generated on the server:
$ find bin/META-INF/ -type f
bin/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
$ cat bin/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
lookup
threadlookupog4j2.plugins.ThreadLookup
threadLookup
$ vi bin/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat
^A^Flookup^A^Lthreadlookup^[log4j2.plugins.ThreadLookup^LthreadLookup
$ find logs -type f -name "thread-*"
logs/thread-${threadLookup:threadName}.log
logs/thread-${threadLookup:threadName}-1.log.gz</pre>
Thanks
I ended up finding out that the issue was that my Plugin name cannot be camelCased.
I was debugging through PluginManager.java (Log4j2 - 2.3), on line 169 in private static void mergeByName(final Map<String, PluginType<?>> newPlugins, final List<PluginType<?>> plugins), and I saw that I had the following properties to go into newPlugins.put(key, pluginType);
key: threadlookup,
pluginType: PluginType [pluginClass=class log4j2.plugins.ThreadLookup, key=threadlookup, elementName=threadLookup, isObjectPrintable=false, isDeferChildren==false, category=Lookup]
After seeing that, I modified my Routing appender in my log4j2.xml config to the following (without needing to change the annotation in my Plugin class that implemented StrLookup) and it worked:
<Routing name="routing">
<Routes pattern="$${threadlookup:threadName}">
<Route>
<RollingFile name="RollingFile-${threadlookup:threadName}"
fileName="${sys:log4j.dir}/thread-${threadlookup:threadName}.log"
filePattern="${sys:log4j.dir}/thread-${threadlookup:threadName}-%i.log.gz">
<PatternLayout pattern="${logMsgPattern}"/>
<Policies>
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
Hopefully this can help others out, as I had to spend a few days to figure this out and I didn't find this in any of the documentation or questions I was reviewing for Log4j2.
Thanks!
I need to configure logger to roll the logs by time (hourly or daily), by size and on start. I searched a lot and red log4j docs, now i am confused.
Looks like it can be done by following ways:
using Simon library
creating custom classes
using log4j 2 beta
I can't add new 3rd party libs to my project (log4j 2 beta is already added) so i consider the last two options.
What exactly classes should i create if i go with the 2nd option? Should it be appender, rollingPolicy or triggeringPolicy?
Will log4j2 really support that?
Thanks for help, Yuri
I don't think you need to create any classes, you should be able to achieve what you want with configuration only. The Log4J2 documentation provides a number of examples.
Here is a good place to start: http://logging.apache.org/log4j/2.x/manual/appenders.html#RollingFileAppender
If this is not sufficient, don't hesitate to ask a question on the log4j-2 user mailing list.
Config example:
<?xml version="1.0" encoding="UTF-8"?><configuration name="install" status="info">
<appenders>
<!-- ################# InstallAppender ############################### -->
<RollingFile name="InstallAppender"
fileName="${sys:installation.path}/installation/logs/post_install.log"
filePattern="${sys:installation.path}/installation/logs/post_install.log">
<PatternLayout>
<pattern>%d{dd/MM/yyyy HH:mm:ss} %-5p [%t] [%c{1}] %m%n</pattern>
</PatternLayout>
<Policies>
<OnStartupTriggeringPolicy/>
</Policies>
<DefaultRolloverStrategy max="1"/>
</RollingFile>
</appenders>
<loggers>
<root level="info">
<appender-ref ref="InstallAppender"/>
</root>
</loggers>