Develop Reference

How to properly configure log4j2 programmatically?

I am trying to setup log4j2 to write logs using the RollingFileAppender. I want to configure the logging system programmatically instead of using XML files.
Here is what I tried (mostly the same as the docs at https://logging.apache.org/log4j/2.x/manual/customconfig.html#Configurator):
public static void configure(String rootLevel, String packageLevel) {
ConfigurationBuilder<BuiltConfiguration> builder = ConfigurationBuilderFactory
.newConfigurationBuilder();
builder.setConfigurationName("RollingBuilder");
builder.setStatusLevel(Level.TRACE);
// Create a rolling file appender
LayoutComponentBuilder layoutBuilder = builder.newLayout("PatternLayout")
.addAttribute("pattern", "%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c{1}:%L - %m%n");
ComponentBuilder triggeringPolicy =
builder
.newComponent("Policies")
.addComponent(
builder
.newComponent("SizeBasedTriggeringPolicy")
.addAttribute("size", "200M")
);
AppenderComponentBuilder appenderBuilder =
builder
.newAppender("rolling", "RollingFile")
.addAttribute("fileName", "log")
.addAttribute("filePattern", "log.%d.gz")
.add(layoutBuilder)
.addComponent(triggeringPolicy);
builder.add(appenderBuilder);
// Create new logger
LoggerComponentBuilder myPackageLoggerBuilder =
builder.newLogger("com.mypackage", packageLevel)
.add(builder.newAppenderRef("rolling"))
.addAttribute("additivity", false);
builder.add(myPackageLoggerBuilder);
RootLoggerComponentBuilder rootLoggerBuilder =
builder
.newRootLogger(rootLevel)
.add(builder.newAppenderRef("rolling"));
builder.add(rootLoggerBuilder);
// Initialize logging
Configurator.initialize(builder.build());
}
I call the configure() method at the start of the main method. A file named log is created when I run my program, but all log output goes to standard out and the log file remains empty.
Can someone help figure out what is wrong with my config?
I am not using any log4j configuration file, if that makes a difference. Also using the slf4j API in my code. Dependencies -
org.apache.logging.log4j:log4j-api:2.11.1
org.apache.logging.log4j:log4j-core:2.11.1
org.apache.logging.log4j:log4j-slf4j-impl:2.11.1
org.slf4j:slf4j-api:1.7.25

First, this answer is in response to the additional information provided in your comment.
My requirement is that I want to control log levels for different
packages via command line flags when I launch my program
Since you want to use program arguments to control your log level I suggest you take a look at the Main Arguments Lookup and the Routing Appender. Using these two features together you can set up your logging configuration to send log events to the appropriate appender based on program arguments.
I will provide a simple example to help guide you.
First here is an example Java class that generates some log events:
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.core.lookup.MainMapLookup;
public class SomeClass {
private static Logger log = LogManager.getLogger();
public static void main(String[] args){
MainMapLookup.setMainArguments(args);
if(log.isDebugEnabled())
log.debug("This is some debug!");
log.info("Here's some info!");
log.error("Some error happened!");
}
}
Next the configuration file for log4j2 (see comments in the code for details):
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
<Appenders>
<Routing name="myRoutingAppender">
<!-- log events are routed to appenders based on the logLevel program argument -->
<Routes pattern="$${main:logLevel}">
<!-- If the logLevel argument is followed by DEBUG this route is used -->
<Route ref="DebugFile" key="DEBUG" />
<!-- If the logLevel argument is omitted or followed by any other value this route is used -->
<Route ref="InfoFile" />
</Routes>
</Routing>
<!-- This appender is not necessary, was used to test the config -->
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
</Console>
<!-- Below are the 2 appenders used by the Routing Appender from earlier -->
<File name="DebugFile" fileName="logs/Debug.log" immediateFlush="true"
append="false">
<PatternLayout
pattern="%d{yyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
</File>
<File name="InfoFile" fileName="logs/Info.log" immediateFlush="true"
append="false">
<PatternLayout
pattern="%d{yyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
<LevelRangeFilter minLevel="FATAL" maxLevel="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
</File>
</Appenders>
<Loggers>
<!-- Root logger is set to DEBUG intentionally so that debug events are generated.
However, events may be ignored by the LevelRangeFilter depending on where they
are routed by the Routing Appender
-->
<Root level="DEBUG">
<AppenderRef ref="Console" />
<AppenderRef ref="myRoutingAppender" />
</Root>
</Loggers>
</Configuration>
Using this configuration if you do not provide a "logLevel" argument then by default log events are routed to the "InfoFile" appender and any events that are more specific than INFO are ignored via the LevelRangeFilter.
If a "logLevel" argument is provided and is followed by "DEBUG" then log events are routed to the "DebugFile" appender and none of the events are ignored.
Note that I did try to use lookups to set the log level but it appears that log level parameters can't be configured via lookups. That is why I had to use this alternative approach. My one concern with this approach is that as I noted in the comments within the configuration file the log level must be kept at DEBUG which means you always generate the DEBUG events even if you don't use them. This might impact performance. A workaround would be to use your program argument to determine whether you need to generate the debug events. For example:
Normally you would use:
if(log.isDebugEnabled())
log.debug("This is some debug!");
but when using the configuration above you would use something like:
if("DEBUG".equals(args[1]))
log.debug("This is some debug!");
and you could make that more efficient using an enum (maybe even use the Level class provided by log4j2) if you needed to.
I hope this helps get you started.

Please find the sample Root log and finally needs to call reconfigure function.
RootLoggerComponentBuilder rootLogger = builder
.newRootLogger(Level.ALL)
.add(builder.newAppenderRef("LogToRollingFile"));
LoggerComponentBuilder logger = builder
.newLogger("MyClass",Level.ALL)
.addAttribute("additivity", false);
Configurator.reconfigure(builder.build());

Log4J API to log long messages in temporary files

Am trying to understand if there are any best practices/utilities/out of the box functionality for logging messages that are quite long (typically json file responses/xmls/CSVs etc).
While logging as part of my application I do something like
log.info("Incompatible order found {}", order.asJson())
The problem being that the asJson() representation could be pretty long. In the log files the actual json is only relevant say 1% of the time. So it is important enough to retain but big enough to make me lose focus when I am trawling through logs.
Is there anyway I can could do something like
log.info("Incompatible order found, file dumped at {}", SomeUtility.dumpString(order.asJson()));
where the utility dumps the file into a location consistent with other log files and then in my log file I can see the following
Incompatible order found, file dumped at /abc/your/log/location/tmpfiles/xy23nmg
Key things to note being
It would be preferable to simply use the existing logging api to somehow configure this so that the location of these temp files are the same as log itself this way it goes through the cleanup cycle after N days etc just like the actual log files.
I can obviously write something but I am keen on existing utilities or features if already available withing log4j
I am aware that when logs such as these are imported into analysis systems like Splunk then only the filenames will be present without the actual files and this is okay.

(Suggestion was given based on logback instead of log4j2. However I believe similar facilities exists in log4j2)
In Logback, there is a facility called SiftingAppender, which allow separate appender (e.g. file appender) to be created according to some discriminator (e.g. in MDC).
So, by configuring an appender (e.g. orderDetailAppender) which separates file based on a discriminator (e.g. by putting order ID in MDC), and make use of a separate logger to connect to the appender, this should give you the result you want:
pseudo code:
logback config:
<appender name="ORDER_DETAIL_APPENDER" class="ch.qos.logback.classic.sift.SiftingAppender">
<!-- use MDC disciminator by default, you may choose/develop other appropriate discrimator -->
<sift>
<appender name="ORDER-DETAIL-${ORDER_ID}" class="ch.qos.logback.core.FileAppender">
....
</appender>
</sift>
</appender>
<logger name="ORDER_DETAIL_LOGGER">
<appender-ref name="ORDER_DETAIL_APPENDER"/>
</logger>
and your code looks like:
Logger logger = LoggerFactory.getLogger(YourClass.class); // logger you use normally
Logger orderDetailLogger = LoggerFactory.getLogger("ORDER_DETAIL_LOGGER");
.....
MDC.put("ORDER_ID", order.getId());
logger.warn("something wrong in order {}. Find the file in separate file", order.getId());
orderDetailLogger.warn("Order detail:\n{}", order.getJson());
// consider making order.getJson() an lambda, or wrap the line with logger
// level checking, to avoid json construction even not required to log
MDC.remove("ORDER_ID");

The simplest approach is to use a separate logger for the JSON dump, and configure that logger to put its output in a different file.
As an example:
private static final Logger log = LogManager.getLogger();
private static final Logger jsonLog = LogManager.getLogger("jsondump");
...
log.info("Incompatible order found, id = {}", order.getId());
jsonLog.info("Incompatible order found, id = {}, JSON = {}",
order.getId(), order.asJson());
Then, configure the logger named jsondump to go to a separate file, possibly with a separate rotation schedule. Make sure to set additivity to false for the JSON logger, so that messages sent to that logger won't be sent to the root logger as well. See Additivity
in the Log4J docs for details.
Example (XML configuration, Log4J 2):
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
<Appenders>
<!-- By setting only filePattern, not fileName, all log files
will use the pattern for naming. Files will be rotated once
an hour, since that's the most specific unit in the
pattern. -->
<RollingFile name="LogFile" filePattern="app-%d{yyyy-MM-dd HH}00.log">
<PatternLayout pattern="%d %p %c [%t] %m %ex%n"/>
<Policies>
<TimeBasedTriggeringPolicy/>
</Policies>
</RollingFile>
<!-- See comment for appender named "LogFile" above. -->
<RollingFile name="JsonFile" filePattern="json-%d{yyyy-MM-dd HH}00.log">
<!-- Omitting logger name for this appender, as it's only used by
one logger. -->
<PatternLayout pattern="%d %p [%t] %m %ex%n"/>
<Policies>
<TimeBasedTriggeringPolicy/>
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<!-- Note that additivity="false" to prevent JSON dumps from being
sent to the root logger as well. -->
<Logger name="jsondump" level="debug" additivity="false">
<appender-ref ref="JsonFile"/>
</Logger>
<Root level="debug">
<appender-ref ref="LogFile"/>
</Root>
</Loggers>
</Configuration>

Log4j2 does not change logging level at runtime

Yes, I've read all the related questions. I am using log4j2 (tried both version 2.4 and updating to latest, version 2.6.2).
I have a small utility program for customers. I'm keen to keep exposed configurations at minimum. But for problematic cases, I'd also want to add a -debug flag to enable debug logs at runtime.
Here is my code to enable debug logging
private static void enableDebugLogs(){
LoggerContext ctx = (LoggerContext) LogManager.getContext();
LoggerConfig log = ctx.getConfiguration().getRootLogger();
System.out.println(log.getLevel()); // INFO
log.setLevel(Level.DEBUG);
System.out.println(log.getLevel()); // DEBUG
ctx.updateLoggers();
System.out.println(ctx.getRootLogger().getLevel()); // DEBUG, hey it works, right?
}
But it does not actually work with any of these cases:
enableDebugLogs();
logger.debug("Debug mode on"); // static, already made logger. Level did not change
LogManager.getLogger(Main.class).debug("Debug"); // Nope, not printing
Logger root = LogManager.getRootLogger();
root.info("Level: " + root.getLevel()); // Level: INFO, should be DEBUG
The utility program is finished usually in less than 30 seconds, so the change should be instant. Here is the log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} %-5level - %msg%n"/>
</Console>
<RollingFile name="File" fileName="program_name.log" filePattern="program_name-archived.log">
<PatternLayout>
<Pattern>%d{HH:mm:ss.SSS} %-5level - %msg%n</Pattern>
</PatternLayout>
<Policies>
<SizeBasedTriggeringPolicy size="10 KB" />
</Policies>
<DefaultRolloverStrategy min="1" max="1"/>
</RollingFile>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="Console"/>
<AppenderRef ref="File"/>
</Root>
</Loggers>
</Configuration>
Is the problem with usings AppenderRefs? Can I somehow tell the Appenders to update logging level from Root logger?

Found the real issue. Had to use:
LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
instead of
LoggerContext ctx = (LoggerContext) LogManager.getContext();
API stating the difference being "returns the LoggerContext" and "returns the current LoggerContext". And I clearly missed this bit of information for the version without boolean parameter:
"WARNING - The LoggerContext returned by this method may not be the LoggerContext used to create a Logger for the calling class."

You can change switch the logging configuration between two or multiple log4j2.xml files.
For example, create two log4j2.xml files with different configurations. log4j2.xml & log4j2-debug.xml and pass it to below code.
ConfigurationFactory configFactory = XmlConfigurationFactory.getInstance();
ConfigurationFactory.setConfigurationFactory(configFactory);
LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
ClassLoader classloader = Thread.currentThread().getContextClassLoader();
InputStream inputStream = classloader.getResourceAsStream(logFileName);
ConfigurationSource configurationSource = new ConfigurationSource(inputStream);
ctx.start(configFactory.getConfiguration(ctx, configurationSource));

To reconfigure log4j2 after it's initialization, the documentation give you two ways :
Using the config file, you can enable the automatic reconfiguration (it's the preferred way) so that modifying the file after initialization will be reflected on your runtime : http://logging.apache.org/log4j/2.x/manual/configuration.html#AutomaticReconfiguration
Using ConfigurationBuilder you can reconfigure log4j programatically (I think it's what you are looking for), see the "Reconfigure Log4j Using ConfigurationBuilder with the Configurator" paragraph of this page : http://logging.apache.org/log4j/2.x/manual/customconfig.html

In addition to the above, to debug log4j initialization, set <Configuration status="trace" in the beginning of the configuration file. Log4j2's internal status logs will print to the console. This may help troubleshooting.

Here is a little known fact.
If you set your ROOT_LOGGER level in your log4j2.xml or equivalent properties to say DEBUG, then setting the level dynamically to something lower did not work for me; in other words, setting dynamically had no effect.
ONLY when I change the setting in log4j2.xml to TRACE and then using the below to change the log level did it work for me:
Level level = Level.valueOf(this.consoleLogLevel.toUpperCase());
//Dynamically set the log level for ALL loggers
LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
Configuration config = ctx.getConfiguration();
LoggerConfig loggerConfig = config.getLoggerConfig(LogManager.ROOT_LOGGER_NAME);
loggerConfig.setLevel(level);
ctx.updateLoggers();
And here is my simple configuration file:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Root level="TRACE">
<AppenderRef level="TRACE" ref="Console" />
</Root>
</Loggers>
</Configuration>
And here is a TestNG test you can refactor to verify the above:
#Test
public void givenCut_whenLoggingWithERRORLevel_LoggingOutputIsAtERRORLevelAndHigherOnly() {
try {
lcaProperties.setLogLevel(Level.ERROR.name());
GlobalLogger globalLogger = GlobalLogger.builder().lcaServiceProperties(lcaProperties).build();
assertNotNull(globalLogger.getConsoleLogger());
assertNotNull(globalLogger.getFunctionalDbLogger());
assertEquals(globalLogger.getConsoleLogger().getLevel(), Level.ERROR);
assertEquals(log.getLevel(), Level.ERROR);
System.out.println("log.getLogLevel() = " + log.getLevel());
System.out.println("globalLogger.getConsoleLogger().getLevel() = " + globalLogger.getConsoleLogger().getLevel());
log.fatal("logLevel::"+log.getLevel()+"::log.fatal");
log.error("logLevel::"+log.getLevel()+"::log.error");
log.warn("logLevel::"+log.getLevel()+"::log.warn");
log.info("logLevel::"+log.getLevel()+"::log.info");
log.debug("logLevel::"+log.getLevel()+"::log.debug");
log.trace("logLevel::"+log.getLevel()+"::log.trace");
globalLogger.getConsoleLogger().fatal("logLevel::"+globalLogger.getConsoleLogger().getLevel()+"::globalLogger.getConsoleLogger().fatal");
globalLogger.getConsoleLogger().error("logLevel::"+globalLogger.getConsoleLogger().getLevel()+"::globalLogger.getConsoleLogger().debug");
globalLogger.getConsoleLogger().warn("logLevel::"+globalLogger.getConsoleLogger().getLevel()+"::globalLogger.getConsoleLogger().debug");
globalLogger.getConsoleLogger().info("logLevel::"+globalLogger.getConsoleLogger().getLevel()+"::globalLogger.getConsoleLogger().debug");
globalLogger.getConsoleLogger().debug("logLevel::"+globalLogger.getConsoleLogger().getLevel()+"::globalLogger.getConsoleLogger().debug");
globalLogger.getConsoleLogger().trace("logLevel::"+globalLogger.getConsoleLogger().getLevel()+"globalLogger.getConsoleLogger()::log.trace");
} catch (Exception e) {
fail();
log.error(e);
}
}

Log4j 2.0 - No Logs appearing in the Log Files - Trying Multiple Loggers in the same Class using log4j2.xml

Below is the log4j2.xml file that I have created. I have configured async_file.log for Asynchronous Logging and regular_file.log for regular and synchronous logging. The problem is that the log files get created, but the size of the files is zero and with no logs. All logs are coming to server.log file (JBOSS) and not to the 2 files that I had got configured for (async_file.log and regular_file.log).
Please let me know why the logs are NOT going to the log files that I have configured. Please help me with this or give me some direction or hint.
I am calling the two different loggers in the same class file by name DCLASS as shown below:
private static final transient Logger LOG = Logger.getLogger(DCLASS.class);
private static final transient Logger ASYNC_LOG = Logger.getLogger("ASYNC");
I have included the following jars in the Class Path:
1. log4j-api-2.0-beta8.jar
2. log4j-core-2.0-beta8.jar
3. disruptor-3.0.0.beta1.jar
My log4j2.xml is as below:
<?xml version="1.0" encoding="UTF-8"?>
<configuration status="INFO">
<appenders>
<!-- Async Loggers will auto-flush in batches, so switch off immediateFlush. -->
<FastFile name="AsyncFastFile" fileName="../standalone/log/async_file.log"
immediateFlush="false" append="true">
<PatternLayout>
<pattern>%d %p %class{1.} [%t] %location %m %ex%n</pattern>
</PatternLayout>
</FastFile>
<FastFile name="FastFile" fileName="../standalone/log/regular_file.log"
immediateFlush="true" append="true">
<PatternLayout>
<pattern>%d %p %class{1.} [%t] %location %m %ex%n</pattern>
</PatternLayout>
</FastFile>
</appenders>
<loggers>
<!-- pattern layout actually uses location, so we need to include it -->
<asyncLogger name="ASYNC" level="trace" includeLocation="true">
<appender-ref ref="AsyncFastFile"/>
</asyncLogger>
<root level="info" includeLocation="true">
<appender-ref ref="FastFile"/>
</root>
</loggers>
</configuration>

The reason why the logs were not coming to the log files is because, I was using 'Logger' instead of 'LogManager'.
In the code, I had
private static final transient Logger ASYNC_LOG = Logger.getLogger("ASYNC");
The code should have been
private static final transient Logger ASYNC_LOG = Logmanager.getLogger("ASYNC");
When it is 'logger', then the compiler is looking into 'Log4j API' and when it is 'LogManager' it is looking into 'Log4j2 API'. Since I have configured everything to use Log4j2, by changing logger to LogManager, the logs started coming to the log files as expected.

How to log multiple threads in different log files?

I have a JAVA class that starts various threads that have unique IDs.
Each thread should log into a unique log file, named after the ID.log.
Because I only get the unique ID at runtime, I have to configure Log4J programatically:
// Get the jobID
myJobID = aJobID;
// Initialize the logger
myLogger = Logger.getLogger(myJobID);
FileAppender myFileAppender;
try
{
myFileAppender = new FileAppender(new SimpleLayout(), myJobID + ".log", false);
BasicConfigurator.resetConfiguration();
BasicConfigurator.configure(myFileAppender);
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
Now this works fine if I start jobs sequentially - but when I start 2 threads (of the same class) simultaneously, both logs are created but the logs are mixed up: The second thread logs into the first as well as the second log.
How could I make sure that each instance is unique ?
I already tried to give a unique name to each logger instance, but it did not change anything.

Logback has a special appender called SiftingAppender which provides a very nice solution to the type of problems you describe. A SiftingAppender can be used to separate (or sift) logging according to any runtime attribute, including thread id.

For log4j v2 you can use RoutingAppender to dynamically route messages. You can put value for key 'threadId' into the ThreadContext map and then use this id as a part of the file name. There is an example which I have easily applied for the same purpose as yours. See http://logging.apache.org/log4j/2.x/faq.html#separate_log_files
Be aware when putting values into ThreadContext map: "A child thread automatically inherits a copy of the mapped diagnostic context of its parent." So if you have put a value for key 'threadId' into the parent thread and eventually created multiple threads from it, then all child threads will inherit the value of 'threadId' value. I was not able to simply override this value by using put() one more time - you need to use ThreadContext.clear() or explicitly remove() the value from thread context map.
Here is my working log4j.xml:
<?xml version="1.0" encoding="UTF-8"?>
<configuration status="WARN">
<properties>
<property name="logMsgPattern">%d{HH:mm:ss} %-5level - %msg%n</property>
<property name="logDir">test logs</property><!-- ${sys:testLogDir} -->
</properties>
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${logMsgPattern}"/>
</Console>
<Routing name="Routing">
<Routes pattern="$${ctx:threadId}">
<Route>
<RollingFile name="RollingFile-${ctx:threadId}" fileName="${logDir}/last-${ctx:threadId}.log" filePattern="${logDir}/%d{yyyy-MM-dd}/archived_%d{HH-mm}-${ctx:threadId}.log">
<PatternLayout pattern="${logMsgPattern}"/>
<Policies>
<OnStartupTriggeringPolicy />
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
</appenders>
<loggers>
<root level="debug">
<appender-ref ref="Console" level="debug" />
<appender-ref ref="Routing" level="debug"/>
</root>
</loggers>
</configuration>

#havexz 's approach is quite good: writing everything to the same log file and using nested diagnostic contexts.
If your concern is about several JVMs writing to the same FileAppender, then i'd suggest two things:
using SLF4J as a logging facade
using logback as logging implementation, in prudent mode
In prudent mode, FileAppender will safely write to the specified file,
even in the presence of other FileAppender instances running in
different JVMs, potentially running on different hosts.

Here is the snippet of the Routing code from a working log4j.xml file.
<Appenders>
<Console name="ConsoleAppender" target="SYSTEM_OUT">
<PatternLayout>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %-50logger{4}: %msg%n</pattern>
</PatternLayout>
</Console>
<Routing name="RoutingAppender">
<Routes pattern="${ctx:logFileName}">
<!-- This route is chosen if ThreadContext has a value for logFileName.
The value dynamically determines the name of the log file. -->
<Route>
<RollingFile name="Rolling-${ctx:logFileName}"
fileName="${sys:log.path}/${ctx:logFileName}.javalog"
filePattern="./logs/${date:yyyy-MM}/${ctx:logFileName}_%d{yyyy-MM-dd}-%i.log.gz">
<PatternLayout>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %-50logger{4}: %msg%n</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="6"
modulate="true" />
<SizeBasedTriggeringPolicy size="10 MB" />
</Policies>
</RollingFile>
</Route>
<!-- This route is chosen if ThreadContext has no value for key logFileName. -->
<Route key="${ctx:logFileName}" ref="ConsoleAppender" />
</Routes>
</Routing>
</Appenders>
<loggers>
<root level="debug">
<appender-ref ref="RoutingAppender" level="debug" />
</root>
</loggers>
The key 'logFileName' can be added to the thread context map in the run() method of the Runnable class as follows,
public class SomeClass implements Runnable{
private int threadID;
public SomeClass(int threadID){
this.threadID=threadID;
}
#Override
public void run() {
String logFileName = "thread_log_"+ threadID;
ThreadContext.put("logFileName", logFileName);
//Some code
ThreadContext.remove("threadId");
}
}
In addition, the correct log4j packages must be imported, as shown below.
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;
Please note that the following imports will not work. LogManager and Logger must also come from org.apache.logging.log4j.
import org.apache.log4j.LogManager;
import org.apache.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;

As far as I can tell ThreadLocal API was designed to do what you describe.
Code like below would establish per-thread loggers each using own (per-thread) FileAppender:
/**
* usage: threadLocalLogger.get().info("hello thread local logger")
*/
static ThreadLocal<Logger> threadLocalLogger = newThreadLocalLogger("myJobId");
private static ThreadLocal<Logger> newThreadLocalLogger(final String myJobID) {
return new ThreadLocal<Logger>() {
#Override
protected Logger initialValue() {
return logger(myJobID, Thread.currentThread().getId());
}
};
}
private static Logger logger(String myJobID, long threadId) {
// Initialize the logger
String loggerId = myJobID + "-" + threadId;
Logger myLogger = Logger.getLogger(loggerId);
FileAppender myFileAppender;
try
{
myFileAppender = new FileAppender(new SimpleLayout(),
loggerId + ".log", false);
BasicConfigurator.resetConfiguration();
BasicConfigurator.configure(myFileAppender);
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
return myLogger;
}

What about adding a static instance counter variable to your class. Then you would need a synchronized method which increases the counter for each object created and create the log file name from that value. Something like this:
class yourClass {
private static int cnt = 0;
public yourClass(){
...
initLogger();
}
private synchronized initLogger(){
yourClass.cnt++;
myJobid = yourClass.cnt;
//include your logging code here
}
}