Develop Reference

Log4j2.17.2 xml config file in a multithreaded environment throws errors when included routing tags

Thanks for trying to help the question. I have been trying to upgrade the log4j from version 2.2.x to 2.17.2 for that I have included following starters, the spring boot version is 2.6.6 -
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
<version>${spring.boot.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
<version>${spring.boot.version}</version>
</dependency>
<dependency>
<groupId>com.lmax</groupId>
<artifactId>disruptor</artifactId>
<version>3.4.4</version>
</dependency>
but when I include the routing tag as per my simplified (& partial) logging-config as below, it throws several runtime errors -
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" monitorInterval="30">
<property name="log-path" value="/path/" />
<!-- log levels precedence : ALL < DEBUG < INFO < WARN < ERROR < FATAL <
OFF -->
<property name="log-level" value="INFO" />
<Appenders>
<appender name="console" target="SYSTEM_OUT"
class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n"
</pattern>
</encoder>
</appender>
<Routing name="Routing">
<Routes pattern="$${ctx:docURN}">
<Route>
<appender name="LogFile"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log-path}/filename.log</file>
<rollingPolicy
class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log-path}/filename-%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
<encoder>
<pattern>[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} tx.id=
${ctx:docURN} - %msg%n</pattern>
</encoder>
</appender>
</Route>
</Routes>
</Routing>
</Appenders>
<Loggers>
<logger name="com.name" level="${log-level}" additivity="false">
<appender-ref ref="LogFile" level="${log-level}" />
<appender-ref ref="Routing" level="${log-level}" />
</logger>
<root level="info" additivity="false">
<appender-ref ref="Routing" />
</root>
</Loggers>
</Configuration>
I apologize, if the question appears duplicate but I have searched all over the stackoverflow and internet and the apache documentation is so overwhelming, it seems to be correct as per the documentation.
Thank you again for your help.
Edit - runtime errors thrown are for all the nodes in the logging-config.xml -
java.lang.IllegalStateException: Logback configuration error detected: ERROR in ch.qos.logback.core.joran.spi.Interpreter#10:13 - no applicable action for [Appenders], current ElementPath is [[Configuration][Appenders]]
Edit2 - I tried following configuration example as it is from https://logging.apache.org/log4j/2.x/manual/configuration.html , still it throws those errors.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="debug" name="RoutingTest" packages="org.apache.logging.log4j.test">
<Properties>
<Property name="filename">target/rolling1/rollingtest-$${sd:type}.log</Property>
</Properties>
<ThresholdFilter level="debug"/>
<Appenders>
<Console name="STDOUT">
<PatternLayout pattern="%m%n"/>
<ThresholdFilter level="debug"/>
</Console>
<Routing name="Routing">
<Routes pattern="$${sd:type}">
<Route>
<RollingFile name="Rolling-${sd:type}" fileName="${filename}"
filePattern="target/rolling1/test1-${sd:type}.%i.log.gz">
<PatternLayout>
<pattern>%d %p %c{1.} [%t] %m%n</pattern>
</PatternLayout>
<SizeBasedTriggeringPolicy size="500" />
</RollingFile>
</Route>
<Route ref="STDOUT" key="Audit"/>
</Routes>
</Routing>
</Appenders>
<Loggers>
<Logger name="EventLogger" level="info" additivity="false">
<AppenderRef ref="Routing"/>
</Logger>
<Root level="error">
<AppenderRef ref="STDOUT"/>
</Root>
</Loggers>
</Configuration>
Edit3 - This is the resultant logging-config.xml I have now -
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" monitorInterval="30">
<Properties>
<property name="log-path">/logfiles//</property>
<!-- log levels precedence : ALL < DEBUG < INFO < WARN < ERROR < FATAL
< OFF -->
<property name="log-level">INFO</property>
</Properties>
<Appenders>
<Console name="console" target="SYSTEM_OUT">
<PatternLayout
pattern="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n" />
</Console>
<Routing name="Routing">
<Routes pattern="$${ctx:docURN}">
<Route>
<RollingFile name="LogFile" append="true"
fileName="${log-path}/-services.log" filePattern="${log-path}/-services-%d{yyyy-MM-dd}.log">
<PatternLayout>
<pattern>[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} tx.id= ${ctx:docURN} - %msg%n"></pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"
modulate="true" />
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
<Routing name="ErrorRouting">
<Routes pattern="$${ctx:docURN}">
<Route>
<RollingFile name="ErrorLogFile" append="true"
fileName="${log-path}/-services-error.log" filePattern="${log-path}/-services-error-%d{yyyy-MM-dd}.log">
<PatternLayout>
<pattern>[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} tx.id= ${ctx:docURN} - %msg%n"></pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"
modulate="true" />
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
<RollingFile name="ServerLogFile" fileName="${log-path}/Server.log" filePattern="${log-path}/Server-%d{yyyy-MM-dd}.log">
<PatternLayout>
<Pattern>[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true" />
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Root level="info" additivity="false">
<AppenderRef ref="ServerLogFile" />
<AppenderRef ref="Routing" />
</Root>
<Logger name="com.name" level="${log-level}"
additivity="false">
<appender-ref ref="LogFile" level="${log-level}" />
<appender-ref ref="ErrorLogFile" level="error" />
<appender-ref ref="ServerLogFile" level="${log-level}" />
<appender-ref ref="Routing" level="${log-level}" />
<AppenderRef ref="ErrorRouting" level="error"/>
</Logger>
</Loggers>
</Configuration>

Log4j2 not rolling over log files

I am making a game with LWJGL3 and am using log4j2 for logging. I am trying to setup my logger which works however each time the program is executed it just overwrites the lastest.log file instead of compressing it and then overwriting latest.log. I tried setting the minSize of the OnStartupTriggeringPolicy because it defaults to 1 and my file size is currently extremely small but that didn't help. Does anyone have any idea on how to get this working.
log4j2.xml
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{YYYY-MM-dd HH:mm:ss} [%level] [%t] - %msg%n" />
</Console>
<RollingFile name="RollingFile" filename="log/latest.log"
filePattern="${logPath}/%d{YYYYMMddHHmmss}.log.gz"
append="false">
<PatternLayout pattern="%d{YYYY-MM-dd HH:mm:ss} [%level] [%t] - %msg%n" />
<Policies>
<OnStartupTriggeringPolicy minSize="0"/>
<SizeBasedTriggeringPolicy size="250 MB"/>
</Policies>
<DefaultRolloverStrategy max="8"/>
</RollingFile>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="Console" />
<AppenderRef ref="RollingFile" />
</Root>
</Loggers>
</Configuration>

Have you tried adding:
<TimeBasedTriggeringPolicy interval="1" modulate="true"/>
Here is an example of log4j2 configuration that I currently used in my projects.
<RollingFile name="file-log" fileName="${log-path}/autodeployer.log"
filePattern="${log-path}/autodeployer-%d{yyyy-MM-dd}-%i.log">
<PatternLayout>
<Pattern>${LOG_PATTERN}</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true"/>
<SizeBasedTriggeringPolicy size="50MB"/>
</Policies>
<DefaultRolloverStrategy max="20"/>
</RollingFile>

Rolling file is not creating the next day and previous day file is being written with new logs

I have implemented log4j2 to log my records each day in a new file. But on next day previous day file is getting new records and old records file is getting archived. Instead of creating a new file with today's date. Once I restart the server, a new file with today's date gets created.
<Appenders>
<RollingFile name="LogToRollingFile"
fileName="logs/validategoods/appHeader-${env:hostName}-${date:yyyy-MM-dd}.log"
filePattern="logs/validategoods/${env:hostName}-${date:yyyy-MM}/appHeader-%d{yyyy-MM-dd}-%i.log.gz">
<PatternLayout>
<Pattern>%d %p [%t] %m%n</Pattern>
<!-- <Pattern>%d %p %c{1.} [%t] %m%n</Pattern> -->
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
</RollingFile>
<RollingFile name="LogToRollingFile1"
fileName="logs/validategoods/appPayload-${env:hostName}-${date:yyyy-MM-dd.HH-mm}.log"
filePattern="logs/validategoods/${env:hostName}-${date:yyyy-MM}/appPayload-%d{yyyy-MM-dd}-%i.log.gz">
<PatternLayout>
<Pattern>%d %p [%t] %m%n</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1" />
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
</RollingFile>
<Async name="Async">
<AppenderRef ref="LogToRollingFile" />
</Async>
<Async name="Async">
<AppenderRef ref="LogToRollingFile1" />
</Async>
</Appenders>
<Loggers>
<!-- avoid duplicated logs with additivity=false -->
<Logger name="no.posten.header" level="info" additivity="false">
<AppenderRef ref="LogToRollingFile" />
</Logger>
<Logger name="no.posten.payload" level="info"
additivity="false">
<AppenderRef ref="LogToRollingFile1" />
</Logger>
<Root level="debug">
</Root>
</Loggers>
I want to get a new file created each day without restarting the server.

Delete old Logs with Log4j2.xml is not working

Deleting the 7 days old log files with log4j.xml
I am trying to delete old log files by using DefaultRolloverStrategy but its not working.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Properties>
<Property name="log-path">Logs</Property>
<Property name="LOG_PATTERN">
[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n
</Property>
</Properties>
<Appenders>
<Console name="ConsoleAppender" target="SYSTEM_OUT">
<PatternLayout pattern="${LOG_PATTERN}" />
</Console>
<RollingFile name="InfoFileAppender" fileName="${log-path}/demo-app.log"
filePattern="${log-path}/demo-app-%d{yyyy-MM-dd}.log">
<PatternLayout>
<pattern>
[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n
</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"
modulate="true" />
<!--<SizeBasedTriggeringPolicy size="10 MB" /> -->
</Policies>
<DefaultRolloverStrategy>
<Delete basePath="${log-path}/" maxDepth="1">
<IfFileName glob="demo-app-*.log" />
<IfLastModified age="7d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
<RollingFile name="ErrorFileAppender" fileName="${log-path}/demo-err.log"
filePattern="${log-path}/demo-err-%d{yyyy-MM-dd}.log">
<PatternLayout>
<pattern>
[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n
</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"
modulate="true" />
<!-- <SizeBasedTriggeringPolicy size="5 MB"/> -->
</Policies>
<DefaultRolloverStrategy>
<Delete basePath="${log-path}/" maxDepth="1">
<IfFileName glob="demo-err-*.log" />
<IfLastModified age="7d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
</Appenders>
<Loggers>
<Root level="debug">
<AppenderRef ref="InfoFileAppender" level="info" />
<AppenderRef ref="ConsoleAppender" level="info" />
<AppenderRef ref="ErrorFileAppender" level="error" />
</Root>
</Loggers>
</Configuration>
But its working in my computer but its not working on Linux server. Log files storing path in Linux server is /opt/Logs.
can anyone please help on this.

your file name pattern should be "demo-err-*.log.gz", you missed ".gz".

How to write different logs in different files with log4j2 (MDC in xml)?

Now I'm using structure like this:
Appender:
<RollingFile name="user.log" append="true" fileName="users/%MDC{USERNAME}.txt"
filePattern="users/archive/%MDC{USERNAME}-%d{MM-dd-yyyy}-%i.txt.gz">
<PatternLayout pattern="%-5p %d{MMMM-dd HH:mm:ss} %X: %c - %m%n"/>
<Policies>
<TimeBasedTriggeringPolicy/>
<SizeBasedTriggeringPolicy size="50 MB"/>
</Policies>
</RollingFile>
Logger:
<appender-ref ref="user.log">
<ThreadContextMapFilter onMatch="ACCEPT" onMismatch="DENY" operator="or">
<KeyValuePair key="USERNAME" value="%X{USERNAME}"/>
<KeyValuePair key="IP" value="%X{IP}"/>
</ThreadContextMapFilter>
</appender-ref>
But it doesn't work with MDC keys. How could I use MDC keys in xml to config RollingFileAppender?

Take a look at RoutingAppender. Maybe this can get you started:
<?xml version="1.0" encoding="UTF-8"?>
<configuration status="DEBUG" name="MyApp" packages="">
<appenders>
<Routing name="Routing">
<Routes pattern="$${ctx:USERNAME}">
<Route>
<RollingFile name="user.log" append="true" fileName="users/${ctx:USERNAME}.txt"
filePattern="users/archive/${ctx:USERNAME}-%d{MM-dd-yyyy}-%i.txt.gz">
<PatternLayout>
<pattern>%d{ISO8601} [%t] %p %c %L - %m%n</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy/>
<SizeBasedTriggeringPolicy size="50 MB"/>
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
</appenders>
<loggers>
<root level="TRACE">
<appender-ref ref="Routing" level="DEBUG" />
</root>
</loggers>
</configuration>