I am new to java and need to place logs using slf4j, I need to put sensitive data logs into a different file.
Example:
package in.com.my.service.here
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;
public static void main( args[] ) {
Logger log = LoggerFactory.getLogger(POSServiceImpl.class);
log.info("log without MDC");
MDC.put('isEncrypted',"true");
log.info("log line 1");
log.info("log line 2");
log.info("log line 3");
MDC.clear();
log.info("log again without MDC");
}
Important: please provide the logback.xml configuration such that the logs under MDC are store in different file when isEncrypted value is true.
<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="true">
<timestamp key="bySecond" datePattern="yyyy-MM-dd_HHmmss" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n
</pattern>
</encoder>
</appender>
<appender name="ROLLING"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.folder}/terminal-${bySecond}.log</file>
<rollingPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${log.folder}/Archive-%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<maxFileSize>10MB</maxFileSize>
<maxHistory>30</maxHistory>
<totalSizeCap>1GB</totalSizeCap>
</rollingPolicy>
<encoder>
<pattern>My ID [%X{myId}] %d{HH:mm:ss.SSS} [%thread]
%-5level %logger{5} - %msg%n</pattern>
</encoder>
</appender>
<logger name="in.com.my.service" level="debug"
additivity="false">
<appender-ref ref="ROLLING" />
<appender-ref ref="STDOUT" />
</logger>
<!-- By default, the level of the root level is set to DEBUG -->
<root level="info">
<appender-ref ref="ROLLING" />
<appender-ref ref="STDOUT" />
</root>
</configuration>
This looks like a candidate for Logback's SiftingAppender, javadoc. For example:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="SIFTER" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator>
<key>encryptedStatus</key>
<defaultValue></defaultValue>
</discriminator>
<sift>
<appender name="FILE${encryptedStatus}" class="ch.qos.logback.core.FileAppender">
<file>File${encryptedStatus}.log</file>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>...</pattern>
</layout>
</appender>
</sift>
</appender>
<root level="ALL">
<appender-ref ref="SIFTER" />
</root>
</configuration>
The value of encryptedStatus will be substituted into the log file name. So, log events which contain the MDC value encryptedStatus=Sensitive will be written to FileSensitive.log and log events which do not contain a MDC attribute named encryptedStatus will be written to File.log.
You'll likely want to change the values of file name and the suffix for the sensitive files etc but the above extract shows you how you can direct log events to specific log files based on an MDC attribute.
Related
I have my current logback file, which to my understanding is a classic logback implementation:
<configuration debug="true">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>Hello %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="STDOUT" />
</root>
</configuration>
I have a specific log:
log.info(String.format("ProxyICA %s has an integrity violation. %s.",
processorId.getProxyIca(),
fkViolationMsg)
);
that I am trying to get in a text file. The only configuration I need in my logback is to put every instance of this log into a text file. Every other log can default log to the console (which it was doing without the logback.xml).
Thank you in advanced!
Here is an example of setting it up log to a specific file "Custom-Errors.txt", and to have the logs be deleted if they reach a certain size 20MB or 182 days elapse
<appender name="Custom_Logger
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>Custom-Errors.txt</file>
<rollingPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>Custom-Errors-%d{yyyy-MM-dd}.%i.txt</fileNamePattern>
<!-- each file should be at most 20MB, keep 182 days worth of history,but
at most 500MB -->
<maxFileSize>20MB</maxFileSize>
<maxHistory>182</maxHistory>
<totalSizeCap>500MB</totalSizeCap>
</rollingPolicy>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO, DEBUG, TRACE</level>
</filter>
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%date %level [%thread] [%file:%line] %msg%n%throwable
</Pattern>
</layout>
</appender>
<logger name="CUSTOM_LOGGER" additivity="false">
<appender-ref ref="Custom_Logger" />
</logger>
Then in your java class make a private variable to use the logger like this
private static final Logger CUSTOM_LOGGER =
LoggerFactory.getLogger("CUSTOM_LOGGER");
logging a dependency, or use to log anything you want based on package
here is an example of logging a library
<appender name="ConsoleOutForSpring" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%black(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1.}): %msg%n%throwable
</Pattern>
</layout>
</appender>
<logger name="org.springframework" additivity="false">
<appender-ref ref="ConsoleOutForSpring" />
</logger>
I do admit there are a lot of tags and settings to this logger that are kind of complicated, but reading docs and looking at others examples has helped me.
Here were some docs I noted in my logback.xml
<!-- http://logback.qos.ch/manual/configuration.html this helped me with this weird looking stuff i.e. ===> %date %level [%thread] [%file:%line] %msg%n%throwable -->
<!-- http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy
helped me set this up:
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>Errors-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<maxFileSize>20MB</maxFileSize>
<maxHistory>182</maxHistory>
<totalSizeCap>500MB</totalSizeCap>
</rollingPolicy>
I have configured logback.xml to change log level during build time using "scan" property. Using this I can change log level without rebuilding the code. I can change log level by updating logback.xml file
<?xml version="1.0" encoding="UTF-8"?><!--
For more configuration information and examples see
http://logback.qos.ch/manual/configuration.html-->
<configuration scan="true" scanPeriod="10 seconds">
<!--<statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" />-->
<!-- Debugging appender (duplicates the normal log, PLUS any debug messages) -->
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%date{ISO8601} %level{5} %c{3} --- %message%n</pattern>
</encoder>
</appender>
<!---->
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>./log/elk-file.log</file>
<append>true</append>
<!--cleanHistoryOnStart>true</cleanHistoryOnStart-->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>./log/elk-file_%d{yyyyMMdd}-%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>200MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>1</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%date{ISO8601} [%thread] %level{5} %c{3} - %message%n</pattern>
</encoder>
</appender>
<!-- Our logger writes to file, console and sends the data to Logstash -->
<logger name="ro.fortsoft.elk.testdata" level="DEBUG" additivity="false">
<appender-ref ref="STASH"/>
</logger>
<logger level="INFO" name="rollingFileLogger">
<appender-ref ref="FILE" />
</logger>
<!-- ROOT logger setup -->
<root level="DEBUG">
<appender-ref ref="CONSOLE"/>
</root>
How can I do this in Jenkins, i-e change logback.xml and change log level without rebuilding the code.
Currently my Spring-boot application logs to a file named: myLog.log, this is working as intended, however I would like the log file to have a timestamp at the end of it and create a new file each time it is ran.
I have tried to implement this in my logback-test.xml file shown below, but it is just giving me the filename: myLog.log without a timestamp.
How can I fix this?
Logback-test.xml:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<logger name="org.springframework.web" level="INFO"/>
<!-- Send debug messages to System.out -->
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- By default, encoders are assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} - %msg%n</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>path/to/my/file/mylog.log</file>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>%d{yyyy-MM-dd_HH:mm:ss.SSS} - %msg%n</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<FileNamePattern>mylog.%i{yyyy-MM-dd_HH:mm:ss.SSS}}.log</FileNamePattern>
<MinIndex>1</MinIndex>
<MaxIndex>10</MaxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<MaxFileSize>2MB</MaxFileSize>
</triggeringPolicy>
</appender>
<logger name="com.my.package" level="INFO" additivity="false">
<appender-ref ref="STDOUT" />
<appender-ref ref="FILE" />
</logger>
<!-- By default, the level of the root level is set to DEBUG -->
<root level="INFO">
<appender-ref ref="STDOUT" />
</root>
</configuration>
You can define a variable like this:
<timestamp key="myTimestamp" datePattern="yyyy-MM-dd'_'HH-mm-ss.SSS"/>
(note: don't use colons in datePattern)
Then use it directly in your appender's file element:
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>path/to/my/file/mylog-${myTimestamp}.log</file>
...
</appender>
Or in a simple FileAppender:
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>path/to/my/file/mylog-${myTimestamp}.log</file>
<encoder>
<Pattern>%d{yyyy-MM-dd_HH:mm:ss.SSS} - %msg%n</Pattern>
</encoder>
</appender>
Docx4j gives me a bunch of messages like this
[AWT-EventQueue-0] INFO org.docx4j.model.listnumbering.Emulator -
How to turn that off?
The following log configuration gets loaded, but doesnt turns the logging off.
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<param name="Target" value="System.out"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p %c{1} - %m%n"/>
</layout>
</appender>
<root>
<priority value ="OFF" />
</root>
</log4j:configuration>
Add the following line to your configuration:
<logger name="org.docx4j.model.listnumbering.Emulator">
<level value="ERROR" />
<appender-ref ref="console"/>
</logger>
This will configure log4j to log only ERROR messages originating from logger org.docx4j.model.listnumbering.Emulator. If you want to turn off INFO messages from all classes/packages under org.docx4j, use the following:
<logger name="org.docx4j" >
<level value="ERROR" />
<appender-ref ref="console"/>
</logger>
More information here : http://www.javabeat.net/baisc-steps-to-configure-log4j-using-xml-and-properties-file/
I'm using docx4j 8.3.2, logged by slf4j 1.7.30 + logback 1.2.3.
The annoying log messages disappeared after adding a logback.xml under src/main/resources. The logback.xml content:
<configuration>
<appender name="STDOUT"
class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder
by default -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} %line - %msg%n
</pattern>
</encoder>
</appender>
<!-- docx4j loggers -->
<logger name="org.docx4j" level="ERROR" />
<root level="DEBUG">
<appender-ref ref="STDOUT" />
</root>
</configuration>
I am using logback/slf4j to do my logging. I want to parse my log file to analyze some data, so instead of parsing a great big file (mostly consisting of debug statements) I want to have two logger instances which each log to a separate file; one for analytics and one for all purpose logging. Does anyone know if this is possible with Logback, or any other logger for that matter?
It's very possible to do something like this in logback. Here's an example configuration:
<?xml version="1.0"?>
<configuration>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>logfile.log</file>
<append>true</append>
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg %n</pattern>
</encoder>
</appender>
<appender name="ANALYTICS-FILE" class="ch.qos.logback.core.FileAppender">
<file>analytics.log</file>
<append>true</append>
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg %n</pattern>
</encoder>
</appender>
<!-- additivity=false ensures analytics data only goes to the analytics log -->
<logger name="analytics" level="DEBUG" additivity="false">
<appender-ref ref="ANALYTICS-FILE"/>
</logger>
<root>
<appender-ref ref="FILE"/>
</root>
</configuration>
Then you'd setup two separate loggers, one for everything and one to log analytics data like so:
Logger analytics = LoggerFactory.getLogger("analytics");
You can have as many loggers as you wish. But, it's better you have one for each package that you need to log differently. Then all the classes in that package and its sub-packages will get the that specific logger. They all can share the root logger and send their log data to root logger appender using additivity="true". Here's an example:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="pattern" value="%date{HH:mm:ss.SSS} %-5p %logger{36}
%X{akkaSource} [%file:%line] - %m%n" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%date{HH:mm:ss.SSS} %-5p %logger{36} %X{akkaSource} [%file:%line] - %m%n</pattern>
</encoder>
</appender>
<appender name="abc" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${catalina.base}/logs/worker.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${catalina.base}/logs/worker-%d{yyyy-MM-dd_HH}.log</fileNamePattern>
<maxHistory>360</maxHistory>
</rollingPolicy>
<encoder>
<pattern>${pattern}</pattern>
</encoder>
</appender>
<appender name="xyz" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${catalina.base}/logs/transformer.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${catalina.base}/logs/transformer-%d{yyyy-MM-dd_HH}.log</fileNamePattern>
<maxHistory>360</maxHistory>
</rollingPolicy>
<encoder>
<pattern>${pattern}</pattern>
</encoder>
</appender>
<logger name="com.xxx.yyy.zzz" level="INFO" additivity="true">
<appender-ref ref="xyz"/>
</logger>
<logger name="com.aaa.bbb.ccc" level="INFO" additivity="true">
<appender-ref ref="abc"/>
</logger>
<root>
<level value="INFO" />
<appender-ref ref="STDOUT" />
</root>
in my case I wanted to leave class names as log name
private static final Logger log = LoggerFactory.getLogger(ScheduledPost.class);
and as I had few such classes, so my logback.xml
<!--additivity=false ensures this log data only goes to the this log, and no one more -->
<logger name="xxx.xxx.xxx.ScheduledPost" level="DEBUG" additivity="false">
<appender-ref ref="ASYNC_SCHEDULE_LOG_FILE"/>
</logger>
<logger name="xxx.xxx.xxx.GcmPost" level="DEBUG" additivity="false">
<appender-ref ref="ASYNC_SCHEDULE_LOG_FILE"/>
</logger>
<logger name="xxx.xxx.xxx.PushUtils" level="DEBUG" additivity="false">
<appender-ref ref="ASYNC_SCHEDULE_LOG_FILE"/>
</logger>