My requirement is to define a log level which is in between INFO and DEBUG. In other words INFO plus something but less than debug. The level needs to be enabled in the root level of log4j.xml file in the tomcat configuration directory. I have done the following :
Set the root log level to custom one STAT like :
<root>
<priority value="STAT"/>
<appender-ref ref="CONSOLE"/>
<appender-ref ref="FILE"/>
<appender-ref ref="LOGSTASH"/>
</root>
Defined a log level class :
package com.log4j.logger;
import org.apache.log4j.Level;
/**
* #author shantha
*
*/
#SuppressWarnings("serial")
public class StatLogLevel extends Level {
public static final int STAT_INT = 15000;
public static final Level STAT = new StatLogLevel(STAT_INT, "STAT", 6);
public StatLogLevel(int level, String levelStr, int syslogEquivalent) {
super(level, levelStr, syslogEquivalent);
}
public static Level toLevel(String logArgument) {
if (logArgument != null && logArgument.toUpperCase().equals("STAT")) {
return STAT;
}
return (Level) toLevel(logArgument);
}
public static Level toLevel(int val) {
if (val == STAT_INT) {
return STAT;
}
return (Level) toLevel(val, Level.DEBUG);
}
public static Level toLevel(int val, Level defaultLevel) {
if (val == STAT_INT) {
return STAT;
}
return Level.toLevel(val, defaultLevel);
}
public static Level toLevel(String logArgument, Level defaultLevel) {
if (logArgument != null && logArgument.toUpperCase().equals("STAT")) {
return STAT;
}
return Level.toLevel(logArgument, defaultLevel);
}
}
In the code, logs are sent using the code :
logger.log(StatLogLevel.STAT, "Custom log message");
And also added an entry to the log4j.xml :
<category name="stats">
<priority value="STAT" class="com.log4j.logger.StatLogLevel" />
<appender-ref ref="CONSOLE" />
</category>
The result that I am getting is :
...
log4j: Level value for root is [STAT].
log4j: root level set to DEBUG
log4j: Adding appender named [CONSOLE] to category [root].
...
and all the debug messages are printed.
Appreciate if someone could help on this.
The environment is java 1.8, tomcat 8.5.15 and log4j-1.2.17.jar.
Related
I have set up a custom appender that will log to memory here it is below
#Plugin(name = "MemoryAppender",
category = Core.CATEGORY_NAME,
elementType = Appender.ELEMENT_TYPE)
class InMemoryAppender(name: String,
filter: Filter?) : AbstractAppender(name, filter, null, true, Property.EMPTY_ARRAY) {
companion object {
#PluginFactory
#JvmStatic
fun createAppender(
#PluginAttribute("name") name: String,
#PluginElement("Filter") filter: Filter?): InMemoryAppender {
return InMemoryAppender(name, filter)
}
}
private val buffer = CircularOverwritingBuffer(allowNonPowerOfTwoSizedBuffer = false)
override fun append(event: LogEvent?) {
event?.let {
buffer.add("[${event.level}] ${DateTimeFormatter.ISO_INSTANT.format(Instant.now())} ${event.message.formattedMessage}")
}
}
fun inOrder(numberOfItems: Int): Iterator<String> {
return buffer.emitInOrder(numberOfItems).iterator()
}
}
And I have added it to my root logger in my log4j conf file, first by creating the appender
<appenders>
.... other appenders
<MemoryAppender name="In-Memory-Appender"/>
</appenders>
and then loaded it into the root logger
<Loggers>
<Root level="${defaultLogLevel}">
.... opther appenders
<AppenderRef ref="In-Memory-Appender"/>
</Root>
</Loggers>
However when I now get a logger for a class like this, I get nothing in my buffer
private val log = loggerFor<AClass>()
// or
private val log = contextLog()
But if I get the logger like so I can
LoggerContext.getContext().rootLogger.info("REQUESTING LOGS")
How do I make my appender capture the logs correctly?
My Question was solved by the following answer: https://stackoverflow.com/a/21336499/2341393
The problem was an incorrect logger context being returned, and that slf4j was wrapping the Log4J Logger.
I have to mask the passwords in log messages without using logback.xml file in spring boot.
sample Log:
LOGGER.info("user password : {}", pwd);
expected Output:
2019-11-26 18:27:15,951 [http-nio-8080-exec-2] INFO com.test.controller.TestController - user password: ***********
I able to achieve the same using logback.xml file. as shown below.
but without logback file need to do using application.properties configuration file in spring boot.
Note: Don't use log4j xml file. we should using slf4j or MDC or any filters and application.properties
<configuration>
<property name="DEV_HOME" value="c:/logs" />
<appender name="CONSOLE"
class="ch.qos.logback.core.ConsoleAppender">
<encoder
class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="com.test.config.MaskingPatternLayout">
<patternsProperty>(SSN)</patternsProperty>
<pattern>%d [%thread] %-5level %logger{35} - %msg%n</pattern>
</layout>
</encoder>
</appender>
<logger name="com.test" level="debug" additivity="false">
<appender-ref ref="CONSOLE" />
</logger>
<root level="error">
<appender-ref ref="CONSOLE" />
</root>
</configuration>
Is it possible to achieve this without using logback.xml file and log4j.xml file?
can we able to mention the pattern layout java class in application properties file instead of mentioning the same in logback.xml file?
" in the above example, I have mention the java file in logback"
added MaskingPatternLayout for reference:
package com.test.config;
import java.util.Optional;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.springframework.stereotype.Component;
import ch.qos.logback.classic.PatternLayout;
import ch.qos.logback.classic.spi.ILoggingEvent;
#Component
public class MaskingPatternTest extends PatternLayout {
private String patternsProperty;
private Optional<Pattern> pattern;
public String getPatternsProperty() {
return patternsProperty;
}
public void setPatternsProperty(String patternsProperty) {
this.patternsProperty = patternsProperty;
if (this.patternsProperty != null) {
this.pattern = Optional.of(Pattern.compile(patternsProperty, Pattern.MULTILINE));
} else {
this.pattern = Optional.empty();
}
}
#Override
public String doLayout(ILoggingEvent event) {
final StringBuilder message = new StringBuilder(super.doLayout(event));
if (pattern.isPresent()) {
Matcher matcher = pattern.get().matcher(message);
while (matcher.find()) {
int group = 1;
while (group <= matcher.groupCount()) {
if (matcher.group(group) != null) {
for (int i = matcher.start(group); i < matcher.end(group); i++) {
message.setCharAt(i, '*');
}
}
group++;
}
}
}
return message.toString();
}
}
Kindly help on this.
As soon as you want to use advanced logging features (other than setting log levels), you have to use logging-library specific configuration, such as logback.xml for Logback, log4j.xml for log4j and so on.
However, Logback does have an API you can call. For example, you can set up your ConsoleAppender with beans:
#Bean
public LoggerContext loggerContext() {
return (LoggerContext) LoggerFactory.getILoggerFactory();
}
#Bean
public MaskPatternLayout maskPatternLayout(LoggerContext context) {
MaskPatternLayout layout = new MaskPatternLayout();
layout.setPatternsProperty("(SSN)");
layout.setPattern("%d [%thread] %-5level %logger{35} - %msg%n");
layout.setContext(context);
layout.start();
return layout;
}
#Bean
public LayoutWrappingEncoder<ILoggingEvent> maskEncoder(MaskPatternLayout layout) {
LayoutWrappingEncoder<ILoggingEvent> encoder = new LayoutWrappingEncoder<>();
encoder.setLayout(layout);
return encoder;
}
#Bean
public ConsoleAppender<ILoggingEvent> maskConsoleAppender(LoggerContext context, LayoutWrappingEncoder<ILoggingEvent> maskEncoder) {
ConsoleAppender<ILoggingEvent> appender = new ConsoleAppender<>();
appender.setContext(context);
appender.setEncoder(maskEncoder);
appender.start();
return appender;
}
Now you could create your own LoggerFactory:
#Component
public class MaskLoggerFactory {
private final Appender<ILoggingEvent> appender;
public MaskLoggerFactory(Appender<ILoggingEvent> appender) {
this.appender = appender;
}
public org.slf4j.Logger getLogger(String name) {
Logger logger = (Logger) LoggerFactory.getLogger(name);
logger.addAppender(appender);
logger.setLevel(Level.ALL);
logger.setAdditive(false);
return logger;
}
public org.slf4j.Logger getLogger(Class<?> cls) {
return getLogger(cls.getName());
}
}
And after that you can autowire MaskLoggerFactory to get a proper Logger. However, this doesn't make things easier to use, and if your only justification is to avoid creating a separate XML file, I would encourage you to keep using that XML file.
Is there a way to print a different logging.pattern given a specific logging.level?
The yml file I'm using in my Spring Boot project looks like this
logging:
level:
com.netflix: warn
org.springframework: warn
org.apache: warn
org.mongodb: warn
com.twocatholler: debug
pattern:
console: "%d{dd-MM-yyyy HH:mm:ss.SSS} %white([%thread]) %highlight(%-5level) %white(%logger) - %white(%msg) %n"
If having multiple appenders is acceptable, a straightforward way to have different patterns for different levels would be using two different appenders with ThresholdFilters.
There can be issues when more appenders are writing to the same file. Check this issue for example: https://jira.qos.ch/browse/LOGBACK-114
However, you can have your own appender which formats logging events differently, based on their level.
Check this LevelPatternLayout demo: https://github.com/riskop/logback_LevelPatternLayout
example logback.xml:
<configuration scan="true" debug="true">
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="pack.LevelPatternLayout">
<defaultPattern>DEFAULT PATTERN %c - %m\n</defaultPattern>
<debugPattern>DEBUG PATTERN %c - %m\n</debugPattern>
</layout>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="CONSOLE" />
</root>
</configuration>
test code:
public void test() throws Exception{
log.debug("some debug level log");
log.info("some info level log");
log.error("some error level log");
}
output:
DEBUG PATTERN pack.TestStart - some debug level log
DEFAULT PATTERN pack.TestStart - some info level log
DEFAULT PATTERN pack.TestStart - some error level log
LevelPatternLayout code:
package pack;
import java.util.HashMap;
import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.PatternLayout;
import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.LayoutBase;
/**
* A logback layout which can be configured with different pattern for different levels.
*
* E.g. you can configure a pattern for DEBUG messages and an *other* pattern
* for INFO messages and so on.
*
*
*/
public class LevelPatternLayout extends LayoutBase<ILoggingEvent> {
private boolean started;
// store for patternLayouts per level. The layout stored with 'null' key is the fallback
private HashMap<Level, PatternLayout> layouts = new HashMap<>();
public LevelPatternLayout() {
}
public void setDefaultPattern(String pattern) {
setLevelPattern(null, pattern);
}
public void setTracePattern(String pattern) {
setLevelPattern(Level.TRACE, pattern);
}
public void setDebugPattern(String pattern) {
setLevelPattern(Level.DEBUG, pattern);
}
public void setInfoPattern(String pattern) {
setLevelPattern(Level.INFO, pattern);
}
public void setWarnPattern(String pattern) {
setLevelPattern(Level.WARN, pattern);
}
public void setErrorPattern(String pattern) {
setLevelPattern(Level.ERROR, pattern);
}
private void setLevelPattern(Level level, String pattern) {
PatternLayout layout = new PatternLayout();
layout.setContext(context);
layout.setPattern(pattern);
layouts.put(level, layout);
}
#Override
public String doLayout(ILoggingEvent event) {
PatternLayout matchingLayout = layouts.get(event.getLevel());
if(matchingLayout != null) {
return matchingLayout.doLayout(event);
}
return layouts.get(null).doLayout(event);
}
#Override
public void start() {
if(layouts.get(null) == null) {
throw new RuntimeException("default layout is not initialized, probably no 'defaultPattern' is configured!");
}
layouts.values().forEach(layout -> layout.start());
this.started = true;
}
#Override
public void stop() {
layouts.values().forEach(layout -> layout.stop());
this.started = false;
}
#Override
public boolean isStarted() {
return started;
}
}
I'm using slf4j with either log4j 2.0 or logback as the implementation. For example, my servlet has a logger with level ERROR, and my server spawns 100 threads of the servlet. I will get a list of special users at runtime. When I detect some of the special users connected in. I want to change the log level for those special users/threads to DEBUG, and leave other threads' log level unaffected (still ERROR).
I know the TurboFilter in logback and DynamicThresholdFilter in log4j 2.0, but since I will only get the special users list at runtime, I cannot use them.
Here is my application:
package com.example.logging;
import java.util.HashMap;
import java.util.Map;
import javax.servlet.http.HttpServlet;
import org.slf4j.*;
public class App extends HttpServlet {
private final Logger Logger = LoggerFactory.getLogger(App.class);
Map<String, String> map = new HashMap<String, String>();
public App() {
map.put("user1", "DEBUG");
map.put("user2", "DEBUG");
map.put("user3", "ERROR");
}
public void writeToLogFile(String userName) {
if (map.containsKey(userName)) {
// do something so that I can change the logger to the corresponding log level
}
Logger.error(userName + " error message");
// the logger is of level ERROR, so by default, this log event will not happen
// but I want it to happen for special users
if (Logger.isDebugEnabled()) {
Logger.debug(userName + " debug message");
}
}
}
Here is my log configuration in log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="ERROR">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%-5level %class{36} %M %msg%xEx%n" />
</Console>
</Appenders>
<Loggers>
<Logger name="com.example.logging.App" level="ERROR" additivity="false">
<AppenderRef ref="Console" />
</Logger>
<Root level="DEBUG">
<AppenderRef ref="Console" />
</Root>
</Loggers>
</Configuration>
If I call the methods below:
App myApp = new App();
// assume the below 4 methods are called concurrently
myApp.writeToLogFile("user1");
myApp.writeToLogFile("user2");
myApp.writeToLogFile("user3");
myApp.writeToLogFile("user4");
The expected output should be:
ERROR com.example.logging.App writeToLogFile - user1 error message
DEBUG com.example.logging.App writeToLogFile - user1 debug message
ERROR com.example.logging.App writeToLogFile - user2 error message
DEBUG com.example.logging.App writeToLogFile - user2 debug message
ERROR com.example.logging.App writeToLogFile - user3 error message
ERROR com.example.logging.App writeToLogFile - user4 error message
I've met the same problem, and I end up using my own filter by making changes to DynamicThresholdFilter
Changes to the application:
package com.example.logging;
import java.util.HashMap;
import java.util.Map;
import javax.servlet.http.HttpServlet;
import org.slf4j.*;
public class App extends HttpServlet {
private final Logger Logger = LoggerFactory.getLogger(App.class);
Map<String, String> map = new HashMap<String, String>();
public App() {
map.put("user1", "Debug");
map.put("user2", "Debug");
map.put("user3", "Error");
}
public void writeToLogFile(String userName) {
// if the user is in the map, we put it into ThreadConext for filtering
if (map.containsKey(userName)) {
MDC.put("level", map.get(userName));
}
Logger.error(userName + " error message");
if (Logger.isDebugEnabled()) {
Logger.debug(userName + " debug message");
}
// remember to remove it
MDC.remove("level");
}
}
Here is the newly defined filter based on DynamicThresholdFilter, let's call it DynamicThresholdUserFilter, you can compare it to the source code of DynamicThresholdFilter
package com.example.logging.log4j2.plugin;
import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.Marker;
import org.apache.logging.log4j.ThreadContext;
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.Logger;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.config.plugins.PluginAttribute;
import org.apache.logging.log4j.core.config.plugins.PluginFactory;
import org.apache.logging.log4j.core.filter.AbstractFilter;
import org.apache.logging.log4j.message.Message;
/**
* Compare against a log level that is associated with an MDC value.
*/
#Plugin(name = "DynamicThresholdUserFilter", category = "Core", elementType = "filter", printObject = true)
public final class DynamicThresholdUserFilter extends AbstractFilter {
private Level defaultThreshold = Level.ERROR;
private final String key;
private DynamicThresholdUserFilter(final String key, final Level defaultLevel,
final Result onMatch, final Result onMismatch) {
super(onMatch, onMismatch);
if (key == null) {
throw new NullPointerException("key cannot be null");
}
this.key = key;
this.defaultThreshold = defaultLevel;
}
public String getKey() {
return this.key;
}
#Override
public Result filter(final Logger logger, final Level level, final Marker marker, final String msg,
final Object... params) {
return filter(level);
}
#Override
public Result filter(final Logger logger, final Level level, final Marker marker, final Object msg,
final Throwable t) {
return filter(level);
}
#Override
public Result filter(final Logger logger, final Level level, final Marker marker, final Message msg,
final Throwable t) {
return filter(level);
}
#Override
public Result filter(final LogEvent event) {
return filter(event.getLevel());
}
/* biggest change here */
private Result filter(final Level level) {
final String value = ThreadContext.get(key);
if (value != null) {
Level ctxLevel = Level.toLevel(value);
if (ctxLevel == null) {
// in case the level is invalid
ctxLevel = defaultThreshold;
}
return level.isAtLeastAsSpecificAs(ctxLevel) ? onMatch : onMismatch;
}
return Result.NEUTRAL;
}
#Override
public String toString() {
final StringBuilder sb = new StringBuilder();
sb.append("key=").append(key);
sb.append(", default=").append(defaultThreshold);
return sb.toString();
}
/**
* Create the DynamicThresholdFilter.
* #param key The name of the key to compare.
* #param pairs An array of value and Level pairs.
* #param levelName The default Level.
* #param match The action to perform if a match occurs.
* #param mismatch The action to perform if no match occurs.
* #return The DynamicThresholdFilter.
*/
#PluginFactory
public static DynamicThresholdUserFilter createFilter(
#PluginAttribute("key") final String key,
#PluginAttribute("defaultThreshold") final String levelName,
#PluginAttribute("onMatch") final String match,
#PluginAttribute("onMismatch") final String mismatch) {
final Result onMatch = Result.toResult(match);
final Result onMismatch = Result.toResult(mismatch);
final Level level = Level.toLevel(levelName, Level.ERROR);
return new DynamicThresholdUserFilter(key, level, onMatch, onMismatch);
}
}
Add the DynamicThresholdUserFilter and package name to your configuration file
<?xml version="1.0" encoding="UTF-8"?>
<!-- add the package name of the filter-->
<Configuration status="ERROR" packages="com.example.logging.plugin">
<!-- configuration of the new defined filter -->
<DynamicThresholdUserFilter key="level" defaultThreshold="ERROR" onMatch="ACCEPT" onMismatch="NEUTRAL" />
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%-5level %class{36} %M %msg%xEx%n"/>
</Console>
</Appenders>
<Loggers>
<Logger name="com.example.logging.App" level="ERROR" additivity="false">
<AppenderRef ref="Console" />
</Logger>
<Root level="debug">
<AppenderRef ref="Console" />
</Root>
</Loggers>
</Configuration>
The newly defined filter is pretty similar to DynamicThresholdFilter. The difference is DynamicThresholdFilter uses the predefined level in configuration file as the dynamic threshold, while this filter uses the level programmatically defined in the map.
While the already existing answer might work (haven't tried it personally), after intensive searching, I found a very easy and neat trick to do what you are requesting.
The DynamicThresholdFilter can be used with conditions to switch the log level at run time. This, combined with log4j2's ThreadContext, you can do quite nifty things.
You would have to populate a particular key in the ThreadContext at the beginning of a server call processing (somewhere in doFilter method of your HttpServlet class) based on your custom logic of user names. This would look something like:
ThreadContext.put("customLogLevel", "debug");
Then in your log4j2.xml file, you put this as a global filter, right below the root Configuration tag:
<DynamicThresholdFilter key="customLogLevel" onMatch="ACCEPT" onMismatch="NEUTRAL">
<KeyValuePair key="debug" value="DEBUG"/>
<KeyValuePair key="error" value="ERROR"/>
<KeyValuePair key="info" value="INFO"/>
</DynamicThresholdFilter>
Now based on the value of the key customLogLevel in the ThreadContext that you set at the beginning of a call, all the log calls in that thread will have log level corresponding to the matching KeyValuePair line. So in the example above, all log calls in the thread would have level as DEBUG.
I have written class for custom logging level i.e. INIT
import org.apache.log4j.Logger;
import org.apache.log4j.Level;
public class InitLoggingLevel extends Level {
public static final String INITLOGGING_LEVEL = "INITLOGGING";
public static final Level INIT_LOGGING = new InitLoggingLevel(
DEBUG_INT - 4, INITLOGGING_LEVEL, 7);
protected InitLoggingLevel(int level, String levelStr, int syslogEquivalent){
super(level, levelStr, syslogEquivalent);
}
}
Now what are changes I need to make in log4j.properties and how I am going to use this INIT logging level in My Java class?
You have to:
Override the method toLevel(String logArgument) like this:
public static Level toLevel(String logArgument) {
if (logArgument != null && logArgument.toUpperCase().equals("INITLOGGING")) {
return INIT_LOGGING;
}
return (Level) toLevel(logArgument);
}
Write a configuration line in the log4j.properties like this:
log4j.logger.[MY_CATEGORY]=INITLOGGING#your-package.InitLoggingLevel
That's all!
P.S.: The '#' syntax is described in org.apache.log4j.helpers.OptionConverter source class.
You can try this
log4j.rootLogger=DEBUG, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d %c:%L - %m%n
log4j.category.YOUR_PACKAGE=INFO,YOUR_PACKAGE.InitLoggingLevel
or can view log4j category
or http://veerasundar.com/blog/2009/08/log4j-tutorial-how-to-send-log-messages-to-different-log-files/
It can be like this :
<category name="xxx" additivity="false">
<priority class="your class" value="info"/>
<appender-ref ref="xxxlog"/>
</category>
Don't know if you already figured it out, but I found the following article, which helped me.
https://www.owasp.org/index.php/How_to_add_a_security_log_level_in_log4j
The new level is printed with the expected log tag, but log4j will not recognize the new level in my property file. I will see, what I can do about that.