Develop Reference

All logs are not going to file when file logging enabled

I am trying to configure log4j2 programmatically and i am facing issue with logs coming from my cluster.
Here is the case, when user enable file logging (by passing log path as parameter in this case) all logs should go to File, no console logs. When use disable file logging, all logging should go to console. So I am trying in this:
public static void initializeLogging(
boolean vNewVerboseDebugLogging, String vLoggingPath, String vNewWarehouseQueriesLogPath,
boolean vNoConsoleLogging, boolean vIsEmbeddedHive, boolean isTestSubmission, boolean YarnLogs, boolean debugLogs) throws IOException
{
String[] packageGrp = {"org.apache","hive.ql","com.cdcb.cstone"};// this i am using to turn off logging from other packages/classes
if(vLoggingPath!=null) {
ConfigurationBuilder<BuiltConfiguration> builder = ConfigurationBuilderFactory.newConfigurationBuilder();
RootLoggerComponentBuilder rootLogger = builder.newRootLogger(debugLogs?Level.DEBUG:Level.INFO).addAttribute("additivity", true);
LayoutComponentBuilder layoutComponentBuilder = builder.newLayout("PatternLayout").addAttribute("pattern",
(isTestSubmission)?"%-5p [%t]: %m%n":"%d %-5p [%t]: %m%n");
AppenderComponentBuilder fileAppenderBuilder;
fileAppenderBuilder = builder.newAppender("LogToRollingFile", "RollingFile")
.addAttribute("fileName", vLoggingPath)
.addAttribute("filePattern", vLoggingPath + "-%d{MM-dd-yy-HH}.log")
.add(layoutComponentBuilder)
.addComponent(builder.newComponent("Policies")
.addComponent(builder.newComponent("TimeBasedTriggeringPolicy").addAttribute("interval", "1")));
/*builder.newLogger("com.cdcb.idn.HadoopCluster", (vVerboseDebugLogging)?Level.DEBUG:Level.INFO)
.add(builder.newAppenderRef("LogToRollingFile"))
.addAttribute("additivity", false);*/
for(String packNm: packageGrp)
addNewLoggerComponent(builder,packNm,builder.newAppenderRef("LogToRollingFile"),Level.OFF);
builder.add(fileAppenderBuilder);
rootLogger.add(builder.newAppenderRef("LogToRollingFile"));
builder.add(rootLogger);
Configurator.initialize(builder.build()).updateLoggers();
} else {
ConfigurationBuilder<BuiltConfiguration> builder = ConfigurationBuilderFactory.newConfigurationBuilder();
RootLoggerComponentBuilder rootLogger = builder.newRootLogger(debugLogs?Level.DEBUG:Level.INFO);
LayoutComponentBuilder layoutComponentBuilder = builder.newLayout("PatternLayout").addAttribute("pattern",
(isTestSubmission)?"%-5p [%t]: %m%n":"%d %-5p [%t]: %m%n");
AppenderComponentBuilder vConsoleInfo;
AppenderComponentBuilder vConsoleProblems;
if (!vIsEmbeddedHive){
vConsoleProblems = builder.newAppender("consoleProb", "CONSOLE")
.addAttribute("target", ConsoleAppender.Target.SYSTEM_ERR)
.add(layoutComponentBuilder);
builder.newLogger("com.cdcb.idn.HadoopCluster", (vVerboseDebugLogging)?Level.DEBUG:Level.INFO)
.add(builder.newAppenderRef("consoleProb"))
.addAttribute("additivity", false);
for(String packNm: packageGrp)
addNewLoggerComponent(builder,packNm,builder.newAppenderRef("consoleProb"),Level.OFF);
builder.add(vConsoleProblems);
rootLogger.add(builder.newAppenderRef("consoleProb"));
builder.add(rootLogger);
}
vConsoleInfo = builder.newAppender("consoleInfo", "CONSOLE")
.addAttribute("target", ConsoleAppender.Target.SYSTEM_OUT)
.add(layoutComponentBuilder);
/*builder.newLogger("com.cdcb.idn.HadoopCluster", (vVerboseDebugLogging)?Level.DEBUG:Level.INFO)
.add(builder.newAppenderRef("consoleInfo"))
.addAttribute("additivity", false);*/
for(String packNm: packageGrp)
addNewLoggerComponent(builder,packNm,builder.newAppenderRef("consoleInfo"),Level.OFF);
builder.add(vConsoleInfo);
rootLogger.add(builder.newAppenderRef("consoleInfo"));
builder.add(rootLogger);
Configurator.initialize(builder.build()).updateLoggers();
}
LOGGER = LogManager.getLogger(HadoopClusterLogging.class);
}
public static ConfigurationBuilder addNewLoggerComponent(ConfigurationBuilder builder, String name, AppenderRefComponentBuilder appenderReferences, Level level) {
return builder.add(builder.newLogger(name, level)
.add(appenderReferences)
.addAttribute("additivity", false));
}
but the issue here is, when file logging enabled, not all logs going to File, some going to console(for example, logs coming from hadoop cluster). What am i doing wrong here and how can i capture all logs to file when file logging enabled? can come one please help?
One more thing is, how can i disable transitive dependencies logging. Meaning, my logger should log my application logs only, not from from my dependencies.
Thank you!

Log4j2 logs are not directing to correct log file

I am facing an issue with configuring log4j2 logs programmatically.
Please see the below code, I am creating 2 objects of the class App3, and I want to create 2 debug log files(a log file per App3 object), thereafter each object should be able to log to corresponding log file.
But my code is logging all the logs to the second log file after 2nd log is created. Can someone help on this.
Output of the program
file Name: app3_logger1.log_debug.log
2020-06-16 16:19:31,100 DEBUG app3_logger1.log [main] Hello from Log4j 2app3_logger1
file Name: app3_logger2.log_debug.log
2020-06-16 16:19:31,211 DEBUG app3_logger2.log [main] Hello from Log4j 2app3_logger2
2020-06-16 16:19:31,216 DEBUG app3_logger2.log [main] Hello from Log4j 2app3_logger2
2020-06-16 16:19:31,216 DEBUG app3_logger1.log [main] Hello from Log4j 2app3_logger1
2020-06-16 16:19:31,216 DEBUG app3_logger1.log [main] Hello from Log4j 2app3_logger1
2020-06-16 16:19:31,217 DEBUG app3_logger2.log [main] Hello from Log4j 2app3_logger2
Java Class - you just need to add log4j2 dependencies to compile
public class App3 {
public Logger logger;
public static void main(String[] args) {
App3 app1 = new App3();
app1.initializeYourLogger("app3_logger1.log", "%d %p %c [%t] %m%n");
app1.testProg("app3_logger1");
App3 app2 = new App3();
app2.initializeYourLogger("app3_logger2.log", "%d %p %c [%t] %m%n");
app2.testProg("app3_logger2");
app2.testProg("app3_logger2");
app1.testProg("app3_logger1");
app1.testProg("app3_logger1");
app2.testProg("app3_logger2");
}
public void testProg(String s) {
logger.debug("Hello from Log4j 2" + s);
}
public void initializeYourLogger(String fileName, String pattern) {
this.logger = LogManager.getLogger(fileName);
ConfigurationBuilder<BuiltConfiguration> builder = ConfigurationBuilderFactory.newConfigurationBuilder();
builder.setStatusLevel(Level.DEBUG);
builder.setConfigurationName(fileName);
AppenderComponentBuilder componentBuilder = builder.newAppender("log", "File");
componentBuilder.add(builder.newLayout("PatternLayout").addAttribute("pattern", pattern));
RootLoggerComponentBuilder rootLogger = builder.newRootLogger(Level.DEBUG);
LayoutComponentBuilder layoutBuilder = builder.newLayout("PatternLayout").addAttribute("pattern", pattern);
ComponentBuilder triggeringPolicy = builder.newComponent("Policies")
.addComponent(builder.newComponent("SizeBasedTriggeringPolicy").addAttribute("size", "10MB"));
componentBuilder = builder.newAppender("LogToRollingErrorFile", "RollingFile")
.add(builder.newFilter("ThresholdFilter", Filter.Result.ACCEPT, Filter.Result.DENY)
.addAttribute("level", Level.ERROR))
.addAttribute("fileName", fileName + "_error.log")
.addAttribute("filePattern", fileName + "-%d{MM-dd-yy-HH-mm-ss}_error.log").add(layoutBuilder)
.addComponent(triggeringPolicy);
builder.add(componentBuilder);
componentBuilder = builder.newAppender("LogToRollingDebugFile", "RollingFile")
.add(builder.newFilter("ThresholdFilter", Filter.Result.ACCEPT, Filter.Result.DENY)
.addAttribute("level", Level.DEBUG))
.addAttribute("fileName", fileName + "_debug.log")
.addAttribute("filePattern", fileName + "-%d{MM-dd-yy-HH-mm-ss}_debug.log").add(layoutBuilder)
.addComponent(triggeringPolicy);
builder.add(componentBuilder);
AppenderRefComponentBuilder rollingError = rootLogger.getBuilder().newAppenderRef("LogToRollingErrorFile");
AppenderRefComponentBuilder rollingDebug = rootLogger.getBuilder().newAppenderRef("LogToRollingDebugFile");
rootLogger.add(rollingError);
rootLogger.add(rollingDebug);
builder.add(rootLogger);
Configurator.reconfigure(builder.build());
}
}
This is exactly what I want to do in log4j older version, I am still struggling with log4j2,
private void initLogger(String serviceName, String instance) throws IOException {
String loggerName = serviceName+"_"+instance;
this.logger = Logger.getLogger(loggerName);
this.logger.removeAllAppenders();
PatternLayout layout = new PatternLayout();
layout.setConversionPattern("%d: %m%n");
String loggingFolder = this.properties.getLoggingFolder();
String debugFileName = loggingFolder+"/"+loggerName+"_debug.log";
String errorFileName = loggingFolder+"/"+loggerName+"_error.log";
RollingFileAppender debugAppender = new RollingFileAppender(layout, debugFileName, true);
debugAppender.setThreshold(Level.DEBUG);
debugAppender.setMaxFileSize("10000000");
debugAppender.setMaxBackupIndex(49);
logger.addAppender(debugAppender);
RollingFileAppender errorAppender = new RollingFileAppender(layout, errorFileName, true);
errorAppender.setThreshold(Level.ERROR);
errorAppender.setMaxFileSize("10000000");
errorAppender.setMaxBackupIndex(49);
logger.addAppender(debugAppender);
}

Actually, I am quite sure your Logger is being updated correctly. The problem is that both application instances are going to use the same Logging configuration.
If you look at Log4j's architecture you will see that the Loggers and the configuration are anchored in the LoggerContext. By default, Log4j uses the ClassLoaderContextSelector, which means that every ClassLoader can have its own LoggerContext. Your sample application only has a single ClassLoader and so will only have a single LoggerContext and, therefore, only a single Configuration.
So when you reconfigured the second time you simply replaced the prior configuration with the new one. Since the root logger is handling all log events it will route the events from both Loggers you have created to the file created in the second configuration.
If you want logs to end up in two different files then create a configuration with both files and figure out a way to route them to the correct file, either via logger names or filters.

As per observation, your logger is updated with the last initialization so after app2.initializeYourLogger() your object-level logger object updated with App2 configuration.
You need to return the logger object from initializeYourLogger() and also create separate Logger object.

log4j2 Logger Only Works in Primary Class - Spring Boot

I'm trying to use log4j2's rolling logs functionality for a new app, but I can't get the logger to work outside the main class, trying various combos of static, final, and autowired configs in the secondary classes.
All classes and config below. What am I missing or incorrectly calling?
POM.XML
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.3.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
...
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
Root Class:
#SpringBootApplication
public class DocumentServiceApplication {
private static final Logger logger = LogManager.getLogger(DocumentServiceApplication.class);
public static void main(String[] args) {
//these work.
logger.debug("Logger correctly configured debug.");
logger.error("Logger correctly configured error.");
logger.info("Logger correctly configured info.");
SpringApplication.run(DocumentServiceApplication.class, args);
}
}
Secondary (Web API) Class:
#RestController
public class DocumentController {
#Autowired
private PPAFileService ppaFileService;
private static final Logger logger = LogManager.getLogger(DocumentController.class);
#GetMapping("/ppa/process")
#ResponseBody
public ResponseEntity<String> process( ) throws Exception
{
try {
//this logging gives no error and no entry on console or file.
logger.debug("Received request for PPA file.");
...
} catch (Exception e) {
return new ResponseEntity<>(e.getMessage(), HttpStatus.BAD_REQUEST);
}
}
}
Log Output
2019-04-10 15:37:56 DEBUG Logger correctly configured debug.
2019-04-10 15:37:56 ERROR Logger correctly configured error.
2019-04-10 15:37:56 INFO Logger correctly configured info.
log4j2.properties
status = error
name = PropertiesConfig
#Make sure to change log file path as per your need
property.filename = D:\\projects\\app\\application.log
filters = threshold
filter.threshold.type = ThresholdFilter
filter.threshold.level = debug
appenders = rolling
appender.rolling.type = RollingFile
appender.rolling.name = RollingFile
appender.rolling.fileName = ${filename}
appender.rolling.filePattern = application-backup-%d{MM-dd-yy-HH-mm-ss}-%i.log.gz
appender.rolling.layout.type = PatternLayout
appender.rolling.layout.pattern = %d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1
appender.rolling.policies.time.modulate = true
appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size=10MB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.max = 20
loggers = rolling
logger.rolling.name = com.app.documentservice
logger.rolling.level = debug
logger.rolling.additivity = false
logger.rolling.appenderRef.rolling.ref = RollingFile

log4j2 doesn't save to log file

I have configuration for log4j2 in properties file. Logger sends output to the console but not to the file. It creates log file but it doesn't update it with content. What can be missing in my configuration file?
log4j2.properties
status = error
name = PropertiesConfig
property.filename = /usr/local/logs/kit.log
filters = threshold
filter.threshold.type = ThresholdFilter
filter.threshold.level = debug
appenders = rolling
appender.rolling.type = RollingFile
appender.rolling.name = RollingFile
appender.rolling.fileName = ${filename}
appender.rolling.filePattern = debug-backup-%d{MM-dd-yy-HH-mm-ss}-%i.log.gz
appender.rolling.layout.type = PatternLayout
appender.rolling.layout.pattern = %d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1
appender.rolling.policies.time.modulate = true
appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size=10MB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.max = 20
loggers = rolling
logger.rolling.name = org.kit
logger.rolling.level = debug
logger.rolling.additivity = false
logger.rolling.appenderRef.rolling.ref = RollingFile
Test class
package org.kit.log;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
public class LogTest {
private static final Logger logger = LogManager.getLogger(LogTest.class);
public static void main(String[] args) {
logger.debug("Hello world - debug log");
logger.info("Hello world - info log");
logger.warn("Hello world - warn log");
logger.error("Hello world - error log");
}
}

You can try setting the first line in the configuration to status = TRACE to show Log4j2 internal logging on the console. This may help determine any problems with the configuration.
Also, can you try configuring a rootLogger instead of a named logger?
rootLogger.level = info
rootLogger.appenderRef.rolling.ref = RollingFile

Log4j 2: How to add references from my classes in my log4j.properties [duplicate]

This question already has answers here:
Log4j Log at all Classes Level
(2 answers)
Closed 6 years ago.
is my first question, sorry If I make a mistake asking my question.
Well this is my doubt about log4j2.(I use spring framework java)
My goal is write in a FILE all logs generated in classes wich have a Logger instance.
For this purpose I have the following log4j.properties file:
name=PropertiesConfig
property.filename = C:/sts-bundle/Tomcat 7.0/logs
appenders = console,rolling
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%-5level] %d{dd MMM yyyy HH:mm:ss,SSS} [%t] %c{5} - %msg%n
appender.rolling.type = RollingFile
appender.rolling.name = RollingFile
appender.rolling.fileName = ${filename}/fnelLog-${date:dd-MM-yyyy}.log
appender.rolling.filePattern = ${filename}/fnelLog-%d{dd-MM-yyyy}-%i.log
appender.rolling.layout.type = PatternLayout
appender.rolling.layout.pattern = [%-5level] %d{dd MMM yyyy HH:mm:ss,SSS} [%t] %c{5} - %msg%n
appender.rolling.policies.type = Policies
#appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
#appender.rolling.policies.time.interval = 2
#appender.rolling.policies.time.modulate = true
appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size=50MB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.max = 4
loggers = rolling
############ HERE IS MY PROBLEM, I Only reference to this class, but I want to add another,
############ so I can write all logs in a file described in lines before
logger.rolling.name = fnel.bcp.org.main.TestMain
logger.rolling.level = debug
#logger.rolling.additivity = false
logger.rolling.appenderRefs = rolling
logger.rolling.appenderRef.rolling.ref = RollingFile
rootLogger.level = debug
rootLogger.appenderRefs = stdout
rootLogger.appenderRef.stdout.ref = STDOUT
And Now here are the two clases which I used the Logger interface.
public class FooClass {
private static Logger logger = LogManager.getLogger(FooClass.class);
public void testLog4j() {
// TODO Auto-generated method stub
logger.debug("This is a debug message");
logger.info("This is an info message");
logger.warn("This is a warn message");
logger.error("This is an error message");
logger.fatal("This is a fatal message");
}
}
public class TestMain {
private static Logger logger = LogManager.getLogger(TestMain.class);
public static void main(String[] args) {
FooClass foo = new FooClass();
//Calling foo method
foo.testLog4j();
// TODO Auto-generated method stub
logger.debug("This is a debug message");
logger.info("This is an info message");
logger.warn("This is a warn message");
logger.error("This is an error message");
logger.fatal("This is a fatal message");
}
}
And these are the output in console
So, my question is how can I add FooClass in my log4j.properties file. Because I added next to TestMain class like this " fnel.bcp.org.main.TestMain , fnel.bcp.org.main.FooClass" , but not work.
I tried to separate in different logger.rolling.name, but didn't work too.

Change the line
logger.rolling.name = fnel.bcp.org.main.TestMain
to
logger.rolling.name = fnel.bcp.org.main
this will cause any class in the fnel.bcp.org.main package to be directed to the RollingFile appender. Alternatively, if you want everything in the file you can just remove your logger definition and change the root logger to be directed to RollingFile instead of STDOUT.