Develop Reference

log4j2 log file is not generating

the log file is generated when I run the code within IDE (Intellij IDEA).
as soon as I create runnable jar of the code and then try to run the jar then the logs are not generating.
I have made sure the log4j2.xml file is a part of classpath.
is there anything extra I have to do while creating jar in the Intellij IDEA?

Taken from the FAQ: How do I debug my configuration?
First, make sure you have the right jar files on your classpath. You need at least log4j-api and log4j-core.
Next, check the name of your configuration file. By default, log4j2 will look for a configuration file named log4j2.xml on the classpath. Note the “2” in the file name! (See the configuration manual page for more details.)
From log4j-2.9 onward
From log4j-2.9 onward, log4j2 will print all internal logging to the console if system property log4j2.debug is either defined empty or its value equals to true (ignoring case).
Prior to log4j-2.9
Prior to log4j-2.9, there are two places where internal logging can be controlled:
If the configuration file is found correctly, log4j2 internal status logging can be controlled by setting in the configuration file. This will display detailed log4j2-internal log statements on the console about what happens during the configuration process. This may be useful to trouble-shoot configuration issues. By default the status logger level is WARN, so you only see notifications when there is a problem.
If the configuration file is not found correctly, you can still enable log4j2 internal status logging by setting system property -Dorg.apache.logging.log4j.simplelog.StatusLogger.level=TRACE.

How to know if logback configuration file has loaded?

I am new to SLF4j and I don't know if the logback.xml file has loaded properly or not. The logback.xml file is in PROJECTNAME/src/main/java where all my packages are found.
My questions are:
How can I know if the configuration file has properly loaded or not
?
How can restrict the logging only from an explicit set of class,
only to avoid logging from libraries

You can add the debug="true" attribute to the <configuration> element to enable debug of the logback configuration. It will print the configuration to the console. See https://logback.qos.ch/manual/configuration.html#dumpingStatusData.

Simple answer, if the configuration file is loaded properly, you will see results in log file or console, depending on your configuration.
By default, logback searches file in src/main/resources instead of src/main/java if I remember correctly.
In the configuration file, you can define log lever on a specific logger. Normally you'll still want to see logs of the libraries, but maybe only WARN or ERROR, so you could set the root level to WARN/ERROR, and add a logger of your root package with DEBUG/INFO level.
Also, use a logback-test file (under src/test/resources) for your own dev environment.

Cannot seem to redirect Tomcat logs to log4j

I am using this doc to change my Tomcat loggers to log4j:
https://tomcat.apache.org/tomcat-7.0-doc/logging.html
my log4j.properties file is as shown in the document, except I changed this line to confirm it's working:
log4j.appender.CATALINA.File = ${catalina.base}/logs/test_catalina
I still see a file logs/catalina.out and not logs/test_catalina, which makes me think either log4j is not being used or my properties file is not read.
I have tomcat-juli-adapters.jar, log4j-1.2.17.jar and log4j.properties in lib
and overwrote bin/tomcat-juli.jar
running set -x catalina.sh start shows:
eval "/a/java64/jdk1.8.0/bin/java" "-Dnop" -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dlog4j.debug=true -Djava.endorsed.
dirs="/opt/apache-tomcat-7.0.62/endorsed" -classpath "/opt/apache-tomcat-7.0.62/bin/bootstrap.jar:/opt/apache-tomcat-7.0.62/bin/tomcat-juli.jar" -Dcatalina.base="/opt/apache-tomcat-7.0.62" -Dcatalina.home="/opt/apache-tom
cat-7.0.62" -Djava.io.tmpdir="/opt/apache-tomcat-7.0.62/temp" org.apache.catalina.startup.Bootstrap start &
I also have no logging.properties file on disk:
ls conf/*.properties
conf/catalina.properties
I would really appreciate advice on how to debug this -- I'm not sure if a jar is not being found properly, if my log4j.properties has a problem, or Tomcat just plain can't tell I'd like log4j.
Using Tomcat version Server number: 7.0.62.0

I would really appreciate advice on how to debug this
The switch of sending Tomcat log messages from java.util.logging to log4j 1.x is performed by Apache Commons Logging library. (Its compiled package-renamed version is in those tomcat-juli.jar and tomcat-juli-adapters.jar)
There is a debug setting available in Apache Commons Logging. After package renaming the setting name becomes org.apache.juli.logging.diagnostics.dest.
I do not see any evident error in those steps that you performed. Dummy issues may be: are the files readable?
Notes:
It is possible to put tomcat-juli-adapters.jar into $CATALINA_HOME/bin instead of /lib, but you need to update CLASSPATH variable (in your bin/setenv.sh) so that it is included in the value of -classpath argument to java. Repacking the two jars into a single jar file is also an option.
As mentioned by tomcat-7.0-doc/logging.html, deleting conf/logging.properties makes java.util.logging to fallback to default configuration. You may wish to explicitly configure that file.
Source code for tomcat-juli.jar and tomcat-juli-adapters.jar can be found at Maven Central. Those libraries have groupId=org.apache.tomcat.extras

hadoop flume log4j configuration

If you run a hadoop flume node, as default it generates logs under /var/log/flume using log4j. The files will look like
/var/log/flume/flume-$FLUME_IDENT_STRING-$command-$HOSTNAME.log
According to the flume user guide here, the only way to change the flume log configuration is via flume-daemon.sh which runs flume node using the Flume Environment Variables like:
export FLUME_LOGFILE=flume-$FLUME_IDENT_STRING-$command-$HOSTNAME.log
export FLUME_ROOT_LOGGER="INFO,DRFA"
export ZOOKEEPER_ROOT_LOGGER="INFO,zookeeper"
export WATCHDOG_ROOT_LOGGER="INFO,watchdog"
The questions are:
if I want to change the log level from INFO to DEBUG, this is the only place to do it?
Is there a configuration somewhere I can do this?
what about I want to set some packages' log level to DEBUG while others stay INFO?

Check if the log4j.properties or log* related files exist to set the variables -- which will also allow you to check and have some components of the logging part do excessive/DEBUG while other do INFO.

Noticed that under /etc/flume/conf.empty, there is a log4j.properties. Copied it to /etc/flume/conf, restart the flume node service, the log4j.properties file starts taking effect.
The overriding order is like flume-env.sh->flume-daemon.sh->log4j.properties.
eg. if you set up your flume_root_logger to DEBUG in flue-daemon.sh, it will override whatever setting you have for root_logger in your log4j.properties.

Java Application: Getting Log4j To Work Within Eclipse Environment

I've done my best to setup Eclipse and my Java application to use a log4j.properties file. However, it does not seem to be using the properties file and I'm not sure why.
Libraries: slf4j-api-1.6.1, slf4j-jdk14-1.6.1
Within the application the logging works fine. I am able to print info, warnings, and errors into the Eclipse console.
What I would like to be able to do is change the log level to debug and print all logging messages to both the console and a log file.
I have created a log4j.properties file that looks like this:
log4j.rootLogger=DEBUG,console,file
log4j.rootCategory=DEBUG, R, O
# Stdout
log4j.appender.O=org.apache.log4j.ConsoleAppender
# File
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=log4j.log
# Control the maximum log file size
log4j.appender.R.MaxFileSize=100KB
# Archive log files (one backup file here)
log4j.appender.R.MaxBackupIndex=5
log4j.appender.file.File=checkLog.log
log4j.appender.file.threshold=DEBUG
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.O.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=[%d{ISO8601}]%5p%6.6r[%t]%x - %C.%M(%F:%L) - %m%n
log4j.appender.O.layout.ConversionPattern=[%d{ISO8601}]%5p%6.6r[%t]%x - %C.%M(%F:%L) - %m%n
My directory structure looks like this:
My Project
--src/
----MYProject/
------*.java
--bin/
----MYProject/
------*.class
--log4j/
----log4j.properties
In Eclipse I this this:
Run Configurations -> Classpath (tab) ->, right clicked on User Entries -> Added "log4j" as a new folder, and saved.
Then in my code I call the logger like this (sample code to demonstrate my approach so it may have syntax errors):
package MYProject;
import org.slf4j.LoggerFactory;
public class MyClass{
final org.slf4j.Logger test_logger = LoggerFactory.getLogger(MyClass.class);
public MyClass(){}
public someMethod(){
test_logger.debug("Some Debug");
test_logger.info("Some Info");
test_logger.warn("Some Warning");
test_logger.error("An Error");
}
}
I then call someMethod and it prints INFO, WARN, ERROR to the Eclipse console. It won't print DEBUG and won't print to a file.
I'd appreciate any suggestions on what I may be doing wrong.

There may be another log4j.properties or log4j.xml file in the classpath ahead of your log4j.properties. Open the run configuration for your project, and add -Dlog4j.debug=true as a VM Argument for your project. This will instruct log4j to print a lot of additional information on the console, including the config file that it is using.

If you are using the logging façade slf4j, then you need to specify exactly one logging backend by including the corresponding jar file for that backend. In your case, you have installed slf4j-jdk14-x.x.x.jar on your classpath, which is just a generic logger backend.
In order to use the log4j backend, you need to remove slf4j-jdk14-x.x.x.jar and replace it with slf4j-log4j12-x.x.x.jar. If you don't remove it, slf4j must choose only one backend jar, and probably not the one you want.
Of course you will also need the actual log4j-x.x.x.jar file on your classpath too.
Once these jars are properly in place, then the VM parameter of -Dlog4j.debug will actually work and be useful in debugging where your logging configs are coming from.

You need to tell your code to use the properties file. Before any logging is done please put
PropertyConfigurator.configure("log4j/log4j.properties");

You might have an old version in your target directory.
Clean the project and try again.
Apart from that you should make sure that you refresh the eclipse project if you didn't add the log4j.properties via eclipse.

delete jre(JRE System Library) of project in path project properties/library tab and set jre again!