Develop Reference

How can I straighten out my Tomcat logging?

Am working on a Tomcat configuration I did not do and would like any help to fix the logging. For reasons unknown, there are three -D parameters related to logging passed into the startup. The process looks like:
./bin/java -Djava.util.logging.config.file=CATALINA_BASE_DIR_HERE/tomcat/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Dlog4j.configuration=file:CATALINA_BASE_DIR_HERE/tomcat/conf/log4j.properties ...
The relevant section of the logging.properties file looks as shown below. I believe this is standard.
handlers = 1catalina.org.apache.juli.FileHandler, 2localhost.org.apache.juli.FileHandler, 3manager.org.apache.juli.FileHandler, 4host-manager.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler
...
2localhost.org.apache.juli.FileHandler.level = FINE
2localhost.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
2localhost.org.apache.juli.FileHandler.prefix = localhost.
For completeness, here is relevant portion of the log4j.properties file:
log4j.rootLogger=INFO, logfile
log4j.appender.logfile=org.apache.log4j.DailyRollingFileAppender
log4j.appender.logfile.DatePattern='.'yyyy-MM-dd
log4j.appender.logfile.File=MY_DIR/my_log.txt
log4j.appender.logfile.layout=org.apache.log4j.PatternLayout
log4j.appender.logfile.layout.ConversionPattern = [%d{ABSOLUTE}] [%t] %-5p [%c] - %m%n
log4j.appender.logfile.Append=true
# per first answer given below, added:
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost] = INFO, logfile
Prior to today, there were no real issues - all the log data went to the log4j file as desired. The war file deployed in this Tomcat uses Spring Data JPA and Hibernate. To debug, showSql was enabled. All the queries came out, but to the localhost.DATE.log file. Can anyone tell how to fix so that ALL the statements go to the log4j designated file?

If you set showSql to true, hibernate will print SQL statement to SystemOut. You should add log4j.logger.org.hibernate.SQL=DEBUG to log4j config, so hibernate can also log the SQL statement. (The reason can be found in this answer)
By default, Tomcat use java.util.logging API for all internal logging. So the output goto localhost.DATE.log as you mentioned.
You can change the configuration, please refer the section Using Log4j
(for Tomcat 6.x~8.x)

Why I obtain this FileNotFoundException when I try to use log4j to write into a file?

I am absolutly new using log4j and I have the following problem.
I am trying to print the logging line into a file named log.out.
So I create the following log4j.properties configuration file:
# Define the root logger with appender file
log4j.rootLogger = DEBUG, FILE
# Define the file appender
log4j.appender.FILE=org.apache.log4j.FileAppender
log4j.appender.FILE.File=${log}/log.out
# Define the layout for file appender
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.conversionPattern=%m%n
The problem is that, when I perform my application and when it incurs in a logging operation, something like this:
logger.debug("INTO main()");
I obtain the following exception into the console (the error message related to the log.out file is access denied (in italian Accesso negato):
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: \log.out (Accesso negato)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(Unknown Source)
at java.io.FileOutputStream.<init>(Unknown Source)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:2
56)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.j
ava:132)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.j
ava:96)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigura
tor.java:654)
and don't write nothing into my log.out file (that I have created manually). This log.out file is at the same level of the performed jar file that represent my application.
Why? What am I missing? How can I solve this issue?
Tnx

I think $log is empty and it's trying to create a file on root and you are running program as a normal user.
give it a check.

For logging when using properties file following config should work. For the tomcat application server, please use ${catalina.base} to get the tomcat base directory.
log4j.appender.FILE.File=${catalina.base}/logs/debug.log
For XML config the syntax might be like the following
<param name="file" value="${catalina.base}/logs/debug.log"/>
If you are using Tomcat as your application server, I hope it will work.

How to change log level of userlogs in Hadoop 2?

I've been trying to change the log level on userlogs i.e the files that appear under /var/log/hadoop-yarn/userlogs/application_<id>/container_<id> on CDH 5.2.1. However, no matter what I try, only INFO level logs will appear. I want to enable TRACE level logs for debugging.
Things I have tried so far:
Setting all loggers to TRACE level in /etc/hadoop/conf/log4j.properties.
Setting mapreduce.map.log.level and mapreduce.reduce.log.level in mapred-site.xml.
Setting mapreduce.map.log.level and mapreduce.reduce.log.level in the job configuration before submitting it.
Including a log4j.properties in my job jar file that sets the root Log4j logger to TRACE.
Modifying yarn-env.sh to specify YARN_ROOT_LOGGER=TRACE,console
None of these worked -- they didn't break anything, but they didn't have any effect on the log outputs under the userlogs directory. Modifying yarn-env.sh did cause the ResourceManager and NodeManager logs to enter trace level. Unfortunately these are not useful for my purpose.
I get the following error appearing in /var/log/hadoop-yarn/userlogs/application_<id>/container_<id>/stderr that may be relevant.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/hadoop-yarn/nm-local-dir/usercache/tomcat/appcache/application_1419961570089_0001/filecache/10/job.jar/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
log4j:WARN No appenders could be found for logger (org.apache.hadoop.ipc.Server).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
I don't understand why the log4j "no configuration" message would happen, given that there is a log4j.properties file at the root of the job jar file that specifies a root logger:
log4j.rootLogger=TRACE, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] %m%n
My code does not knowingly use SLF4J for logging, it purely uses Log4j.

The actual answer was to set yarn.app.mapreduce.am.log.level to the level you need, but, crucially, it needs to be set in the Hadoop job configuration at submission time. It cannot be set on the cluster globally. The cluster global will always default to INFO, as it is hardcoded.
Using container-log4j.properties alone will not work as YARN will override the log level value on the command line. See the method addLog4jSystemProperties of org.apache.hadoop.mapreduce.v2.util.MRApps and cross reference with org.apache.hadoop.mapreduce.MRJobConfig.
container-log4j.properties will indeed be honoured, but it can't override the level set by this property.

On CDH5.8, I am overwriting the application log level at job submission time. To modify the log level of the mappers and reducers, I had to specify them explicitly using a command like this:
hadoop jar my-mapreduce-job.jar -libjars ${LIBJARS} -Dyarn.app.mapreduce.am.log.level=DEBUG,console -Dmapreduce.map.log.level=DEBUG,console -Dmapreduce.reduce.log.level=DEBUG,console
The logs are written to the applications's syslog which can be browsed using the Yarn ResourceManager Web UI.

You should try to edit (or create if it does'nt exist) : /etc/hadoop/conf/container-log4j.properties.
You can see it with a simple ps aux : the command line includes -Dlog4j.configuration=container-log4j.properties.

Log4J log does not create back up files

I have configured my application log in the generic JVM arguments of my Websphere application server as :
-Dlog4j.configuration=file:D:/app_logs/log4j/log4j.properties
The log4j.properties file looks like:
log4j.appender.app.layout=org.apache.log4j.PatternLayout
log4j.appender.app.layout.ConversionPattern=%d{ISO8601} %5p %c{1}:%L - %m%n
log4j.appender.app=org.apache.log4j.RollingFileAppender
log4j.appender.app.File=D:/app_logs/log4j/logs/app.log
log4j.appender.app.MaxFileSize=8000KB
log4j.appender.app.MaxBackupIndex=10
#root logger option
log4j.rootLogger=debug,app
# restrictions
log4j.logger.net.sf.hibernate=info
#log4j.logger.net.sf.hibernate.type=debug
#log4j.logger.net.sf.hibernate.ps.PreparedStatementCache=debug
log4j.logger.com.abc.wm.eu.app.interop=off
log4j.logger.com.abc.wm.eu.app.webservice.servlet.ADOServlet=off
#
log4j.logger.org.apache.axis=error
log4j.logger.org.apache.commons.httpclient=error
log4j.logger.httpclient.wire.header=error
log4j.logger.httpclient.wire.content=error
log4j.logger.org.htmlparser=error
Problem:
The log rolls over. But the back up logs are not created. The same configuration works fine for another application running on the same server.
I have no idea what is wrong here.
All suggestions and solutions are welcome.
Thanks

log4j.appender.FILE.Append=true
This might do the trick

Application is not logging log, how to debug log 4j starting or not

I have created a very simple application, where I am trying to use the Log4J, but my application is not logging any log.
Can anyone please tell me how can I debug the same as my log4j started or not?
I have kept the file in classes folder of WEB-INF/classes
Thanks
following is my log4j.properties
log4j.rootLogger=debug, stdout, ABC
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
# Pattern to output the caller's file name and line number.
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n
log4j.appender.ABC=org.apache.log4j.RollingFileAppender
log4j.appender.ABC.File=D://abc//dams_workflow_application.log
log4j.appender.ABC.MaxFileSize=3000KB
# Keep one backup file
log4j.appender.ABC.MaxBackupIndex=10
log4j.appender.ABC.layout=org.apache.log4j.PatternLayout
log4j.appender.ABC.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n
following is the sample line for using log 4j..
Logger log = Logger.getLogger("ABC");
log.info("my message");

If you're writing a web application, you can view the Log4J Default Initialization Under Tomcat. Even it states Tomcat, the same applies in other Containers/Web application servers.

Console will print this if log4j is not started :
log4j:WARN No log4j configuration information found.
log4j:WARN Changed non-configured level from DEBUG to ERROR.
log4j:WARN The log4j system is not properly configured!
You could have a look at Log4j - Looking for a good 'Getting started' tutorial or blog
to get started.
Regards,
Stéphane

Make sure the log4j jar is in the lib directory for your webapp or in the lib directory in tomcat.
Log4j doesn't seem to work properly for me unless i put the log4j properties file in the default package in the webapp's source directory within the project.
All of my log4j properties files typically look similar to the following
# Define a logger called "processLog" using the FileAppender implementation of the
# Log interface
log4j.appender.processLog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.processLog.File.DateFormat='.'yyyy-ww
# Define the output location
log4j.appender.processLog.File=C:/logs/myapp.log
# Define what the output is going to look like for your log
log4j.appender.processLog.layout=org.apache.log4j.PatternLayout
log4j.appender.processLog.layout.ConversionPattern=%d{yyyy-MM-dd hh:mm a} %5p %c{1}: Line#%L - %m%n
# log4j.rootLogger specifies the default logging level and output location.
# The first parameter is the level (debug > info > warn > error > fatal).
log4j.rootLogger=INFO, processLog
That's how I do it anyway. There may be a better way but this one always works for me.