I've been trying to change the log level on userlogs i.e the files that appear under /var/log/hadoop-yarn/userlogs/application_<id>/container_<id> on CDH 5.2.1. However, no matter what I try, only INFO level logs will appear. I want to enable TRACE level logs for debugging.
Things I have tried so far:
Setting all loggers to TRACE level in /etc/hadoop/conf/log4j.properties.
Setting mapreduce.map.log.level and mapreduce.reduce.log.level in mapred-site.xml.
Setting mapreduce.map.log.level and mapreduce.reduce.log.level in the job configuration before submitting it.
Including a log4j.properties in my job jar file that sets the root Log4j logger to TRACE.
Modifying yarn-env.sh to specify YARN_ROOT_LOGGER=TRACE,console
None of these worked -- they didn't break anything, but they didn't have any effect on the log outputs under the userlogs directory. Modifying yarn-env.sh did cause the ResourceManager and NodeManager logs to enter trace level. Unfortunately these are not useful for my purpose.
I get the following error appearing in /var/log/hadoop-yarn/userlogs/application_<id>/container_<id>/stderr that may be relevant.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/hadoop-yarn/nm-local-dir/usercache/tomcat/appcache/application_1419961570089_0001/filecache/10/job.jar/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
log4j:WARN No appenders could be found for logger (org.apache.hadoop.ipc.Server).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
I don't understand why the log4j "no configuration" message would happen, given that there is a log4j.properties file at the root of the job jar file that specifies a root logger:
log4j.rootLogger=TRACE, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] %m%n
My code does not knowingly use SLF4J for logging, it purely uses Log4j.
The actual answer was to set yarn.app.mapreduce.am.log.level to the level you need, but, crucially, it needs to be set in the Hadoop job configuration at submission time. It cannot be set on the cluster globally. The cluster global will always default to INFO, as it is hardcoded.
Using container-log4j.properties alone will not work as YARN will override the log level value on the command line. See the method addLog4jSystemProperties of org.apache.hadoop.mapreduce.v2.util.MRApps and cross reference with org.apache.hadoop.mapreduce.MRJobConfig.
container-log4j.properties will indeed be honoured, but it can't override the level set by this property.
On CDH5.8, I am overwriting the application log level at job submission time. To modify the log level of the mappers and reducers, I had to specify them explicitly using a command like this:
hadoop jar my-mapreduce-job.jar -libjars ${LIBJARS} -Dyarn.app.mapreduce.am.log.level=DEBUG,console -Dmapreduce.map.log.level=DEBUG,console -Dmapreduce.reduce.log.level=DEBUG,console
The logs are written to the applications's syslog which can be browsed using the Yarn ResourceManager Web UI.
You should try to edit (or create if it does'nt exist) : /etc/hadoop/conf/container-log4j.properties.
You can see it with a simple ps aux : the command line includes -Dlog4j.configuration=container-log4j.properties.
Related
I'm usually running stuff from JUnit but I've also tried running from main and it makes no difference.
I have read nearly two dozen SO questions, blog posts, and articles and tried almost everything to get Spark to stop logging so much.
Things I've tried:
log4j.properties in resources folder (in src and test)
Using spark-submit to add a log4j.properties which failed with "error: missing application resources"
Logger.getLogger("com").setLevel(Level.WARN);
Logger.getLogger("org").setLevel(Level.WARN);
Logger.getLogger("akka").setLevel(Level.WARN);Logger.getRootLogger().setLevel(Level.WARN);spark.sparkContext().setLogLevel("WARN");
In another project I got the logging to be quiet with:
Logger.getLogger("org").setLevel(Level.WARN);
Logger.getLogger("akka").setLevel(Level.WARN);
but it is not working here.
How I'm creating my SparkSession:
SparkSession spark = SparkSession
.builder()
.appName("RS-LDA")
.master("local")
.getOrCreate();
Let me know if you'd like to see more of my code.
Thanks
I'm using IntelliJ and Spark and, this work for me:
Logger.getRootLogger.setLevel(Level.ERROR)
You could change Log Spark configurations too.
$ cd SPARK_HOME/conf
$ gedit log4j.properties.template
# find this lines in the file
# Set everything to be logged to the console
log4j.rootCategory=INFO, console
and change to ERROR
log4j.rootCategory=ERROR, console
In this file you have other options tho change too
# Set the default spark-shell log level to WARN. When running the spark-shell, the
# log level for this class is used to overwrite the root logger's log level, so that
# the user can have different defaults for the shell and regular Spark apps.
log4j.logger.org.apache.spark.repl.Main=WARN
# Settings to quiet third party logs that are too verbose
.....
And finally rename the log4j.properties.template file
$ mv log4j.properties.template log4j.properties
You can follow this link for further configuration:
Logging in Spark with Log4j
or this one too:
Logging in Spark with Log4j. How to customize the driver and executors for YARN cluster mode.
It might be an old question, but I just ran by the same problem.
To fix it what I did was:
Adding private static Logger log = LoggerFactory.getLogger(Spark.class);
as a field for the class.
spark.sparkContext().setLogLevel("WARN"); after creating the spark session
Step 2 will work only after step 1.
I am using the log4j.properties in My Selenium Package.
Every Time I run the Module the below 3 Lines are always been added in the Console and the Applications.log file.
log4j:WARN No appenders could be found for logger (org.apache.http.client.protocol.RequestAddCookies).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
For Reference, below is the log4j.properties Code:
#Application Logs
log4j.logger.devpinoyLogger = DEBUG,dest1
log4j.appender.dest1=org.apache.log4j.RollingFileAppender
log4j.appender.dest1.maxFileSize= 5000KB
log4j.appender.dest1.maxBackupIndex=3
log4j.appender.dest1.layout=org.apache.log4j.PatternLayout
log4j.appender.dest1.layout.ConversionPattern=%d{dd/MM/YYYY HH:mm:ss} %m%n
log4j.appender.dest1.File=D:\\Automation\\src\\Logs\\Application.log
#do not append the old file .Create a new log File everytime
log4j.appender.dest1.Append=false
Please let me know what needs to be change in order to remove the 3 lines of Warnings from the console output and Application logs.
Please include the following snippet to your code:
System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.Jdk14Logger");
Hope this helps!
This will resolve your errors and the message appears in the console:
System.setProperty("org.apache.commons.logging.Log",
"org.apache.commons.logging.impl.Jdk14Logger");
Trying to run my GWT app, I get
EntryPoint initialization exception
Exception while loading module ch.swisstph.mortqual.mqui.client.MqInput. See Development Mode for details.
com.google.gwt.core.ext.UnableToCompleteException: (see previous log entries)
at com.google.gwt.dev.shell.ModuleSpace.rebindAndCreate(ModuleSpace.java:513)
at com.google.gwt.dev.shell.ModuleSpace.onLoad(ModuleSpace.java:385)
at com.google.gwt.dev.shell.OophmSessionHandler.loadModule(OophmSessionHandler.java:200)
at com.google.gwt.dev.shell.BrowserChannelServer.processConnection(BrowserChannelServer.java:526)
at com.google.gwt.dev.shell.BrowserChannelServer.run(BrowserChannelServer.java:364)
at java.lang.Thread.run(Thread.java:722)
in the web browser, and just
/usr/lib/jvm/java-6-openjdk/bin/java -Xmx256m -Didea.launcher.port=7537 -Didea.launcher.bin.path=/home/dhardy/code/download/idea-IU-129.239/bin -Dfile.encoding=UTF-8 -classpath /home/install/gwt/gwt-dev.jar:/home/dhardy/p/mortqual/mqui/src:/usr/lib/jvm/java-6-openjdk/jre/lib/compilefontconfig.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/jsse.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/rhino.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/jce.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/management-agent.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/rt.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/javazic.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/resources.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/charsets.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/ext/java-atk-wrapper.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/ext/pulse-java.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/ext/zipfs.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/ext/dnsns.jar:/home/dhardy/p/mortqual/mqui/out/test/mqui:/home/dhardy/p/mortqual/mqui/out/production/mqui:/home/install/appengine-java-sdk/lib/shared/jsp-api.jar:/home/install/appengine-java-sdk/lib/shared/appengine-local-runtime-shared.jar:/home/install/appengine-java-sdk/lib/shared/el-api.jar:/home/install/appengine-java-sdk/lib/shared/servlet-api.jar:/home/install/appengine-java-sdk/lib/user/appengine-api-1.0-sdk-1.7.5.jar:/home/install/gwt/gwt-user.jar:/home/dhardy/p/mortqual/anacod/target/test-classes:/home/dhardy/p/mortqual/anacod/target/classes:/home/dhardy/.m2/repository/net/sourceforge/jexcelapi/jxl/2.6.12/jxl-2.6.12.jar:/home/dhardy/.m2/repository/log4j/log4j/1.2.14/log4j-1.2.14.jar:/home/dhardy/.m2/repository/com/beust/jcommander/1.30/jcommander-1.30.jar:/home/dhardy/.m2/repository/org/apache/poi/poi/3.9-20130311/poi-3.9-20130311.jar:/home/dhardy/.m2/repository/org/apache/poi/poi-ooxml/3.9/poi-ooxml-3.9.jar:/home/dhardy/.m2/repository/org/apache/poi/poi-ooxml-schemas/3.9/poi-ooxml-schemas-3.9.jar:/home/dhardy/.m2/repository/org/apache/xmlbeans/xmlbeans/2.3.0/xmlbeans-2.3.0.jar:/home/dhardy/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/home/dhardy/.m2/repository/dom4j/dom4j/1.6.1/dom4j-1.6.1.jar:/home/dhardy/.m2/repository/xml-apis/xml-apis/1.0.b2/xml-apis-1.0.b2.jar:/home/install/gwt/validation-api-1.0.0.GA-sources.jar:/home/install/gwt/validation-api-1.0.0.GA.jar:/home/dhardy/code/download/idea-IU-129.239/lib/idea_rt.jar com.intellij.rt.execution.application.AppMain com.google.gwt.dev.DevMode -war /home/dhardy/.IntelliJIdea12/system/gwt/mqui.5589a251/mqui.33ff3210/run/www -remoteUI 7901:IntelliJIdea -startupUrl mqInput.html ch.swisstph.mortqual.mqui.mqui
log4j:WARN No appenders could be found for logger (org.apache.jasper.compiler.JspRuntimeContext).
log4j:WARN Please initialize the log4j system properly.
Dev Mode initialized. Startup URL:
http://127.0.0.1:8888/mqInput.html?gwt.codesvr=127.0.0.1:9997
on the command line. There are no logs I can find (I tried configuring log4j via a .properties file, which removed its warnings but still didn't give me any logs).
So how do I solve this?
The two most likely causes are renaming of my start-up page and pushing some code out to a library.
try putting -logLevel SPAM in your command line arguments, this will print detailed logs, also you can put explicit GWT.log("message") in your entry point code, that will tell your how far it is going (normal logging doesn't work directly with gwt).
This page describes how to debug GWT in general, how ever my suggestion would be run debug this directly in eclipse with gwt plugin, GWT support in eclipse is amazing.
I'm having the same issue described here: Log4j Warning while initializing?
My application has a log4j.properties that allows me to configure log4j.rootLogger=INFO, stdout correctly. However I still see:
log4j:WARN No appenders could be found for logger (org.jboss.logging).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
When running the application.
My log4j.properties currently reads:
log4j.rootLogger=INFO, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %t %c:%L - %m%n
#log4j.category.org.springframework=DEBUG,stdout
I've also tried adding the properties file to my config via:
#PropertySource(value= {
"classpath:/log4j.properties",
"classpath:/application.properties"
})
How can I make this error go away? I can toggle between DEBUG and INFO just fine via this file, so I know it's being read.
You can make this error go away by adding the line
log4j.logger.org.jboss.logging=INFO, stdout
to your log4j.properties.
I don't know why, but it seems like something sets the additivity of org.jboss.logging to false, causing it to ignore the root appender(s). So you can either find that magical spot or you add the the line above.
I have created a very simple application, where I am trying to use the Log4J, but my application is not logging any log.
Can anyone please tell me how can I debug the same as my log4j started or not?
I have kept the file in classes folder of WEB-INF/classes
Thanks
following is my log4j.properties
log4j.rootLogger=debug, stdout, ABC
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
# Pattern to output the caller's file name and line number.
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n
log4j.appender.ABC=org.apache.log4j.RollingFileAppender
log4j.appender.ABC.File=D://abc//dams_workflow_application.log
log4j.appender.ABC.MaxFileSize=3000KB
# Keep one backup file
log4j.appender.ABC.MaxBackupIndex=10
log4j.appender.ABC.layout=org.apache.log4j.PatternLayout
log4j.appender.ABC.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n
following is the sample line for using log 4j..
Logger log = Logger.getLogger("ABC");
log.info("my message");
If you're writing a web application, you can view the Log4J Default Initialization Under Tomcat. Even it states Tomcat, the same applies in other Containers/Web application servers.
Console will print this if log4j is not started :
log4j:WARN No log4j configuration information found.
log4j:WARN Changed non-configured level from DEBUG to ERROR.
log4j:WARN The log4j system is not properly configured!
You could have a look at Log4j - Looking for a good 'Getting started' tutorial or blog
to get started.
Regards,
Stéphane
Make sure the log4j jar is in the lib directory for your webapp or in the lib directory in tomcat.
Log4j doesn't seem to work properly for me unless i put the log4j properties file in the default package in the webapp's source directory within the project.
All of my log4j properties files typically look similar to the following
# Define a logger called "processLog" using the FileAppender implementation of the
# Log interface
log4j.appender.processLog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.processLog.File.DateFormat='.'yyyy-ww
# Define the output location
log4j.appender.processLog.File=C:/logs/myapp.log
# Define what the output is going to look like for your log
log4j.appender.processLog.layout=org.apache.log4j.PatternLayout
log4j.appender.processLog.layout.ConversionPattern=%d{yyyy-MM-dd hh:mm a} %5p %c{1}: Line#%L - %m%n
# log4j.rootLogger specifies the default logging level and output location.
# The first parameter is the level (debug > info > warn > error > fatal).
log4j.rootLogger=INFO, processLog
That's how I do it anyway. There may be a better way but this one always works for me.