Develop Reference

Empty Output file after doing the mapreduce (hadoop)

Hi so I got this problem that happened when I tried to use a jar on hadoop. But my Output folder is empty and I got this error message. I was wondering how can I fix this problem ? I saw some post on stack that said that I needed to change a property to this one -> mapreduce.map.failures.maxpercent. But I can't find where it's located I tried to see in mapred-site.xml but there is no line similar to this one. Also I am not really sure if this line would fix anything.

From the screenshot, it is evident that the job failed. The map task has failed (Failedmap tasks=4). Check additional logs to correct the issue with the code.
A folder with the same application ID will be created in the logs/userlogs of your Hadoop installation directory. For example:
HADOOP_INSTALLATION_DIR/logs/userlogs/application_xxxxxxxxxxxxxxx_xxxx
You can check the syslog and sysout messages.

Java System Level deployment Malformed deployment.config file

I am trying to install Java jre 1.8u31 from the command line. I am using system level install configuration by using the deployment.config file and deployment.properties.
I have tried the following:
deployment.system.config=file\:C\:/WINDOWS/Sun/Java/Deployment/deployment.properties
deployment.system.config.mandatory=true
I have also tried the following
deployment.system.config=file:///C:/Windows/Sun/Java/Deployment/deployment.properties
deployment.system.config.mandatory=true
I have swapped the entries around in hopes of getting a better error describing what I am doing wrong. I have also made the first line blank in the deployment.config file. I have googled and tried all examples I could find online. In all the cases, I am being presented a dialog box with an error that states the deployment.config file's line 1 is malformed.
Any suggestions would be greatly appreciated.
Russ
I have tried all of these formats:

The path you have given should be in below format
deployment.system.config=file:/C:/Windows/Sun/Java/Deployment/deployment.proper‌​ties

I got the install to work correctly. What I did was put the deployment.config file in the C:\Windows\Sun\Java\Deployment directory. The property in the file was setup as so:
deployment.system.config=file\:C\:\\Sup\\Java\\UPGRADE\\Deployment\\deployment.properties
The exceptionlist file was in the same directory as the deployment.properties file.

How is user.dir System Property interpreted under commons-daemon procrun & log4j?

I have installed a java batch process using the version of procrun that ships with tomcat 5.5.33:
Commons Daemon Service Runner version 1.0.5.0/Win32 (Jan 5 2011)
Copyright (c) 2000-2011 The Apache Software Foundation.
In the installation, I specify (among other JVM options):
--JvmOptions="-Duser.dir=C:\LOCAL\serverapps"
My log4j.properties configuration includes:
log4j.appender.InfoLogFile.File=../logs/info.log
However, the info.log file is being written to:
C:\WINDOWS\logs
I've checked the value of user.dir at many different points and it's always C:\LOCAL\serverapps.
But, log4j is behaving as if user.dir=C:\Windows\System32 (or some other subir of C:\Windows).
From what I can tell from the log4j source (1.2.16), the FileAppender deals only with the java.io.FileOutputStream and File classes which claim to make paths relative from the user.dir location.
I've worked around the issue, but I am curious: has anyone else has encountered this type of behaviour? If so, what's really going on?

FileAppender, when given a relative path, creates a file withing the current working directory - not the user home directory.
You need to pass the ${user.dir} within the filename.
SRC:
http://svn.apache.org/viewvc/logging/log4j/trunk/src/main/java/org/apache/log4j/FileAppender.java?view=markup
EDIT: see comment below for correction - user.dir != user.home
http://bugs.sun.com/view_bug.do?bug_id=4117557

I have used ${user.dir} in the lo4j.properties and it has worked. Have you tried?
log4j.appender.InfoLogFile.File=${user.dir}/logs/info.log

PhilW's comment points to the correct answer to the original question. That is, Oracle/Sun declares an issue http://bugs.sun.com/view_bug.do?bug_id=4117557 when user.dir is set via the command line. That is the reason why the relative path is not properly understood when logging files are written out.
By using a an absolute path (even prefixing with ${user.dir} -- which can be trusted at that point - even if the JVM gets the value wrong internally) as Phil, Amir and I all suggest, you avoid the issue altogether.

Where is jboss.server.config.url set in JBoss 5?

I am facing a rather strange problem with an installation of JBoss 5.0.1 I have to deploy a webapp on.
In my webapp I try:
System.getProperty("jboss.server.config.url")
to get at the "conf"-dir. (And subsquently read a few files) On my development-machine this worked just fine, but in the deployment environment the aforementioned property appears to be the empty String ("").
Does anyone know why that could be and how to change it?
Thank you

I don't know why it does not work properly at yours, but you can change any JBoss property, just open run.conf file, and add line like at the end (at least JAVA_OPTS should be defined before):
JAVA_OPTS="$JAVA_OPTS -Djboss.server.config.url=new_path"
for Windows should work (in run.conf.bat):
set JAVA_OPTS = %JAVA_OPTS% -Djboss.server.config.url=new_path
this way you can change any property. Like Kevin D. already wrote, take a look at JBoss Properties, jboss.server.config.url is just $jboss.server.home.url/conf, so maybe something wrong with jboss.server.home.url variable, you can specify its value.. In its turn its value is $jboss.server.base.url/$jboss.server.name, check them etc. So you could find real cause of it... Hope it helps

It may rely on your JBOSS_Home being set in your environment variables.
This...article/list seems to indicate that all the JBOSS properties are built up from this http://community.jboss.org/wiki/JBossProperties

Where/how does log4j look for a log4j.properties file?

I'm trying out log4j in a simple test app. I create a new Java project in eclipse and add the log4j JAR (v1.2.16) to my build path. I then create a simple class that prints Hello World. Then I use the log4j Logger class to log a info message. When I run the app, I see the log message, using what I assume is the default appender and layout. Great. What I'm having trouble with is adding my own configuration. This is what I've done:
Created a log4j.properties file with a custom appender and log level and placed it into the src folder (which upon compilation gets copied to the bin folder). Run the app - no change.
I try adding PropertyConfigurator.configure("log4j.properties"). Run the app - no change. No errors, but no change.
What do I have to do to get log4j to load my configuration file?

Look in the manual under the heading Default Initialization Procedure, where you'll find the following:
The exact default initialization algorithm is defined as follows:
Setting the log4j.defaultInitOverride system property to any other value than "false" will cause log4j to skip the default
initialization procedure (this procedure).
Set the resource string variable to the value of the log4j.configuration system property. The preferred way to specify
the default initialization file is through the log4j.configuration
system property. In case the system property log4j.configuration is
not defined, then set the string variable resource to its default
value "log4j.properties".
Attempt to convert the resource variable to a URL.
If the resource variable cannot be converted to a URL, for example due to a MalformedURLException, then search for the resource from the
classpath by calling
org.apache.log4j.helpers.Loader.getResource(resource, Logger.class)
which returns a URL. Note that the string "log4j.properties"
constitutes a malformed URL. See
Loader.getResource(java.lang.String)
for the list of searched locations.
If no URL could not be found, abort default initialization. Otherwise, configure log4j from the URL. The
PropertyConfigurator
will be used to parse the URL to configure log4j unless the URL ends
with the ".xml" extension, in which case the
DOMConfigurator
will be used. You can optionaly specify a custom configurator. The
value of the log4j.configuratorClass system property is taken as the
fully qualified class name of your custom configurator. The custom
configurator you specify must implement the Configurator interface.

Argh. I discovered the problem was that eclipse had imported the wrong Logger class. It had imported java.util.logging.Logger which of course has it's own configuration that is different from log4j. Oh well, hope somebody else does this and gets it solved by reading this question.

You can enable log4j internal debugging by setting the log4j.debug system property. Among other things, this will cause log4j to show how it is configuring itself.
You can try explicitly setting the URL to the configuration file with the log4j.configuration system property.
See also: this question.

The problem may be in the classpath, if the classpath was defined.
The reason it wasn't loading (in my case): There was a conflicting log4j.properties file in one of my jars, and it was overloading the one in my classpath.
In short, if your log4j.properties file isn't loading, there might be another one somewhere else overriding it.
Just thought I'd throw this in too, in case anyone else runs into this. I just spent the last 5 hours trying to figure out why my default log4j.properties wouldn't load.

log4j.properties should be in your classpath. The "src folder" which is copied to the "bin folder" (I assume you are speaking of a Eclipse setup here), normally belongs to your classpath, so it should be found (are you placing it at the top of the "src" folder, right?)

I know this is a couple of months old, but I feel the need to point out that the scr folder isn't "copied" to the bin folder, nor is it part of your runtime classpath....(build path is not runtime classpath!). Eclipse compiles the source files in the src folder to the bin (or whatever you like) folder. It's the bin folder that is part of your runtime classpath.
Just wanted to point this out as these threads are often read by very junior programmers as well, and I'm always frustrated that most of them don't grasp the finesse of the Java classpath, and hence make avoidable mistakes against it.

Just throwing it out there to anyone who is struggling to get log4j.properties filed loaded.
I fixed mine by adding the following line:
-Dlog4j.defaultInitOverride=TRUE
Turns out using that JVM parameter, it possible to tell Log4j not to use the Default Initialization Procedure and create your own.
I struggled with mine. I did from clean JAR build to entire REPO pull, nothing work. Combination of that and clean up of class file made it work.