Installing Hadoop on MacOS resulting in build failure with CMake - java

I am trying to build the hadoop native for the first time as given below
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.3:cmake-compile (cmake-compile) on project hadoop-common: Error executing CMake: Cannot run program "cmake" (in directory "/Users/krishnanand/projects/hadoop/hadoop-common-project/hadoop-common/target/native"): error=2, No such file or directory -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.3:cmake-compile (cmake-compile) on project hadoop-common: Error executing CMake
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Error executing CMake
at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.runCMake(CompileMojo.java:176)
at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.execute(CompileMojo.java:97)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: java.io.IOException: Cannot run program "cmake" (in directory "/Users/krishnanand/projects/hadoop/hadoop-common-project/hadoop-common/target/native"): error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.runCMake(CompileMojo.java:166)
... 23 more
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
... 24 more
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
The requirements to build the native code states that CMake v3.1 is required. I have installed CMake with brew and my list of installed packages is given below
hadoop (release_3_0_3) $ brew list
autoconf cmake gzip libtool protobuf#2.5
automake gcc isl mpfr snappy
bzip2 gmp libmpc openssl zlib
More information about cmake
hadoop (release_3_0_3) $ brew list cmake
/usr/local/Cellar/cmake/3.12.0/bin/ccmake
/usr/local/Cellar/cmake/3.12.0/bin/cmake
/usr/local/Cellar/cmake/3.12.0/bin/cmakexbuild
/usr/local/Cellar/cmake/3.12.0/bin/cpack
/usr/local/Cellar/cmake/3.12.0/bin/ctest
/usr/local/Cellar/cmake/3.12.0/share/aclocal/cmake.m4
/usr/local/Cellar/cmake/3.12.0/share/cmake/ (2376 files)
/usr/local/Cellar/cmake/3.12.0/share/doc/ (8 files)
/usr/local/Cellar/cmake/3.12.0/share/emacs/site-lisp/cmake/cmake-mode.el
/usr/local/Cellar/cmake/3.12.0/share/man/ (20 files)
I am trying to install Hadoop v3.0.3. Any help would be gratefully appreicated.

Related

Maven Invalid maximum heap size error -Xmx4g on mahout-math-scala_2.10

I am trying to install mahout through maven but i get Invalid maximum heap size error:
heap size error
--- scalatest-maven-plugin:1.0:test (test) # mahout-math-scala_2.10 ---
Invalid maximum heap size: -Xmx4g
The specified size exceeds the maximum representable size.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit
I am using 64 bit laptop and 32 bit ubuntu 16.04 os on virtual box. This is full error stack trace:
Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project mahout-math-scala_2.10: There are test failures -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project mahout-math-scala_2.10: There are test failures
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures
at org.scalatest.tools.maven.TestMojo.execute(TestMojo.java:107)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :mahout-math-scala_2.10

Sonarqube scan from Jenkins : ClassNotFoundException: org.eclipse.jgit.api.Git

I have a problem when doing a SonarQube Scan from Jenkins. See the stacktrace in debug mode :
[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar (default-cli) on project scbm-order-parent: Unable to register extension org.sonarsource.scm.git.JGitBlameCommand from plugin 'scmgit': org/eclipse/jgit/api/Git: org.eclipse.jgit.api.Git -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar (default-cli) on project scbm-order-parent: Unable to register extension org.sonarsource.scm.git.JGitBlameCommand from plugin 'scmgit'
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:120)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:347)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:154)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:582)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Unable to register extension org.sonarsource.scm.git.JGitBlameCommand from plugin 'scmgit'
at org.sonarsource.scanner.maven.bootstrap.ExceptionHandling.handle(ExceptionHandling.java:36)
at org.sonarsource.scanner.maven.bootstrap.ScannerBootstrapper.execute(ScannerBootstrapper.java:81)
at org.sonarsource.scanner.maven.SonarQubeMojo.execute(SonarQubeMojo.java:122)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 19 more
Caused by: java.lang.IllegalStateException: Unable to register extension org.sonarsource.scm.git.JGitBlameCommand from plugin 'scmgit'
at org.sonar.core.platform.ComponentContainer.addExtension(ComponentContainer.java:249)
at org.sonar.scanner.bootstrap.ExtensionInstaller.doInstall(ExtensionInstaller.java:76)
at org.sonar.scanner.bootstrap.ExtensionInstaller.install(ExtensionInstaller.java:57)
at org.sonar.scanner.scan.ProjectScanContainer.addBatchExtensions(ProjectScanContainer.java:227)
at org.sonar.scanner.scan.ProjectScanContainer.doBeforeStart(ProjectScanContainer.java:115)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:141)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:128)
at org.sonar.scanner.task.ScanTask.execute(ScanTask.java:47)
at org.sonar.scanner.task.TaskContainer.doAfterStart(TaskContainer.java:86)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:143)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:128)
at org.sonar.scanner.bootstrap.GlobalContainer.executeTask(GlobalContainer.java:118)
at org.sonar.batch.bootstrapper.Batch.executeTask(Batch.java:117)
at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
at com.sun.proxy.$Proxy31.execute(Unknown Source)
at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:233)
at org.sonarsource.scanner.api.EmbeddedScanner.runAnalysis(EmbeddedScanner.java:151)
at org.sonarsource.scanner.maven.bootstrap.ScannerBootstrapper.execute(ScannerBootstrapper.java:78)
... 22 more
Caused by: java.lang.NoClassDefFoundError: org/eclipse/jgit/api/Git
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.getDeclaredMethods(Class.java:1975)
at org.picocontainer.injectors.AdaptingInjection$1.run(AdaptingInjection.java:203)
at java.security.AccessController.doPrivileged(Native Method)
at org.picocontainer.injectors.AdaptingInjection.injectionMethodAnnotated(AdaptingInjection.java:200)
at org.picocontainer.injectors.AdaptingInjection.methodAnnotatedInjectionAdapter(AdaptingInjection.java:171)
at org.picocontainer.injectors.AdaptingInjection.createComponentAdapter(AdaptingInjection.java:70)
at org.picocontainer.behaviors.AbstractBehaviorFactory.createComponentAdapter(AbstractBehaviorFactory.java:44)
at org.picocontainer.behaviors.OptInCaching.createComponentAdapter(OptInCaching.java:45)
at org.picocontainer.DefaultPicoContainer.addComponent(DefaultPicoContainer.java:536)
at org.picocontainer.DefaultPicoContainer.access$300(DefaultPicoContainer.java:84)
at org.picocontainer.DefaultPicoContainer$AsPropertiesPicoContainer.addComponent(DefaultPicoContainer.java:1149)
at org.sonar.core.platform.ComponentContainer.addExtension(ComponentContainer.java:247)
... 44 more
Caused by: java.lang.ClassNotFoundException: org.eclipse.jgit.api.Git
at org.sonar.classloader.ParentFirstStrategy.loadClass(ParentFirstStrategy.java:39)
at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:87)
at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:76)
... 58 more
Used versions :
Jenkins 1.624 (ok... old one but cannot upgrade it now)
sources scan with Java 8
Sonarqube 6.3 (Git plugin version 1.2, java plugin version 4.6.0.8784)
Extract from the pom.xml :
<pluginManagement>
<plugin>
<groupId>org.sonarsource.scanner.maven</groupId>
<artifactId>sonar-maven-plugin</artifactId>
<version>3.2</version>
</plugin>
</pluginManagement>
mvn command line :
clean install -U -X org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar -Dsonar.host.url=http://x.x.x.x:9000 -Dsonar.working.directory=/tmp/.sonar/ -Dsonar.login=xxxxxxxxxxxxxx -DskipTests=true site
Any idea ?
I have also tried to add the JGit artifact in the plugin management part in my pom.xml, without success... :(
Ok, it works now. Just disable Git SCM plugin. Don't know why...
Then disable Sonar XML Plugin because of the next error :
Caused by: java.lang.ClassNotFoundException: org.sonar.squidbridge.api.AnalysisException
If anyone know why, please answer :)
I have also seen that some guys have updated JGit dependency version in Sonar Git plugin : https://groups.google.com/forum/#!topic/sonarqube/TcRrB9XBk8o.
Not tested...
We ran into a similar issue with Sonarqube v5.6.7. The root cause appears to have been that the local cache of the plugin library that gets downloaded by the sonar scanner was corrupted.
Content found under "/logs/ssc/sonar/cache/cache":
./675ef60911b1d93e17e84ab20261e18a
./675ef60911b1d93e17e84ab20261e18a/sonar-xml-plugin-1.5-SNAPSHOT.jar_unzip
./675ef60911b1d93e17e84ab20261e18a/sonar-xml-plugin-1.5-SNAPSHOT.jar_unzip/META-INF
./675ef60911b1d93e17e84ab20261e18a/sonar-xml-plugin-1.5-SNAPSHOT.jar_unzip/META-INF/lib
Note that the "lib" dir was empty. Deleting the entire content of the "/logs/ssc/sonar/cache/cache" cache directory and running another project scan caused it to re-download and the lib dir was now populated with the list of dependency libraries for that plugin.
To find where the local cache of plugin jars lived I ran "strace -f -e trace=open,close,read,write -o strace.log " and then searched "strace.log" for a plugin name.

maven builds fails inside ubuntu vagrant machine as well as docker instance

I am developing an api using jaxb. The maven build (using jdk-7) works on my local windows machine. Now if I try to build it on my vagrant box (ubuntu/trusty64) after setting up maven and openjdk7 on the vm, I get following error on performing mvn clean install:
java.io.FileNotFoundException (operation not permitted)
There are in all 5 projects as part of my pom.xml, the first 3 (business and projects) gets built successfully, only the 4th one (which generates the war) file fails with this error when it is trying to generate the war file. Maven was successful in copying other files into the target folder for other projects.
Also, I am able to do perform maven builds on the same machine without errors on simple java projects (from https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html)
My guess is the Vagrant Virtualbox VM is not is not able to allocate required resources to JVM but i am clueless on how i could do it. I did try to increase the memory footprint by setting the maven properties on my vm
export MAVEN_OPTS="-Xms1024m -Xmx4096m -XX:PermSize=1024m"
Detailed error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.4:war (default-war) on project '<app>' : Could not copy webapp classes [/usr/src/<app>/target/classes]: /usr/src/<app>/target/<app>-03.00.00.01-SNAPSHOT/WEB-INF/classes/<my-app-src>/rest/config/ResourceConfig.class (Operation not permitted) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.4:war (default-war) on project <my-app>: Could not copy webapp classes [/vagrant_data/<my-app-src>/<my-app>/target/classes]
[/vagrant_data/<my-app-src>/<my-app>/target/classes]
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Could not copy webapp classes [/vagrant_data/<my-apps-src>/target/classes]
at org.apache.maven.plugin.war.packaging.ClassesPackagingTask.performPackaging(ClassesPackagingTask.java:80)
at org.apache.maven.plugin.war.packaging.WarProjectPackagingTask.handleClassesDirectory(WarProjectPackagingTask.java:207)
at org.apache.maven.plugin.war.packaging.WarProjectPackagingTask.performPackaging(WarProjectPackagingTask.java:104)
at org.apache.maven.plugin.war.AbstractWarMojo.buildWebapp(AbstractWarMojo.java:505)
at org.apache.maven.plugin.war.AbstractWarMojo.buildExplodedWebapp(AbstractWarMojo.java:433)
at org.apache.maven.plugin.war.WarMojo.performPackaging(WarMojo.java:213)
at org.apache.maven.plugin.war.WarMojo.execute(WarMojo.java:175)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
... 19 more
Caused by: java.io.FileNotFoundException: /vagrant_data/<my-apps-src>/target/<my-app>-03.00.00.01-SNAPSHOT/WEB-INF/classes/<my-app-src>/rest/config/ResourceConfig.class (Operation not permitted)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
at org.codehaus.plexus.util.FileUtils.doCopyFile(FileUtils.java:1068)
at org.codehaus.plexus.util.FileUtils.copyFile(FileUtils.java:1049)
at org.apache.maven.plugin.war.packaging.AbstractWarPackagingTask.copyFile(AbstractWarPackagingTask.java:334)
at org.apache.maven.plugin.war.packaging.AbstractWarPackagingTask$1.registered(AbstractWarPackagingTask.java:154)
at org.apache.maven.plugin.war.util.WebappStructure.registerFile(WebappStructure.java:207)
at org.apache.maven.plugin.war.packaging.AbstractWarPackagingTask.copyFile(AbstractWarPackagingTask.java:149)
at org.apache.maven.plugin.war.packaging.AbstractWarPackagingTask.copyFiles(AbstractWarPackagingTask.java:104)
at org.apache.maven.plugin.war.packaging.ClassesPackagingTask.performPackaging(ClassesPackagingTask.java:75)
... 27 more
it seems that you don't have permision to write in that folder.
you could try using chmod 777 . -R in the vagrant shared folder. Also before doing this you could check if you can create a new file in the target directory by "touch target/newFile".

Hadoop 2.6.0 Maven Compilation Error on hadoop-yarn-server-nodemanager

I am running into a particular error I haven't seen around on the internet
when compiling hadoop from source.
It seems to be a problem with some missing java packages the error occurs
when trying to compile hadoop-yarn-server-nodemanager
Looks like some java packages are missing or were not gotten by maven.
The build log of everything after the error ocurred is on this gist: https://gist.github.com/grplyler/46025ba50664260fee75
Here is the command I used to compile and package hadoop on the
root of its source:
mvn package -Pdist,native -DskipTest -Dtar -X
My system and environment details:
OS: Linux Mint 17 (Ubuntu 14.10)
Java: java version "1.7.0_75"
Protobuf: 2.5.0
Maven: 3.0.5
Hadoop: 2.6.0
Also installed so I can build the native code (because i want the fuse-dfs part) are:
cmake
zlib
zlib1g-dev
libcurl4-openssl-dev
A similar problem is detailed in this question But it was cause by windows maximum path lengths which is not my case here.
I have been working on this for 2 days now, any feedback or advice as to a solution to the problem would be much appreciated.
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-yarn-server-nodemanager: Compilation failure
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.CompilationFailureException: Compilation failure
at org.apache.maven.plugin.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:729)
at org.apache.maven.plugin.TestCompilerMojo.execute(TestCompilerMojo.java:161)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
... 19 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
I would try and update to the newest version of maven(which is no longer picked up by apt-get)
sudo apt-get remove maven
sudo apt-get install gdebi
wget http://ppa.launchpad.net/natecarlson/maven3/ubuntu/pool/main/m/maven3/maven3_3.2.1-0~ppa1_all.deb
sudo gdebi maven3_3.2.1-0~ppa1_all.deb
sudo ln -s /usr/share/maven3/bin/mvn /usr/bin/maven

org.apache.maven.plugin.MojoExecutionException: protoc failure

I meet some problems when i use maven to complie hadoop from source code.Here is the error, can anyone help me? Thanks.
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.0-SNAPSHOT:protoc (compile-protoc) on project hadoop-c
[Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.0-SNAP
plugin.MojoExecutionException: protoc failure
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: org.apache.maven.plugin.MojoExecutionException: org.apache.maven.plugin.MojoExecutionException: protoc failure
at org.apache.hadoop.maven.plugin.protoc.ProtocMojo.execute(ProtocMojo.java:81)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
... 19 more
**Caused by: org.apache.maven.plugin.MojoExecutionException: protoc failure**
at org.apache.hadoop.maven.plugin.protoc.ProtocMojo.execute(ProtocMojo.java:78)
... 21 more
[ERROR]
[ERROR]
The instructions at http://wiki.apache.org/hadoop/HowToContribute explain some of the less obvious aspects of building hadoop - including this one. The main problem here is that Protocol Buffers are required by hadoop's YARN framework, and you most likely do not have them installed.
It seems like the problem might be that your not interpretting the error correctly, so --
How to Isolate the Build error
Any mojo which fails should be associated with a class.
In this case, the mojo that is failing is here :
https://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/protoc/ProtocMojo.java
Looking at the code, its clear that this Mojo requires the "protoc" program to run.
http://code.google.com/p/protobuf/
Solution
To build hadoop, you need protocol buffers. This can be installed on a *NIX machine from source, by running the following commands to grab the code from googlecode (taken from http://numbers.brighterplanet.com/2012/04/14/how-to-install-mosh-on-amazon-ec2/).
wget http://protobuf.googlecode.com/files/protobuf-2.4.1.tar.gz
tar xzf protobuf-2.4.1.tar.gz
cd protobuf-2.4.1
./configure
make
sudo make install
sudo ldconfig

Categories

Resources