Develop Reference

Intellij + Bazel Error when compiling Java headers

I am learning Spring Boot and am working through this example using Intellij.
Problem:
Intellij's Bazel button --> errors (see example below)
This is the small green icon top right "Sync Project with BUILD Files"
#CrossOrigin annotation causes the error during sandbox compilation or something.
Running command manually bazel build //src/main/java/hello:app --> builds just fine
My understanding of the problem so far:
Intellij's bazel build command includes a flag, --output_groups=intellij-resolve-java-direct-deps, that checks implicit dependencies (see full command + error below).
Normal bazel build //src/main/java/hello:app obviously doesn't include this flag and builds + runs fine.
Questions:
Does anyone know what exactly this flag does?
The error is confusing, stating that it's looking for "another annotation parameter" but I don't believe this to be accurate. If not, what exactly is the parameter I'm supposed to supply? ...and what is <clinit>?
Error Command + Output
/usr/bin/bazel build --tool_tag=ijwb:IDEA:community --keep_going --build_event_binary_file=/tmp/intellij-bep-ef1a54af-ac18-480c-b813-050e261d0a4f --nobuild_event_binary_file_path_conversion --curses=no --color=yes --progress_in_terminal_title=no --noexperimental_run_validations --aspects=#intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect --override_repository=intellij_aspect=/home/jordan/.local/share/JetBrains/IdeaIC2021.2/ijwb/aspect --output_groups=intellij-resolve-java-direct-deps -- //src/main/java/hello:app
Loading:
Loading: 0 packages loaded
Analyzing: target //src/main/java/hello:app (0 packages loaded, 0 targets configured)
INFO: Analyzed target //src/main/java/hello:app (5 packages loaded, 184 targets configured).
INFO: Found 1 target...
[0 / 39] [Prepa] BazelWorkspaceStatusAction stable-status.txt
ERROR: /home/jordan/Documents/swe/rules_jvm_external/examples/spring_boot/src/main/java/hello/BUILD:1:13: Compiling Java headers src/main/java/hello/liblib-hjar.jar (2 source files) failed: (Exit 1): java failed: error executing command external/remotejdk11_linux/bin/java -Xverify:none -XX:+UseParallelOldGC -XX:-CompactStrings '--patch-module=java.compiler=external/remote_java_tools_linux/java_tools/java_compiler.jar' ... (remaining 42 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
src/main/java/hello/HelloController.java:7: error: missing required annotation argument: <clinit>
#CrossOrigin
^
src/main/java/hello/HelloController.java:11: error: missing required annotation argument: <clinit>
#CrossOrigin
^
Aspect #intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect of //src/main/java/hello:app failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.460s, Critical Path: 0.33s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully
INFO: Build Event Protocol files produced successfully.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
Successful Command + Output
(removed flag --output_groups=intellij-resolve-java-direct-deps)
/usr/bin/bazel build --tool_tag=ijwb:IDEA:community --keep_going --build_event_binary_file=/tmp/intellij-bep-ef1a54af-ac18-480c-b813-050e261d0a4f --nobuild_event_binary_file_path_conversion --curses=no --color=yes --progress_in_terminal_title=no --noexperimental_run_validations --aspects=#intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect --override_repository=intellij_aspect=/home/jordan/.local/share/JetBrains/IdeaIC2021.2/ijwb/aspect -- //src/main/java/hello:app
Loading:
Loading: 0 packages loaded
Analyzing: target //src/main/java/hello:app (0 packages loaded, 0 targets configured)
INFO: Analyzed target //src/main/java/hello:app (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
[0 / 1] [Prepa] BazelWorkspaceStatusAction stable-status.txt
Aspect #intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect of //src/main/java/hello:app up-to-date (nothing to build)
INFO: Elapsed time: 0.034s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
INFO: Build Event Protocol files produced successfully.
INFO: Build completed successfully, 1 total action

Bazel is still working to support Java 11 and there are Java annotation processors which appear to be hardcoded to JDK 8.
See this GitHub issue which refers to a
Temporary fix
--java_header_compilation=false
Note that this won't solve the issue when using Intellij's bazel button. This will only fix running the command manually (with the --java_header_compilation=false flag).
Long-term fix in the upcoming release (6.0.0)
I installed the pre-release 6.0.0 Bazel from the source code using these directions, which includes the implemented fix. This solved the problem. Now I can use two different releases of bazel - 6.0.0 by calling bazel_dev or the latest main release (4.2.1) by simply calling bazel.
Point to /some/path/to/bazel_dev binary in Intellij's Bazel settings.

"Class Not Found: JDBC driver org.apache.derby.jdbc.EmbeddedDriver could not be loaded" despite having included required JARs in classpath

While following this tutorial, in order to test if everything is set up correctly, I ran the 'setup' Ant target from Eclipse. The following is the error message I got:
Buildfile: C:\Users\bingis_khan\eclipse-workspace\JDBCTutorial\build.xml
setup:
drop-tables:
[echo] C:\Users\bingis_khan\eclipse-workspace\JDBCTutorial\classes;C:\Users\bingis_khan\Downloads\db-derby-10.15.2.0-bin\db-derby-10.15.2.0-bin\lib\derby.jar;C:\Users\bingis_khan\eclipse-workspace\JDBCTutorial\lib\JDBCTutorial.jar;C:\apache\xalan-j_2_7_2\xalan.jar
BUILD FAILED
C:\Users\bingis_khan\eclipse-workspace\JDBCTutorial\build.xml:141: The following error occurred while executing this line:
C:\Users\bingis_khan\eclipse-workspace\JDBCTutorial\build.xml:84: Class Not Found: JDBC driver org.apache.derby.jdbc.EmbeddedDriver could not be loaded
Total time: 374 milliseconds
This [echo] here is set up to print the CLASSPATH. As the next section shows, the path(s) in classpath match the actual locations of these archives:
Directory of C:\Users\bingis_khan\Downloads\db-derby-10.15.2.0-bin\db-derby-10.15.2.0-bin\lib
04/20/2021 08:57 PM <DIR> .
04/20/2021 08:57 PM <DIR> ..
04/20/2021 08:57 PM 3,683,035 derby.jar
04/20/2021 08:57 PM 1,490 derby.war
04/20/2021 08:57 PM 601,103 derbyclient.jar
04/20/2021 08:57 PM 83,942 derbyLocale_cs.jar
04/20/2021 08:57 PM 99,882 derbyLocale_de_DE.jar
*snip*
What I've tried:
Experimented with changing backslashes to slashes and vice-versa.
Tried changing derby.jar to derbyclient.jar.
Just in case imported the archive (outside of Ant) to the Eclipse project.
None of the above methods worked (or even changed the output of "setup").
Where I've looked:
here - Poster did not add derby.jar to classpath.
also here - Different circumstances, but still applied the solution - nothing.
Some others, but most referred to runtime ClassNotFoundException. I haven't even gotten that far...
EDIT:
Also attaching the relevant Ant target:
<target name="drop-tables">
<echo>${toString:CLASSPATH}</echo>
<sql driver="${DB.DRIVER}" url="${DB.URL}" userid="${DB.USER}"
password="${DB.PASSWORD}" classpathref="CLASSPATH"
delimiter="${DB.DELIMITER}" autocommit="false" onerror="continue">
<transaction src="./sql/${DB.VENDOR}/drop-tables.sql"></transaction>
</sql>
</target>

verry important
Since you're using Derby 10.15, you have to also have derbytools.jar in your classpath for the embedded driver.
db-derby-10.15.2.0-bin\lib\derbytools.jar\org\apache\derby\jdbc\EmbeddedDriver.class
Use 7zip to analyse!

Since you're using Derby 10.15, you have to also have derbyshared.jar in your classpath for the embedded driver.
More information here: http://db.apache.org/derby/releases/release-10.15.1.3.cgi#Release%20Notes%20for%20Apache%20Derby%2010.15.1.3
In particular, note this part:
A new jar file (derbyshared.jar) has been added. All Derby configurations require it. In addition, the derbytools.jar library is now required when running the network server, when using Derby DataSources, and when directly referencing the JDBC drivers.

How to correctly configure the property "sonar.java.binaries"?

We are using SonarQube 5.1.2 using Ant runner 2.2 and Java pluging 3.12 for the analysis. I can succesfully analyse my project. I just keep getting this error:
Java bytecode has not been made available to the analyzer. The org.sonar.java.bytecode.visitor.DependenciesVisitor#d678716, org.sonar.java.checks.unused.UnusedPrivateMethodCheck#58e28efd, CycleBetweenPackages rule are disabled.
So I need to configure my sonar.java.binaries and sonar.java.test.binaries properties (following http://docs.sonarqube.org/display/PLUG/Java+Plugin+and+Bytecode).
Which I think I have done correctly:
<property name="project.dir" value="${basedir}/xalg.prj/h3_service_fo" />
<property name="sonar.java.binaries" location="${project.build.dir}/classes/main" />
<property name="sonar.java.test.binaries" value="${project.build.dir}/classes/test" />
Which resolve to the following valid directories for the above properties:
basedir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj
project.dir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo
sonar.java.binaries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\xalg.prj\\h3_service_fo\\build\\classes\\main
sonar.java.test.binaries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/build/classes/test
But I keep getting:
Java bytecode has not been made available to the analyzer. The org.sonar.java.bytecode.visitor.DependenciesVisitor#d678716, org.sonar.java.checks.unused.UnusedPrivateMethodCheck#58e28efd, CycleBetweenPackages rule are disabled.
And for the life of me, I can not figure out what values I need to give the sonar.java.binaries and sonar.java.test.binaries properties. I even tried using sonar.binaries, which gave me the following output:
Binary dirs: xalg.prj/h3_service_fo/build/classes
Which I did not get using either sonar.java.binaries or sonar.java.test.binaries. I also got:
JavaClasspath initialization...
sonar.binaries and sonar.libraries are deprecated since version 2.5 of sonar-java-plugin, please use sonar.java.binaries and sonar.java.libraries instead
Which is to be expected for a deprecated property. But using the sonar.java.binaries property I did not get the "Binary dirs" line in my log.
Using sonar.java.binaries:
Language is forced to java
Load rules
Load rules (done) | time=761ms
Code colorizer, supported languages: cs,plsql
Initializers :
Base dir: D:\appl\BuildAgent\work\H3\src.prj\java.prj
Working dir: D:\appl\BuildAgent\work\H3\src.prj\java.prj\.sonar
Source paths: xalg.prj/h3_service_fo/src/main/java
Test paths: xalg.prj/h3_service_fo/src/test/java
Source encoding: windows-1252, default locale: en_US
Index files
Versus using sonar.binaries:
Language is forced to java
Load rules
Load rules (done) | time=736ms
Code colorizer, supported languages: cs,plsql
Initializers :
Base dir: D:\appl\BuildAgent\work\H3\src.prj\java.prj
Working dir: D:\appl\BuildAgent\work\H3\src.prj\java.prj\.sonar
Source paths: xalg.prj/h3_service_fo/src/main/java
Test paths: xalg.prj/h3_service_fo/src/test/java
Binary dirs: xalg.prj/h3_service_fo/build/classes
Source encoding: windows-1252, default locale: en_US
Index files
I also looked into the source code of SonarQube, SonarQube Java Plugin and the SonarQube Scanner for instances of either "Java bytecode has not been made available to the analyzer." or sonar.java.binaries. I found plenty on sonar.java.binaries, but nothing on "Java bytecode has not been made available to the analyzer." So I have no clue what conditions exactly trigger that error.
I also tried the following permutations on sonar.java.binaries:
<property name="sonar.java.binaries" location="${project.build.dir}/classes" />
<property name="sonar.java.binaries" location="${project.build.dir}/classes/main/nl" />
But that did nothing either.
What is weird is that Squid seems to resolve the classpath just fine:
----- Classpath analyzed by Squid:
D:\appl\BuildAgent\work\H3\src.prj\java.prj\xalg.prj\h3_service_fo\build\classes\main
So, what am I missing? What am I doing wrong? Thanks in advance.
Update 2016-09-08:
Removed the entire log, the post become to long.
A subset with the (I think) relevant paths:
project.build.dir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/build
project.dir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo
project.src.dir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/src
sonar.dir=D\:/appl/sonarqube-5.1.2
sonar.working.directory=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\.sonar
sonar.projectBaseDir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj
sonar.jacoco.reportPath=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/build/jacoco/test.exec
sonar.junit.reportsPath=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/build/test-results
sonar.sources=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/src/main/java
sonar.java.binaries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\xalg.prj\\h3_service_fo\\build\\classes\\main
sonar.java.libraries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_deploy/*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_repos/lib/*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_repos/provided/*.jar
sonar.tests=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/src/test/java
sonar.java.test.binaries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/xalg.prj/h3_service_fo/build/classes/test
sonar.java.test.libraries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_deploy/*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_repos/lib/*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj/_repos/provided/*.jar
The paths have exactly the same format as in my post. Could it be that the Sonar Ant runner can't figure out a path with both backslashes and slashes?
Update 2016-09-16:
Removed the entire log, the post become to long.
A subset with the (I think) relevant paths:
project.build.dir=xalg.prj\\\\h3_service_fo\\\\build
project.dir=xalg.prj\\\\h3_service_fo
project.src.dir=xalg.prj\\\\h3_service_fo\\\\src
sonar.dir=D\:\\\\appl\\\\sonarqube-5.1.2
sonar.working.directory=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\.sonar
sonar.projectBaseDir=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj
sonar.jacoco.reportPath=xalg.prj\\\\h3_service_fo\\\\build\\\\jacoco/test.exec
sonar.junit.reportsPath=xalg.prj\\\\h3_service_fo\\\\build\\\\test-results
sonar.sources=xalg.prj\\\\h3_service_fo\\\\src\\\\main\\\\java
sonar.java.binaries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\xalg.prj\\h3_service_fo\\build\\classes\\main
sonar.java.libraries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_deploy\\\\*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_repos\\\\lib\\\\*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_repos\\\\provided\\\\*.jar
sonar.tests=xalg.prj\\\\h3_service_fo\\\\src\\\\test\\\\java
sonar.java.test.binaries=xalg.prj\\\\h3_service_fo\\\\build\\\\classes\\\\test
sonar.java.test.libraries=D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_deploy\\\\*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_repos\\\\lib\\\\*.jar,D\:\\appl\\BuildAgent\\work\\H3\\src.prj\\java.prj\\\\_repos\\\\provided\\\\*.jar
Some paths have become relative, but I think that is because TeamCity changed the Ant file to the file in SVN.
The sonar.java.binaries is absolute and it definitely points to the correct directory.
But I still get this error:
09:17:52.299 INFO - Java Main Files AST scan done: 1579 ms
09:17:52.301 INFO - 2/2 source files have been analyzed
09:17:52.305 WARN - Java bytecode has not been made available to the analyzer. The org.sonar.java.bytecode.visitor.DependenciesVisitor#757a48f9, org.sonar.java.checks.unused.UnusedPrivateMethodCheck#1adf492b, CycleBetweenPackages rule are disabled.
The classpath is still interpreted just fine:
[sonar:sonar] 09:17:51.971 DEBUG - ----- Classpath analyzed by Squid:
[sonar:sonar] 09:17:51.972 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\xalg.prj\h3_service_fo\build\classes\main
[sonar:sonar] 09:17:51.973 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\batch.daemon.jar
[sonar:sonar] 09:17:51.974 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\buildinfo.jar
[sonar:sonar] 09:17:51.975 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h2_shared.jar
[sonar:sonar] 09:17:51.975 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_generator.jar
[sonar:sonar] 09:17:51.976 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_loadtest.jar
[sonar:sonar] 09:17:51.977 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_common.jar
[sonar:sonar] 09:17:51.977 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xalg.jar
[sonar:sonar] 09:17:51.978 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xalg_dao.jar
[sonar:sonar] 09:17:51.979 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xalg_mappers.jar
[sonar:sonar] 09:17:51.979 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xalg_procedures.jar
[sonar:sonar] 09:17:51.980 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xcare.jar
[sonar:sonar] 09:17:51.981 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xcare_dao.jar
[sonar:sonar] 09:17:51.982 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xcare_mappers.jar
[sonar:sonar] 09:17:51.982 DEBUG - D:\appl\BuildAgent\work\H3\src.prj\java.prj\_deploy\h3_model_xcare_procedures.jar
Could the Sonar Ant runner have a problem with the escaped back slashes?

This is what I use, and it works.
sonar-scanner -Dsonar.projectKey=projectName
-Dsonar.gitlab.commit_sha=$CI_BUILD_REF
-Dsonar.gitlab.ref_name=$CI_BUILD_REF_NAME -Dsonar.sources=directory\src\
-Dsonar.java.binaries=.build\libs\
You need to point sonar.java.binaries to a directory that contains jars. I would't try to get fancy with wildcards and file extensions, that's not the same thing. You need a directory as an argument, not a regular expression for files.
I don't know everything about sonar scanner, but my configuration works.
Take my example into context, I use it in a gitlab runner pipeline. Your double backslashes should be fine, but you can always substitute them for forwards slashes in your config files.

Here are some options for a Gradle multi-project. I'm running in CI (Jenkins) but the principles apply.
Options:
Hand-maintain a hardcoded sonar.java.binaries entry, eg:
sonar.java.binaries=subproject1/build/classes,subproject2/build/classes, (etc)
Set it to the top-level directory of the project:
sonar.java.binaries=.
Use a script to generate your own props file at runtime, for example:
cat > sonar-project.properties <<EOF
sonar.projectKey=XXXX
sonar.projectVersion=YYYY
sonar.sources=$(find . -path '*/src/main' | xargs | tr ' ' ',')
sonar.java.binaries=$(find . -path '*/build/classes/java' | xargs | tr ' ' ',')
EOF
Note that the above also applies to sonar.sources, and it is likely going to need to be in sync with sonar.java.binaries. Having binaries be a superset of source seems to be OK. Obviously, if your sources value is too broad (ie includes tests, etc), you'll get more noise. The advantage of using a script is that you can customize these heuristics to your taste.

you have to provide the parameter sonar.java.libraries.
if you use maven you can add these inside the pom.xml as a property
<project>
...
<properties>
...
<sonar.java.libraries>target</sonar.java.libraries>
</properties>
</project>
or as run parameter -Dsonar.java.libraries=target

Based on the documentation at https://docs.sonarqube.org/latest/analysis/languages/java/
The purpose of sonar.java.binaries path is:
Comma-separated paths to directories containing the compiled bytecode files corresponding to your source files.
For Gradle, the default path is at build/classes, so I would set:
sonar.java.binaries=build/classes

IDK but adding this sonar.java.binaries= **/*.bin to sonar-project.properties helped me

Errors for running Mahout example

I downloaded the examples of latest version for chapter 09 of “Mahout in Action”. I can successfully run several examples, but for three files, NewsKMeansClustering.java, ReutersToSparseVectors.java, and NewsFuzzyKMeansClusteing.java. Running these three programs gives similar error messages:
Aug 3, 2011 2:03:54 PM org.apache.hadoop.metrics.jvm.JvmMetrics init
INFO: Initializing JVM Metrics with processName=JobTracker, sessionId=
Aug 3, 2011 2:03:54 PM org.apache.hadoop.mapred.JobClient configureCommandLineOptions
WARNING: Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
Aug 3, 2011 2:03:54 PM org.apache.hadoop.mapred.JobClient configureCommandLineOptions
WARNING: No job jar file set. User classes may not be found. See JobConf(Class) or
JobConf#setJar(String).
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/home/user1/workspaceMahout1/recommender/inputDir
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:224)
at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:55)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:241)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.mahout.vectorizer.DocumentProcessor.tokenizeDocuments(DocumentProcessor.java:93)
at mia.clustering.ch09.NewsKMeansClustering.main(NewsKMeansClustering.java:54)
For the above messages, I do not quite understand what do those two warnings mean? Moreover, it looks like that “input path” should have been created, how can I create this type of input? Thanks.

You can ignore the warnings. The error is that the input directory you have specified does not exist. Does it exist? What is your command line?

I ran into a similar mismatch. The MiA files at https://github.com/tdunning/MiA have some cases where a .csv file is left in the same dir as the Java source. For example https://github.com/tdunning/MiA/tree/master/src/main/java/mia/recommender/ch02 ... however via Eclipse, loading it using DataModel model = new FileDataModel(new File("intro.csv")); ...doesn't find it.
Adding
System.out.println("CWD: "+System.getProperty("user.dir"));
...will reveal where Eclipse is looking (in my case, a couple levels up the filetree, but this might vary depending on how exactly you've set things up).

What am I doing wrong when trying to run the this java test command for LWJGL?

I'm attempting to use the lwjgl library and I'm starting from scratch on a new Windows 7 install.
I downloaded the latest JDK 6 from the Oracle website. After installing it, I found that commands like "java" or "javac" weren't being recognized from a windows cmd prompt. So, I edited my path variable and appended the jdk's bin folder to it.
Now the java commands work.
So, I download the latest lwjgl, extract it and read the installation instructions on their website:
Download the distribution Unpack the
archive, file contents (in sub
folders) should include (amongst other
things):
lwjgl.dll lwjglaudio.dll lwjgl.jar
lwjgl_util.jar lwjgl_test.jar
Test
LWJGL by opening a command prompt, and
navigating to the folder where the
archive was extracted. Once navigated,
issue the following command: (all in
one line, space before each -option)
java -cp .;res;jar\lwjgl.jar;jar\lwjgl_test.jar;jar\lwjgl_util.jar;jar\lwjgl_fmod3.jar;jar\lwjgl_devil.jar;jar\jinput.jar;-Djava.library.path=native\windows org.lwjgl.test.WindowCreationTest
A window should appear and you should
see the following output:
Found display modes 240, 320,
WindowCreationTest Display created
Moving to 100, 100 Window created 600,
800, Game
So, I extracted it and navigated to the extracted folder in a cmd prompt.
I then executed the test command specified above and I get the following error:
C:\Users\Nestor\Downloads\lwjgl-2.6\lwjgl-2.6>java
-cp .;res;jar\lwjgl.jar;jar\ lwjgl_test.jar;jar\lwjgl_util.jar;jar\lwjgl_fmod3.jar;jar\lwjgl_devil.jar;jar\ji
nput.jar;-Djava.library.path=native\windows
org.lwjgl.test.WindowCreationTest
The
following keys are available: ESCAPE:
Exit test ARROW Keys: Move window
when in non-fullscreen mode L:
List selectable display modes 0-8:
Selection of display modes F:
Toggle fullscreen SHIFT-F:
Toggle fullscreen with
Display.destroy()/create() cycle
Exception in thread "main"
java.lang.UnsatisfiedLinkError: no
lwjgl in java.libr ary.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
at java.lang.Runtime.loadLibrary0(Runtime.java:823)
at java.lang.System.loadLibrary(System.java:1028)
at org.lwjgl.Sys$1.run(Sys.java:73)
at java.security.AccessController.doPrivileged(Native
Method)
at org.lwjgl.Sys.doLoadLibrary(Sys.java:66)
at org.lwjgl.Sys.loadLibrary(Sys.java:82)
at org.lwjgl.Sys.(Sys.java:99)
at org.lwjgl.opengl.Display.(Display.java:130)
at org.lwjgl.test.WindowCreationTest.initialize(WindowCreationTest.java:
82)
at org.lwjgl.test.WindowCreationTest.main(WindowCreationTest.java:286)
C:\Users\Nestor\Downloads\lwjgl-2.6\lwjgl-2.6>
Why am I getting that error? I don't understand why there should be linking errors. In the command that I attempted to execute it clearly spells out the path to those native dll's it needs:
C:\Users\Nestor\Downloads\lwjgl-2.6\lwjgl-2.6>java
-cp .;res;jar\lwjgl.jar;jar\ lwjgl_test.jar;jar\lwjgl_util.jar;jar\lwjgl_fmod3.jar;jar\lwjgl_devil.jar;jar\ji
nput.jar;-Djava.library.path=native\windows
org.lwjgl.test.WindowCreationTest
I've confirmed that the relative path "native\windows" contains those dependencies:
C:\Users\Nestor\Downloads\lwjgl-2.6\lwjgl-2.6\native\windows>dir
Volume in drive C has no label.
Volume Serial Number is 2061-75F6
Directory of C:\Users\Nestor\Downloads\lwjgl-2.6\lwjgl-2.6\native\windows
11/24/2010 12:35 AM .
11/24/2010 12:35 AM ..
10/18/2010 08:44 PM 31,232 jinput-dx8.dll
10/18/2010 08:44 PM 65,024 jinput-dx8_64.dll
10/18/2010 08:44 PM 29,696 jinput-raw.dll
10/18/2010 08:44 PM 62,464 jinput-raw_64.dll
10/18/2010 08:44 PM 197,120 lwjgl.dll
10/18/2010 08:44 PM 305,664 lwjgl64.dll
10/18/2010 08:44 PM 56,832 OpenAL32.dll
10/18/2010 08:44 PM 157,184 OpenAL64.dll
8 File(s) 905,216 bytes
2 Dir(s) 155,163,058,176 bytes free
Can anyone help point out what I'm doing wrong? Can anyone reproduce this by downloading the LWJGL library and attempting to run the test command given in the installation instructions?

It seems that you do not have a space between your classpath argument (-cp jar1.jar;jar2.jar) and your system property setting (-D..).
E.g. your classpath looks like this-cp .;res;jar\lwjgl.jar;jar\lwjgl_test.jar;jar\lwr...;-Djava.library.path=native\windows. This way java will interpret your property setting of native library path argument like a classpath!
Just add a space between those arguments and you should be up and running, this is the corrected command (also tested on Windows 7):
java -cp jar\lwjgl.jar;jar\lwjgl_test.jar;jar\lwjgl_util.jar -Djava.library.path=native\windows org.lwjgl.test.WindowCreationTest
Note that I removed the unused jars from the classpath since you only want to run the WindowCreationTest example.

Check again that directory
C:\Users\Nestor\Downloads\lwjgl-2.6\lwjgl-2.6\native\windows
exists and contains lwjgl.dll and lwjglaudio.dll
I believe that something is wrong with your installation, i.e. the directory does not exist or files are not there.

Just throwing this out there, because Ive had some issues related to this. Go to your Java/JRE/BIN folder. Right click on Java, and go to properties. Under Privilege Level, check the box by run as an administrator.