I am passing parameters to hadoop jar as below:
hadoop jar C:\Pointerfile.jar main.com.asos.recommendations.Pointerfile -D Signal=Signaltest -D ProductData=ProductData -D CustomerSegment=CustomerSegment -D CustomerCategory=CustomerCategory -D P2PSimilarity=P2PSimilarity -D ProductCategory=ProductCategory/test -D ProductSegment=ProductSegment/test -D Customer=Customer -D ProductDataCN=ProductDatab -D CustomerSegmentCN=CustomerSegmentb -D CustomerCategoryCN=CustomerCategoryb -D P2PSimilarityCN=P2PSimilarityb -D ProductCategoryCN=ProductCategoryb -D ProductSegmentCN=ProductSegmentb -D SignalCN=Signalb -D CustomerCN=Customerb -D SAN=SAN -D version=v0.1
In java code I am trying to access those parameters:
Configuration conf = new Configuration();
System.out.println("parameters are" + conf.get("Signal"));
but it returns null.
I have also tried by removing space:
-DSignal=Signal and "-DSignal=Signal"
Use this kind of class :
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;
public class MyClass extends Configured implements Tool {
#Override
public int run(String[] args) throws Exception {
Configuration conf = getConf();
System.out.println("parameters are" + conf.get("Signal"));
// Some code ...
return 0;
}
public static void main(String[] args) throws Exception {
int exitCode = ToolRunner.run(new MyClass(), args);
System.exit(exitCode);
}
}
In this class, getConf() will be filled with the parameters.
For call, don't use spaces nor quotes :
hadoop jar C:\Pointerfile.jar main.com.asos.recommendations.Pointerfile -DSignal=Signaltest -DProductData=ProductData -DCustomerSegment=CustomerSegment -DCustomerCategory=CustomerCategory -DP2PSimilarity=P2PSimilarity -DProductCategory=ProductCategory/test -DProductSegment=ProductSegment/test -DCustomer=Customer -DProductDataCN=ProductDatab -DCustomerSegmentCN=CustomerSegmentb -DCustomerCategoryCN=CustomerCategoryb -DP2PSimilarityCN=P2PSimilarityb -DProductCategoryCN=ProductCategoryb -DProductSegmentCN=ProductSegmentb -DSignalCN=Signalb -DCustomerCN=Customerb -DSAN=SAN -Dversion=v0.1
Related
The following command works well in command line
shp2pgsql -s 4326 /Users/abc.shp | psql -U user1 -h localhost -p 5432 -d postgis
However when I run the following command in Java using ProcessBuilder it says command not found. Here is the code:
public static void main(String arg[]) throws Exception {
ProcessBuilder pb =
new ProcessBuilder("/bin/sh -c shp2pgsql /Users/abc.shp | psql -U user1 -h localhost -p 5432 -d postgis");
Process p = pb.start();
showOutput(p.getInputStream(), System.out);
showOutput(p.getErrorStream(), System.err);
}
private static void showOutput(final InputStream src, final PrintStream dest) {
new Thread(new Runnable() {
public void run() {
Scanner sc = new Scanner(src);
while (sc.hasNextLine()) {
dest.println(sc.nextLine());
}
}
}).start();
}
It seems that Java does not understand where the path of your environment (psql or shp2pgsql ..) is
You need to specify the path so that it can execute. It is usually in /usr/local/bin or usr/bin. Also, note the argument for "/bin/sh and "-c" (this you specify the command your going to execute is in string format)is separate. Just modify the following snippet. It should work!!
String env = "/usr/local/bin/";
ProcessBuilder pb =
new ProcessBuilder("/bin/sh", "-c", env +"shp2pgsql /Users/abc.shp | "+env+"psql -U user1 -h localhost -p 5432 -d postgis");
Process p = pb.start();
I run a command from terminal and it work, the command is:
docker exec mysql-container /bin/sh -c "./build/update-time.sh 1559652300000"
I want to be able to do this in JAVA, so i have tried the following but failed. Any ideas how to make it work?
public class DockerUtils {
public static DockerComposeRule docker;
public static DockerComposeExecOption option = DockerComposeExecOption.noOptions();
public static void updateTime() {
//also tried "bin/sh", "-c"
DockerComposeExecArgument args = DockerComposeExecArgument.arguments("bash", "-c", "./build/update-time.sh 1559652300000");
//this fails
docker.exec(option, "mysql-container", args);
}
}
Throws java.lang.NullPointerException
$> docker ps
(I had to cut the output below, but there is a container with the name!)
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ce1e1d33dcef xx xx xx xx xx mysql-container
I failed specifying the class path. Here's my setup:
File: "root/src/hello/German.java"
package hello;
public class German {
public void greet() { System.out.println("Hallo"); }
}
I compile this in "root":
> javac root/src/hello/German.java -d root/package
where "root/package/hello" exists as an empty directory. Fine. Now I want to test and write
File: "root/test/testHello.java"
import hello.German;
public class helloTest {
public static void main(String[] args) {
German guy = new German();
guy.greet();
}
}
I compile
> javac testHello.java -cp ../package
To summarize, I have:
root/package/hello/German.class
root/test/helloTest.class
I execute in "root/test/":
> java testHello => class not found except.
> java testHello -cp ../package => class not found except.
> java testHello -cp ../package/hello => class not found except.
However, copying the 'hello' directory into test such that there is
root/test/hello/German.class
root/test/helloTest.class
I can execute in "root/test/"
> java testHello
and it greets friendly in German. I want to specify the classpath, though. But, I have no idea why '-cp' and '-classpath' is not accepted.
Try this:
java -classpath .:../package testHello
.:../package to use the current directory and ../package as classpath.
I'm trying to run a Java program from another Java application. Here is my code:
public class Main {
public static int Exec() throws IOException {
Process p = Runtime.getRuntime().exec("javac -d C:/Users/Dinara/Desktop/D/bin "
+ "C:/Users/Dinara/Desktop/D/src/test.java");
Process p1 = Runtime.getRuntime().exec("java -classpath C:/Users/Dinara/Desktop/D/bin test");
return 0;
}
public static void main(String[] args) throws IOException {
Exec();
}
}
javac works fine and creates test.class file in bin directory. However java -classpath C:/Users/Dinara/Desktop/D/bin test does not run the test.class file.
the content of the test.java:
import java.io.*;
class test {
public static void main(String args[]) {
try {
FileWriter fstream = new FileWriter("out.txt");
BufferedWriter out = new BufferedWriter(fstream);
out.write("Hello Java");
out.close();
} catch (Exception e) {
System.err.println("Error: " + e.getMessage());
}
}
}
I suppose that something wrong with recognizing Java command. Could you please give me a sample code for fixing this problem or share idea? I'm using Netbeans to run Main class and the location of the application folder is C:\Users\Dinara\Main
Use
System.getProperty("java.home") + "/bin/java -classpath C:/Users/Dinara/Desktop/D/bin test"
instead of
"java -classpath C:/Users/Dinara/Desktop/D/bin test"
You need to supply the full path to the javac, exec won't use the ath to find it for you
I need to do some housekeeping.I accidentally setup my classpath same as my codebase and all classes are placed along with my code.I need to write a quick java program to select all files of the type .class and .class alone and delete it immediately.Has anyone done something related to this?
Why don't you use the shell to do that, something like:
Linux:
find . -name *.class -print -exec rm {} \;
Windows:
for /r %f in (*.class) do del %f
find . -name "*.class" -exec rm '{}' \;
This might work. Untested. Those find/for commands by the others look promising, too, but just in case you are on an OS/390 mainframe here is the Java. ;-)
import java.io.File;
import java.io.IOException;
public class RemoveClass {
public static void main(String[] args) throws Exception {
File f = new File(".");
deleteRecursive(f);
}
public static void deleteRecursive(File f) throws IOException {
if (f.isDirectory()) {
for (File file : f.listFiles()) {
deleteRecursive(file);
}
} else if (f.isFile() && f.getName().endsWith(".class")) {
String path = f.getCanonicalPath();
// f.delete();
System.out.println("Uncomment line above to delete: [" + path + "]");
}
}
}