I'm developer of a website where programmers can submit bots that compete against each other in a game. I'm trying to add Scala to our list of supported languages, but I'm having trouble here.
The problem is that every bot runs from it's own jail and I can't get scala to run from the jail (Linux system btw) because I get an error saying the executable is not found. Outside of the jail, everything works perfectly. So I'm missing some stuff to put in to the jail. I have all the Java dependency libraries, the whole jvm folder and the whole scala folder.. but there's some other stuff still missing and I'm clueless what it could be.
Here are the important lines of the compile script (which works as it should I think)
#compile
scalac -sourcepath src/ -d bin/ `find src/ -name '*.scala'`
#create runscript
echo "#!/bin/sh" > bin/run_ai
echo "cd / && ./scala -Djava.security.manager -cp bin/ -J-Xss8m -J-Xmx450m '$MAIN'" >> bin/run_ai
run_ai is the script that is called when running the bot from the jail, $MAIN is the main scala file to be ran. As I said, this all works fine outside of the jail.
Here's the script that creates everything that's needed inside of the jail. Here I'm missing some important stuff obviously.
mkdir -p lib64 bin lib/x86_64-linux-gnu usr/lib/x86_64-linux-gnu scala jvm proc
cp /lib64/ld-linux-x86-64.so.2 lib64/
cp /bin/sh bin/
#These dependency libraries are actually copied from the Java mkjail script.
#Using 'ldd /usr/bin/scala' returns "not a dynamic executable" in the shell.
#For other languages, it gives a nice list of dependencies, scala not :(
cp /lib/x86_64-linux-gnu/libgcc_s.so.1 lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libdl.so.2 lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libz.so.1 lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libc.so.6 lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libpthread.so.0 lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libm.so.6 lib/x86_64-linux-gnu/
cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 lib/x86_64-linux-gnu/
mount -o bind /proc proc
echo "bound /proc"
mount -o remount,ro proc
echo "remounted /proc"
#adding jvm to jail
mount -o bind /usr/lib/jvm jvm
mount -o remount,ro jvm
#adding scala to jail
mount -o bind /usr/share/scala scala
mount -o remount,ro scala
#some extra stuff to run Java from the jail, not actually needed here I think
if [ -f jvm/java-7-openjdk-amd64/jre/bin/java ]
then
cp jvm/java-7-openjdk-amd64/jre/lib/amd64/jli/libjli.so lib/x86_64-linux-gnu/
ln -s jvm/java-7-openjdk-amd64/jre/bin/java .
else
cp jvm/java-6-openjdk-amd64/jre/lib/amd64/jli/libjli.so lib/x86_64-linux-gnu/
ln -s jvm/java-6-openjdk-amd64/jre/bin/java .
fi
#creating a soft link to run scala. trying to run the bots without the soft link gives the same error, so this is not the problem.
ln -s scala/bin/scala ./scala
So in short: running scala from the jail results in an error: "./scala not found", if I try other stuff like: /scala/bin/scalac, I get the same error. (running the Java that is added in the jail works fine). So I'm missing some stuff to run Scala.
My question is: What is exactly needed to run Scala?
The 'scala' executable references /usr/bin/env, which resides outside the jail.
This might work:
java -cp /absolute/path/to/scala-library.jar:path/to/scala/classfiles/ MyMainClass
scala and scalac are shell scripts. They might be missing the env and bash executables.
You can further debug the scripts by adding trace code (e.g. echo "trace01") to pinpoint a problem.
Related
Let's say we have the following Dockerfile for the purpose of creating a java image and compiling two scripts.
FROM openjdk:latest
COPY src JavaDocker
WORKDIR JavaDocker
RUN mkdir -p bin
RUN javac -d bin ./com/myapp/HelloWorld1.java
RUN javac -d bin ./com/myapp/HelloWorld2.java
WORKDIR bin
ENTRYPOINT java
How can I run any of these two scripts that have been compiled?
I'm using the command: docker run myapp-image "com.myapp.Server"
And I get:
Usage: java [options] <mainclass> [args...]
(to execute a class)
or java [options] -jar <jarfile> [args...]
(to execute a jar file)
or java [options] -m <module>[/<mainclass>] [args...]
java [options] --module <module>[/<mainclass>] [args...]
(to execute the main class in a module)
or java [options] <sourcefile> [args]
(to execute a single source-file program)
Arguments following the main class, source file, -jar <jarfile>,
-m or --module <module>/<mainclass> are passed as the arguments to
main class.
I'd suggest building a separate image per application; that can help clarify what the image is supposed to do. I also generally recommend using CMD over ENTRYPOINT.
So a Dockerfile that runs only the first application could look like:
FROM openjdk:latest
# Prefer an absolute path for clarity.
WORKDIR /JavaDocker
# Set up the Java class path.
RUN mkdir bin
ENV CLASSPATH=/JavaDocker/bin
# Use a relative path as the target, to avoid repeating it.
# (If you change the source code, repeating `docker build` will
# skip everything before here.)
COPY src .
# Compile the application.
RUN javac -d bin ./com/myapp/HelloWorld1.java
# Set the main container command.
CMD ["java", "com.myapp.HelloWorld1"]
What if you do have an image that contains multiple applications? If you use CMD here, it's very easy to provide an alternate command when you run the image:
docker run myapp-image \
java com.myapp.HelloWorld2
# Wait, what's actually in this image?
docker run --rm myapp-image \
ls -l bin/com/myapp
I generally recommend reserving ENTRYPOINT for a wrapper script that does some first-time setup, then runs exec "$#" to run a normal CMD. There's an alternate pattern of giving a complete command in ENTRYPOINT, and using CMD to provide its arguments. In both of these cases ENTRYPOINT needs to be JSON-array syntax, not shell syntax.
ENTRYPOINT ["java", "com.myapp.HelloWorld1"] # <-- JSON-array syntax
CMD ["-argument", "to-program-1"]
docker run myapp-image \
-argument=different -options
but it's harder to make that image do something else
docker run \
--entrypoint ls \ # <-- first word of the command is before the image name
myapp-image \
-l bin/com/myapp # <-- and the rest after
docker run \
--entrypoint java \
myapp-image \
com.myapp.HelloWorld2
Your original Dockerfile will probably work if you change the ENTRYPOINT line from shell to JSON-array syntax; using shell syntax will cause the CMD part to be ignored (including a command passed after the docker run image-name). You might find it easier to make one complete application invocation be the default and include the java command if you need to run the other.
I try to make a simple shell script to make a jar file. The jar command combined with -C does not work with wildcards. Therefor I use a wildcard to find the files I want. Write them to a file, and loop over them.
It looks something like this:
the_classes=''
cd "$bin_folder"
tmp_dir=$(mktemp -d -t java_sucks)
find "imui/core/" -type f -name "IMUI_Widget_Agent*.class" >"$tmp_dir/classes.txt"
while IFS="" read -r p || [ -n "$p" ]
do
the_classes="${the_classes} -C '$bin_folder' '$p'"
done < "$tmp_dir/classes.txt"
Using the above I complete the command:
cmd='jar cfm build/IMUI_Widget_Agent.jar'
cmd="${cmd} \"$bin_folder/imui/core/IMUI_Widget_Agent_MANIFEST.MF\" $the_classes"
printf "\n\n\ncmd\n\n\n"
echo $cmd
Now if I copy and paste this command to execute it works!
But I want to avoid the manual labour of doing the copy and paste by hand every time.
Now I have:
eval "$("$cmd")"
But I get an error File name too long. No matter what I try, every fix I do creates a new problem. I have been working 6 hours now to make this script.
What would be a good step forward?
Since you cd "$bin_folder" you don't actually need -C "$bin_folder":
#!/bin/bash
shopt -s globstar
cd "$bin_folder"
jar cfm build/IMUI_Widget_Agent.jar \
imui/core/IMUI_Widget_Agent_MANIFEST.MF \
imui/core/**/IMUI_Widget_Agent*.class
However, if you still want to add them as part of a larger script, you can easily and robustly build your command in an array:
#!/bin/bash
shopt -s globstar
cmd=(jar cfm build/IMUI_Widget_Agent.jar imui/core/IMUI_Widget_Agent_MANIFEST.MF)
cd "$bin_folder"
for file in imui/core/**/IMUI_Widget_Agent*.class
do
cmd+=(-C "$bin_folder" "$file")
done
echo "About to execute: "
printf "%q " "${cmd[#]}"
echo
"${cmd[#]}"
Alternatively, you can simply do eval "$cmd" with your code, which is equivalent to echo and copy-pasting. However, be aware that this is fragile and error prone because it requires careful escaping of the filenames which you're not currently doing.
Hy. I'm trying to compile a .java file using docker. I read the files on docker's website, also I read these links:
docker's website
about volumes
and another question I had put up for gcc compiler
I understood the concept for the gcc compiler since it doesn't create any extra file for compiling.
But the java one does. It creates a Main.class file on my /home directory if I use the following command and compile a file named Main.java
sudo docker run --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp java:7 javac Main.java
after learning from the above links I was able to successfully compile a java file with my own path using:
docker run --rm -v /mypathhere/mycode.java:/mycode.java: java:7 javac mycode.java"
if there is any error it shows an error but if there isn't it just compiles and gives me no output, and that's justified because it creates a Main.class file.
My problem is that I am unable to find that Main.class file. I don't know where docker is creating it and I have zero understanding for it. Please help me out.
The .class file will be inside the container, under the root directory.
The best plan is to mount the whole source directory and have javac put the result to the same directory e.g:
docker run --rm -v /mypathhere:/mycode java:7 sh -c "cd mycode; javac mycode.java"
That way, you should get the class file written to the mypathhere directory.
Apologies if that doesn't quite work - it's off the top of my head. Hopefully you get the idea though.
I'm trying to compile a simple WordCount.java map-reduce example on a linux (CentOS) installation of Cloudera 4. I keep hitting compiler errors when I reference any of the hadoop classes, but I can't figure out which jars of the hundreds under /usr/lib/hadoop I need to add to my classpath to get things to compile. Any help would be greatly appreciated! What I'd like most is a java file for word count (just in case the one I found is bad for some reason) along with the associated command to compile and run it.
I am trying to do this using just javac rather than Eclipse. My main issue either way is what exactly are the Hadoop libraries from the Cloudera 4 install which I need to include in order to get the classic WordCount example to compile. Basically, I need to put the Java MapReduce API classes (Mapper, Reducer, etc.) in my classpath.
I have a script that builds my hadoop classes. Try:
#!/bin/bash
program=`echo $1 | awk -F "." '{print $1}'`
if [ ! -d "${program}_classes" ]
then mkdir ${program}_classes/;
fi
javac -classpath /usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop/client/h\
adoop-mapreduce-client-core-2.0.0-cdh4.0.1.jar -d ${program}_classes/ $1
jar -cvf ${program}.jar -C ${program}_classes/ .;
You were probably missing the key jars:
/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.1.jar
and
/usr/lib/hadoop/client/hadoop-mapreduce-client-core-2.0.0-cdh4.0.1.jar
If you are running the Cloudera CDH4 Virtual Machine then the following should get you running:
javac -classpath /usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop/client/hadoop-mapreduce-client-core-2.0.0-cdh4.0.0.jar -d wordcount_classes WordCount.java
Or you can export environment:
export JAVA_HOME=/usr/java/default
export PATH=${JAVA_HOME}/bin:${PATH}
export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar
and use the commands below:
$ bin/hadoop com.sun.tools.javac.Main WordCount.java
$ jar cf wc.jar WordCount*.class
If you are using Eclipse please do add Hadoop packages. you may get it from java2s or any similar sites. I couldn't say without know anything about what you did till now.
I am trying to build RPM for my java code.RPM build and install sections works fine.But,later i have added a shell script in %post section to run shell script file that creates some files and starts the java application.The file gets copied fine.But, in %post section I get the failed dependencies bin/sh error.I am building the rpm on linux machine and target machines are centOS.I have also tried AutoReqProv: no in spec file.I am not running the install as a root user.also,target machines are not expected to have ANT(iam using it to build the java code-this might be not related to the problem) on them.I am attaching the code below.Default location of installation is in tmp folder.( for testing).
Summary: test Summary: test
Name: test
Version: 1
Release: 1
License: Restricted
Group: Applications/System
BuildRoot: ~/rpm/BUILD/helloworld-root
URL: http://mycompany.net/helloworld.tar
Vendor: Mycompany
Packager: Mycompany
Prefix: /tmp
BuildArchitectures: x86_64
%description
Hello World
%prep
pwd
%build
pwd
echo "changing directory"
echo $RPM_BUILD_ROOT
cd %{_sourcedir}
pwd
ant -f testbuild.xml
%install
pwd
echo "in install"
echo $RPM_BUILD_ROOT
rm -rf $RPM_BUILD_ROOT
mkdir -p $RPM_BUILD_ROOT/tmp/test/
cd $RPM_BUILD_ROOT/tmp/test/
tar -xf %{_builddir}/test.tar
%post
sh /tmp/test/createdb.sh
%clean
%files
/tmp/test
%changelog
i have also searched on google but couldn't find how to run shell script from spec.with oust post section, I have tried to run through command line it executes fine.I appreciate the help if somebody can point where the problem is?
%post -p /bin/bash
That will give your post script a terminal to run bash commands and all that you are doing into it.