Copying Java from one build to another using docker-compose - java

I need some help to put two Docker images together.
First, I have this Dockerfile/docker-compose files, for a Docker container with ROS. Alone, it works fine.
Dockerfile:
ARG FROM_IMAGE=ros:foxy
ARG OVERLAY_WS=/opt/ros/overlay_ws
# multi-stage for caching
FROM $FROM_IMAGE AS cacher
# clone overlay source
ARG OVERLAY_WS
WORKDIR $OVERLAY_WS/src
RUN echo "\
repositories: \n\
ros2/codigo_gustavo: \n\
type: git \n\
url: https://github.com/GustavoLLima/codigo_gustavo.git \n\
version: master \n\
" > ../overlay.repos
RUN vcs import ./ < ../overlay.repos
# copy manifests for caching
WORKDIR /opt
RUN mkdir -p /tmp/opt && \
find ./ -name "package.xml" | \
xargs cp --parents -t /tmp/opt && \
find ./ -name "COLCON_IGNORE" | \
xargs cp --parents -t /tmp/opt || true
# multi-stage for building
FROM $FROM_IMAGE AS builder
# install overlay dependencies
ARG OVERLAY_WS
WORKDIR $OVERLAY_WS
COPY --from=cacher /tmp/$OVERLAY_WS/src ./src
RUN . /opt/ros/$ROS_DISTRO/setup.sh && \
apt-get update && rosdep install -y \
--from-paths \
src \
--rosdistro \
foxy \
&& rm -rf /var/lib/apt/lists/*
# build overlay source
COPY --from=cacher $OVERLAY_WS/src ./src
ARG OVERLAY_MIXINS="release"
RUN . /opt/ros/$ROS_DISTRO/setup.sh && \
colcon build \
--packages-select \
codigo_gustavo \
--mixin $OVERLAY_MIXINS
# source entrypoint setup
ENV OVERLAY_WS $OVERLAY_WS
RUN sed --in-place --expression \
'$isource "$OVERLAY_WS/install/setup.bash"' \
/ros_entrypoint.sh
Docker-compose.yml:
version: '3'
services:
talker:
build: ./
command: ros2 run codigo_gustavo talker
listener:
build: ./
environment:
- "PYTHONUNBUFFERED=1"
command: ros2 run codigo_gustavo listener
deploy:
mode: replicated
replicas: 2
Then, I have another Dockerfile/Docker-Compose files for a Docker container running NetLogo. Again, alone it works fine.
Dockerfile:
FROM openjdk:8-jdk
LABEL maintainer="Allen Lee <allen.lee#asu.edu>"
ARG NETLOGO_HOME=/opt/netlogo
ARG NETLOGO_VERSION=6.0.4
ENV LC_ALL=C.UTF-8 \
LANG=C.UTF-8 \
NETLOGO_TARBALL=NetLogo-$NETLOGO_VERSION-64.tgz
ENV NETLOGO_URL=https://ccl.northwestern.edu/netlogo/$NETLOGO_VERSION/$NETLOGO_TARBALL
WORKDIR /opt
RUN wget $NETLOGO_URL && tar xzf $NETLOGO_TARBALL && ln -sf "NetLogo $NETLOGO_VERSION" netlogo \
&& rm -f $NETLOGO_TARBALL
Docker-compose.yml:
version: '3'
services:
modelo1:
build: ./
command: tail -f /dev/null
volumes:
- ./teste:/teste
volumes:
teste:
Now the problem: I'm trying to merge these two, to use just one Dockerfile/Docker-Compose files, making it easier to run. The problem is that I managed to make the two containers, but when I try to run the NetLogo (via netlogo headless), the container can't find Java (dependency of NetLogo). I checked in the directory, and it seems like Java isn't installed. The workspace comes with both folders, NetLogo and ROS.
Dockerfile:
#--------------Netlogo Instalation--------------
ARG FROM_IMAGE=ros:foxy
ARG OVERLAY_WS=/opt/ros/overlay_ws
#FROM openjdk:8-jdk AS netlogo_cache
FROM openjdk:8-jdk
LABEL maintainer="Allen Lee <allen.lee#asu.edu>"
ARG NETLOGO_HOME=/opt/netlogo
ARG NETLOGO_VERSION=6.0.4
ENV LC_ALL=C.UTF-8 \
LANG=C.UTF-8 \
NETLOGO_TARBALL=NetLogo-$NETLOGO_VERSION-64.tgz
ENV NETLOGO_URL=https://ccl.northwestern.edu/netlogo/$NETLOGO_VERSION/$NETLOGO_TARBALL
WORKDIR /opt
RUN wget $NETLOGO_URL && tar xzf $NETLOGO_TARBALL && ln -sf "NetLogo $NETLOGO_VERSION" netlogo \
&& rm -f $NETLOGO_TARBALL
#--------------ROS------------------------
#ARG FROM_IMAGE=ros:foxy
#ARG OVERLAY_WS=/opt/ros/overlay_ws
# multi-stage for caching
FROM $FROM_IMAGE AS cacher
# clone overlay source
ARG OVERLAY_WS
WORKDIR $OVERLAY_WS/src
RUN echo "\
repositories: \n\
ros2/codigo_gustavo: \n\
type: git \n\
url: https://github.com/GustavoLLima/codigo_gustavo.git \n\
version: master \n\
" > ../overlay.repos
RUN vcs import ./ < ../overlay.repos
# copy manifests for caching
WORKDIR /opt
RUN mkdir -p /tmp/opt && \
find ./ -name "package.xml" | \
xargs cp --parents -t /tmp/opt && \
find ./ -name "COLCON_IGNORE" | \
xargs cp --parents -t /tmp/opt || true
# multi-stage for building
FROM $FROM_IMAGE AS builder
# install overlay dependencies
ARG OVERLAY_WS
WORKDIR $OVERLAY_WS
COPY --from=cacher /tmp/$OVERLAY_WS/src ./src
RUN . /opt/ros/$ROS_DISTRO/setup.sh && \
apt-get update && rosdep install -y \
--from-paths \
src \
--rosdistro \
foxy \
&& rm -rf /var/lib/apt/lists/*
# build overlay source
COPY --from=cacher $OVERLAY_WS/src ./src
ARG OVERLAY_MIXINS="release"
RUN . /opt/ros/$ROS_DISTRO/setup.sh && \
colcon build \
--packages-select \
codigo_gustavo \
--mixin $OVERLAY_MIXINS
# source entrypoint setup
ENV OVERLAY_WS $OVERLAY_WS
RUN sed --in-place --expression \
'$isource "$OVERLAY_WS/install/setup.bash"' \
/ros_entrypoint.sh
#COPY --from=netlogo_cache /opt/netlogo ./netlogo
COPY --from=0 /opt/netlogo ./netlogo
Docker-compose.yml:
version: '3'
services:
talker:
build: ./
command: ros2 run codigo_gustavo talker
listener:
build: ./
environment:
- "PYTHONUNBUFFERED=1"
command: ros2 run codigo_gustavo listener
deploy:
mode: replicated
replicas: 2
modelo1:
build: ./
command: tail -f /dev/null
volumes:
- ./teste:/teste
volumes:
teste:
Error context:
cd /opt/ros/overlay_ws
./netlogo/netlogo-headless.sh --model /teste/WS2.nlogo --experiment experiment1 --spreadsheet /teste/teste2.csv
JAVA_HOME undefined, using java from path. For control over exact java version, set JAVA_HOME
./netlogo/netlogo-headless.sh: line43: java: command not found
Any ideas? What should I do? What am I doing wrong?

Related

Error: Could not find or load main class in Docker

I try to containerized my selenium code. In order to do the same I created a jar of my code that contains the .class file and dependencies.
My Dockerfile is
FROM ubuntu:18.
FROM openjdk:8u191-jre-alpine3.8
RUN apk add --no-cache curl tar bash procps
ARG MAVEN_VERSION=3.6.3
ARG USER_HOME_DIR="/root"
ARG BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries
# Install Java.
#
RUN apk --update --no-cache add openjdk8 curl
RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
&& curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache- maven-${MAVEN_VERSION}-bin.tar.gz \
&& tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip- components=1 \
&& rm -f /tmp/apache-maven.tar.gz \
&& ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
ENV JAVA_HOME /usr/lib/jvm/java-1.8-openjdk
ENV CLASSPATH /usr/lib/jvm/java-1.8-openjdk/lib
# Define default command.
CMD ["mvn", "--version"]
WORKDIR E:/Advanced_Selenium_Test_Automation_Framework-master
COPY target/AdavencedLevel.QDPM-0.0.1-SNAPSHOT-jar-with-dependencies.jar AdavencedLevel.QDPM-0.0.1-SNAPSHOT-jar-with-dependencies.jar
ENTRYPOINT ["java","-jar","/E:/Advanced_Selenium_Test_Automation_Framework-master/AdavencedLevel.QDPM-0.0.1-SNAPSHOT-jar-with-dependencies.jar"]
after running the docker image getting Error:
Error: Could not find or load main class TestRunner
I try to set ENV through Dockerfile to avoid this issue
I am expecting that as jdk is already present in the Image then that issue should not occur.

java command is not found using jlink in docker alpine

I'm using jlink to create a customized JRE to be used in docker
FROM gradle:7.5.1-jdk17-jammy As build
WORKDIR /build
COPY my-source my-source
RUN cd my-source && gradle clean build
RUN jlink \
--module-path /... \
--add-modules ... \
--output jre-custom \
--strip-debug \
--no-header-files \
--no-man-pages \
--compress 2
FROM alpine:latest
WORKDIR /deployment
COPY --from=build /build/jre-custom jre-custom/
COPY --from=build /build/my-source/build/libs/*.jar build/app.jar
# ERROR at line bellow: /bin/sh: jre-custom/bin/java: not found
CMD ["jre-custom/bin/java","-jar","build/app.jar"]
EXPOSE 3009
When I'm running jre-custom/bin/java -version in alpine image I've got:
/bin/sh: jre-custom/bin/java: not found
Is there anything in alpine image to be installed?
NOTE: I can run jre-custom/bin/java -version in first stage docker successfully.
I've got the solution changing the first stage image to alpine-based image as follow
# using image on alpine-based
FROM gradle:7.5.1-jdk17-alpine As build
WORKDIR /build
COPY my-source my-source
# install required packages
RUN apk update && apk add gcompat binutils
RUN gradle -p my-source clean build
RUN jlink \
--module-path /opt/java/openjdk/jmods \
--add-modules $(jdeps --ignore-missing-deps --print-module-deps --multi-release 17 my-source/build/libs/*.jar ) \
--output jre-custom \
--strip-debug \
--no-header-files \
--no-man-pages \
--compress 2
# reduce image size a little bit more (-4MB)
RUN strip -p --strip-unneeded jre-custom/lib/server/libjvm.so && \
find jre-custom -name '*.so' | xargs -i strip -p --strip-unneeded {}
FROM alpine:latest
WORKDIR /deployment
COPY --from=build /build/jre-custom jre-custom/
COPY --from=build /build/my-source/build/libs/*.jar build/app.jar
CMD ["jre-custom/bin/java","-jar","build/app.jar"]
EXPOSE 3009

Tweak Any Errors with Docker Build for Maven-3.6.3 and Java SDK 8 Application

New to the java community, and would appreciate all of the help that I can get with my Dockerfile and Docker Compose setup.
These are the requirements of my project.
Use Maven 3.6.3 and Java 8 SDK,
create a workdir named /app
create a non root user with root permissions (max -777)
install basic linux tools and utilities
change to the root folder as a root user and cd into .m2/settings.xml and edit it with the code below.
Additionally, I need to copy over the pom.xml, the /target, the /src, and settings.xml and the app dir itself as illustrated in my attempt below.
And then run and execute this entire application as a non-root user with root permissions to create, delete, and update files and dirs in the root dir and throughout /app for the build with mvn.
# Custom image from Maven on DockerHub
# Language: dockerfile
FROM maven:3.6.3-amazoncorretto-8
# Set the working dir
WORKDIR /app
# Create a non root user
ARG USERNAME=jefferson
ARG USER_UID=1000
ARG USER_GID=$USER_UID
# Add linux dependenciesq
RUN yum install wget -y
RUN yum install shadow-utils -y
# Create the user
RUN groupadd --gid $USER_GID $USERNAME \
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
&& yum install sudo -y \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
&& chmod 777 /etc/sudoers.d/$USERNAME \
&& sudo groupadd docker \
&& sudo usermod -aG docker $USERNAME \
&& newgrp docker
# Change to the root folder and edit the settings.xml for Maven
WORKDIR /root/.m2
RUN rm -rf settings.xml
RUN echo '<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" \
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" \
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 \
http://maven.apache.org/xsd/settings-1.0.0.xsd"> \
</settings>' >> settings.xml
WORKDIR /app
COPY . ./
USER $USERNAME
# Run the application
CMD ["mvn", "clean", "verify", "-Pcargo.run", "-X"]
version: '1.0'
services:
app:
build: .
command: sh -c "mvn clean verify && mvn -Pcargo.run -X"
ports:
- 3100:3100
working_dir: /app
volumes:
# give absolute path of the workdir app working_dir
- .:/app
# give the absolute bath of src starting at app
- ./src:/app/src
# give the absolute path of target
- ./target:/app/target
# give the absolute path of the maven repo
- ~/.m2:/root/.m2
# give thepom.xml
- ./pom.xml:/app/pom.xml
That being said the docker compose up command should both build and run the container on port 3100. Can somebody give me a push in the right direction?

How to configure Dockerfile for non-root user but give them access privileges to write to a file in the root dir?

New to linux and working on containerizing our stack and essentially here is the problem that I am running into with the code below:
a) I have to execute this dockerfile as a non-root user for elastic search to work (requirement)
b)If I add USER $USERNAME to the bottom of the script before CMD i get the error:
"mkdir: cannot create directory ‘/root’: Permission denied
Can not write to /root/.m2/copy_reference_file.log. Wrong volume permissions? Carrying on"
c) If I remove the USER $USERNAME from the bottom of the file then I get the elastic search issue referenced above.
What I am asking is, how can I fix this in my dockerfile?
# Custom image from Maven on DockerHub
# Language: dockerfile
FROM maven:3.6.3-amazoncorretto-8
# Set the working dir
WORKDIR /app
# Create a non root user
ARG USERNAME=jefferson
ARG USER_UID=1000
ARG USER_GID=$USER_UID
# Add linux dependenciesq
RUN yum install wget -y
RUN yum install shadow-utils -y
# Create the user
RUN groupadd --gid $USER_GID $USERNAME \
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
&& yum install sudo -y \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
&& chmod 777 /etc/sudoers.d/$USERNAME \
&& sudo groupadd docker \
&& sudo usermod -aG docker $USERNAME \
&& newgrp docker
# Change to the root folder and edit the settings.xml for Maven
WORKDIR /root/.m2
RUN rm -rf settings.xml
RUN echo '<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" \
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" \
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 \
http://maven.apache.org/xsd/settings-1.0.0.xsd"> \
</settings>' >> settings.xml
WORKDIR /app
COPY . ./
USER $USERNAME
# Run the application
CMD ["mvn", "clean", "verify", "-Pcargo.run", "-X"]
If i understand well your needs, on the official docker maven image give you have a way to follow :
Why not fully use a non-root user to run maven (and not trying to write in the root directory) ?
at this page : https://hub.docker.com/_/maven under the header named
Running as non-root
it tells you to use a MAVEN_CONFIG env var and to add the -Duser.home= flag when calling maven to run maven without using the root user
here is the full Dockerfile modified using this way (from your own Dockerfile):
# Custom image from Maven on DockerHub
# Language: dockerfile
FROM maven:3.6.3-amazoncorretto-8
# Set the working dir
WORKDIR /app
# Create a non root user
ARG USERNAME=jefferson
ARG USER_UID=1000
ARG USER_GID=$USER_UID
# Add linux dependenciesq
RUN yum install wget -y
RUN yum install shadow-utils -y
ENV MAVEN_CONFIG=/var/maven/.m2
# Create the user
RUN groupadd --gid $USER_GID $USERNAME \
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
&& yum install sudo -y \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
&& chmod 600 /etc/sudoers.d/$USERNAME \
&& sudo groupadd docker \
&& sudo usermod -aG docker $USERNAME \
&& newgrp docker
# Change to the root folder and edit the settings.xml for Maven
WORKDIR "/var/maven/.m2"
RUN rm -rf settings.xml \
&& chown $USER_UID:$USER_GID .
RUN echo '<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" \
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" \
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 \
http://maven.apache.org/xsd/settings-1.0.0.xsd"> \
</settings>' >> settings.xml
WORKDIR /app
COPY . ./
USER $USERNAME
# Run the application
CMD ["mvn", "clean", "verify", "-Duser.home=/var/maven", "-Pcargo.run", "-X"]
the rights of the sudoers file you added was too permissive so i changed it to 600.
as an example :
with the following command line (you don't have to run this command line in your case because the example command line given (on the maven docker image site) is just starting a new container with interactive mode and mounting volumes)
docker run -v ~/.m2:/var/maven/.m2 -ti --rm -u 1000 -e MAVEN_CONFIG=/var/maven/.m2 maven mvn -Duser.home=/var/maven archetype:generate

cannot copy from one stage to another in multi stage docker build

I want to create and execute performance tests against the application. For now, my idea was to use multi-stage build on the first stage to build the application, on the second - build the performance testing and start both application and performance tests in the same container.
My Dockerfile looks like this
# build stage
FROM gradle:jdk11
ARG ARTIFACT_PATH=json-comparison-application/build/libs
ARG ARTIFACT_NEW=app.jar
ARG ARTIFACT_OLD=json-comparison-application-0.0.1-SNAPSHOT.jar
RUN echo ${ARTIFACT_PATH}
RUN apt-get install git && git clone https://github.com/xxxx/json-comparison.git
WORKDIR json-comparison
RUN chmod +x gradlew && \
./gradlew clean build -x pmdMain -x spotbugsMain -x checkstyleMain --no-daemon && \
cd ${ARTIFACT_PATH} && mv ${ARTIFACT_OLD} ${ARTIFACT_NEW}
# performance test stage
FROM ubuntu:18.04
# simplified adoptopenjdk/11 without CMD entry point, probably better move to some external dockerfile
ARG ESUM='6dd0c9c8a740e6c19149e98034fba8e368fd9aa16ab417aa636854d40db1a161'
ARG BINARY_URL='https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.5%2B10/OpenJDK11U-jdk_x64_linux_hotspot_11.0.5_10.tar.gz'
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'
ENV JAVA_VERSION jdk-11.0.5+10
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl ca-certificates fontconfig locales \
&& echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen \
&& locale-gen en_US.UTF-8 \
&& rm -rf /var/lib/apt/lists/*
RUN set -eux; \
curl -LfsSo /tmp/openjdk.tar.gz ${BINARY_URL}; \
echo "${ESUM} */tmp/openjdk.tar.gz" | sha256sum -c -; \
mkdir -p /opt/java/openjdk; \
cd /opt/java/openjdk; \
tar -xf /tmp/openjdk.tar.gz --strip-components=1; \
rm -rf /tmp/openjdk.tar.gz;
ENV JAVA_HOME=/opt/java/openjdk \
PATH="/opt/java/openjdk/bin:$PATH"
# custom part of the stage
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y python default-jre-headless python-tk python-pip python-dev \
libxml2-dev libxslt-dev zlib1g-dev net-tools && \
pip install bzt
WORKDIR /home/tmp
ARG ARTIFACT_PATH=json-comparison/json-comparison-application/build/libs
ARG ARTIFACT_NEW=app.jar
RUN echo ${ARTIFACT_PATH}
COPY --from=0 /${ARTIFACT_PATH}/${ARTIFACT_NEW} .
# prototype for test
CMD ["bzt", "quick_test.yml"]
But it fails during build with error
Step 23/24 : COPY --from=0 /${ARTIFACT_PATH}/${ARTIFACT_NEW} .
COPY failed: stat /var/lib/docker/overlay2/f227e7b77fba105ba0769aa355458900d202add59c98583f0fd0936cbf4dfc11/merged/json-comparison/json-comparison-application/build/libs/app.jar: no such file or directory
What is the problem?
At first sight the problem comes from the (absolute vs. relative) paths of the .jar in your two stages.
Can you verify that they are the same?
For example, you may replace the last command of the first stage with
RUN […] && \
cd ${ARTIFACT_PATH} && mv ${ARTIFACT_OLD} ${ARTIFACT_NEW} && readlink -f ${ARTIFACT_NEW}
and relaunch the build. If you don't obtain
/json-comparison/json-comparison-application/build/libs/app.jar
but a longer path, then you'll know the correct path to put in ARTIFACT_PATH within the second stage…
Alternatively, you could just get rid of the relative path in the first stage and replace WORKDIR json-comparison with:
WORKDIR /json-comparison
As an aside, it can be useful to name your build stages in the following way:
FROM gradle:jdk11 as builder
[…]
FROM ubuntu:18.04 as test
[…]
Then, this allows you to only build the first stage by running:
$ docker build --target builder -t $IMAGE_NAME .

Categories

Resources