I am new to docker-compose and probably don't understand the ideology behind.
Short question -- I see there is docker-compose build command. I'd like to trigger a (gradle) script outside of the Dockerfile file as part of running it.
Let's say that I have a java web service inside Docker that I build in advance by gradle.
I don't add gradle agent to the Dockerfile (as I am expected to keep it small, right?), I only COPY binaries
FROM openjdk:17-jdk-alpine
RUN apk add --no-cache bash
VOLUME /var/log/app/
WORKDIR /app
ARG EXTRACTED
COPY ${EXTRACTED}/application/ ./
...
ENTRYPOINT ["java","org.springframework.boot.loader.JarLauncher"]
And so I build this image by a script
./gradlew build
java -Djarmode=layertools -jar build/libs/*.jar extract --destination build/libs-extracted/
docker build --build-arg EXTRACTED=build/libs-extracted -t my-server .
I can define the following compose.yml. How do I trigger gradle inside of it? Or, same as with my single Dockerfile, I am expected to wrap docker-compose build into a build script?
version: '3.1'
services:
db:
image: postgres:alpine
restart: always
my-server:
image: my-server
restart: always
depends_on:
- db
Maybe I am asking for a hack, but actually I am happy to take another approach if it's cleaner
I discovered multi-stage Dockerfile feature that addresses my task, and I can use it for both individual Dockerfile builds and docker-compose builds.
https://stackoverflow.com/a/61131308/1291049.
I will probably lose gradle daemon optimisations though.
Change your docker compose file:
version: '3.1'
services:
db:
image: postgres:alpine
restart: always
my-server:
build:
context: ./my-server
dockerfile: Dockerfile
restart: always
depends_on:
- db
I've generated a Dockerfile with Visual Studio. It runs in Visual Studio just fine and now I'm trying to build it from Windows itself (docker build ., and I tried many combinations). Yet I get the following error:
When I change copy to ./client.csproj it does continue and then I get:
What am I doing wrong? I changed Docker Linux to Windows, changed WSL, and restarted everything.
Dockerfile client
Check your .dockerignore file. Possible it ignores needed files for copy command and you get failed to compute cache key error.
.dockerignore may be configured to minimize the files sent to docker for performance and security:
*
!dist/
The first line * disallows all files. The second line !dist/ allows the dist folder
This can cause unexpected behavior:
FROM nginx:latest
# Fails because of * in .dockerignore
# failed to compute cache key: "/nginx.conf.spa" not found: not found
# Fix by adding `!nginx.conf.spa` to .dockerignore
COPY nginx.conf.spa /etc/nginx/nginx.conf
RUN mkdir /app
# Works because of !dist/ in .dockerignore
COPY dist/spa /app
Belts and suspenders.
The way Visual Studio does it is a little bit odd.
Instead of launching docker build in the folder with the Dockerfile, it launches in the parent folder and specifies the Dockerfile with the -f option.
I was using the demo project (trying to create a minimal solution for another question) and struck the same situation.
Setup for my demo project is
\WorkerService2 ("solution" folder)
+- WorkerService2.sln
+- WorkserService2 ("project" folder)
+- DockerFile
+- WorkerService2.csproj
+- ... other program files
So I would expect to go
cd \Workerservice2\WorkerService2
docker build .
But I get your error message.
=> ERROR [build 3/7] COPY [WorkerService2/WorkerService2.csproj, WorkerService2/] 0.0s
------
> [build 3/7] COPY [WorkerService2/WorkerService2.csproj, WorkerService2/]:
------
failed to compute cache key: "/WorkerService2/WorkerService2.csproj" not found: not found
Instead, go to the parent directory, with the .sln file and use the docker -f option to specify the Dockerfile to use in the subfolder:
cd \Workerservice2
docker build -f WorkerService2\Dockerfile --force-rm -t worker2/try7 .
docker run -it worker2/try7
Edit (Thanks Mike Loux, tblev & Goku):
Note the final dot on the docker build command.
For docker the final part of the command is the location of the files that Docker will work with. Usually this is the folder with the Dockerfile in, but that's what's different about how VS does it. In this case the dockerfile is specified with the -f. Any paths (such as with the COPY instruction in the dockerfile) are relative to the location specified. The . means "current directory", which in my example is \WorkerService2.
I got to this stage by inspecting the output of the build process, with verbosity set to Detailed.
If you choose Tools / Options / Projects and Solutions / Build and Run you can adjust the build output verbosity, I made mine Detailed.
Edit #2 I think I've worked out why Visual Studio does it this way.
It allows the project references in the same solution to be copied in.
If it was set up to do docker build from the project folder, docker would not be able to COPY any of the other projects in the solution in. But the way this is set up, with current directory being the solution folder, you can copy referenced projects (subfolders) into your docker build process.
I had the same issue, I set the Docker environment to Windows in when adding Docker support. Even running in Visual Studio threw error to that. I changed the environment to Linux as my Docker is running in the Windows Subsystem for Linux (WSL).
Then I moved back to the terminal to run the commands.
I was able to resolve this by moving to the Solutions folder (Root folder).
And I did docker build like this:
docker build -t containername/tag -f ProjectFolder/Dockerfile .
Then I did docker run:
docker run containername/tag
Asking for a directory that does not exist throws this error.
In my case, I tried
> [stage-1 7/14] COPY /.ssh/id_rsa.pub /.ssh/:
------
failed to compute cache key: "/.ssh/id_rsa.pub" not found: not found
I had forgotten to add the /.ssh folder to the project directory. In your case you should check whether /client is really a subfolder of your Dockerfile build context.
The following command was failing with failed to compute cache key: not found:
docker build -t tag-name:v1.5.1 - <Dockerfile
Upon changing the command to the following it got fixed:
docker build -t tag-name:v1.5.1 -f Dockerfile .
In my case I found that docker build is case sensitive in directory name, so I was writing /bin/release/net5.0/publish in the COPY instruction and failed with the same error, I've just changed to /bin/Release/net5.0/publish and it worked
Error : failed to compute cache key: "src" not found: not found
in my case , folder/file excluded in .dockerignore
after resolving file from dockerignore able to create image.
In my case, I had something like this:
FROM mcr.microsoft.com/dotnet/aspnet:5.0
COPY bin/Release/net5.0/publish/ app/
WORKDIR /app
ENTRYPOINT ["dotnet", "MyApi.dll"]
And I finally realized that I had the bin folder in my .dockerignore file.
I had the same issue. In my case there was a wrong directory specified.
My Dockerfile was:
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS publish
WORKDIR /app
COPY . .
RUN dotnet publish -c Release -o publish/web src/MyApp/MyApp.csproj
FROM mcr.microsoft.com/dotnet/aspnet:5.0
WORKDIR /app
COPY --from=publish publish/web .
EXPOSE 80
CMD ASPNETCORE_URLS=http://*:$PORT dotnet MyApp.dll
Then I realised that in the second build stage I am trying to copy project files from directory publish/web:
COPY --from=publish publish/web .
But as I specified workdir /app in the first stage, my files are located in that directory in image filesystem, so changing path from publish/web to app/publish/web resolved my issue.
So my final working Dockerfile is:
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS publish
WORKDIR /app
COPY . .
RUN dotnet publish -c Release -o publish/web src/MyApp/MyApp.csproj
FROM mcr.microsoft.com/dotnet/aspnet:5.0
WORKDIR /app
COPY --from=publish app/publish/web .
EXPOSE 80
CMD ASPNETCORE_URLS=http://*:$PORT dotnet MyApp.dll
In my case there was a sneaky trailing whitespace in the file name.
------
> [3/3] COPY init.sh ./:
------
failed to compute cache key: "/init.sh" not found: not found
So the file was actually called "init.sh " instead of "init.sh".
I had a similar issues: Apparently, docker roots the file system during build to the specified build directory for security reasons. As a result, COPY and ADD cannot refer to arbitrary locations on the host file system. Additionally, there are other issues with syntax peculiarities. What eventually worked was the following:
COPY ./script_file.sh /
RUN /script_file.sh
I had faced the same issue.
The reason was the name of the DLL file in the Docker file is case sensitive.
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
WORKDIR /src
COPY MyFirstMicroService.csproj .
RUN dotnet restore
COPY . .
RUN dotnet publish -c release -o /app
FROM mcr.microsoft.com/dotnet/aspnet:5.0
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "**MyFirstMicroService.dll**"]
This .dll name should match your .csproj file.
This also happens when you don't provide the proper path to your COPY command input. The most important clue I had is that WORKDIR command opens a folder for the container, not in the windows explorer (so it doesn't affect the path you need to specify for the COPY command).
In my Case,
i was doing mistake in '/' and ''. Let me explain
Open your dockerfile (it should be named as dockerfile only, not DockerFile or Dockerfile).
You may have something like this-
FROM mcr.microsoft.com/dotnet/runtime:5.0
COPY bin\Release\net5.0\publish .
ENTRYPOINT ["dotnet", "HelloDocker.dll"]
Replace COPY bin\Release\net5.0\publish . to COPY bin/Release/net5.0/publish .
in my case, it was a wrong Build with PATH configuration e.g. Docker build context
Simple docker script
docker build .
where . is path to build context
Gradle+Docker
docker {
dependsOn build
dependsOn dockerFilesCopy
name "${project.name}:${project.version}"
files "build" // path to build context
}
Gradle+GitHub action
name: Docker build and push
on:
push:
branches: [ main ]
# ...
jobs:
build:
runs-on: ubuntu-latest
# ...
steps:
- name: Checkout
uses: actions/checkout#v2
# ...
- name: Build and export to Docker
uses: docker/build-push-action#v2
with:
# ...
file: src/main/docker/Dockerfile
context: ./build # path to build context
In my case, with Angular project, my project was in the folder called ex: My-Folder-Project and I was putting on Dockerfile COPY --from=publish app/dist/My-Folder-Project .
But of course the correct thing is put the "name" in your package.json like COPY --from=publish app/dist/name-in-package.json .
In my case I changed context, and path of Dockerfile within docker-compose.yml config:
services:
server:
# inheritance structru
extends:
file: ../../docker-compose.server.yml
# I recommend you to play with this paths
build:
context: ../../
dockerfile: ./apps/${APP_NAME}/Dockerfile
...
I'm trying to deploy my spring boot application with docker compose but get this error:
Step 14/15 : COPY ${JAR_FILE} manager.jar
ERROR: Service 'manager' failed to build : When using COPY with more than one source file, the destination must be a directory and end with a /
But if I do a docker build using the Dokerfile it works correctly. The question is, why fail with docker-compose up?.
C:\Push\Workspace\manager>docker build --tag "docker-manager:latest" .
[+] Building 7.3s (8/8) FINISHED
I have tried looking for examples but I am using windows 10
The Dockerfile:
FROM openjdk:8-jdk-alpine
RUN addgroup -S spring && adduser -S spring -G spring
USER spring:spring
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} example.jar
ENTRYPOINT ["java", "-jar", "example.jar"]
The docker-compose.yml
services:
oracle:
image: container-registry-london.oracle.com/database/enterprise:12.2.0.1
ports:
- "8080:8080"
- "1521:1521"
manager:
build: .
ports:
- "8181:8181"
depends_on:
- oracle
links:
- oracle
restart: always
I am guessing there might be mulitple jar files in your target folder.
In stead of using the *.jar , please use full name of the jar and then rebuild the dokcer compose file with --build flag.
ARG JAR_FILE=target/test-0.0.1-SNAPSHOT.jar
Then build with
docker-compose up --build
I am trying make a docker image of a java project. I first created a directory and in that I created a docker.txt file. The files contains this
FROM java:8
# Install maven
RUN apt-get update
RUN apt-get install -y maven
WORKDIR /home/mmt/CouchBaseClient/CB-RestAPI/CouchBaseThinClient
# Prepare by downloading dependencies
ADD pom.xml /home/mmt/CouchBaseClient/CB-RestAPI/CouchBaseThinClient/pom.xml
RUN ["mvn", "dependency:resolve"]
RUN ["mvn", "verify"]
# Adding source, compile and package into a fat jar
ADD src /home/mmt/CouchBaseClient/CB-RestAPI/CouchBaseThinClient/src
RUN ["mvn", "package"]
EXPOSE 4567
CMD ["/usr/lib/jvm/java-8-openjdk-amd64/bin/java", "-jar", "target/sparkexample-jar-with-dependencies.jar"]
and then I run in terminal the following command
docker build -t API .
I get the following error
invalid value "API" for flag -t: Error parsing reference: "API" is not a valid repository/tag
See 'docker build --help'.
Docker is complaining about "API" in the sense that it's not allowed to have a tag name with one or more character in uppercase:
$ docker build -t FOO .
repository name component must match "[a-z0-9](?:-*[a-z0-9])*(?:[._][a-z0-9](?:-*[a-z0-9])*)*"
Usually "recipes" to build Docker images are written in a file named Dockerfile, anyway you can continue to use docker.txt using the -f option:
docker build -f docker.txt -t api .
I'm trying to deploy our java web application to aws elastic beanstalk using docker, the idea is to be able to run the container locally for development and testing and eventually push it up to production using git.
I've created a base image that has tomcat8 and java8 installed, the image that performs the gradle builds inherit from this base image, speeding up build process.
All works well, except for the fact that the inheriting application container that gets built using docker doesn't seem to cache the gradle dependencies, it downloads it every time, including gradlew. We build our web application using the following command:
./gradlew war
Is there some way that i can cache the files in ~/.gradle this would speed my build up dramatically.
This isn't so much of an issue on beanstalk but is a big problem for devs trying to build and run locally as this does take a lot of time, as you can imagine.
The base image dockerfile:
FROM phusion/baseimage
EXPOSE 8080
RUN apt-get update
RUN add-apt-repository ppa:webupd8team/java
RUN apt-get update
RUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | sudo /usr/bin/debconf-set-selections
RUN apt-get -y install oracle-java8-installer
RUN java -version
ENV TOMCAT_VERSION 8.0.9
RUN wget --quiet --no-cookies http://archive.apache.org/dist/tomcat/tomcat-8/v${TOMCAT_VERSION}/bin/apache-tomcat-${TOMCAT_VERSION}.tar.gz -O /tmp/catalina.tar.gz
# Unpack
RUN tar xzf /tmp/catalina.tar.gz -C /opt
RUN mv /opt/apache-tomcat-${TOMCAT_VERSION} /opt/tomcat
RUN ln -s /opt/tomcat/logs /var/log/tomcat
RUN rm /tmp/catalina.tar.gz
# Remove unneeded apps
RUN rm -rf /opt/tomcat/webapps/examples
RUN rm -rf /opt/tomcat/webapps/docs
RUN rm -rf /opt/tomcat/webapps/ROOT
ENV CATALINA_HOME /opt/tomcat
ENV PATH $PATH:$CATALINA_HOME/bin
ENV CATALINA_OPTS $PARAM1
# Start Tomcat
CMD ["/opt/tomcat/bin/catalina.sh", "run"]
The application dockerfile:
FROM <tag name here for base image>
RUN mkdir ~/.gradle
# run some extra stuff here to add things to gradle.properties file
# Add project Source
ADD . /var/app/myapp
# Compile and Deploy Application, this is what is downloading gradlew and all the maven dependencies every time, if only there was a way to take the changes it makes to ~/.gradle and persist it as a cache layer
RUN cd /var/app/myapp/ && ./gradlew war
RUN mv /var/app/myapp/build/libs/myapp.war /opt/tomcat/webapps/ROOT.war
# Start Tomcat
CMD ["/opt/tomcat/bin/catalina.sh", "run"]
I faced this issue. As you might agree, it is a best practice to download dependencies alone as a separate step while building the docker image. It becomes little tricky with gradle, since there is no direct support for downloading just dependencies.
Option 1 : Using docker-gradle Docker image
We can use pre-built gradle docker image to build the application. This ensures that it's not a local system build but a build done on a clean docker image.
docker volume create --name gradle-cache
docker run --rm -v gradle-cache:/home/gradle/.gradle -v "$PWD":/home/gradle/project -w /home/gradle/project gradle:4.7.0-jdk8-alpine gradle build
ls -ltrh ./build/libs
gradle cache is loaded here as a volume. So subsequent builds will reuse the downloaded dependencies.
After this, we could have a Dockerfile to take this artifact and generate application specific image to run the application.
This way, the builder image is not required. Application build flow and Application run flow is separated out.
Since the gradle-cache volume is mounted, we could reuse the downloaded dependencies across different gradle projects.
Option 2 : Multi-stage build
----- Dockerfile -----
FROM openjdk:8 AS TEMP_BUILD_IMAGE
ENV APP_HOME=/usr/app/
WORKDIR $APP_HOME
COPY build.gradle settings.gradle gradlew $APP_HOME
COPY gradle $APP_HOME/gradle
RUN ./gradlew build || return 0
COPY . .
RUN ./gradlew build
FROM openjdk:8
ENV ARTIFACT_NAME=your-application.jar
ENV APP_HOME=/usr/app/
WORKDIR $APP_HOME
COPY --from=TEMP_BUILD_IMAGE $APP_HOME/build/libs/$ARTIFACT_NAME .
EXPOSE 8080
CMD ["java","-jar",$ARTIFACT_NAME]
In the above Dockerfile
First we try to copy the project's gradle files alone, like
build.gradle, gradlew etc.,
Then we copy the gradle directory itself
And then we try to run the build. At this point, there is no other
source code files exists in the directory. So build will fail. But
before that it will download the dependencies.
Since we expect the
build to fail, I have tried a simple technique to return 0 and allow
the docker to continue execution
this will speed up the subsequent build flows, since all the dependencies are downloaded and docker cached this layer. Comparatively, Volume mounting the gradle cache directory is still the best approach.
The above example also showcases multi-stage docker image building, which avoid multiple docker build files.
I
Add resolveDependencies task in build.gradle:
task resolveDependencies {
doLast {
project.rootProject.allprojects.each { subProject ->
subProject.buildscript.configurations.each { configuration ->
configuration.resolve()
}
subProject.configurations.each { configuration ->
configuration.resolve()
}
}
}
}
and update Dockerfile:
ADD build.gradle /opt/app/
WORKDIR /opt/app
RUN gradle resolveDependencies
ADD . .
RUN gradle build -x test --parallel && \
touch build/libs/api.jar
II
Bellow is what I do now:
build.gradle
ext {
speed = project.hasProperty('speed') ? project.getProperty('speed') : false
offlineCompile = new File("$buildDir/output/lib")
}
dependencies {
if (speed) {
compile fileTree(dir: offlineCompile, include: '*.jar')
} else {
// ...dependencies
}
}
task downloadRepos(type: Copy) {
from configurations.all
into offlineCompile
}
Dockerfile
ADD build.gradle /opt/app/
WORKDIR /opt/app
RUN gradle downloadRepos
ADD . /opt/app
RUN gradle build -Pspeed=true
You might want to consider splitting your application image to two images: one for building the myapp.war and the other for running your application. That way, you can use docker volumes during the actual build and bind the host's ~/.gradle folder into the container performing the build. Instead of only one step to run your application, you would have more steps, though. Example:
builder image
FROM <tag name here for base image including all build time dependencies>
# Add project Source
# -> you can use a project specific gradle.properties in your project root
# in order to override global/user gradle.properties
ADD . /var/app/myapp
RUN mkdir -p /root/.gradle
ENV HOME /root
# declare shared volume path
VOLUME /root/.gradle
WORKDIR /var/app/myapp/
# Compile only
CMD ["./gradlew", "war"]
application image
FROM <tag name here for application base image>
ADD ./ROOT.war /opt/tomcat/webapps/ROOT.war
# Start Tomcat
CMD ["/opt/tomcat/bin/catalina.sh", "run"]
How to use in your project root, assuming the builder Dockerfile is located there and the application Dockerfile is located at the webapp subfolder (or any other path you prefer):
$ docker build -t builder .
$ docker run --name=build-result -v ~/.gradle/:/root/.gradle/ builder
$ docker cp build-result:/var/app/myapp/myapp.war webapp/ROOT.war
$ cd webapp
$ docker build -t application .
$ docker run -d -P application
I haven't tested the shown code, but I hope you get the idea. The example might even be improved by using data volumes for the .gradle/ cache, see the Docker user guide for details.
The current version of Docker supports mounting a "cache" and it's local to the Docker environment (so it's not shared with your OS which is both good and bad; good in that there's nothing about your system in the build process, bad in that you have to download again)
This code is from my Spring Docker Swarm integration rework
FROM gradle:7.4-jdk17 AS builder
WORKDIR /w
COPY ./ /w
RUN --mount=type=cache,target=/home/gradle/.gradle/caches gradle build --no-daemon -x test
FROM openjdk:17-jdk as extractor
WORKDIR /w
COPY bin/extract.sh /w/extract.sh
COPY --from=builder /w/*/build/libs/*.jar /w/
RUN sh ./extract.sh
FROM openjdk:17-jdk as sample-service
WORKDIR /w
COPY --from=extractor /w/sample-service/* /w/
ENTRYPOINT ["java", "-XX:MaxRAMPercentage=80", "org.springframework.boot.loader.JarLauncher"]
HEALTHCHECK --interval=5s --start-period=60s \
CMD curl -sfo /dev/null http://localhost:8080/actuator/health
USER 5000
EXPOSE 8080
What this does is from my current folder which is a multi-module gradle build I run the build. extractor stage unbundles the JAR file using extract.sh script below.
Then assembles the relevant component
The relevant contents of extract.sh
#!/bin/sh
set -e
set -x
# Remove support projects that won't be a Spring Boot
# rm buildSrc.jar
# rm gateway-common-*.jar
for jar in *.jar
do
DIR=$(basename $jar -0.0.1-SNAPSHOT.jar)
mkdir $DIR
java -Djarmode=layertools -jar $jar extract --destination $DIR
done
try changing the gradle user home directory
RUN mkdir -p /opt/gradle/.gradle
ENV GRADLE_USER_HOME=/opt/gradle/.gradle