Develop Reference

Best practice for dockerizing a springboot app in a CICD pipeline

Context: I'm able to dockerize and run my Springboot app locally just fine with the below, following the most common recommendations of using a generated jar file to run the app via Docker.
Question: Since it's bad practice to push the jar file to the repo as it's going to potentially contain secrets from the local application.yml files, and the docker file depends on the jar file, how can I have my app be dockerized not just locally but on the cloud in anywhich cicd pipeline? Would my next step be modifying the dockerfile to copy over the project directory, and handle generating the jar file itself? Or should I not be using a jar at all and copying over the directory and using a CMD [Some Spring Run command)]
DockerFile:
FROM maven:3.8.5-openjdk-17
ADD target/xyz.jar xyz.jar
ENV RESTFUL_PORT 8080
ENV PORT_POSTGRES 5432
EXPOSE $RESTFUL_PORT
EXPOSE $PORT_POSTGRES
ENTRYPOINT ["java", "-jar","/xyz.jar"]
The pom.xml plugin which generates the jar file:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.10.1</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
Docker Steps
Build the Jar:
mvn clean install
Build the Image:
docker build -t xyz -f Dockerfile
Run the Image:
docker run -d -p 8080:8080 -e "SPRING_PROFILES_ACTIVE=dev" xyz

The Dockerfile can be like this:
# Build Stage
FROM maven:3.8-openjdk-17-slim AS build
COPY src /home/app/src
COPY pom.xml /home/app
RUN mvn -f /home/app/pom.xml clean package
# Execution Stage
FROM openjdk:17-oracle
COPY --from=build /home/app/target/demo-0.1-SNAPSHOT.jar /usr/local/lib/demo.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/usr/local/lib/demo.jar"]
To build docker image
$ docker build -t demo-app:1.0 .
Run docker image
$ docker run -p 8080:8080 demo-app:1.0
Simple build part of pom.xml will work
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>

FROM maven:3.8-openjdk-17-slim AS build
WORKDIR /workspace/app
COPY mvnw .
COPY .mvn .mvn
COPY pom.xml .
COPY src src
RUN mvn -f /workspace/app/pom.xml install
RUN mkdir -p target/dependency && (cd target/dependency; jar -xf ../*.jar)
FROM openjdk:17-oracle
VOLUME /tmp
ARG DEPENDENCY=/workspace/app/target/dependency
COPY --from=build ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY --from=build ${DEPENDENCY}/META-INF /app/META-INF
COPY --from=build ${DEPENDENCY}/BOOT-INF/classes /app
ENTRYPOINT ["java","-cp","app:app/lib/*","com.demo.DemoServiceApplication"]
Got it working with this, only needing to change the last line.
Docker Command:
docker build -t demo -f Dockerfile .
docker run -d -p 8080:8080 -e "SPRING_PROFILES_ACTIVE=develop" demo:latest
This results in a DockerFile that is not dependent on an external jar file and can be run in isolation against a repo for a cicd pipeline.
Source: Toptal - Multi-Stage Build

getting "Handler java.util.logging.ConsoleHandler is not defined" error on jenkins console when jboss is trying to restart

I have generated a war file using maven for one of my java projects, and tried to deploy it to jboss application server using jenkins CI/CD.
Getting below error on the jenkins console and it runs forever and doesn't come out of the build when jboss is restarted.
"Handler java.util.logging.ConsoleHandler is not defined"
Below is the command used to start jboss:
cd /apps/jboss-eap-7.2/bin
nohup ./standalone.sh -c standalone-full.xml -b 0.0.0.0 -Djboss.as.management.blocking.timeout=5000 > nohup.out &
I have tried below options:
created a logging.properties file in src/main/resources folder and added below line:
handlers = java.util.logging.ConsoleHandler.level= INFO
tried adding below dependancies in my pom.xml file:
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.2.3</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>1.2.3</version>
</dependency>
None of the above options worked.
I have uploaded the jenkins console log image of jboss-restart error part.
jboss-restart-log
Can anyone help me with this. Thanks in advance.

install mysql and java jdk-11 in Dockerfile and run my spring boot jar file in container

I am going to install MySQL and Jdk-11 and run the jar file(spring boot project) on the container. If anyone has experience in this field, please help.
Thanks
this is my sql config
host='localhost',
port=3306,
user='root',
passwd='password',
FROM ubuntu
RUN apt-get update
RUN apt-get -y install mysql-server
RUN apt-get -y install openjdk-11-jdk
COPY target/orderCodeBackEnd-0.0.1-SNAPSHOT.jar /usr/app/
WORKDIR /usr/app
ENTRYPOINT ["java", "-jar", "orderCodeBackEnd-0.0.1-SNAPSHOT.jar"]

//Dockerfile
FROM openjdk:11
ADD target/*.jar app.jar
ENTRYPOINT ["java","-jar","app.jar"]
//Dockerfile just desame to other one
FROM openjdk:11
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
//docker-compose.yaml
services:
yourapp:
image: yourAppJarName:latest
container_name: yourapp
depends_on:
- mysqldb
restart: always
build:
context: ./
dockerfile: Dockerfile
ports:
- "9090:9090"
environment:
MYSQL_HOST: mysqldb
MYSQL_USER: root
MYSQL_PASSWORD: root
MYSQL_PORT: 3306
mysqldb:
image: mysql:8.0.28
restart: unless-stopped
container_name: mysqldb
ports:
- "3307:3306"
cap_add:
- SYS_NICE
environment:
MYSQL_DATABASE: dbname
MYSQL_ROOT_PASSWORD: root
//application.properties or yaml
spring:
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://${MYSQL_HOST:localhost}:${MYSQL_PORT:3306}/dbname
username: root
password: root
//customize you jar name in pom.xml
</dependency>
<dependency>
..........
</dependency>
<dependency>
..........
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
<finalName>yourAppJarName</finalName>
</build>
</project>
Then click Project file then "Run as" then click maven "Install"
you must also open your mysql then connect to 3307 since 3307 is expose

Create a container of MySQL / MariaDB by pulling image from MySQL Docker repository.
sudo docker run --detach --env MARIADB_PASSWORD=secret --env MARIADB_ROOT_PASSWORD=secret -p 3306:3306 --add-host=YOUR_DESIRED_HOSTNAME:YOUR_LOCAL_MACHINE_IP mariadb:latest
--detach
Will run the server in detached mode.
--env MARIADB_PASSWORD=secret --env MARIADB_ROOT_PASSWORD=secret
Setting up environment variables for your DB server passwords. You can define the password as you wish. For me, I set it to secret
-p 3306:3306
Port mapping, container internal port 3306 will be mapped to the port 3306 outside container.
--add-host=YOUR_DESIRED_HOSTNAME:YOUR_LOCAL_MACHINE_IP
Don't forget to change YOUR_DESIRED_HOSTNAME and YOUR_LOCAL_MACHINE_IP values if you want to establish a remote connection with the docker host machine. Usually, the hostname can be localhost if you are running docker on the same development machine. In such case, we don't even need this --add-host flag.
Now you can use the following connection parameters for connecting your application with the database if you run it in local.
host: YOUR_LOCAL_MACHINE_IP
port: 3306
username: root
password: secret
However, if you want to access the db for your spring boot application from inside a docker container, you may have to use additional tool, docker-compose. The reason is because your host machine IP address may not work inside your docker container.
I think, this following git repository will be helpful for you to understand how to write your first docker-compose. This repository has a readme.md file, which you can take help from.
https://gitlab.com/mainul35/traver-auth

It is good practice to separate different services in independent containers, thus creating a less related architecture.
Next think, in docker hub we can find usefull, ready to use, images.
We can pull all images from command line and create all services but there is better way - docker compose. So first file, that u need is docker-compose.yml:
version: '2'
services:
mysql:
image: mysql
environment:
- MYSQL_ROOT_PASSWORD=password
ports:
- 3306:3306
app:
build: .
ports:
- 8080:8080
depends_on:
- mysql
in this file we describe this 2 services:
mysql:
image: image is from docker hub it's offical docker mysql image
environment variable: u can find all possible variable in image docs
ports: there we can specify what port will be expose
app:
build: path to Dockerfile
depends_on: before you can create this service, create mysql first
Dockerfile for your app:
FROM openjdk:11-jre
COPY target/orderCodeBackEnd-0.0.1-SNAPSHOT.jar /usr/app/
WORKDIR /usr/app
ENTRYPOINT ["java", "-jar", "orderCodeBackEnd-0.0.1-SNAPSHOT.jar"]
now you can easily start these services from the terminal
docker compose up -d
you must be in the directory where docker-compose.yml is located or specife -f parametr

The answer of #ConRed is not complete. I have done lots of changes from it in my application (which is shared here: https://github.com/Aliuken/JobVacanciesApp_Java11).
These are my final files:
docker-compose.yaml:
version: "3.9"
services:
app-db-service:
image: mysql:latest
container_name: app-db-container
ports:
- "3307:3306"
environment:
- MYSQL_DATABASE=job-vacancies-app-db
- MYSQL_ROOT_PASSWORD=admin
networks:
- internal-net
restart: on-failure
volumes:
- app-db-data:/var/lib/mysql
- ./src/main/resources/META-INF/db_dumps_folder:/docker-entrypoint-initdb.d
cap_add:
- SYS_NICE
healthcheck:
test: "mysql -uroot -padmin -e 'select 1'"
interval: 1s
retries: 120
app-service:
image: job-vacancies-app:latest
container_name: app-container
ports:
- "9090:8080"
environment:
- MYSQL_HOST=app-db-container
- MYSQL_PORT=3306
- MYSQL_USER=root
- MYSQL_PASSWORD=admin
networks:
- internal-net
- external-net
restart: on-failure
volumes:
- /AppData:/AppData
depends_on:
app-db-service:
condition: service_healthy
build:
context: .
dockerfile: Dockerfile
networks:
external-net:
external: true
internal-net:
driver: bridge
volumes:
app-db-data:
driver: local
where:
./src/main/resources/META-INF/db_dumps_folder contains my db dump file: db-dump.sql.
/AppData is the folder in my PC (which is Linux) that has images and documents used in the application.
healthcheck and service_healthy are used joint to determine when the db-dump.sql file was executed, to start the Spring Boot application after that.
internal-net is used to communicate the Spring Boot application with the database.
external-net is used to communicate the Spring Boot application with the user.
Dockerfile:
FROM adoptopenjdk/openjdk11-openj9:alpine
USER root
RUN mkdir /opt/apps
RUN mkdir /opt/apps/jobVacanciesApp
ARG JAR_FILE=lib/*.jar
COPY ${JAR_FILE} /opt/apps/jobVacanciesApp/jobVacanciesApp.jar
RUN addgroup -S jobVacanciesAppGroup && adduser -S jobVacanciesAppUser -G jobVacanciesAppGroup
USER jobVacanciesAppUser:jobVacanciesAppGroup
CMD ["java", "-jar", "/opt/apps/jobVacanciesApp/jobVacanciesApp.jar"]
docker-compose-start.sh:
docker volume prune -f
docker network create "external-net"
docker-compose build
docker-compose up
docker-compose start
docker-compose-stop.sh:
docker-compose stop
docker-compose down
docker volume prune -f
docker network rm "external-net"
application.yaml:
spring:
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://${MYSQL_HOST:localhost}:${MYSQL_PORT:3306}/job-vacancies-app-db?useSSL=false&serverTimezone=Europe/Madrid&allowPublicKeyRetrieval=true
username: root
password: admin
pom.xml:
...
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>com.aliuken.jobvacanciesapp.MainApplication</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-installed</id>
<phase>install</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>${project.groupId}</groupId>
<artifactId>${project.artifactId}</artifactId>
<version>${project.version}</version>
<type>${project.packaging}</type>
<outputDirectory>lib</outputDirectory>
<destFileName>job-vacancies-app.jar</destFileName>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<finalName>job-vacancies-app</finalName>
</build>
...
To run the application:
Execute in a terminal: ./docker-compose-start.sh
Open in a web browser: http://localhost:9090
To stop the application:
Press in the terminal previously opened: Ctrl + C
Execute in the terminal: ./docker-compose-stop.sh

Logback and Graylog cannot communicate on a Mac using syslog

I want to send logmessages from a Java application to Graylog, using slf4j on top of logback with a logback GELF-appender on one side and a Graylog GELF-input on the other. To test it, i'm running Graylog in a Docker container (using Docker for Mac) and run my Java application locally. The gist of my story is that the Graylog GELF-input does not receive anything from the Java application. Somehow the Java application and Graylog don't seem to be able to communicate. The same applies when i switch to a different appender/input combination (one based on syslog records). However, when echoing a message from the commandline to a different Graylog input, namely the RAW input that's listening to port 5555, that message is received fine.
Any idea what the problem is?
This is my setup using GELF:
Java app:
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class LogDemo {
public static void main(String[] args) {
Logger logger = LoggerFactory.getLogger(LogDemo.class);
logger.error("Hello World 2");
}
}
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>logdemo</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>de.appelgriepsch.logback</groupId>
<artifactId>logback-gelf-appender</artifactId>
<version>1.5</version>
</dependency>
</dependencies>
</project>
logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="GELF" class="de.appelgriepsch.logback.GelfAppender">
<server>localhost</server>
<port>12201</port>
<protocol>TCP</protocol>
</appender>
<root level="error">
<appender-ref ref="GELF"/>
</root>
</configuration>
Graylog docker startup:
$ docker run --name mongo -d mongo:3
$ docker run --name elasticsearch \
-e "http.host=0.0.0.0" \
-e "ES_JAVA_OPTS=-Xms512m -Xmx512m" \
-d docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.10
$ docker run --link mongo --link elasticsearch \
-p 9000:9000 -p 12201:12201 -p 1514:1514 -p 5555:5555 \
-e GRAYLOG_HTTP_EXTERNAL_URI="http://127.0.0.1:9000/" \
-d graylog/graylog:3.3
Graylog GELF tcp input (running):
bind_address: 0.0.0.0
decompress_size_limit: 8388608
max_message_size: 2097152
number_worker_threads: 4
override_source: <empty>
port: 12201
recv_buffer_size: 1048576
tcp_keepalive: false
tls_cert_file: <empty>
tls_client_auth: disabled
tls_client_auth_cert_file: <empty>
tls_enable: false
tls_key_file: <empty>
tls_key_password:********
use_null_delimiter: true
As stated, when i run the java app and Graylog is running as a Docker container in the background, Graylog does not receive the logmessage i sent. However, when i type the following on my commandline (using Terminal on Mac), the message IS received by the Graylog RAW input:
$ echo "Testmessage" | nc localhost 5555
Does somebody got a clue what i'm doing wrong?

I found a solution, though i'm not sure what the exact cause of the problem was. The solution was to use a different Gelf appender. Instead of the one i mentioned above, i'm now using the following one:
<dependency>
<groupId>de.siegmar</groupId>
<artifactId>logback-gelf</artifactId>
<version>2.2.0</version>
</dependency>
That did the trick, but as i said, i'm unsure why the one i used earlier did not work.

How to speedup java maven build on Google Cloud Build (100s of dependencies)

I am using Google Cloud Build to build a java project which has 100s of dependencies. By default the local maven repository cache will be empty and it downloads all dependencies each time there is a build.
The google documentation only suggests "Caching directories with Google Cloud Storage" https://cloud.google.com/cloud-build/docs/speeding-up-builds but it takes a long time to sync 7000 files (which means the build is slower)
just one dependency is 5 files
repository/org/mockito
repository/org/mockito/mockito-core
repository/org/mockito/mockito-core/2.15.0
repository/org/mockito/mockito-core/2.15.0/mockito-core-2.15.0.jar
repository/org/mockito/mockito-core/2.15.0/mockito-core-2.15.0.jar.sha1
repository/org/mockito/mockito-core/2.15.0/mockito-core-2.15.0.pom
repository/org/mockito/mockito-core/2.15.0/mockito-core-2.15.0.pom.sha1
repository/org/mockito/mockito-core/2.15.0/_remote.repositories
An example cloudbuild.yaml file
steps:
- name: gcr.io/cloud-builders/gsutil
args: ['rsync', '-r', 'gs://my-mavencache-bucket/repository', '.']
- name: 'gcr.io/$PROJECT_ID/mvn'
args: ['package']
...
I would like to mount gs://my-mavencache-bucket at as a volume - but I dont see an option to do that

After much experimentation, this solution seems to work quite well. google-storage-wagon. This maven plugin provides a mechanism to read and publish maven artifacts from a google
data bucket
Maven pom.xml contains
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
...
<repositories>
<repository>
<id>my-repo-bucket-release</id>
<url>gs://bucket-ave-build-artifact/external</url>
<releases>
<enabled>true</enabled>
<!-- TODO figure out why checksums do not match when artifact pulled from GCP -->
<checksumPolicy>ignore</checksumPolicy>
</releases>
</repository>
</repositories>
<distributionManagement>
<snapshotRepository>
<id>my-repo-bucket-snapshot</id>
<url>gs://my-build-artifact-bucket/snapshot</url>
</snapshotRepository>
<repository>
<id>my-repo-bucket-release</id>
<url>gs://my-build-artifact-bucket/release</url>
</repository>
</distributionManagement>
...
<extensions>
<extension>
<groupId>com.gkatzioura.maven.cloud</groupId>
<artifactId>google-storage-wagon</artifactId>
<!-- version 1.8 seems to produce exception, ticket logged -->
<version>1.7</version>
</extension>
</extensions>
</build>
and cloudbuild.yaml is simply
steps:
- name: 'gcr.io/cloud-builders/mvn'
# -X here simply for verbose maven debugging
args: ['deploy', '-X']
this will:
maven publish artifacts to a data bucket
gs://my-build-artifact-bucket/release
download external dependencies
from gs://my-build-artifact-bucket/external (if they exist in this directory)

I found the package google-storage-wagon very nice, but lacking in terms of authentication and timing of the synchronization.
I implemented it myself like follows. For more information about service accounts refer to this answer: https://stackoverflow.com/a/56610260/1236401
So assuming you have your service account key.json file handy and you have the name of your SERVICE_ACCOUNT as well as a storage bucket BUCKET_PATH, this is the basic Dockerfile:
FROM maven:3.6.1-jdk-12
ENV MAVEN_PATH="/root/.m2" \
BUCKET_PATH="gs://mugen-cache/maven"
COPY key.json /key.json
# install gcloud sdk
RUN mkdir -p $MAVEN_PATH && \
yum install -y curl which && \
curl https://sdk.cloud.google.com | bash > /dev/null
ENV PATH="${PATH}:/root/google-cloud-sdk/bin" \
SERVICE_ACCOUNT="mugen-build#mugen.iam.gserviceaccount.com"
# authenticate service account and install crcmod - https://cloud.google.com/storage/docs/gsutil/addlhelp/CRC32CandInstallingcrcmod
RUN gcloud auth activate-service-account $SERVICE_ACCOUNT --key-file=/key.json && \
yum install -y gcc python-devel python-setuptools redhat-rpm-config
RUN curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py" && \
python get-pip.py && \
pip uninstall crcmod && \
pip install --no-cache-dir -U crcmod
RUN echo "Syncing m2 in..." && \
gsutil -q -m rsync -r $BUCKET_PATH $MAVEN_PATH && \
echo "Downloaded $(find $MAVEN_BUCKET -type f -name "*.pom" | wc -l) packages"
# ... build and stuff
RUN echo "Syncing m2 out..." && \
gsutil -q -m rsync -r $MAVEN_PATH $BUCKET_PATH
Some of the instructions here are specific to the base image (which is the REHL-based Oracle Linux Server), but you should be able to extract the important details in order to make it work in your case.