Develop Reference

Error creating a docker-compose connecting a java and a mysql containers

I am trying to connect the container of my springboot application with the container of a mysql image using docker-compose, however when I run docker-compose up my terminal starts a loop where it starts the spring application, try to connect with the MySQL container, fails and keep trying. The error that I get is com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failures
docker-compose file:
version: '3.8'
services:
mysqldb:
image: mysql
platform: linux/x86_64
env_file: ./.env
restart: always
environment:
- MYSQL_ROOT_PASSWORD=$MYSQLDB_ROOT_PASSWORD
- MYSQL_DATABASE=$MYSQLDB_DATABASE
ports:
- $MYSQLDB_LOCAL_PORT:$MYSQLDB_DOCKER_PORT
volumes:
- db:/var/lib/mysql
app:
depends_on:
- mysqldb
build: .
restart: always
env_file: ./.env
ports:
- $APP_LOCAL_PORT:$APP_DOCKER_PORT
environment:
- DB_HOST=mysqldb
- DB_USER=$MYSQLDB_USER
- DB_PASSWORD=$MYSQLDB_ROOT_PASSWORD
- DB_NAME=$MYSQLDB_DATABASE
- DB_PORT=$MYSQLDB_DOCKER_PORT
stdin_open: true
tty: true
volumes:
db:
.env:
MYSQLDB_USER=root
MYSQLDB_ROOT_PASSWORD=12345678
MYSQLDB_DATABASE=dronefeederdb
MYSQLDB_LOCAL_PORT=3306
MYSQLDB_DOCKER_PORT=3306
APP_LOCAL_PORT=8080
APP_DOCKER_PORT=8080
Application.yaml:
server:
port: 8080
spring:
datasource:
username: ${DB_USER}
password: ${DB_PASSWORD}
url: jdbc:mysql://${DB_HOST}:${DB_PORT}/${DB_NAME}
jpa:
hibernate:
ddl-auto: update
show-sql: true
open-in-view: false
#https://ia-tec-development.medium.com/lombok-e-spring-data-jpa-142398897733
security.user:
name: dronefeeder
password: dronefeeder
#https://www.baeldung.com/spring-boot-security-autoconfiguration
resilience4j.circuitbreaker:
configs:
default:
waitDurationInOpenState: 10s
failureRateThreshold: 10
#instances:
#estudantes:
#baseConfig: default
Dockerfile:
FROM openjdk:11.0-jdk as build-image
WORKDIR /app
COPY . .
RUN ./mvnw clean package -DskipTests
FROM openjdk:11.0-jre
COPY --from=build-image /app/target/*.jar /app/app.jar
EXPOSE 8080
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom", "-jar", "/app/app.jar"]
Repository link:
https://github.com/julia-baptista/dronefeeder/tree/docker-configuration

I believe the issue is your application's use of localhost for the SQL URL in the Application.yaml property file. Since your app runs on a container by itself it tries to look at localhost of the container, while your SQL server is in another container, with its own localhost. Localhost in docker container do not refer to the host, they refer to the localhost within the container itself. If you want to access the host machine, this is an excellent answer From inside of a Docker container, how do I connect to the localhost of the machine?
url: jdbc:mysql://localhost:3306/dronefeederdb
localhost should not be used, you need to use the sql continainer url.
The fastest option is to use host.docker.internal instead of localhost. But it's not the best.
Another quick option is to run the two containers on the same docker network. Define that in your compose file the same way as the volumes. Then set each container to that network. See Networking in Compose. Then you can set your SQL url to use the SQL container name instead of localhost. So this..
url: jdbc:mysql://localhost:3306/dronefeederdb becomes url: jdbc:mysql://mysql/dronefeederdb
Neither option is robust, since you're hardcoding the container name in the application property file. A better solution is to have an environment variable in your webApp image that can accept the URL to the SQL server. Then you can provide the SQL location when running the container, or in your compose file (Environment variables in Compose). This way the SQL server can be anywhere.
Update:
There were a couple of issues in the compose and env files that caused mySQL container to fail startup. Thus the webApp was not able to connect.
Credentials
MYSQL_USER was set to root. mySql already creates the user root. You cannot create it again. I changed that to foo. See the Environment Variables section in the official docker image readme for more.
MYSQL_PASSWORD was not set. This is the password for the user your app will use. I set this to pass!123
The apps DB_PASSWORD was set to user root. That would have been ok if sql had started and it was using the root user I guess. But I changed that to the non-root user since were setting DB_USER=foo
Network was not defined
The two containers need to be on the same "docker network" if they are to run together in docker in the same machine. There's more to this which is beyond my experience. But in this case it needs to be on the same network for app to access mysqldb by its container name. I created dronefeederNet and added each container to it.
Files:
.env
MYSQLDB_USER=foo
MYSQLDB_PASSWORD=pass!123
MYSQLDB_ROOT_PASSWORD=12345678
MYSQLDB_DATABASE=dronefeederdb
MYSQLDB_LOCAL_PORT=3307
MYSQLDB_DOCKER_PORT=3306
APP_LOCAL_PORT=8081
APP_DOCKER_PORT=8080
docker-compose.yml
version: '3.8'
services:
mysqldb:
image: mysql
platform: linux/x86_64
env_file: ./.env
restart: always
environment:
- MYSQL_USER=$MYSQLDB_USER
- MYSQL_PASSWORD=$MYSQLDB_PASSWORD
- MYSQL_ROOT_PASSWORD=$MYSQLDB_ROOT_PASSWORD
- MYSQL_DATABASE=$MYSQLDB_DATABASE
ports:
- $MYSQLDB_LOCAL_PORT:$MYSQLDB_DOCKER_PORT
volumes:
- db:/var/lib/mysql
networks:
- dronefeederNet
app:
depends_on:
- mysqldb
build: .
restart: always
env_file: ./.env
ports:
- $APP_LOCAL_PORT:$APP_DOCKER_PORT
environment:
- DB_HOST=mysqldb
- DB_USER=$MYSQLDB_USER
- DB_PASSWORD=$MYSQLDB_PASSWORD
- DB_NAME=$MYSQLDB_DATABASE
- DB_PORT=$MYSQLDB_DOCKER_PORT
stdin_open: true
tty: true
networks:
- dronefeederNet
volumes:
db:
networks:
dronefeederNet:
Give this a try and I hope it runs. I was able to start it up ok.

You need to add in the app definition block a depends on: sentence, to make docker compose to not boot the application until the database is up.
Check this documentation: Docker Compose Startup Order

Cannot access Github Action MSSQL database in tests

I have a Spring Boot application with MSSQL database. I would like to run github action and run tests for pull requests and merges to master. However I have problem with connecting to database from GA tests. My application uses YAML configuration and I have separate config file for CI tests.
Here is workflow:
name: Java CI with Maven
on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
jobs:
build:
runs-on: ubuntu-latest
services:
mssql:
image: mcr.microsoft.com/mssql/server:2019-latest
env:
SA_PASSWORD: myPassword
ACCEPT_EULA: 'Y'
DBNAME: test
ports:
- 1433:1433
steps:
- uses: actions/checkout#v3
- name: Set up JDK 11
uses: actions/setup-java#v3
with:
java-version: '11'
distribution: 'temurin'
cache: maven
- name: Build with Maven
run: mvn -ntp -U clean test -P junit-ci
And junit-ci config file:
spring:
datasource:
driverClassName: com.microsoft.sqlserver.jdbc.SQLServerDriver
url: jdbc:sqlserver://mssql:1433;database=test;
username: sa
password: myPassword
And here is error:
[main] ERROR com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Exception during pool initialization.
com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host mssql, port 1433 has failed. Error: "mssql. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:234)
at com.microsoft.sqlserver.jdbc.SQLServerException.ConvertConnectExceptionToSQLServerException(SQLServerException.java:285)
at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2434)

Suggest to use Testcontainers instead. This way it is the testcontainers Java library which takes care of starting a database to use it for Integration Testing. It is sooo much easier than the path you are currently on. IMHO.
Your GitHub Action YAML then becomes simpler. It will just be pure Maven actions. You also make your test code less dependent on GitHub Actions as your CI system.
Using Testcontainers has other advantages: Your test can have full control over the database (or the container in which it runs). For example you can have a test where you kill the container during the test, thereby similating the effect on your application on a database which suddenly is lost.
Btw: Strictly speaking, what you are attempting is not Unit Tests, but Integration Tests. Maven makes a distinction. It is advisable to make this distinction. Integration Tests are often quite heavy. By using Testcontainers approach the database container will only be started when you tell Maven to execute Integration Tests. In your example, the database container is always started, regardless if it is needed.

Microservice can not connect to MySQL running in Docker container

I am trying to run my Java microservice locally on MacOs and connect to MySQL db running in Docker container but I get an error:
java.sql.SQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
My docker-compose.yml looks like it:
version: "3.1"
services:
mysql:
image: mysql
container_name: mysql_v.0.1
command: --default-authentication-plugin=mysql_native_password
restart: always
env_file:
- .mysql-dev-env
ports:
- 33060:33060
However I am able to connect to mysql and create dbs trough docker itself with:
docker exec -it mysql_v.0.1
mysql -u root -p
Microservice has such application.dev.yml config:
db:
pool.size.maximum: 30
connection.idle.minimum: 10
widget:
url: jdbc:mysql://localhost:33060/dev_widget_platform?characterEncoding=UTF-8&useUnicode=yes&autoReconnect=true
user: root
password: pass
I am trying to connect to db via MySQL Workbench it has no success too.
Seems like I need to apply some network trick but i am new to docker and MySQl.

Unless you're using SSL locally, add this to the end of your connection string
&useSSL=false
So it becomes
url: jdbc:mysql://localhost:33060/dev_widget_platform?characterEncoding=UTF-8&useUnicode=yes&autoReconnect=true&useSSL=false

Flyway can't find migrations when database is run on docker container

I have been following this tutorial trying to set up a database using flyway migration scripts. The only difference with the tutorial is that I have been trying to use it in a Spring Boot application. For some reason, when I run "docker-compose up" I always get the the following logging in my terminal.
flyway_1 | Flyway Community Edition 7.5.3 by Redgate flyway_1 |
Database: jdbc:postgresql://postgres:5432/db-name (PostgreSQL 12.2)
flyway_1 | Successfully validated 0 migrations (execution time
00:00.041s) flyway_1 | WARNING: No migrations found. Are your
locations set up correctly? flyway_1 | Current version of schema
"public": << Empty Schema >> flyway_1 | Schema "public" is up to
date. No migration necessary.
However, I have a migration script under src/main/resources/db/migration. I am not sure why it is not able to find it, as it seems that is where flyway is supposed to look for them by default.
Here is my docker-compose.yml file
version: '3'
services:
flyway:
image: flyway/flyway:7.5.3
command: -configFiles=/flyway/conf/flyway.config -locations=filesystem:/flyway/sql -connectRetries=60 migrate
volumes:
- ${PWD}/src/main/java/resources/db/migration
- ${PWD}/docker-flyway.config:/flyway/conf/flyway.config
depends_on:
- postgres
postgres:
image: postgres:12.2
restart: always
ports:
- "5432:5432"
environment:
- POSTGRES_USER=example-username
- POSTGRES_PASSWORD=pass
- POSTGRES_DB=db-name
And here is my docker-flyway.config file.
flyway.url=jdbc:postgresql://postgres:5432/db-name
flyway.user=example-username
flyway.password=pass
flyway.baselineOnMigrate=false

The Flyway files need to have the .sql extension. Mentioned here: https://flywaydb.org/documentation/concepts/migrations#naming

I realized that I had one of the volume mappings written incorrectly in my docker-compose.yml file. I don't exactly understand the mapping itself, but after copying what was in this post it ran the migration script properly.

Postgres container connection refused

Docker-compose with 2 containers. 1st is a Postgres database and 2nd is a Java Spring Boot application. For running, I use further docker-compose config file:
docker-compose.yml
version: "3.7"
services:
db-service:
image: postgres
restart: always
volumes:
- /home/ec2-user/dbdata:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: 1
POSTGRES_DB: auth
ports:
- 5432:5432
auth-service:
build: .
restart: always
depends_on:
- db-service
links:
- db-service
ports:
- 80:80
I suppose to use /home/ec2-user/dbdata to containing database data and after all, data is created. successfully. And log of postgres container is:
PostgreSQL init process complete; ready for start
2021-01-07 01:36:16.786 UTC
[1] LOG: starting PostgreSQL 13.1 (Debian 13.1-1.pgdg100+1) on
x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2021-01-07 01:36:16.786 UTC [1] LOG: listening on
IPv4 address "0.0.0.0", port 5432 2021-01-07
01:36:16.786 UTC [1] LOG: listening on IPv6 address "::", port 5432
2021-01-07 01:36:16.790 UTC [1] LOG: listening on
Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-01-07 01:36:16.795 UTC [76] LOG: database system was shut down
at 2021-01-07 01:36:16 UTC 2021-01-07 01:36:16.800
UTC [1] LOG: database system is ready to accept connections
But Java app throws an error:
org.postgresql.util.PSQLException: Connection to 127.0.0.1:5432
refused. Check that the hostname and port are correct and that the
postmaster is accepting TCP/IP connections.
But in port mapping is 5432:5432.
And data source properties into Java app is:
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://127.0.0.1:5432/auth
spring.datasource.username=postgres
spring.datasource.password=1
What is can be the reason for this error?

It's not working because the java app is pointed to 127.0.0.1 which is local to the java container and postgres is not running in that container.
In your properties file change this line:
spring.datasource.url=jdbc:postgresql://127.0.0.1:5432/auth
to
spring.datasource.url=jdbc:postgresql://db-service:5432/auth
Using db-service as the host comes from the name of the service in your docker-compose.yml file. See the docker compose networking page for more information.

Another option, since both containers are linked (as per docker-compose file) is to use localhost (instead of 127.0.0.1) as the DB host in the connection string -> spring.datasource.url=jdbc:postgresql://localhost:5432/auth