Docker Compose - Flyway - Unable to obtain Jdbc connection from DataSource - java

I'm traying to deploy a microserivice and a flyway service with Docker Compose in Ubuntu. The docker-compose.yml looks like this:
version: '2'
services:
mysqldb:
image: mysql:5.6.26
environment:
MYSQL_USER: user
MYSQL_PASSWORD: password
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: base
ports:
- "3306:3306"
flyway-service-i:
image: mialk/flyway-service
volumes:
- "../resources/db/migration:/migrations/ro"
depends_on:
- mysqldb
links:
- mysqldb
command: migrate -url=jdbc:mysql://mysqldb:3306/base -user=user -password=password -baselineOnMigrate=true - locations='filesystem:/migrations'
service1:
image: my/service
ports:
- "8080:8080"
links:
- mysqldb
environment:
- SPRING_DATASOURCE_URL=jdbc:mysql://mysqldb:3306/base
- SPRING_DATASOURCE_USERNAME=user
- SPRING_DATASOURCE_PASSWORD=password
But when I run the command: sudo docker-compose up, I have this message:
flyway-service-i_1 | ERROR: Unable to obtain Jdbc connection from DataSource (jdbc:mysql://mysqldb:3306/base) for user 'user': Host '172.18.0.4' is not allowed to connect to this MySQL server
Each time that I run the command, I get a different host, e.g:
ERROR: Unable to obtain Jdbc connection from DataSource (jdbc:mysql://mysqldb:3306/base) for user 'user': Host '172.18.0.6' is not allowed to connect to this MySQL server
The data base was created in that way:
CREATE DATABASE base CHARACTER SET utf8 COLLATE utf8_bin;
GRANT ALL PRIVILEGES ON base.* To 'user'#'localhost';
The ifconfig command shows that Docker has the IP 172.17.0.1.
I don't know why flyway can't connect to the data base, and why the host changes en each call,can you help me please?
Thanks! :)

Issue
In this line
GRANT ALL PRIVILEGES ON base.* To 'user'#'localhost';
you grant priviledges to user on localhost but your service1 and flyaway-service-1 services are running in their own separate containers and therefore are not running on localhost (with respect to the mysqldb container). If these services were running inside the mysqldb container then maybe they could be running on localhost.
Possible Resolution?
You may want to try using the hostname of the services/containers you're granting access to instead of localhost?
ie :
GRANT ALL PRIVILEGES ON base.* To 'user'#'service1';
Docker Networks instead of Container Links
Also, I would suggest using Docker's new networking feature instead of the legacy container linking method. See rationale behind docker compose "links" order

Related

Error creating a docker-compose connecting a java and a mysql containers

I am trying to connect the container of my springboot application with the container of a mysql image using docker-compose, however when I run docker-compose up my terminal starts a loop where it starts the spring application, try to connect with the MySQL container, fails and keep trying. The error that I get is com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failures
docker-compose file:
version: '3.8'
services:
mysqldb:
image: mysql
platform: linux/x86_64
env_file: ./.env
restart: always
environment:
- MYSQL_ROOT_PASSWORD=$MYSQLDB_ROOT_PASSWORD
- MYSQL_DATABASE=$MYSQLDB_DATABASE
ports:
- $MYSQLDB_LOCAL_PORT:$MYSQLDB_DOCKER_PORT
volumes:
- db:/var/lib/mysql
app:
depends_on:
- mysqldb
build: .
restart: always
env_file: ./.env
ports:
- $APP_LOCAL_PORT:$APP_DOCKER_PORT
environment:
- DB_HOST=mysqldb
- DB_USER=$MYSQLDB_USER
- DB_PASSWORD=$MYSQLDB_ROOT_PASSWORD
- DB_NAME=$MYSQLDB_DATABASE
- DB_PORT=$MYSQLDB_DOCKER_PORT
stdin_open: true
tty: true
volumes:
db:
.env:
MYSQLDB_USER=root
MYSQLDB_ROOT_PASSWORD=12345678
MYSQLDB_DATABASE=dronefeederdb
MYSQLDB_LOCAL_PORT=3306
MYSQLDB_DOCKER_PORT=3306
APP_LOCAL_PORT=8080
APP_DOCKER_PORT=8080
Application.yaml:
server:
port: 8080
spring:
datasource:
username: ${DB_USER}
password: ${DB_PASSWORD}
url: jdbc:mysql://${DB_HOST}:${DB_PORT}/${DB_NAME}
jpa:
hibernate:
ddl-auto: update
show-sql: true
open-in-view: false
#https://ia-tec-development.medium.com/lombok-e-spring-data-jpa-142398897733
security.user:
name: dronefeeder
password: dronefeeder
#https://www.baeldung.com/spring-boot-security-autoconfiguration
resilience4j.circuitbreaker:
configs:
default:
waitDurationInOpenState: 10s
failureRateThreshold: 10
#instances:
#estudantes:
#baseConfig: default
Dockerfile:
FROM openjdk:11.0-jdk as build-image
WORKDIR /app
COPY . .
RUN ./mvnw clean package -DskipTests
FROM openjdk:11.0-jre
COPY --from=build-image /app/target/*.jar /app/app.jar
EXPOSE 8080
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom", "-jar", "/app/app.jar"]
Repository link:
https://github.com/julia-baptista/dronefeeder/tree/docker-configuration
I believe the issue is your application's use of localhost for the SQL URL in the Application.yaml property file. Since your app runs on a container by itself it tries to look at localhost of the container, while your SQL server is in another container, with its own localhost. Localhost in docker container do not refer to the host, they refer to the localhost within the container itself. If you want to access the host machine, this is an excellent answer From inside of a Docker container, how do I connect to the localhost of the machine?
url: jdbc:mysql://localhost:3306/dronefeederdb
localhost should not be used, you need to use the sql continainer url.
The fastest option is to use host.docker.internal instead of localhost. But it's not the best.
Another quick option is to run the two containers on the same docker network. Define that in your compose file the same way as the volumes. Then set each container to that network. See Networking in Compose. Then you can set your SQL url to use the SQL container name instead of localhost. So this..
url: jdbc:mysql://localhost:3306/dronefeederdb becomes url: jdbc:mysql://mysql/dronefeederdb
Neither option is robust, since you're hardcoding the container name in the application property file. A better solution is to have an environment variable in your webApp image that can accept the URL to the SQL server. Then you can provide the SQL location when running the container, or in your compose file (Environment variables in Compose). This way the SQL server can be anywhere.
Update:
There were a couple of issues in the compose and env files that caused mySQL container to fail startup. Thus the webApp was not able to connect.
Credentials
MYSQL_USER was set to root. mySql already creates the user root. You cannot create it again. I changed that to foo. See the Environment Variables section in the official docker image readme for more.
MYSQL_PASSWORD was not set. This is the password for the user your app will use. I set this to pass!123
The apps DB_PASSWORD was set to user root. That would have been ok if sql had started and it was using the root user I guess. But I changed that to the non-root user since were setting DB_USER=foo
Network was not defined
The two containers need to be on the same "docker network" if they are to run together in docker in the same machine. There's more to this which is beyond my experience. But in this case it needs to be on the same network for app to access mysqldb by its container name. I created dronefeederNet and added each container to it.
Files:
.env
MYSQLDB_USER=foo
MYSQLDB_PASSWORD=pass!123
MYSQLDB_ROOT_PASSWORD=12345678
MYSQLDB_DATABASE=dronefeederdb
MYSQLDB_LOCAL_PORT=3307
MYSQLDB_DOCKER_PORT=3306
APP_LOCAL_PORT=8081
APP_DOCKER_PORT=8080
docker-compose.yml
version: '3.8'
services:
mysqldb:
image: mysql
platform: linux/x86_64
env_file: ./.env
restart: always
environment:
- MYSQL_USER=$MYSQLDB_USER
- MYSQL_PASSWORD=$MYSQLDB_PASSWORD
- MYSQL_ROOT_PASSWORD=$MYSQLDB_ROOT_PASSWORD
- MYSQL_DATABASE=$MYSQLDB_DATABASE
ports:
- $MYSQLDB_LOCAL_PORT:$MYSQLDB_DOCKER_PORT
volumes:
- db:/var/lib/mysql
networks:
- dronefeederNet
app:
depends_on:
- mysqldb
build: .
restart: always
env_file: ./.env
ports:
- $APP_LOCAL_PORT:$APP_DOCKER_PORT
environment:
- DB_HOST=mysqldb
- DB_USER=$MYSQLDB_USER
- DB_PASSWORD=$MYSQLDB_PASSWORD
- DB_NAME=$MYSQLDB_DATABASE
- DB_PORT=$MYSQLDB_DOCKER_PORT
stdin_open: true
tty: true
networks:
- dronefeederNet
volumes:
db:
networks:
dronefeederNet:
Give this a try and I hope it runs. I was able to start it up ok.
You need to add in the app definition block a depends on: sentence, to make docker compose to not boot the application until the database is up.
Check this documentation: Docker Compose Startup Order

Microservice can not connect to MySQL running in Docker container

I am trying to run my Java microservice locally on MacOs and connect to MySQL db running in Docker container but I get an error:
java.sql.SQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
My docker-compose.yml looks like it:
version: "3.1"
services:
mysql:
image: mysql
container_name: mysql_v.0.1
command: --default-authentication-plugin=mysql_native_password
restart: always
env_file:
- .mysql-dev-env
ports:
- 33060:33060
However I am able to connect to mysql and create dbs trough docker itself with:
docker exec -it mysql_v.0.1
mysql -u root -p
Microservice has such application.dev.yml config:
db:
pool.size.maximum: 30
connection.idle.minimum: 10
widget:
url: jdbc:mysql://localhost:33060/dev_widget_platform?characterEncoding=UTF-8&useUnicode=yes&autoReconnect=true
user: root
password: pass
I am trying to connect to db via MySQL Workbench it has no success too.
Seems like I need to apply some network trick but i am new to docker and MySQl.
Unless you're using SSL locally, add this to the end of your connection string
&useSSL=false
So it becomes
url: jdbc:mysql://localhost:33060/dev_widget_platform?characterEncoding=UTF-8&useUnicode=yes&autoReconnect=true&useSSL=false

docker-compose java application connection to mongodb

2 Containers, one Java application and the second mongodb.
If I run my java app locally and mongodb in a container, it connects but if both run inside a container, java app can't connect to mongodb.
docker-compose file is as follows, am I missing something
version: "3"
services:
user:
image: jboss/wildfly
container_name: "user"
restart: always
ports:
- 8081:8080
- 65194:65193
volumes:
- ./User/target/User.war:/opt/jboss/wildfly/standalone/deployments/User.war
environment:
- JAVA_OPTS=-agentlib:jdwp=transport=dt_socket,address=0.0.0.0:65193,suspend=n,server=y -Djava.net.preferIPv4Stack=true
- MONGO_HOST=localhost
- MONGO_PORT=27017
- MONGO_USERNAME=myuser
- MONGO_PASSWORD=mypass
- MONGO_DATABASE=mydb
- MONGO_AUTHDB=admin
command: >
bash -c "/opt/jboss/wildfly/bin/add-user.sh admin Admin#007 --silent && /opt/jboss/wildfly/bin/standalone.sh -b 0.0.0.0 -bmanagement 0.0.0.0"
links:
- mongo
mongo:
image: mongo:4.0.10
container_name: mongo
restart: always
volumes:
- ./assets:/docker-entrypoint-initdb.d/
environment:
- MONGO_INITDB_ROOT_USERNAME=myuser
- MONGO_INITDB_ROOT_PASSWORD=mypass
ports:
- 27017:27017
- 27018:27018
- 27019:27019
Edit
I'm also confused about the following.
links:
- mongo
depends_on:
- mongo
At 2019 July, official docker documentation :
Source: https://docs.docker.com/compose/compose-file/#links
Solution #1 : environment file before start
Basically We centralize all configurations in a file with environment variables and execute it before docker-compose up
The following approach helped me in these scenarios:
Your docker-compose.yml has several containers with complex dependencies between them
Some of your services in your docker-compose needs to connect to another process in the same machine. This process could be a docker container or not.
You need to share variables between several docker-compose files like host, passwords, etc
Steps
1.- Create one file to centralize configurations
This file could be named: /env/company_environments with extension or not.
export MACHINE_HOST=$(hostname -I | awk '{print $1}')
export GLOBAL_LOG_PATH=/my/org/log
export MONGO_PASSWORD=mypass
export MY_TOKEN=123456
2.- Use the env variables in your docker-compose.yml
container A
app_who_needs_mongo:
environment:
- MONGO_HOST=$MACHINE_HOST
- MONGO_PASSWORD=$MONGO_PASSWORD
- TOKEN=$MY_TOKEN
- LOG_PATH=$GLOBAL_LOG_PATH/app1
container B
app_who_needs_another_db_in_same_host:
environment:
- POSTGRESS_HOST=$MACHINE_HOST
- LOG_PATH=$GLOBAL_LOG_PATH/app1
3.- Startup your containers
Just add source before docker-compose commands:
source /env/company_environments
docker-compose up -d
Solution #2 : host.docker.internal
https://stackoverflow.com/a/63207679/3957754
Basically use a feature of docker in which host.docker.internal could be used as the ip of the server in which your docker-compose has started several containers
You probably cant connect because you set the MONGO_HOST as localhost and mongo is a linked service.
In order to use linked services network, you must specify the MONGO_HOST as the name of the service - mongo, like that:
MONGO_HOST=mongo

Not able to connect to MySQL with Docker container name but can connect with localhost

I am not able to connect to MySQL using Docker container name in connection string but can connect with localhost.
docker-compose:
mysql-docker-container:
image: mysql:latest
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=test
- MYSQL_PASSWORD=test
ports:
- 3306:3306
volumes:
- /data/mysql
app:
image: app
build:
context: ./
dockerfile: Dockerfile
depends_on:
- mysql-docker-container
links:
- mysql-docker-container:mysql-docker-container
ports:
- 9090:9090
volumes:
- /data/p2c-app
environment:
- DATABASE_HOST=mysql-docker-container
- DATABASE_USER=testuser
- DATABASE_PASWORD=testuser
- DATABASE_NAME=test
- DATABASE_PORT=3306
spring.datasource.url=jdbc:mysql://localhost:3306/test?useSSL=false&allowPublicKeyRetrieval=true
Above works, but I want with container name like below - I am getting CONNECTION REFUSED
spring.datasource.url=jdbc:mysql://mysql-docker-container:3306/test?useSSL=false&allowPublicKeyRetrieval=true
What am I doing wrong?
You can update /etc/hosts if localhost connection works.
127.0.0.1 localhost mysql-docker-container
To check whether mysql-docker-container is reachable from the app container you can open a tty and ping.
docker exec -it app_container_name bash
ping mysql-docker-container
Everything is ok.
It seems the database service is up but the mysql is in the startup process.
And the app service starts up then and can not reach to the database.
There are some workarounds for this situation.
But the simple one is that you add below to app service.
restart: on-failure
And note that depends_on section just means in docker container context not in underlying services.

grails/mysql with docker compose

I am trying to run a grails app in docker and keep running in some mysql connection problems. I can't figure out where the problem is.
This is my docker-compose file
version: "2"
services:
db:
image: mysql:5.7
volumes:
- db_data:/var/lib/mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: root2017
MYSQL_DATABASE: dbname
MYSQL_USER: Dbuser
MYSQL_PASSWORD: passw
grails:
depends_on:
- db
ports:
- "9001:9001"
restart: always
environment:
DB_HOST: db:3306
DB_PASSWORD: passw
volumes:
db_data:
The grails app does not start with the following error:
ERROR 18:08:05 org.apache.tomcat.jdbc.pool.ConnectionPool - Unable to create initial connections of pool.
grails_1 | com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
You will need to configure your datasource to point to "jdbc:mysql://db:3306/dbname"
You can do this with external config file for production env.
Or you can read the values for host and db name from system environments.
See Deploying grails application war to tomcat with docker and docker compose

Categories

Resources