I have a docker-compose configuration file with the following settings:
version: '3.6'
services:
mongo:
image: mongo
restart: always
ports:
- "27017:27017"
networks:
- devnetwork
discovery:
image: discovery
ports:
- "8761:8761"
networks:
- devnetwork
environment:
- SPRING_PROFILES_ACTIVE=dev
- DISCOVERY_SERVER=discovery
- DISCOVERY_PORT=8761
networks:
devnetwork:
After up compose I build and run the following dockerfile:
FROM openjdk:12
#RUN echo $(grep $(hostname) /etc/hosts | cut -f1) api >> /etc/hosts
ADD gubee-middleware/target/*.jar /usr/share/middlewareservice/middleware-service.jar
EXPOSE 9670
ENTRYPOINT ["/usr/bin/java", "-XX:+UnlockExperimentalVMOptions","-XX:+UseContainerSupport","-Dspring.profiles.active=dev", "-jar", "/usr/share/middlewareservice/middleware-service.jar"]
How do I make my port 9670 service dockerfile see other services that I start with docker-compose?
You need to attach the container to the network created by docker compose when you start it.
This can be done by passing --network=<network-name> (obtain by running docker network ls ) when executing your docker run command.
Once attached, your app will be able to reach the containers started by compose using their names as the host. Eg: http://mongo:27017
Related
So I wrote a simple spring boot rest app. Now I need to create several containers of it and simulate a delay in some so that other's can be accessed using load-balancer (programming team recruitment task). I feel like all should be happening locally but I might be wrong. I'm a beginner programmer.
here is what I tried in docker-compose
version: '3.3'
networks:
frontend:
driver: bridge
backend:
driver: bridge
services:
haproxy:
build:
context: ./
dockerfile: HAProxyDockerfile
container_name: haproxy
restart: always
ports:
- 80:80
depends_on:
- web1_app
- web2_app
networks:
- frontend
- backend
database:
image: mysql:5.7
restart: always
environment:
- MYSQL_ROOT_PASSWORD=pass123
- MYSQL_USER=user
- MYSQL_PASSWORD=pass123
- MYSQL_DATABASE=currencyconverterdb
networks:
- backend
- frontend
web1_app:
image: spring-boot-converter
container_name: web1_app
ports:
- 8082:8080
build:
context: ./
dockerfile: Dockerfile
depends_on:
- database
environment:
- DATABASE_HOST=database
- DATABASE_USER=user
- DATABASE_PASSWORD=pass123
- DATABASE_NAME=currencyconverterdb
- DATABASE_PORT=3306
networks:
- frontend
- backend
web2_app:
image: spring-boot-converter
container_name: web2_app
ports:
- 8081:8080
build:
context: ./
dockerfile: Dockerfile
depends_on:
- database
environment:
- DATABASE_HOST=database
- DATABASE_USER=user
- DATABASE_PASSWORD=pass123
- DATABASE_NAME=currencyconverterdb
- DATABASE_PORT=3306
networks:
- frontend
- backend
Here is how I build the HAProxy container
FROM haproxy:1.8
RUN apt-get update && apt-get install -y \
inetutils-ping \
curl && \
mkdir /srv/logs && \
touch /srv/logs/haproxy
COPY ./haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg
and here is what I write in haproxy.cfg
global
daemon
log /srv/logs/haproxy local0
log /srv/logs/haproxy local1 notice
defaults
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
option http-keep-alive
frontend http-in
bind :80
# springboot 1
acl host_web1 hdr(host) -i 127.0.0.1
use_backend web1_be if host_web1
backend web1_be
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
server node1 web1_app
server node1 web2_app
When i access localhost:80 I get 503 message with "No server is available to handle this request."
Would be thankful for any advices / solutions. I only got one day left.
The 503 "No server is available to handle this request." usually means:
your ACL does not work and you do not have a default_backend defined
your HAProxy is unable to reach any of its upstreams (servers)
your HAProxy is unable to resolve the names "web1_app" and "web2_app" and therefore is unable to reach any of its upstreams
there is no application running on port 80 of "web1_app"/"web2_app"
If you actually want to access with the name "localhost", change your ACL accordingly:
acl host_web1 hdr(host) -i localhost
For HAProxy to be able to do the matching on the host-header sent by your client.
I have a Java application consuming binlogs from Mysql database and interacting with another service which I have hosted on my localhost. How do I create a Docker image with all these services so that I can run them all at once?
From what I've heard, I need to use docker-compose. But I don't really know how to?
Do I create separate Docker containers and link their ports?
Do I create one container?
I have created one Dockerfile for the Java application:
FROM maslick/minimalka:jdk11
WORKDIR /app
EXPOSE 8080
COPY target/foster-maxwell-uber-jar.jar foster-maxwell.jar
COPY config.properties config.properties
CMD java -jar foster-maxwell.jar --config_type MAXWELL_UNSIGNED_KAAS --kaas_config config.yml --zk_timeout 1000
How do I go about it? Sorry for the noob question.
You should read docker-compose documentation: https://docs.docker.com/compose/compose-file/
For example, try with a docker-compose.yml file like this
version: '3.8'
services:
your-java-app1:
build:
context: . <-- (dir where you have your Dockerfile)
dockerfile: Dockerfile
image: yourjavaapp1image:1.0
container_name: your-java-app1
depends_on:
- db
your-java-app2:
build:
context: . <-- (dir where you have your Dockerfile_for_app2)
dockerfile: Dockerfile_for_app2
image: yourjavaapp2image:1.0
container_name: your-java-app2
depends_on:
- db
db:
image: mysql:5.7
volumes:
- db_data:/var/lib/mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: [put here your root pass]
MYSQL_DATABASE: [put here your db name]
MYSQL_USER: [put here your user]
MYSQL_PASSWORD: [put here your pass]
Once you have this defined your Dockerfiles, first, build images with your apps:
docker-compose build
Then, run images (container creations)
docker-compose up -d
I have three services.
Config server
Eureka server
api-gateway
If I run them individually it's working fine. Then I am trying to introduce docker on above services. So I have prepare 3 dockerfile for each services:
VOLUME /tmp
ADD config-server/build/libs/config-server-0.0.1-SNAPSHOT.jar config-server-0.0.1-SNAPSHOT.jar
CMD ["java", "-jar", "config-server-0.0.1-SNAPSHOT.jar"]
VOLUME /var/lib/config-repo
EXPOSE 10270
FROM java:8
VOLUME /tmp
ADD eureka-server/build/libs/eureka-server-1.0-SNAPSHOT.jar eureka-server-1.0-SNAPSHOT.jar
CMD ["java","-jar","eureka-server-1.0-SNAPSHOT.jar"]
EXPOSE 10210
FROM java:8
VOLUME /tmp
ADD api-gateway/build/libs/api-gateway-0.0.1-SNAPSHOT.jar api-gateway-0.0.1-SNAPSHOT.jar
RUN bash -c 'touch /api-gateway-0.0.1-SNAPSHOT.jar'
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/api-gateway-0.0.1-SNAPSHOT.jar"]
And then I have prepared my docker-compose file:
version: '3'
services:
eureka-server:
restart: always
container_name: eureka-server
build:
context: .
dockerfile: eureka-server/Dockerfile_eureka_server
expose:
- 10210
ports:
- 10210:10210
networks:
- servicenet
config-server:
restart: always
container_name: config-server
build:
context: .
dockerfile: config-server/Dockerfile_config_server
expose:
- 10270
ports:
- 10270:10270
healthcheck:
test: ["CMD", "curl", "-f", "http://config-server:10270"]
interval: 30s
timeout: 10s
retries: 5
networks:
- servicenet
api-gateway:
restart: on-failure
container_name: api-gateway
build:
context: .
dockerfile: api-gateway/Dockerfile_api_gateway
expose:
- 10200
ports:
- 10200:10200
networks:
- servicenet
links:
- config-server
- eureka-server
networks:
servicenet:
driver: bridge
But api-gateway start before config-server completely start its service. Thats why api gateway start on 8080 port and looking for eureka server at host localhost and port 8621. though its not getting this ports and hosts in docker its continuing looking for eureka server but not again fetching config from config server. Is there anything wrong with my configuration?
My application.properties file on github like this
server.port=10200
#Eureka configuration
eureka.instance.metadataMap.instanceId=${vcap.application.instance_id:${spring.application.name}:${spring.application.instance_id:${random.value}}}
lombok.equalsAndHashCode.callSuper = call
eureka.instance.instanceId=${spring.application.name}:${spring.application.instance_id:${random.value}}
eureka.client.registryFetchIntervalSeconds=5
eureka.client.serviceUrl.defaultZone=http://eureka-server:10210/eureka
spring.cloud.service-registry.auto-registration.enabled=true
eureka.client.enabled=true
eureka.client.serviceUrl.registerWithEureka=true
lombok.anyConstructor.suppressConstructorProperties = true
#Zuul Configuration
# A prefix that can added to beginning of all requests.
#zuul.prefix=/api
# Disable accessing services using service name (i.e. gallery-service).
# They should be only accessed through the path defined below.
zuul.ignored-services=*
# Map paths to services
zuul.routes.gallery-service.path=/gallery/**
zuul.routes.gallery-service.service-id=gallery-manager
zuul.routes.image-service.path=/image/**
zuul.routes.image-service.service-id=image-service
zuul.routes.book-manager.path=/book-manager/**
zuul.routes.book-manager.service-id=book-manager
zuul.routes.auth-service.path=/auth/**
zuul.routes.auth-service.service-id=auth-manager
zuul.routes.remote-library.path=/remote/**
zuul.routes.remote-library.service-id=remote-library
#zuul.routes.auth-service.strip-prefix=false
# Exclude authorization from sensitive headers
zuul.routes.auth-service.sensitive-headers=Cookie,Set-Cookie
NB: If I try with other services rather than api-gateway its working fine. I am using zuul proxy for api gateway service.
A quick fix would be to add a 'depends_on' clause so that the api-service depends on the config server - that way the api-service won't start until the config server is up.
To make this workable we have to wait still other services are up. So I have written a script for waiting and rewrite docker compose like this
version: '3'
services:
eureka-server:
restart: always
container_name: eureka-server
build:
context: .
dockerfile: eureka-server/Dockerfile_eureka_server
expose:
- 10210
ports:
- 10210:10210
networks:
- servicenet
config-server:
restart: always
container_name: config-server
build:
context: .
dockerfile: config-server/Dockerfile_config_server
expose:
- 10270
ports:
- 10270:10270
healthcheck:
test: ["CMD", "curl", "-f", "http://config-server:10270"]
interval: 30s
timeout: 10s
retries: 5
networks:
- servicenet
api-gateway:
restart: always
container_name: api-gateway
build:
context: .
dockerfile: api-gateway/Dockerfile_api_gateway
expose:
- 10200
ports:
- 10200:10200
networks:
- servicenet
links:
- config-server
depends_on:
- config-server
command: ["./wait-for-it.sh", "config-server:10270","--","./wait-for-it.sh", "eureka-server:10210", "--", "java","-jar", "api-gateway-0.0.1-SNAPSHOT.jar"]
And also rewrite docker files like this(for api-gateway)
FROM java:8
VOLUME /tmp
ADD api-gateway/build/libs/api-gateway-0.0.1-SNAPSHOT.jar api-gateway-0.0.1-SNAPSHOT.jar
ADD wait-for-it.sh wait-for-it.sh
RUN chmod +x wait-for-it.sh
2 Containers, one Java application and the second mongodb.
If I run my java app locally and mongodb in a container, it connects but if both run inside a container, java app can't connect to mongodb.
docker-compose file is as follows, am I missing something
version: "3"
services:
user:
image: jboss/wildfly
container_name: "user"
restart: always
ports:
- 8081:8080
- 65194:65193
volumes:
- ./User/target/User.war:/opt/jboss/wildfly/standalone/deployments/User.war
environment:
- JAVA_OPTS=-agentlib:jdwp=transport=dt_socket,address=0.0.0.0:65193,suspend=n,server=y -Djava.net.preferIPv4Stack=true
- MONGO_HOST=localhost
- MONGO_PORT=27017
- MONGO_USERNAME=myuser
- MONGO_PASSWORD=mypass
- MONGO_DATABASE=mydb
- MONGO_AUTHDB=admin
command: >
bash -c "/opt/jboss/wildfly/bin/add-user.sh admin Admin#007 --silent && /opt/jboss/wildfly/bin/standalone.sh -b 0.0.0.0 -bmanagement 0.0.0.0"
links:
- mongo
mongo:
image: mongo:4.0.10
container_name: mongo
restart: always
volumes:
- ./assets:/docker-entrypoint-initdb.d/
environment:
- MONGO_INITDB_ROOT_USERNAME=myuser
- MONGO_INITDB_ROOT_PASSWORD=mypass
ports:
- 27017:27017
- 27018:27018
- 27019:27019
Edit
I'm also confused about the following.
links:
- mongo
depends_on:
- mongo
At 2019 July, official docker documentation :
Source: https://docs.docker.com/compose/compose-file/#links
Solution #1 : environment file before start
Basically We centralize all configurations in a file with environment variables and execute it before docker-compose up
The following approach helped me in these scenarios:
Your docker-compose.yml has several containers with complex dependencies between them
Some of your services in your docker-compose needs to connect to another process in the same machine. This process could be a docker container or not.
You need to share variables between several docker-compose files like host, passwords, etc
Steps
1.- Create one file to centralize configurations
This file could be named: /env/company_environments with extension or not.
export MACHINE_HOST=$(hostname -I | awk '{print $1}')
export GLOBAL_LOG_PATH=/my/org/log
export MONGO_PASSWORD=mypass
export MY_TOKEN=123456
2.- Use the env variables in your docker-compose.yml
container A
app_who_needs_mongo:
environment:
- MONGO_HOST=$MACHINE_HOST
- MONGO_PASSWORD=$MONGO_PASSWORD
- TOKEN=$MY_TOKEN
- LOG_PATH=$GLOBAL_LOG_PATH/app1
container B
app_who_needs_another_db_in_same_host:
environment:
- POSTGRESS_HOST=$MACHINE_HOST
- LOG_PATH=$GLOBAL_LOG_PATH/app1
3.- Startup your containers
Just add source before docker-compose commands:
source /env/company_environments
docker-compose up -d
Solution #2 : host.docker.internal
https://stackoverflow.com/a/63207679/3957754
Basically use a feature of docker in which host.docker.internal could be used as the ip of the server in which your docker-compose has started several containers
You probably cant connect because you set the MONGO_HOST as localhost and mongo is a linked service.
In order to use linked services network, you must specify the MONGO_HOST as the name of the service - mongo, like that:
MONGO_HOST=mongo
I am not able to connect to MySQL using Docker container name in connection string but can connect with localhost.
docker-compose:
mysql-docker-container:
image: mysql:latest
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=test
- MYSQL_PASSWORD=test
ports:
- 3306:3306
volumes:
- /data/mysql
app:
image: app
build:
context: ./
dockerfile: Dockerfile
depends_on:
- mysql-docker-container
links:
- mysql-docker-container:mysql-docker-container
ports:
- 9090:9090
volumes:
- /data/p2c-app
environment:
- DATABASE_HOST=mysql-docker-container
- DATABASE_USER=testuser
- DATABASE_PASWORD=testuser
- DATABASE_NAME=test
- DATABASE_PORT=3306
spring.datasource.url=jdbc:mysql://localhost:3306/test?useSSL=false&allowPublicKeyRetrieval=true
Above works, but I want with container name like below - I am getting CONNECTION REFUSED
spring.datasource.url=jdbc:mysql://mysql-docker-container:3306/test?useSSL=false&allowPublicKeyRetrieval=true
What am I doing wrong?
You can update /etc/hosts if localhost connection works.
127.0.0.1 localhost mysql-docker-container
To check whether mysql-docker-container is reachable from the app container you can open a tty and ping.
docker exec -it app_container_name bash
ping mysql-docker-container
Everything is ok.
It seems the database service is up but the mysql is in the startup process.
And the app service starts up then and can not reach to the database.
There are some workarounds for this situation.
But the simple one is that you add below to app service.
restart: on-failure
And note that depends_on section just means in docker container context not in underlying services.