I have a new Spring Boot application that I just finished and am trying to deploy it to Docker. Inside the container the application works fine. It uses ports 9000 for user facing requests and 9100 for administrative tasks like health checks. When I start a docker instance and try to access port 9000 I get the following error:
curl: (56) Recv failure: Connection reset by peer
After a lot of experimentation (via curl), I confirmed in with several different configurations that the application functions fine inside the container, but when I try to map ports to the host it doesn't connect. I've tried starting it with the following commands. None of them allow me to access the ports from the host.
docker run -P=true my-app
docker run -p 9000:9000 my-app
The workaround
The only approach that works is using the --net host option, but this doesn't allow me to run more than one container on that host.
docker run -d --net=host my-app
Experiments with ports and expose
I've used various versions of the Dockerfile exposing different ports such as 9000 and 9100 or just 9000. None of that helped. Here's my latest version:
FROM ubuntu
MAINTAINER redacted
RUN apt-get update
RUN apt-get install openjdk-7-jre-headless -y
RUN mkdir -p /opt/app
WORKDIR /opt/app
ADD ./target/oauth-authentication-1.0.0.jar /opt/app/service.jar
ADD config.properties /opt/app/config.properties
EXPOSE 9000
ENTRYPOINT java -Dext.properties.dir=/opt/app -jar /opt/app/service.jar
Hello World works
To make sure I can run a Spring Boot application, I tried Simplest-Spring-Boot-MVC-HelloWorld and it worked fine.
Netstat Results
I've used netstat to do port scans from the host and from the container:
From the host
root#my-docker-host:~# nmap 172.17.0.71 -p9000-9200
Starting Nmap 6.40 ( http://nmap.org ) at 2014-11-14 19:19 UTC Nmap
scan report for my-docker-host (172.17.0.71)
Host is up (0.0000090s latency).
Not shown: 200 closed ports
PORT STATE SERVICE
9100/tcp open jetdirect
MAC Address: F2:1A:ED:F4:07:7A (Unknown)
Nmap done: 1 IP address (1 host up) scanned in 1.48 seconds
From the container
root#80cf20c0c1fa:/opt/app# nmap 127.0.0.1 -p9000-9200
Starting Nmap 6.40 ( http://nmap.org ) at 2014-11-14 19:20 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.0000070s latency).
Not shown: 199 closed ports
PORT STATE SERVICE
9000/tcp open cslistener
9100/tcp open jetdirect
Nmap done: 1 IP address (1 host up) scanned in 2.25 seconds
The container is using Ubuntu
The hosts I've replicated this are Centos and Ubuntu.
This SO question seems similar but had very few details and no answers, so I thought I'd try to document my scenario a bit more.
I had a similar problem, in which specifying a host IP address as '127.0.0.1' wouldn't properly forward the port to the host.
Setting the web server's IP to '0.0.0.0' fixes the problem
eg - for my Node app - the following doesn't work
app.listen(3000, '127.0.0.1')
Where as the following does work:
app.listen(3000, '0.0.0.0')
Which I guess means that docker, by default, is exposing 0.0.0.0:containerPort -> local port
You should run with docker run -P to get the ports to map automatically to the same values to set in the Dockerfile.. Please see http://docs.docker.com/reference/run/#expose-incoming-ports
Related
Hello I have an app in Spring Boot and I am exposing some metrics on Prometheus. My next goal is to provide these metrics on Grafana in order to obtain some beautiful dashboards. I am using docker on WSL Ubuntu and typed the next commands for Prometheus and Grafana:
docker run -d --name=prometheus -p 9090:9090 -v /mnt/d/Projects/Msc-Thesis-Project/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus --config.file=/etc/prometheus/prometheus.yml
docker run -d --name=grafana -p 3000:3000 grafana/grafana
Below I am giving you the Prometheus dashboard in my browser and as you can see, everything is up and running. My problem is in Grafana configuration where I have to configure Prometheus as Data Source.
In the field URL I am providing the http://localhost:9090 but I am getting the following error:
Error reading Prometheus: Post "http://localhost:9090/api/v1/query": dial tcp 127.0.0.1:9090: connect: connection refused
I've searched everywhere and saw some workarounds that don't apply to me. To be specific I used the following: http://host.docker.internal:9090, http://server-ip:9090 and of course my system's IP address via the ipconfig command http://<ip_address>:9090. Nothing works!!!
I am not using docker-compose but just a prometheus.yml file which is as follows.
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: 'prometheus'
scrape_interval: 5s
static_configs:
- targets: ['localhost:9090']
- job_name: 'Spring Boot Application input'
metrics_path: '/actuator/prometheus'
scrape_interval: 2s
scheme: http
static_configs:
- targets: ['192.168.1.233:8080']
labels:
application: "MSc Project Thesis"
Can you advise me something?
You can use the docker inspect command to find the IP address of the Prometheus container and then replace the localhost word with it.
I'll suggest you to use docker-compose, which better supports in DNS resolving and your issues of localhost will get resolved.
It works for https://stackoverflow.com/a/74061034/4841138
Also, if you deploy the stack by docker compose and all dockers are in same network, you can do that:
URL: http://prometheus:9090
In above, prometheus is the domain name of the prometheus docker, which can be resolved by all dockers within same network.
I have installed Apache GUI on Linux Debian in the following directory
/usr/local/apachegui/ApacheGUI/bin
by the following instruction as shown on this webpage:
https://www.howtoforge.com/tutorial/ubuntu-apache-gui/
At the end of the instruction, it's says run the following command:
sudo ./run.sh
when I run the above command I get the following output but with an error
./run.sh: 1: [: -ne: unexpected operator
Bottom of the terminal window its mention tomcat started and as in above mention tutorial webpage claims after running run.sh command
Your ApacheGUI is now starting and listening on port 9999.
Next, open your web browser and type the URL >http://your-server-ip:9999/ApacheGUI. You will >be redirected to the following page (ApachiGUI home page)
In my case when I am typing the above URL with my IP address or localhost my Chrome browser print: output site can not be reach
So I searched to check if port 9999 is enabled and listening in my case I was unable to see port 9999 in my list of ports so ran the following commands and the terminal window didn't produce any output from here I assumed port 9999 is close
pi#Home:~ $ ss -na | grep :9999
After using the above command terminal window return back to ip#home:$
now I open a separate terminal window and ping port 9999 and as result, This terminal window hang now it is clear the port 9999 is not open in my server so to open the close port I run the following command
Open port command:
sudo iptables -A INPUT -p tcp --dport 9999 -j ACCEPT
To update the firewall rules, I restart the iptables service
sudo iptables -L
output:
pi#Home:~ $ sudo iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- anywhere anywhere tcp dpt:9999
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Now I check again a list of open ports on my server and I still can't see port 9999
After taking all these steps executed following command:
pi#Home:/usr/local/apachegui/ApacheGUI/bin $ ./run.sh
Output:
Using CATALINA_BASE: /usr/local/apachegui/ApacheGUI/tomcat Using
CATALINA_HOME: /usr/local/apachegui/ApacheGUI/tomcat Using
CATALINA_TMPDIR: /usr/local/apachegui/ApacheGUI/tomcat/temp Using
JRE_HOME: /usr Using CLASSPATH:
/usr/local/apachegui/ApacheGUI/tomcat/bin/bootstrap.jar:/usr/local/apachegui/ApacheGUI/tomcat/bin/tomcat-juli.jar
Tomcat started.
look like the initial error is resolved but I am still unable access ApacheGUI from my browser still unable to resolve the error mention above can anyone suggest what is the solution to this problem?
The run.sh file is a badly written shell script with content:
if [ $UID -ne 0 ]; then
sudo ../tomcat/bin/startup.sh
exit
fi
exec ../tomcat/bin/startup.sh
Just execute tomcat/bin/startup.sh. This script can fail in many ways:
there is no shebang (#!interpreter_name), so it is executed using /bin/sh or whatever the default shell is,
UID is not a POSIX variable (cf. this question), so the result is blank and test ([) only sees -ne 0. Usually in such cases the variable should be quoted or tested if it is not empty,
the path ../tomcat/bin/startup.sh is relative to the current directory, not the directory of the script.
By executing following tests you will learn port testing and port configuration and at the end of these test you will be able to find out either problem exist within your Server ports or you have to check the setting of your browser. In my case I have taken an unintentional long route, I set my testing plan into three sections as follow
Server Ports and Browser
If you ever experience the same issue my advise is to troubleshoot
Server Browser and then test Ports
By executing tomcat/bin/startup.sh it does start the server successfully just like run.sh (as mention in Question and given answer above) but the browser was still unable to load the localhost:9999/ApacheGUI index page because the server's initial error
./run.sh: 1: [: -ne: unexpected operator
was resolve (as shown in question) but the browser was still unable to load the GUI page so for this reason just to find out if there is a problem with port 9999 itself or there is something else causing localhost not to communicate with the browser via port 9999 I have carried out following tests:
I believe the following information is suitable for a tester who wants to troubleshoot ports but if you think there is nothing wrong with your ports and the problem may lies with the browser then visit the following link Where I had the same problem with port 8080 and I overcome this issue by modifying browser settings:
https://stackoverflow.com/questions/66524719/tomcat10-home-page-can-not-load-by-typing-localhost8080-error-err-connection-r
Reboot my server and execute the following command in terminal
tomcat/bin/startup.sh
output shows
Tomcat started
With this output, I know ApacheGUI is installed and the server is up and running.
Q: if the server is running so whey localhost:9999/ApacheGUI index
page is unable to load?
let's test port 9999 and take the following steps to check either it's port on your machine causing this error or it's something else
StepNO:1
pi#Home:~ $ nc -vz 192.168.0.16 9999
Connection to 192.168.0.16 9999 port [tcp/*] succeeded!
StepNO:2
type command on Linux terminal to test port 9999 (Enter)
pi#Home:~ $ sudo ls | nc -l -p 9999
(Cursor start flashing with no output)
StepNo:3
Then turn ON Window computer and open PuTTY client
Establish SSH connection from window machine by typing the following credentials in PuTTY Client interface:
Server: 192.168.0.16 port 9999
(Press Enter)
if port 9999 is open successfully you will get the following results
Linux Server Terminal (StepNO:2) Where cursor was flashing start responding to
Client request and print following line (where the cursor is still flashing):
SSH-2.0-PuTTY_Release_0.74
StepNO:4
Open a new terminal window and typed another command to make sure port 9999 is definitely open and responding (Linux Server terminal in StepNO:2 still open)
pi#Home:~ $ nmap -p 9999 192.168.0.16
Output:
Starting Nmap 7.70 ( https://nmap.org ) at 2021-03-05 23:44 GMT Nmap
scan report for 192.168.0.16 Host is up (0.00049s latency).
PORT STATE SERVICE 9999/tcp open abyss
Nmap done: 1 IP address (1 host up) scanned in 0.26 seconds
This stage confirmed I have successfully open port 9999 and the port is responding to
TCP protocols
StepNO:5
in the same Linux server terminal (stepNO:2) I type the following command again:
sudo ls | nc -l -p 9999 (Enter)
(Cursor start flashing)
Now I have open the browser and type ApacheGUI URL
192.168.0.16:9999/ApacheGUI (Enter)
Linux server terminal from StepNO:2 where the cursor is flashing start responding to
URL request from the browser and print the following lines in the terminal window:
Output:
GET /ApacheGUI/ HTTP/1.1 Host: 192.168.0.16:9999 Connection:
keep-alive Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (X11;
Linux armv7l) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/86.0.4240.197 Safari/537.36 Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate Accept-Language: en-US,en;q=0.9
But: my ApacheGUI index page still unable to load and showing the following error message
This site can’t be reached192.168.0.16 refused to connect. Try:
Checking the connection Checking the proxy and the firewall
ERR_CONNECTION_REFUSED
At this stage, I am 100% sure that my installed server is up and running. Port 9999 in my server is open and functional so there is nothing else that can cause this error except the browser so I have followed the step to tune my browser setting as described in answer to the following link:
https://stackoverflow.com/questions/66524719/tomcat10-home-page-can-not-load-by-typing-localhost8080-error-err-connection-r
Result: Successful connection ApacheGUI index page finally loaded to the browser.
I have created an image out of my simple spring-boot application and set the port to 7000 for the tomcat , when I run the image and try to hit the service in my local I am not getting any response.
here are the contents of the dockerfile
FROM java:8
VOLUME /tmp
ADD /target/demo-1.0-SNAPSHOT.jar /app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
Am I missing something here ( I am new to docker and I just used the file in spring boot sample docker and created the image and stuck here.)
Apart from publishing the port from docker as mentioned by #Van0SS you need to create a port forwarding rule as well.
Open virtualbox and Navigate to VM -> Settings -> Network -> Advanced -> Port forwarding
Create a new rule:
Name : <Anything - Purpose of port>
Protocol: TCP
HostIP: 127.0.0.1
Host port: 7000
Guest Port: 7000
Not sure that it is only one problem, but at least you have to publish your port in docker. To do so run the container with flag:
-p 7000:7000
Try access on :7000 port.
You need to add:
EXPOSE: 7000
to your docker file, and keep an eye on the console log info when running the docker image to see on which port your app is being served.
Useful Ref:
https://www.youtube.com/watch?v=FlSup_eelYE
Best of luck!
I'm trying to remotely debug a Spring boot application deployed on a cloud foundry Instance. Below are the steps that I'm following to setup remote debug using eclipse:
Setting up JBP_CONFIG_DEBUG: '{enabled: true}' environment variable for the application.
After that I'm setting up the local port forwarding to my app container with cf ssh -N -T -L 8000:localhost:8000 <APP_NAME>
Then I'm setting up the remote debug configuration in eclipse as shown below image:Eclipse remote debug setup
After that when I try to start remote debug session, It tried to connect to the app but fails with the message at the port forwarded terminal:
connect to localhost:8000 failed: ssh: rejected: connect failed (dial
tcp 127.0.0.1:8000: getsockopt: connection refused)
Please help, if anyone has any clue about this error!!
You might have to restage the application for the change to take place, debug configuration to change.
You can use alternatively ssh into the container using cf ssh <application name> and check that the debug port is opened on not using netstat -an
We had some issues as well, but with a specific java buildback, but anyway mabe it'll help you in your case as well, try:
cf set-env <app-name> JBP_CONFIG_JAVA_OPTS '[java_opts: "-XX:+ForceDebuggingOnDemand"]'
cf restage <app-name>
try to change debugging port like this:
JBP_CONFIG_DEBUG "{enabled: true, port: 8001}"
I cannot connect to HBase running in Docker on Windows (banno/hbase-standalone image). However, I can connect to locally installed HBase.
banno/hbase-standalone image is run using:
docker run -d -p 2181:2181 -p 60000:60000 -p 60010:60010 -p 60020:60020 -p 60030:60030 banno/hbase-standalone
I also set up the port forwarding on the boot2docker-vm (which is required when running on Windows):
I can successfully telnet to all those ports on my localhost.
Next, here is a code sample that we use in our tests:
Configuration config = HBaseConfiguration.create();
config.clear();
config.setInt("timeout", 12000);
config.set("zookeeper.znode.parent", "/hbase");
config.set("hbase.zookeeper.quorum", "127.0.0.1");
config.set("hbase.zookeeper.property.clientPort", "2181");
config.set("hbase.master", "127.0.0.1:60000");
final Configuration configuration = HBaseConfiguration.create(config);
JobDefinition.Buildable.dumpProperties(configuration, newArrayList("hbase.*"));
HBaseAdmin.checkHBaseAvailable(config);
Which causes the following exception
Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.net.UnknownHostException: unknown host: a3e6c240af20
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1651)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:900)
at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2366)
at com.xxx.compute.hadoop.jobs.transaction.OurTest.main(OurTest.java:24)
Caused by: com.google.protobuf.ServiceException: java.net.UnknownHostException: unknown host: a3e6c240af20
at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674)
at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715)
at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623)
... 5 more
Caused by: java.net.UnknownHostException: unknown host: a3e6c240af20
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.<init>(RpcClient.java:386)
at org.apache.hadoop.hbase.ipc.RpcClient.createConnection(RpcClient.java:352)
at org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1526)
at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1438)
at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657)
... 10 more
This is explainable. We run Windows, which requires boot2docker-vm virtual machine running using NAT. The Docker container of the image is running inside the boot2docker-vm also using NAT. However, the ports are "visible" to the host machine running tests, since Docker container exports the ports, and the boot2docker-vm forwards the ports the host machine. The name a3e6c240af20 actually comes from the Docker container ID, so probably a3e6c240af20 is a hostname for the Docker container :
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a3e6c240af20 banno/hbase-standalone:latest "/bin/sh -c '/opt/hb 24 minutes ago Up 24 minutes 0.0.0.0:2181->2181/tcp, 0.0.0.0:60000->60000/tcp, 0.0.0.0:60010->60010/tcp, 0.0.0.0:60020->60020/tcp, 0.0.0.0:60030->60030/tcp agitated_wozniak
I am not sure how exactly HBase communication works, but apparently it makes RPC calls to the instance. HBase Docker returns its hostname hoping that the client will call it there. But since both boot2docker-vm and Docker container running using NAT, the host machine does not see the Docker container.
I tried to add a3e6c240af20 to my hosts file:
127.0.0.1 a3e6c240af20
Then I get a different error, also during the RPC call, which actually does not help me much:
Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.lang.NullPointerException
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1651)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:900)
at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2366)
at com.xxx.compute.hadoop.jobs.transaction.OurTest.main(OurTest.java:24)
Caused by: com.google.protobuf.ServiceException: java.lang.NullPointerException
at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674)
at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715)
at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623)
... 5 more
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.writeRequest(RpcClient.java:1051)
at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1440)
at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657)
... 10 more
Does anyone have a suggestion how this can be solved?
Try add [boot2docker IP] a3e6c240af20 instead of 127.0.0.1 because HBase Java client needs to reach your docker's host not exactly localhost to reach zookeeper (CMIIW). Not pretty sure if it will works but it works in my Windows.
I used oddpoet/hbase-cdh5 docker image to avoid this issue.
docker run -d -p 2181:2181 -p 60000:60000 -p 60010:60010 -p 60020:60020 -p 60030:60030 -h hbase oddpoet/hbase-cdh5
fig.yml
hbase:
image: oddpoet/hbase-cdh5
hostname: hbase
ports:
- "3181:2181"
- "60000:60000"
- "60010:60010"
- "60020:60020"
- "60030:60030"
my configuration file
conf.set("hbase.zookeeper.quorum", zkPath);
conf.set("hbase.zookeeper.property.clientPort","2181");
conf.set("zookeeper.znode.parent", "/hbase");
conf.set("hbase.client.retries.number", "3"); // default 35
conf.set("hbase.rpc.timeout", "10000"); // default 60 secs
conf.set("hbase.rpc.shortoperation.timeout", "5000"); // default 10 secs