I have made a Java web application and trying to host it on Heroku with a Dockerfile, CMD as :
CMD ["java", "-Dserver.port=$PORT", "-jar", "/usr/local/lib/backend.jar", "--server.port=$PORT"]
And getting the logs as follows:
2022-05-06T06:53:33.599532+00:00 heroku[web.1]: Starting process with command `java -Dserver.port\=\27061 -jar /usr/local/lib/backend.jar --server.port\=\27061`
2022-05-06T06:53:35.445811+00:00 app[web.1]: 199 [main] INFO org.reflections.Reflections - Reflections took 179 ms to scan 1 urls, producing 4 keys and 22 values
2022-05-06T06:53:41.324114+00:00 app[web.1]: 6078 [main] INFO org.reflections.Reflections - Reflections took 58 ms to scan 1 urls, producing 3 keys and 16 values
2022-05-06T06:53:41.544513+00:00 app[web.1]: May 06, 2022 6:53:41 AM org.javawebstack.httpserver.HTTPServer start
2022-05-06T06:53:41.544514+00:00 app[web.1]: INFO: HTTP-Server started on port 1337
I am not getting able to understand that my argument is providing --server.port=27061 but still the HTTP server is starting on port 1337.
I have already seen all the related available answers on Stack Overflow but none works for me.
I am using javawebstack httpserver
Related
I upload the .jar file of my bot for discord to heroku. Everything works as it should. And then the app breaks down and everything stops working.
Application Logs
heroku[web.1]: Starting process with command `java $JAVA_OPTS -jar target/DiscordBot-1.0-SNAPSHOT-jar-with-dependencies.jar -Dserver.port=43997`
app[web.1]: Setting JAVA_TOOL_OPTIONS defaults based on dyno size. Custom settings will override them.
app[web.1]: Picked up JAVA_TOOL_OPTIONS: -Xmx300m -Xss512k -XX:CICompilerCount=2 -Dfile.encoding=UTF-8
app[web.1]: [main] INFO net.dv8tion.jda.api.JDA - Login Successful!
app[web.1]: [JDA MainWS-ReadThread] INFO net.dv8tion.jda.internal.requests.WebSocketClient - Connected to WebSocket
app[web.1]: [JDA MainWS-ReadThread] INFO net.dv8tion.jda.api.JDA - Finished Loading!
heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 90 seconds of launch
heroku[web.1]: Stopping process with SIGKILL
heroku[web.1]: Process exited with status 137
Procfile
web: java $JAVA_OPTS -jar target/DiscordBot-1.0-SNAPSHOT-jar-with-dependencies.jar -Dserver.port=$PORT
application.properties
server.port=${PORT:8080}
worker: java -jar target/DiscordBot-1.0-SNAPSHOT-jar-with-dependencies.jar
without server.port=${PORT:8080} in application.properties
I have a Spring Boot app in Docker that runs on Heroku.
Recently, after updating Tomcat to 10.1.0-M10, I started getting this error:
Error R10 (Boot timeout) -> Web process failed to bind to $PORT within
60 seconds of launch
The immediate thought of downgrading to lower versions doesn't work due to vulnerabilities in the earlier versions. I have checked possible causes and found Tomcat binding port issue.
I cannot set up fixed config for different ports as I am deploying to Heroku and dependent on their random ports.
My Dockerfile:
FROM azul/zulu-openjdk-alpine:11
ENV PORT=$PORT
COPY /target/app.jar /app.jar
CMD java -Xms256m -Xmx512m \
-Dlog4j2.formatMsgNoLookups=true \
-Djava.security.egd=file:/dev/./urandom \
-Dserver.port=$PORT \
-jar /app.jar
What is the way to solve it? Is there anything I am missing?
UPDATE:
There are more logs from Heroku:
Feb 22 12:50:16 integration-test app/web.1 2022-02-22 20:50:16.057 [main] INFO c.g.s.z.ApplicationKt - Started ApplicationKt in 8.09 seconds (JVM running for 9.062)
Feb 22 12:50:16 integration-test app/web.1 2022-02-22 20:50:16.060 [main] DEBUG o.s.b.a.ApplicationAvailabilityBean - Application availability state LivenessState changed to CORRECT
Feb 22 12:50:16 integration-test app/web.1 2022-02-22 20:50:16.063 [main] DEBUG o.s.b.a.ApplicationAvailabilityBean - Application availability state ReadinessState changed to ACCEPTING_TRAFFIC
Feb 22 12:51:06 integration-test heroku/web.1 Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch
I found a solution that wasn't perfect but seemed to work for me.
Downgraded Spring Boot from 2.6.3 to 2.6.1
Downgraded Tomcat from 10.X.X to 9.X.X
Removed dev tools dependencies
I think the two latest did the magic. Dev tools stopped asking for an extra port in the test/prod environment. Tomcat bound the port in the version 9.X.X but not in 10.X.X.
Even though I found the solution, I don't know why it behaved like this, and it isn't perfect security-wise.
from the error message it seems that $PORT is not resolved to any environment variable.
deploying to heroku you must use .env file to define env vars (you can't use docker run -e PORT=1234) see documentation
When you use heroku locally, you can set config vars in a .env file. When heroku local is run .env is read and each name/value pair is set in the environment. You can use this same .env file when using Docker: docker run -p 5000:5000 --env-file .env <image-name>
I've followed the docs in order to install Jetty9 as a service but whenever I run
service jetty start
It would fail with no messages, my JETTY_HOME is /opt/jetty9, contains the home distribution for version 9.4.14. I've also created my JETTY_BASE at /usr/share/jetty9 with my webapp and modules.
Both Jetty Home and Base are owned by the user jetty. I've then symlinked to my init.d folder as:
ln -s /opt/jetty9/bin/jetty.sh /etc/init.d/jetty
Then I created a /etc/default/jetty file with the following content:
# change to 1 to prevent Jetty from starting
NO_START=0
# change to 'no' or uncomment to use the default setting in /etc/default/rcS
VERBOSE=yes
# Run Jetty as this user ID (default: jetty)
# Set this to an empty string to prevent Jetty from starting automatically
JETTY_USER=jetty
# The home directory of the Java Runtime Environment (JRE). You need at least
# Java 6. If JAVA_HOME is not set, some common directories for OpenJDK and
# the Oracle JDK are tried.
#JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java
# Extra options to pass to the JVM
#JAVA_OPTIONS="-Xmx256m -Djava.awt.headless=true"
# Timeout in seconds for the shutdown of all webapps
#JETTY_SHUTDOWN=30
# Additional arguments to pass to Jetty
#JETTY_ARGS=
# Jetty uses a directory to store temporary files like unpacked webapps
TMPDIR=/opt/jetty9/tmp
JETTY_HOME=/opt/jetty9
JETTY_BASE=/usr/share/jetty9
# Default for number of days to keep old log files in /var/log/jetty9/
#LOGFILE_DAYS=14
# If you run Jetty on port numbers that are all higher than 1023, then you # do not need authbind. It is used for binding Jetty to lower port numbers.
# (yes/no, default: no)
#AUTHBIND=yes
JETTY_HOST=0.0.0.0
If I start Jetty using java -jar $JETTY_HOME/start.jar in my base folder it would work with no problem. Also, if I run
service jetty supervise
It would also run with no issues, but when I call start it fails with:
root#app:/usr/share/jetty9# service jetty start
Job for jetty.service failed because the control process exited with error code.
See "systemctl status jetty.service" and "journalctl -xe" for details.
root#app:/usr/share/jetty9# service jetty status
● jetty.service - LSB: Jetty start script.
Loaded: loaded (/etc/init.d/jetty; generated)
Active: failed (Result: exit-code) since Mon 2018-12-03 15:05:26 UTC; 14s ago
Docs: man:systemd-sysv-generator(8)
Process: 21162 ExecStop=/etc/init.d/jetty stop (code=exited, status=0/SUCCESS)
Process: 21202 ExecStart=/etc/init.d/jetty start (code=exited, status=1/FAILURE)
Dec 03 15:05:22 app systemd[1]: Stopped LSB: Jetty start script..
Dec 03 15:05:22 app systemd[1]: Starting LSB: Jetty start script....
Dec 03 15:05:26 app jetty[21202]: Starting Jetty: FAILED Mon Dec 3 15:05:26 UTC 2018
Dec 03 15:05:26 app systemd[1]: jetty.service: Control process exited, code=exited status=1
Dec 03 15:05:26 app systemd[1]: jetty.service: Failed with result 'exit-code'.
Dec 03 15:05:26 app systemd[1]: Failed to start LSB: Jetty start script..
This is the output of service jetty check:
root#app:/usr/share/jetty9# service jetty check
Jetty NOT running
JAVA = /usr/bin/java
JAVA_OPTIONS = -Djetty.home=/opt/jetty9 -Djetty.base=/usr/share/jetty9 -Djava.io.tmpdir=/opt/jetty9/tmp
JETTY_HOME = /opt/jetty9
JETTY_BASE = /usr/share/jetty9
START_D = /usr/share/jetty9/start.d
START_INI = /usr/share/jetty9/start.ini
JETTY_START = /opt/jetty9/start.jar
JETTY_CONF = /opt/jetty9/etc/jetty.conf
JETTY_ARGS = jetty.state=/usr/share/jetty9/jetty.state jetty-started.xml
JETTY_RUN = /var/run/jetty
JETTY_PID = /var/run/jetty/jetty.pid
JETTY_START_LOG = /var/run/jetty/jetty-start.log
JETTY_STATE = /usr/share/jetty9/jetty.state
JETTY_START_TIMEOUT = 60
RUN_CMD = /usr/bin/java -Djetty.home=/opt/jetty9 -Djetty.base=/usr/share/jetty9 -Djava.io.tmpdir=/opt/jetty9/tmp -jar /opt/jetty9/start.jar jetty.state=/usr/share/jetty9/jetty.state jetty-started.xml
Any ideas?
UPDATE
Changing the user in /etc/default/jetty to root would solve the issue, but this is not a solution, isn't it?
# Run Jetty as this user ID (default: jetty)
# Set this to an empty string to prevent Jetty from starting automatically
JETTY_USER=root
I finally got this working, the jetty user should have permissions to the following folders and /usr/sbin/nologin as shell as described here.
JETTY_HOME
JETTY_BASE
/var/run/jetty <-- couldn't find a reference to this folder in the docs
And add the following to your /etc/default/jetty:
JETTY_SHELL=/bin/sh
JETTY_LOGS=/usr/share/jetty9/logs
JETTY_START_LOG=/usr/share/jetty9/logs/jetty-start-log.log
Also you should double check that there are no remaining log files owned by other user than jetty in your folders.
I am running a war on Heroku using webapp-runner. I deploy the application using the heroku-maven-plugin version 1.2 via the following command: mvn heroku:deploy-war. Initially, the app works and all endpoints return valid responses. However, if I allow the app to idle long enough for Heroku to put it to sleep and then invoke an endpoint which calls into guava I receive a NoSuchMethodError:
2017-09-23T19:19:45.388865+00:00 app[web.1]: SEVERE: Servlet.service() for servlet [jersey-serlvet] in context with path [] threw exception [org.glassfish.jersey.server.ContainerException: java.lang.NoSuchMethodError: com.google.common.base.CharMatcher.ascii()Lcom/google/common/base/CharMatcher;] with root cause
2017-09-23T19:19:45.388866+00:00 app[web.1]: java.lang.NoSuchMethodError: com.google.common.base.CharMatcher.ascii()Lcom/google/common/base/CharMatcher;
2017-09-23T19:19:45.388867+00:00 app[web.1]: at com.google.common.io.BaseEncoding$Alphabet.<init>(BaseEncoding.java:453)
2017-09-23T19:19:45.388868+00:00 app[web.1]: at com.google.common.io.BaseEncoding$Base64Encoding.<init>(BaseEncoding.java:892)
2017-09-23T19:19:45.388869+00:00 app[web.1]: at com.google.common.io.BaseEncoding.<clinit>(BaseEncoding.java:317)
...application specific stack trace
All subsequent calls to the same API produce a NoClassDefFoundError at the same point
2017-09-23T19:22:24.454901+00:00 app[web.1]: SEVERE: Servlet.service() for servlet [jersey-serlvet] in context with path [] threw exception [org.glassfish.jersey.server.ContainerException: java.lang.NoClassDefFoundError: Could not initialize class com.google.common.io.BaseEncoding] with root cause
2017-09-23T19:22:24.454903+00:00 app[web.1]: java.lang.NoClassDefFoundError: Could not initialize class com.google.common.io.BaseEncoding
...application specific stack trace
These issues seem to suggest that the guava jar is present at compile time but not present at runtime. However, I logged-in to the web dyno and verified that the guava jar was included in my warfile
my-mbp:TrickServer me$ heroku ps:exec
Establishing credentials... done
Connecting to web.1 on ⬢ myapp...
~ $ cd target/
~/target $ ls
MyApp.war dependency mvn-dependency-list.log tomcat.52079
~/target $ jar -tf MyApp.war
...lots of dependencies...
WEB-INF/lib/google-oauth-client-1.20.0.jar
WEB-INF/lib/gson-2.2.4.jar
WEB-INF/lib/guava-23.0.jar <---guava
WEB-INF/lib/guava-jdk5-13.0.jar
...lots more dependencies...
I am struggling to explain why the endpoints work immediately after the app is deployed but later produce these errors. To me this behavior seems to suggest that Heroku is potentially supplying a different classpath when my app wakes up from sleep than when it is initially run or that Heroku is moving/cleaning up the guava jarfile.
Contents of my Procfile:
web: java $JAVA_OPTS -jar target/dependency/webapp-runner.jar --port $PORT --expand-war target/MyApp.war
Java Processes runnning on my web dyno:
~/target $ ps -ef | grep java
u30439 4 1 0 18:50 ? 00:00:44 java -Xmx300m -Xss512k -Dfile.encoding=UTF-8 -Duser.timezone=UTC -jar target/dependency/webapp-runner.jar --port 52079 target/MyApp.war
u30439 27 4 0 18:50 ? 00:00:00 bash --login -c java $JAVA_OPTS -jar target/dependency/webapp-runner.jar $WEBAPP_RUNNER_OPTS --port 52079 target/MyApp.war
Update 1
Since I am invoking my webapp with the --expand-war argument I also checked the jarfiles in the expanded directory to verify that guava was present. It is:
~/target/tomcat.55320/webapps/expanded/WEB-INF/lib $ ls
...dependencies...
google-oauth-client-1.20.0.jar
gson-2.2.4.jar
guava-23.0.jar
guava-jdk5-13.0.jar
...more dependencies...
Update 2
I added the following logic to the problematic web service to printout the classpath and the resources on it:
logger.info("System Classpath: " + System.getProperty("java.class.path"));
logger.info("Runtime Classes...");
ClassLoader cl = UserService.class.getClassLoader();
URL[] urls = ((URLClassLoader) cl).getURLs();
for(URL url: urls){
logger.info(url.getFile());
}
The next time the error occurred I examined the logs and to my surprise found that the guava jar was present on the runtime classpath!
2017-09-24T12:07:40.843438+00:00 app[web.1]: [heroku-exec] ERROR: Could not connect to proxy:
2017-09-24T12:07:40.844145+00:00 app[web.1]: [heroku-exec] ERROR: Too many reconnect attempts. Waiting 30 seconds...
2017-09-24T12:07:52.671620+00:00 app[web.1]: Sep 24, 2017 12:07:52 PM org.myorg.server.web.services.MyService authenticate
2017-09-24T12:07:52.671631+00:00 app[web.1]: INFO: System Classpath: target/dependency/webapp-runner.jar
2017-09-24T12:07:52.671931+00:00 app[web.1]: Sep 24, 2017 12:07:52 PM org.myorg.server.web.services.MyService authenticate
2017-09-24T12:07:52.671932+00:00 app[web.1]: INFO: Runtime Classes...
2017-09-24T12:07:52.672277+00:00 app[web.1]: Sep 24, 2017 12:07:52 PM org.myorg.server.web.services.MyService authenticate
2017-09-24T12:07:52.672279+00:00 app[web.1]: INFO: /app/target/tomcat.28304/webapps/expanded/WEB-INF/classes/
....
2017-09-24T12:07:52.690304+00:00 app[web.1]: Sep 24, 2017 12:07:52 PM org.myorg.server.web.services.MyService authenticate
2017-09-24T12:07:52.690306+00:00 app[web.1]: INFO: /app/target/tomcat.28304/webapps/expanded/WEB-INF/lib/google-oauth-client-1.20.0.jar
2017-09-24T12:07:52.690501+00:00 app[web.1]: Sep 24, 2017 12:07:52 PM org.myorg.server.web.services.MyService authenticate
2017-09-24T12:07:52.690503+00:00 app[web.1]: INFO: /app/target/tomcat.28304/webapps/expanded/WEB-INF/lib/guava-23.0.jar <--- Guava!!!
....
What is going on here? How do I debug this?
You probably have two versions of guava or its related jars on your classpath. See NoSuchMethodError exception when using com.google.common.base.Splitter
After some debugging, I discovered that my program had two different versions of Guava on the classpath (guava-23.0.jar & guava-jdk5-13.0.jar). The debugging tip suggested here was necessary but not sufficient for me to get to the bottom of this.
When working with ClassLoaders it's important to remember that the getClassLoader method defined in the .class object returns a reference to the ClassLoader that originally loaded the class. To find the duplicate jar, it was crucial to invoke classLoader.getResource("/com/google/common/base/CharMatcher.class") on the same ClassLoader which loaded the class that later failed with the NoSuchMethodError.
For posterity, the specific dependency which caused the conflict was com.google.api-client. I resolved it by adding the following exclusion to the dependency in my pom.xml
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.22.0</version>
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava-jdk5</artifactId>
</exclusion>
</exclusions>
</dependency>
In my current project, I have been using apache-nifi on Windows operating system.
I have extracted nifi-0.7.0-bin.zip file in C:\. Now, when I run \bin\run-nifi.bat as an administrator I see the following message on the command line and I am not able to run nifi. Could you please suggest what should I do in order to run the nifi.
the content of app.log file as follows
2016-08-08 11:41:57,716 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2016-08-08 11:41:57,851 INFO [main] org.apache.nifi.BootstrapListener Started Bootstrap Listener, Listening for incoming requests on port 50050
2016-08-08 11:41:57,877 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap
2016-08-08 11:42:59,485 WARN [Timer-0] org.apache.nifi.NiFi NiFi has detected that this box is not responding within the expected timing interval, which may cause Processors to be scheduled erratically. Please see the NiFi documentation for more information.
The content of nifi-bootstrap.log file is as follows:
2016-08-08 11:41:57,279 INFO [main] o.a.n.b.NotificationServiceManager Successfully loaded the following 0 services: []
2016-08-08 11:41:57,286 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STARTED
2016-08-08 11:41:57,286 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STOPPED
2016-08-08 11:41:57,286 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_DIED
2016-08-08 11:41:57,298 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi...
2016-08-08 11:41:57,299 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: C:\NIFI-0~1.0
2016-08-08 11:41:57,299 INFO [main] org.apache.nifi.bootstrap.Command Command: C:\Program Files\Java\jdk1.8.0_102\bin\java.exe -classpath C:\NIFI-0~1.0\.\conf;C:\NIFI-0~1.0\.\lib\jcl-over-slf4j-1.7.12.jar;C:\NIFI-0~1.0\.\lib\jul-to-slf4j-1.7.12.jar;C:\NIFI-0~1.0\.\lib\log4j-over-slf4j-1.7.12.jar;C:\NIFI-0~1.0\.\lib\logback-classic-1.1.3.jar;C:\NIFI-0~1.0\.\lib\logback-core-1.1.3.jar;C:\NIFI-0~1.0\.\lib\nifi-api-0.7.0.jar;C:\NIFI-0~1.0\.\lib\nifi-documentation-0.7.0.jar;C:\NIFI-0~1.0\.\lib\nifi-nar-utils-0.7.0.jar;C:\NIFI-0~1.0\.\lib\nifi-properties-0.7.0.jar;C:\NIFI-0~1.0\.\lib\nifi-runtime-0.7.0.jar;C:\NIFI-0~1.0\.\lib\slf4j-api-1.7.12.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dnifi.properties.file.path=C:\NIFI-0~1.0\.\conf\nifi.properties -Dnifi.bootstrap.listen.port=50049 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=C:\NIFI-0~1.0\bin\..\\logs org.apache.nifi.NiFi
2016-08-08 11:41:57,866 WARN [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Failed to set permissions so that only the owner can read status file C:\NIFI-0~1.0\bin\..\run\nifi.pid; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2016-08-08 11:41:57,876 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 50050
I recommend running NiFi with the current directory context in the bin directory:
Open a new command prompt window
Change to the NiFi bin directory, cd c:\nifi-0.7\bin
From c:\nifi-0.7\bin, execute run-nifi.bat
I believe run-nifi.bat depends on finding nifi-env.bat in the current directory, rather than finding it relative to its own path. If you ran C:\...\nifi-0.7.0\bin\run-nifi.bat, I think you would see that error.