I am running Apache Tomcat 7.0.42 on Ubuntu 12.04. When I start up Apache initially it's all fine and I can connect. However after using it for a while I can no longer browse to any site hosted on Apache, not even the root site. The browser just waits for a connection forever. If I go on the machine I can see lots of connections that have been closed and are CLOSE_WAIT on netstat, but nothing open. The Apache log file doesn't tell me anything either.
If I restart Apache the problem doesn't go away, even rebooting the server hasn't solved the problem. After struggling for a while it starts working again only to stop shortly afterwards.
I don't have any network issues between the machines.
I had a look at the known bugs on 7, but don't notice anything that sounds related. I have also reinstalled Apache on the machine, but the problem returned soon after.
I feel like I'm missing something silly, but I'm just not getting it.
I am not sure if this will help, but you can give a try for below options.
The problem may not be for Tomcat7, but for your own site. I have used tomcat for so many days and didnt notice such kind of issue single time.
May be your one of the site is having memory leaks and after running for sometime, it makes server stop because of low memory.
If there are many sites then you can try removing all and adding one by one to check which site is giving the issue.
You can try with simple page first by removing all other sites to check if the issue is with tomcat or your site.
Related
I have a strange behaviour happening in one of our projects and I wonder if anyone encountered anything similar. It is a legacy Java project and we are using ojdbc6.jar to connect to an Oracle DB. Locally we are using a Tomcat 6 app server.
When using IntelliJ IDE the db connection fails in some cases (specially where higher number or more complicated queries are run), with
ORA-12516, TNS:listener could not find available handler with matching protocol stack
The strange thing is, that Eclipse users never have the mentioned problem. (For some reason, their db calls seem much faster too)
We are using an IBM Websphere JDK because of production reasons, its a 64 bit 1.8 jdk.
The issue happens more often when running debug, but also happens while running normally.
I looked up this error and tried the following things:
monitored the number of processes oracle is running, but we have a limit of 400 and it never went above 300
tried changing locally to ojdbc7 and ojdbc8: 7 produces the same issues, 8 throws a could not establish connection with Network Adapter error at the same situation
tried changing the jdk-s under tomcat: no effect
tried all kinds of memory settings for tomcat and for intellij, no effect
It's a pretty annoying issue and I dont understand, why this does not happen with Eclipse.
Thank you for your advice in advance!
Today in the company where I work we received a report about one of our webapps not working.
The first thing we did was look at the hardware utilization:
Processor: 5%;
Memory: 68%;
Disk IO capacity: 4%;
Network: 1Mbps/1Gbps;
After some tests we saw that, indeed, the webapp pages weren't loading, some time after the webapp timeouts.
Other webapps in the exactly same tomcat instance were working fine and fast as ever, no problem with them. We tried to restart/reload the webapp but it still didn't get it working. Finally we restarted tomcat which corrected the issue for now.
There were no restarts or redeploys since yesterday when it was working fine. We believe this may be a periodic bug of some sort so we want to correct it soon.
Does anyone know of any steps we may take to investigate what it might have been?
It seems related to lock on IO.
If you are using linux, the first thing is do is to check open files by using the command lsof . If you see a lot of opened files by your app, you have to check in the code that every InputStream/OutputStream are closed (even in exception handling code).
An other common source of issues is related to thread starvation, see http://tomcat.apache.org/tomcat-9.0-doc/config/valve.html#Stuck_Thread_Detection_Valve
We are trying to access an application from the tomcat which is on a different host, but it is not loading even though the tomcat is running. It was running fine for the past 3 months. We restarted the tomcat now it is working fine.
But, we could not able to zero in on what happened.
Any idea how to trace / what might have caused this?
The CPU usage was normal and the tomcat memory was 1205640.
the memory setting of tomcat are 1024- 2048(min-max)
We are using tomcat 7.
Help much appreciated....thanks in advance.....cheers!!
...also - not sure on Windows - you may be running out of file descriptors. This typically happens when streams are not properly closed in finally blocks.
In addition, check with netstat if you have a lot of sockets remaining open or accumulating in wait state.
Less likely, the application is creating threads and never releasing them.
The application is leaking something (memory, file descriptors, sockets, threads,...) and running over a limit.
There are different ways to track this. A profiler may help or more simply, running JVM dumps at regular intervals and checking what is accumulating. The excellent MAT will help you analyze the dumps.
Memory leak problems are not uncommon. If your Tomcat instance was running for three months and suddenly the contained application became unresponsive maybe that was the case. One solution (and if your resources allow you to do so) could be monitoring that Tomcat instance though JMX using jconsole to see how it behaves
I am currently facing an issue with my java webapp running on Jetty 7.4.5.v20110725 on Linux. My Webapp serving static content is running out of file descriptors after a few days from its start timestamp. I am starting the jetty server with useFileMappedBuffer = true( in webDefaults.xml ) . I am using jdk1.6.0_30 . Please let me know if you'll have any suggestions on how to fix this issue.
Please note that this issue does not occur when useFileMappedBuffer = false (in webDefaults.xml).
If your application has run out of timestamps next time, please try to find out which files are open and if open connections are causing a problem.
Try to list open files by calling (I think it's lsof -p, but try or look at the lsof manpage as I'm writing this out of my mind):
lsof -p <jettypid>
This will show you what files are opened by the jetty process. Look for files which probably should have been closed already, etc.
Then do a:
netstat -an
This will show you established network connections. Look if there's lots of connections in CLOSE_WAIT state or similar indicating that connections are not properly closed.
Also have a look at your system wide OS limits with:
ulimit -a
It'll show you how many file descriptors can be opened by a single process. If you've a site with high traffic and the pretty common default value of 1024 max fd, you might need to raise that. If you think that traffic is the problem have a look at this guide: http://wiki.eclipse.org/Jetty/Howto/High_Load
However you've stated that the problem does occur only after a couple of days. This usually indicates not properly closed connections, file resources, etc.
If unsure what to do with the output of the commands above, feel free to paste them.
Independent of the problem I'd recommend you to upgrade to the latest jetty 7.x.
As soon I click to start (literally immediately) the tomcat server with my web app, it displays the following error:
http://i67.photobucket.com/albums/h282/jamesekki/error.jpg
What's weird is the application still builds successfully and works fine. The problem is it slows down development time since I get this error every time I need to deploy any changes.
Here is my eclipse ini:
http://i67.photobucket.com/albums/h282/jamesekki/Capture.jpg
Any ideas?
Things I have tried:
1. Increased xms/xmx in increments of 128 megs all the way up to 4 gigs (still fail)
2. Increased permsize/maxpermsize similar to increments above (still fail)
Maybe these will help explain:
http://www.mulesoft.com/tomcat-oome-out-of-memory-error
http://baskarfelix.wordpress.com/2008/08/06/out-of-memory-errorjava-heap-space/
I'd also be curious to know if you have the same issue if you leave Eclipse out of the picture. Create a WAR file, add it to the Tomcat /webapps directory, and start Tomcat using a command shell and the start script in the /bin directory. If you still have a problem, I'd say you should look at how Tomcat and your JVM are configured. If you don't, it tells me that Eclipse is the problem.
Thanks everyone for your inputs. I was able to narrow it down to what was the root cause of the whole issue. I had an external api jar file in the src/main/resources/lib of my services layer and Eclipse seems to not like that for whatever reason. As soon as I moved that out of that location, it resolved the issue. Can anyone explain why this would potentially cause an issue? Just for learning purposes. Thanks!
Your settings look good to me. You could try the long way of profiling [1] tomcat to see what happens on startup, or the crude way of deleting your local server and runtime, to see if it helps with a fresh one. If that fails, re-download latest Eclipse Java EE edition, and set up from scratch.
While it's not the best solution (in case the problem resurfaces, you still have no proper solution), it is sometimes worth it if it takes too long otherwise.
[1] You could profile with JProfiler or YourKit Profiler. Both have trial versions. JVisualVM is also an alternative that comes with the JDK. However it may be an issue if Tomcat crashes immediately.