OK, somehow in the log of the server I was not getting the proper error, but after trying different things I got an OutOfMemory:PermGenSpace error, and for that one stackoverflow already has a solution.
Dealing with "java.lang.OutOfMemoryError: PermGen space" error
I use the solution there and my problem is solved :)
Thanks #maximdim for your help
I've been struggling with an issue for two weeks.
I am connecting to test.salesforce.com through a web service in a Java web application using jdk7.
I generated the stubs with JAX-WS wsimport.
I am using STS with VMWare vFabric tc Server v2.6 in my local environment, here connection works fine.
The problem is when I deploy to the test server which is SpringSource tc Runtime 7.0 with jdk 7 I get the following exception after doing the web service call:
Exception in thread "RMI TCP Connection(idle)"
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "RMI TCP Connection(idle)"
Exception in thread "RMI TCP Connection(idle)"
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "RMI TCP Connection(idle)"
Exception in thread "RMI TCP Connection(idle)" Exception in thread "ContainerBackgroundProcessor[StandardEngine[Catalina]]"
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "RMI TCP Connection(idle)"
I have already incremented the memory parameters in the test server. It is starting with 1 GB or memory. I am giving more detail on the way memory has been increased:
vFabric server has a console, so we have the following configuration there:
Min Heap Size: 1,000MB
Max Heap Size: 16,000 MB
Thread Stack Sie: 192 KB.
I also found the file where these parameters are set (setenv.sh) and they are like this:
JVM_OPTS="-Xms1000m -Xmx16000m -Xss192k -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n"
I also know that is not a connectivity issue because sometimes the connection is made successfully but after that I get the error.
I also got a dump of the memory after the OutOfMemoryError and analyze it with Eclipse plug in and the memory leak is in:
com.sun.xml.internal.ws.client.sei.SEIStub
$Proxy51
Tried to post the image here but as I am a new user stackoverflow didn't allow me.
Please HELP!! Any help will be appreciated.
Alan Robles
How exactly did you 'incremented the memory parameters'? There are few different memory regions in JVM so you might increment one of them but the problem is in another - e.g. PermGen vs. Heap.
OK, somehow in the log of the server I was not getting the proper error, but after trying different things I got an OutOfMemory:PermGenSpace error, and for that one stackoverflow already has a solution.
Dealing with "java.lang.OutOfMemoryError: PermGen space" error
I use the solution there and my problem is solved :)
Thanks #maximdim for your help.
Related
We have deployed openam13.0.0 war in Tomcat8.026 version server. If server is idle for 12 to 24 hours, it gets stopped automatically and throws the below error. Please provide your suggestions on this.
org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads
The web application [openam] appears to have started a thread named
[com.google.inject.internal.util.$Finalizer] but has failed to stop
it. This is very likely to create a memory leak. Stack trace of
thread: java.lang.Object.wait(Native Method)
This is related to Tomcat:
Guice + Tomcat potential memory leak
and more than likely this bug:
https://bugster.forgerock.org/jira/browse/OPENAM-3133
The issue you are seeing is happening only when Tomcat is stopping. As long as Tomcat isn't actually prevented from shutting down then these are fairly harmless.
As reported in the bug report, this issue has been there since OpenAM 10 with recent version of Tomcats, it is fair to assume that the errors can be safety ignored as long as the Tomcat is able to shutdown. The discussion in the bug report OpenAM-3133 give a fairly good assessment of the situation.
Cheers
A client application has been built using Jdeveloper 10.1.3.2 and it is running on OC4J server. This application is sending data to external server application. It is working for quite long time without any issue. Lately a connection issue occurred and the following stack trace is generated:
com.sun.xml.ws.client.ClientTransportException: HTTP transport error: java.net.SocketException: Connection reset
at com.sun.xml.ws.transport.http.client.HttpClientTransport.getOutput(HttpClientTransport.java:133)
at com.sun.xml.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:153)
at com.sun.xml.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:93)
at com.sun.xml.ws.transport.DeferredTransportPipe.processRequest(DeferredTransportPipe.java:105)
at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:629)
at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:588)
at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:573)
at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:470)
at com.sun.xml.ws.client.Stub.process(Stub.java:319)
at com.sun.xml.ws.client.sei.SEIStub.doProcess(SEIStub.java:157)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:109)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:89)
at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:140)
at $Proxy44.sendRem(Unknown Source)
After goggling I found out a good discussion about the error sockets - What's causing my java.net.SocketException: Connection reset? .One answer in this link says that the issue mostly from the client side because if it is from the server side the exception will be (SocketException reset by peer).
What I did:
I tried out to increase the socket time out for the OC4J with the help of this form How to change OC4J HTTP Timeout. What I did is I changed the propriety oracle.j2ee.http.socket.timeout to be 5000 instead of 500 (10 times longer)
But the error still there. So, any suggestion to over come this issue?
Note: I able to use telnet command for external server IP and Port and it is working fine.
-------------------------------------------------------- Update 1 --------------------------------------------------------
I increase the server clock skew where the client application is running using the following command on server start up:
-Dweblogic.wsee.security.clock.skew=72000000
-Dweblogic.wsee.security.delay.max=72000000
But no luck, problem is not resolved.
-------------------------------------------------------- Update 2 --------------------------------------------------------
I realized that the problem is not from application at all; I test the external URL using SoapUI and I got the same error Connection rest. I think this new update clreay shows that there is nothing wrong with program code. But I need to know where to go or check now. Where is the starting point now to overcome the issue. Any clue will be helpful.
As you can see from Update 2 in the question, the problem was not from the client application because same error occurred from SoapUI.
The problem was that the machine where the client application was running have low bandwidth which was not enough for APIs communication. Using simple speed test , I found out that the upload bandwidth was low comparing to minimum requirements given by server application team.
I concluded this fact by monitoring the network resource using Resource Monitor in Windows while the client application was running and by using online speed check
To solve the issue, the machine bandwidth has to be increased where the client application is running.
I'm experiencing this kind of exception, can someone help me about this problem?
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1566)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526)
at org.quartz.utils.UpdateChecker.getClientId(UpdateChecker.java:149)
at org.quartz.utils.UpdateChecker.buildParamsString(UpdateChecker.java:120)
at org.quartz.utils.UpdateChecker.buildUpdateCheckUrl(UpdateChecker.java:114)
at org.quartz.utils.UpdateChecker.doCheck(UpdateChecker.java:55)
at org.quartz.utils.UpdateChecker.checkForUpdate(UpdateChecker.java:47)
at org.quartz.utils.UpdateChecker.run(UpdateChecker.java:39)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
I'm also getting the same exception with Could not load java.net.URLEncoder and Could not load java.net.URLConnection. I'm using Eclipse Indigo SR1 and Tomcat V6.0
You can solve this problem after restarting the server, the server iscaching older version of the app.
see here
You cannot resolve this problem by "restarting the server".
The problem is that when you update your app by uploading a new version to Tomcat, some resource is not cleaned up properly. Could be anything from a runaway thread, to some kind of scheduled service, to a listener. When that process tries to load a class, Tomcat will detect and prevent it.
The solution to the problem is to properly keep track of, and on shutdown close, those threads.
I have also experienced this problem recently.
In my case the problem was generated by an unhandled exception in javax.servlet.ServletContextListener which was logged in one localhost-date.log.
Because of that exception the webapp was not deployed, but an already started java.util.concurrent.Executors was still running which caused another chain of exceptions in catalina-date.log:
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load WhatEver.class.
The solution for my case:
check all the Tomcat logs
fix exceptions in javax.servlet.ServletContextListener
I have faced the similar issue. I have solved by increasing the permsize of Server in the VM arguments -XX:MaxPermSize=1024m
Am seeing the below error while trying to connect to Oracle 11g on Red Hat Linux, 64-bit using thin jdbc drivers. Would highly appreciate if anyone can throw more light on how to go about troubleshooting this.
Caused by: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at oracle.net.ns.DataPacket.send(DataPacket.java:199)
at oracle.net.ns.NetOutputStream.flush(NetOutputStream.java:211)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:227)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:175)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:100)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:85)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:123)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:79)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1122)
at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1099)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:288)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:191)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:366)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:752)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:366)
We've had something very similar, moved a program from 32-bit to 64bit OS & a number of concurrent processes would throw the IO Error: Connection Reset.
Stumbled across this which fixed it:
https://community.oracle.com/message/3701989
Basically add the -Djava.security.egd=file:/dev/./urandom parameter and it's good to go :)
You didn't include any details of the problem like what changed? Is this a new configuration for you or did it suddenly stop working? Do you know if you have enough connections available? Does this happen to every connection or is it intermittent?
Considering the error is occurring during the logon process, a few possibilities are:
Network fault
You have exhausted the maximum # of connections, so Oracle hangs up on you.
Firewall restrictions
A problem with the database server or the listener. The processing serving your session could be crashing after it's opened.
Check the following stackoverflow thread about how to check the number of active connections and the max. I would expect an "ORA-00018: maximum number of sessions exceeded" error if that were the problem, so it may not be. But it's worth checking.
How to check the maximum number of allowed connections to an Oracle database?
I faced similar issue with Sqoop Oracle import and I have implemented the fix suggested by this link
This setting helped to resolve issue:
-Dmapred.child.java.opts="-Djava.security.egd=file:///dev/urandom"
and I've changed mapred-site.xml configuration property name value as:
mapreduce.admin.map.child.java.opts -Djava.security.egd=file:///dev/urandom
This is a bit sqoop specific, but I think setting JVM option
-Djava.security.egd=file:///dev/urandom"
will help you to resolve issue.
Since the stack does not indicate any ORAs, you have underlying network problems.
Other thing that was causing me this problem was having the HOSTNAME settings wrong. My connection attempt was hanged at:
"main" prio=10 tid=0x00007f7cc8009000 nid=0x2f3a runnable [0x00007f7cce69e000]
java.lang.Thread.State: RUNNABLE
at java.net.Inet4AddressImpl.getLocalHostName(Native Method)
at java.net.InetAddress.getLocalHost(InetAddress.java:1444)
at sun.security.provider.SeedGenerator$1.run(SeedGenerator.java:176)
at sun.security.provider.SeedGenerator$1.run(SeedGenerator.java:162)
at java.security.AccessController.doPrivileged(Native Method)
So make sure you have an entry for your hostname in /etc/hosts/.
If you issue a hostname command like this:
$ hostname
my.server.com
You need a line in your /etc/hosts:
127.0.0.1 my my.server.com
In my application, I was creating BasicDataSource twice & it was failing with java 1.8 32 bit. Issue got resolved when I moved to java 1.8 64 bit
We have a java application which use JBoss 4.2.0, EJB, Hibernate.
In one of our production server, getting OutOfMemory error at regular interval.
We are seeing user defined exceptions in the jboss and server log files. we intentionally throwing this user defined exception from our code when the user hit the particular piece of code. But the number of users and usage is more for this customer. so it throws 400-500 exceptions. After that we are getting a
WARN org.jboss.mq.SpyJMSException: No pong received;
We have already implemented the ExceptionListener to reconnect the topicConnection in case of failure. And also some of the socketexception as below.
2012-10-12 10:33:57,824 ERROR [org.apache.tomcat.util.net.PoolTcpEndpoint] (http-0.0.0.0-8880-Processor1436) Socket error caused by remote host /125.236.40.205 java.net.SocketException: Connection reset
Have analysed the heap dump generated. 4.5 GB memory is occupied by jmsspyobject exception.
We have got the thread dump from jmx console. Do we need to load the copied thread dump to TDA(Thread Dump Analyzer) tool?
Is OOM because of user defined exception in so many numbers? do we need to reduce it ? or user defined exceptions are not cause for the OOM error?
Please find the attached thread log. I am very new to analyze the jboss thread logs. Could anyone help me to do this?
http://www.4shared.com/file/toRiqF1n/perf-info.html
Thanks.