Develop Reference

"java.sql.SQLException: I/O Error: Connection reset" in linux server [duplicate]

This question already has answers here:
Oracle JDBC intermittent Connection Issue
(13 answers)
Closed 5 years ago.
I have a java program which connect to oracle database sometimes it work and sometimes show me this error message :
Exception in thread "main" java.sql.SQLRecoverableException: IO Error: Connection reset
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:498)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:553)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:254)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:528)
How i can solve this problem?

Answer took from oracle forum, here:
java.security.SecureRandom is a standard API provided by sun. Among various methods offered by this class void nextBytes(byte[]).
This method is used for generating random bytes. Oracle 11g JDBC drivers use this API to generate random number during
login. Users using Linux have been encountering SQLException("Io exception: Connection reset").
The problem is two fold:
The JVM tries to list all the files in the /tmp (or alternate tmp directory set by -Djava.io.tmpdir) when
SecureRandom.nextBytes(byte[]) is invoked. If the number of files is large the
method takes a long time
to respond and hence cause the server to timeout
The method void nextBytes(byte[]) uses /dev/random on Linux and on some machines which lack the random
number generating hardware the operation slows down to the extent of bringing the whole login process to
a halt. Ultimately the the user encounters SQLException("Io exception:
Connection reset")
Users upgrading to 11g can encounter this issue if the underlying OS is Linux which is running on a faulty hardware.
The cause of this has not yet been determined exactly. It could either be a problem in the hardware or the fact
that for some reason the software cannot read from dev/random
a solution seems to add this setting to the jvm
-Djava.security.egd=file:/dev/./urandom

Connection reset usually happens when the connection between you app and the database gets disconnected for example network issue or resources that's required.
Check the following post it should give you an idea of what to check.
SQLRecoverableException: I/O Exception: Connection reset

HTTP transport error: java.net.SocketException: Connection reset

A client application has been built using Jdeveloper 10.1.3.2 and it is running on OC4J server. This application is sending data to external server application. It is working for quite long time without any issue. Lately a connection issue occurred and the following stack trace is generated:
com.sun.xml.ws.client.ClientTransportException: HTTP transport error: java.net.SocketException: Connection reset
at com.sun.xml.ws.transport.http.client.HttpClientTransport.getOutput(HttpClientTransport.java:133)
at com.sun.xml.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:153)
at com.sun.xml.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:93)
at com.sun.xml.ws.transport.DeferredTransportPipe.processRequest(DeferredTransportPipe.java:105)
at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:629)
at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:588)
at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:573)
at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:470)
at com.sun.xml.ws.client.Stub.process(Stub.java:319)
at com.sun.xml.ws.client.sei.SEIStub.doProcess(SEIStub.java:157)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:109)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:89)
at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:140)
at $Proxy44.sendRem(Unknown Source)
After goggling I found out a good discussion about the error sockets - What's causing my java.net.SocketException: Connection reset? .One answer in this link says that the issue mostly from the client side because if it is from the server side the exception will be (SocketException reset by peer).
What I did:
I tried out to increase the socket time out for the OC4J with the help of this form How to change OC4J HTTP Timeout. What I did is I changed the propriety oracle.j2ee.http.socket.timeout to be 5000 instead of 500 (10 times longer)
But the error still there. So, any suggestion to over come this issue?
Note: I able to use telnet command for external server IP and Port and it is working fine.
-------------------------------------------------------- Update 1 --------------------------------------------------------
I increase the server clock skew where the client application is running using the following command on server start up:
-Dweblogic.wsee.security.clock.skew=72000000
-Dweblogic.wsee.security.delay.max=72000000
But no luck, problem is not resolved.
-------------------------------------------------------- Update 2 --------------------------------------------------------
I realized that the problem is not from application at all; I test the external URL using SoapUI and I got the same error Connection rest. I think this new update clreay shows that there is nothing wrong with program code. But I need to know where to go or check now. Where is the starting point now to overcome the issue. Any clue will be helpful.

As you can see from Update 2 in the question, the problem was not from the client application because same error occurred from SoapUI.
The problem was that the machine where the client application was running have low bandwidth which was not enough for APIs communication. Using simple speed test , I found out that the upload bandwidth was low comparing to minimum requirements given by server application team.
I concluded this fact by monitoring the network resource using Resource Monitor in Windows while the client application was running and by using online speed check
To solve the issue, the machine bandwidth has to be increased where the client application is running.

Large messages failing in SoapUI - "Error getting response; java.net.SocketException: Connection reset"

I'm using Soap UI 4.6.0 to hit a WCF web service, and when I have really large message payloads, I'm seeing the following error:
Error getting response; java.net.SocketException: Connection reset
The WCF service has around 10 methods, each with progressively larger inputs (eg, 10 int properties, 50 int properties, 100 int properties, etc). This works with the smaller messages, but as they get around 2000-3000 int properties, the error occurs.
The call appears to succeed on the server side, and with this coming from java, I'm assuming I'm butting up against some size limitation/configuration in the client. Is this something I can tweak within Soap UI, the java runtime, or elsewhere?

For me the trick that worked was adding below entry in SoapUI-5.2.0.vmoptions file (it can be found in the bin directory of installa
-Dsoapui.https.protocols=SSLv3,TLSv1.2

Normally a connection reset means that one of the underlying servers timed out waiting for data from another server/application and it reset the connection.
You should try out the suggestions #kroonwijk gave it'll tell you which server is causing the reset and what is causing the server to reset the connection.
Also see What's causing my java.net.SocketException: Connection reset?

If above solutions won't work for you then try this:
Close SoapUI
Go to SoapUi directory for example: C:\Program Files\SmartBear\SoapUI-5.3.0\
Rename directory "jre" to "jre.ignore"
Done. Open SoapUi and it should work now.

Spring-WS and "Error attempting to save SOAPPart"

I have a few web services on a Weblogic 10 server. Each of these is part of a larger system. Running locally and on our qa environment the system works flawless, replies fast, and as expected. Everything looks to be okay.
Before going into production we're going to stress test the system, thus see how much load we can have before reply time becomes to large. When testing the web services (e.g. using front end or SOAPUI) we hit a certain load (e.g. to many replies per sec or something like that, I'm not sure what exactly triggers the system to fail) we get the error listed below. I haven't got the slightest clue as to why. Seconds later the system replies flawless again, so I'm guessing that it has something to do with the number of requests...
Any ideas or hints is much appreciated! I'm lost here, so please - anything will help.
We're running: Weblogic 10.3.2, Spring 2.5.6 (for architectural reasons we cannot upgrade), Spring-WS 1.5.9 (for architectural reasons we cannot upgrade) and Stripes 1.5.4
<11-11-2011 08:43:58 CET> <Error> <HTTP> <BEA-101017> <[ServletContext#11242741[app:salesoverview-ws-web module:salesoverview-ws-web path:/salesoverview-ws-web spec-version:2.5], request: weblogic.servlet.internal.ServletRequestImpl#1fbbfc5[POST /salesoverview-ws-web/services HTTP/1.1 Accept-Encoding: gzip,deflate Content-Type: text/xml;charset=UTF-8 SOAPAction: "" User-Agent: Jakarta Commons-HttpClient/3.1 Content-Length: 425]] Root cause of ServletException.
org.springframework.ws.soap.saaj.SaajSoapMessageException: Could not write message to OutputStream: Error attempting to save SOAPPart. java.io.IOException: java.net.SocketException: Software caused connection abort: socket write error; nested exception is javax.xml.soap.SOAPException: Error attempting to save SOAPPart. java.io.IOException: java.net.SocketException: Software caused connection abort: socket write error
at org.springframework.ws.soap.saaj.SaajSoapMessage.writeTo(SaajSoapMessage.java:169)
at org.springframework.ws.transport.AbstractWebServiceConnection.send(AbstractWebServiceConnection.java:45)
at org.springframework.ws.transport.support.WebServiceMessageReceiverObjectSupport.handleConnection(WebServiceMessageReceiverObjectSupport.java:97)
at org.springframework.ws.transport.http.WebServiceMessageReceiverHandlerAdapter.handle(WebServiceMessageReceiverHandlerAdapter.java:57)
at org.springframework.ws.transport.http.MessageDispatcherServlet.doService(MessageDispatcherServlet.java:230)
Truncated. see log file for complete stacktrace
Caused By: javax.xml.soap.SOAPException: Error attempting to save SOAPPart. java.io.IOException: java.net.SocketException: Software caused connection abort: socket write error
at weblogic.xml.saaj.SOAPMessageImpl.SOAPPart_writeTo(SOAPMessageImpl.java:1011)
at weblogic.xml.saaj.SOAPMessageImpl.writeTo(SOAPMessageImpl.java:816)
at org.springframework.ws.soap.saaj.Saaj13Implementation.writeTo(Saaj13Implementation.java:292)
at org.springframework.ws.soap.saaj.SaajSoapMessage.writeTo(SaajSoapMessage.java:165)
at org.springframework.ws.transport.AbstractWebServiceConnection.send(AbstractWebServiceConnection.java:45)
Truncated. see log file for complete stacktrace
>
By digging BEA-101017 I found a little info about the from the Weblogic error dok - although this doesn't help me:
Error: [context] Root cause of ServletException.
Description: [context] Root cause of ServletException, which the Web
application container caught while servicing the request.
Cause: The Web application container caught an unexpected exception.
Action: Check the exception for the exact error message.

Assuming that the web service from your example doesn't access other web services (and therefore the above trace corresponds to your web service sending the response):
It seems that your web service, via SAAJ, is trying to write to a disconnected (or otherwise unavailable) socket. An usual cause for this is that the client has disconnected while waiting for the server reply.
I'd suggest to:
Check if your client was waiting for too long before receiving the response, that could have caused it to disconnect.
Check if the operating system might be having issues allocating sockets. Use 'netstat' or other monitoring tool (like TCPView on Windows) to check how many sockets are open (most operating system impose limits on the number of sockets allowed per user or globally).
Ensure there are absolutely no network errors during your tests (shouldn't be the case if you are testing on localhost, but otherwise you need to ensure your network devices (routers, switches, other computers) are not dropping connections or packets. Perhaps this is happening when traffic load is high.
Make sure you have no threading conflicts that could cause your web service to use or close other requests' sockets (this would be a rare situation especially if you are using Spring).
Check this thread Official reasons for "Software caused connection abort: socket write error" and other possible causes of "Software caused connection abort" (note that the issue could be specific to your application server and operating system).
Hope that helps.

After debugging a lot I found out that the problem happened due to DB2 issues - we hit a corner of our database, which triggered an internal stack overflow, which then probagated to the Dao and onwards to the SOAP-part (only making it harder to detect due to Spring JDBC templates in the Dao).
A long story short and the issue was an uncaught exception, which by Spring-WS resulted in a "SaajSoapMessageException". The hint came from "Software caused connection abort: socket write error", but happened on the WS side (not client nor the communication between client/server).
Hint: Surround your database with try/catch and catch Exception thus being able to find the exact exception thrown. In my case it threw a DB2 exception ("SQLCode -1218") and this is normally used when you run out of resources (e.g. data source connections). I my case it was the SQL which DB2 didn't like - and really didn't like under load. I can't explain it, but it has to do with DB2s own internal resources - gah, go figure! :)
Thank you jjmontes, for hints and pointers, but it was not the problem in this case.

Error commiting response java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcher.write0(Native Method)

I encounter this error after a number of https requests. Anyone have any idea what could be the reason? It seens to be related to SSL. But previously it was working fine. I really don't understand what could have caused this issue
Error commiting responsejava.io.IOException: Broken pipe at
sun.nio.ch.FileDispatcher.write0(Native Method) at
sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29) at
sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104) at
sun.nio.ch.IOUtil.write(IOUtil.java:75) at
sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:302) at
com.sun.enterprise.server.ss.ASOutputStream.write(ASOutputStream.java:120) at
com.sun.net.ssl.internal.ssl.OutputRecord.writeBuffer(OutputRecord.java:283) at
com.sun.net.ssl.internal.ssl.OutputRecord.write(OutputRecord.java:272) at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:666) at
com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59) at
org.apache.coyote.http11.InternalOutputBuffer.commit(InternalOutputBuffer.java:602) at
com.sun.enterprise.web.connector.grizzly.ProcessorTask.action(ProcessorTask.java:721) at
org.apache.coyote.Response.action(Response.java:188) at
org.apache.coyote.Response.sendHeaders(Response.java:380) at
org.apache.coyote.tomcat5.OutputBuffer.doFlush(OutputBuffer.java:357) at
org.apache.coyote.tomcat5.OutputBuffer.close(OutputBuffer.java:318) at
org.apache.coyote.tomcat5.CoyoteResponse.finishResponse(CoyoteResponse.java:528) at
org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:192) at
com.sun.enterprise.web.connector.grizzly.ProcessorTask.doProcess(ProcessorTask.java:604) at
com.sun.enterprise.web.connector.grizzly.ProcessorTask.process(ProcessorTask.java:475) at
com.sun.enterprise.web.connector.grizzly.ProcessorTask.doTask(ProcessorTask.java:426) at
com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:281) at
com.sun.enterprise.web.connector.grizzly.WorkerThread.run(WorkerThread.java:83

I don't know about sun.nio.ch. but...
This is a standard annoying error you get sometimes in Java web apps. You get this error when requesting a URL and then either hitting stop in your browser or clicking away to another url. The app is complaining that it wasn't able to send you the complete response.

In my case this was happening when I generated an Excel or csv file on the server for local download (crash at HttpServletResponse getOutputStream().flush())
However it is related to the browser configuration, in my case Chrome 32 bits on Windows 7. Nothing on server side.
For 3 days I've investigated in depth my web application, looking for the cause of the problem, checked many apache librairies etc.
Then I tried to perform the same action from an another computer and finally from the same one with FireFox. No problem.
Finally I discovered that the cause was Chrome’s default Cache Size.
I changed it with -disk-cache-size-2147483648 (at the end of the shortcut's target) and problem disappeared.
I hope it can save time to someone.

A Java NIO Pipe is a one-way data connection between two threads. A Pipe has a source channel and a sink channel. You write data to the sink channel. This data can then be read from the source channel.
Now coming to the problem. Whenever sink channel is FULL (reads are NOT fast enough to leave some space in the buffer ), pipe is closed!!
So any writes coming after this point will fail.