Develop Reference

PDF File Transfer from server to client: "java.io.IOException: An existing connection was forcibly closed by the remote host" [duplicate]

I am working with a commercial application which is throwing a SocketException with the message,
An existing connection was forcibly closed by the remote host
This happens with a socket connection between client and server. The connection is alive and well, and heaps of data is being transferred, but it then becomes disconnected out of nowhere.
Has anybody seen this before? What could the causes be? I can kind of guess a few causes, but also is there any way to add more into this code to work out what the cause could be?
Any comments / ideas are welcome.
... The latest ...
I have some logging from some .NET tracing,
System.Net.Sockets Verbose: 0 : [8188] Socket#30180123::Send() DateTime=2010-04-07T20:49:48.6317500Z
System.Net.Sockets Error: 0 : [8188] Exception in the Socket#30180123::Send - An existing connection was forcibly closed by the remote host DateTime=2010-04-07T20:49:48.6317500Z
System.Net.Sockets Verbose: 0 : [8188] Exiting Socket#30180123::Send() -> 0#0
Based on other parts of the logging I have seen the fact that it says 0#0 means a packet of 0 bytes length is being sent. But what does that really mean?
One of two possibilities is occurring, and I am not sure which,
The connection is being closed, but data is then being written to the socket, thus creating the exception above. The 0#0 simply means that nothing was sent because the socket was already closed.
The connection is still open, and a packet of zero bytes is being sent (i.e. the code has a bug) and the 0#0 means that a packet of zero bytes is trying to be sent.
What do you reckon? It might be inconclusive I guess, but perhaps someone else has seen this kind of thing?

This generally means that the remote side closed the connection (usually by sending a TCP/IP RST packet). If you're working with a third-party application, the likely causes are:
You are sending malformed data to the application (which could include sending an HTTPS request to an HTTP server)
The network link between the client and server is going down for some reason
You have triggered a bug in the third-party application that caused it to crash
The third-party application has exhausted system resources
It's likely that the first case is what's happening.
You can fire up Wireshark to see exactly what is happening on the wire to narrow down the problem.
Without more specific information, it's unlikely that anyone here can really help you much.

Using TLS 1.2 solved this error.
You can force your application using TLS 1.2 with this (make sure to execute it before calling your service):
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12
Another solution :
Enable strong cryptography in your local machine or server in order to use TLS1.2 because by default it is disabled so only TLS1.0 is used.
To enable strong cryptography , execute these commande in PowerShell with admin privileges :
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Wow6432Node\Microsoft\.NetFramework\v4.0.30319' -Name 'SchUseStrongCrypto' -Value '1' -Type DWord
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\.NetFramework\v4.0.30319' -Name 'SchUseStrongCrypto' -Value '1' -Type DWord
You need to reboot your computer for these changes to take effect.

This is not a bug in your code. It is coming from .Net's Socket implementation. If you use the overloaded implementation of EndReceive as below you will not get this exception.
SocketError errorCode;
int nBytesRec = socket.EndReceive(ar, out errorCode);
if (errorCode != SocketError.Success)
{
nBytesRec = 0;
}

Had the same bug. Actually worked in case the traffic was sent using some proxy (fiddler in my case). Updated .NET framework from 4.5.2 to >=4.6 and now everything works fine. The actual request was:
new WebClient().DownloadData("URL");
The exception was:
SocketException: An existing connection was forcibly closed by the
remote host

Simple solution for this common annoying issue:
Just go to your ".context.cs" file (located under ".context.tt" which located under your "*.edmx" file).
Then, add this line to your constructor:
public DBEntities()
: base("name=DBEntities")
{
this.Configuration.ProxyCreationEnabled = false; // ADD THIS LINE!
}

I've got this exception because of circular reference in entity.In entity that look like
public class Catalog
{
public int Id { get; set; }
public int ParentId { get; set; }
public Catalog Parent { get; set; }
public ICollection<Catalog> ChildCatalogs { get; set; }
}
I added [IgnoreDataMemberAttribute] to the Parent property. And that solved the problem.

If Running In A .Net 4.5.2 Service
For me the issue was compounded because the call was running in a .Net 4.5.2 service. I followed #willmaz suggestion but got a new error.
In running the service with logging turned on, I viewed the handshaking with the target site would initiate ok (and send the bearer token) but on the following step to process the Post call, it would seem to drop the auth token and the site would reply with Unauthorized.
Solution
It turned out that the service pool credentials did not have rights to change TLS (?) and when I put in my local admin account into the pool, it all worked.

I had the same issue and managed to resolve it eventually. In my case, the port that the client sends the request to did not have a SSL cert binding to it. So I fixed the issue by binding a SSL cert to the port on the server side. Once that was done, this exception went away.

For anyone getting this exception while reading data from the stream, this may help. I was getting this exception when reading the HttpResponseMessage in a loop like this:
using (var remoteStream = await response.Content.ReadAsStreamAsync())
using (var content = File.Create(DownloadPath))
{
var buffer = new byte[1024];
int read;
while ((read = await remoteStream.ReadAsync(buffer, 0, buffer.Length)) != 0)
{
await content.WriteAsync(buffer, 0, read);
await content.FlushAsync();
}
}
After some time I found out the culprit was the buffer size, which was too small and didn't play well with my weak Azure instance. What helped was to change the code to:
using (Stream remoteStream = await response.Content.ReadAsStreamAsync())
using (FileStream content = File.Create(DownloadPath))
{
await remoteStream.CopyToAsync(content);
}
CopyTo() method has a default buffer size of 81920. The bigger buffer sped up the process and the errors stopped immediately, most likely because the overall download speeds increased. But why would download speed matter in preventing this error?
It is possible that you get disconnected from the server because the download speeds drop below minimum threshold the server is configured to allow. For example, in case the application you are downloading the file from is hosted on IIS, it can be a problem with http.sys configuration:
"Http.sys is the http protocol stack that IIS uses to perform http communication with clients. It has a timer called MinBytesPerSecond that is responsible for killing a connection if its transfer rate drops below some kb/sec threshold. By default, that threshold is set to 240 kb/sec."
The issue is described in this old blogpost from TFS development team and concerns IIS specifically, but may point you in a right direction. It also mentions an old bug related to this http.sys attribute: link
In case you are using Azure app services and increasing the buffer size does not eliminate the problem, try to scale up your machine as well. You will be allocated more resources including connection bandwidth.

I got the same issue while using .NET Framework 4.5. However, when I update the .NET version to 4.7.2 connection issue was resolved. Maybe this is due to SecurityProtocol support issue.

For me, it was because the app server I was trying to send email from was not added to our company's SMTP server's allowed list.
I just had to put in SMTP access request for that app server.
This is how it was added by the infrastructure team (I don't know how to do these steps myself but this is what they said they did):
1. Log into active L.B.
2. Select: Local Traffic > iRules > Data Group List
3. Select the appropriate Data Group
4. Enter the app server's IP address
5. Select: Add
6. Select: Update
7. Sync config changes

Yet another possibility for this error to occur is if you tried to connect to a third-party server with invalid credentials too many times and a system like Fail2ban is blocking your IP address.

I was trying to connect to the MQTT broker using the GO client,
broker address was given as address + port, or tcp://address:port
Example: ❌
mqtt://test.mosquitto.org
which indicates that you wish to establish an unencrypted connection.
To request MQTT over TLS use one of ssl, tls, mqtts, mqtt+ssl or tcps.
Example: ✅
mqtts://test.mosquitto.org

In my case, enable the IIS server & then restart and check again.

We are using a SpringBoot service. Our restTemplate code looks like below:
#Bean
public RestTemplate restTemplate(final RestTemplateBuilder builder) {
return builder.requestFactory(() -> {
final ConnectionPool okHttpConnectionPool =
new ConnectionPool(50, 30, TimeUnit.SECONDS);
final OkHttpClient okHttpClient =
new OkHttpClient.Builder().connectionPool(okHttpConnectionPool)
// .connectTimeout(30, TimeUnit.SECONDS)
.retryOnConnectionFailure(false).build();
return new OkHttp3ClientHttpRequestFactory(okHttpClient);
}).build();
}
All our call were failing after the ReadTimeout set for the restTemplate. We increased the time, and our issue was resolved.

This error occurred in my application with the CIP-protocol whenever I didn't Send or received data in less than 10s.
This was caused by the use of the forward open method. You can avoid this by working with an other method, or to install an update rate of less the 10s that maintain your forward-open-connection.

Odd unknown host exception

I have a server that is experiencing some wierd exceptions.
The server is calling a backend service and during "normal" operation this works without fault, however if that service goes down for a short timespan my server completely dies.
After the backend service has restarted I am getting unknown host exceptions for all my http calls.
If I restart my server it works again, but it seems odd that I even get the error in the first place.
An example of a call I make:
Client client = ClientBuilder.newClient();
response = client.target(uri)
.request(MediaType.APPLICATION_JSON_TYPE)
.header("XApiKey", catalogApiKey)
.get();
and the stacktrace is rather standard:
java.net.UnknownHostException.
at java.net.InetAddress.getAllByName0(InetAddress.java:1280)
at java.net.InetAddress.getAllByName(InetAddress.java:1192)
at java.net.InetAddress.getAllByName(InetAddress.java:1126)
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.resolveHostname(DefaultClientConnectionOperator.java:259)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:159)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.invoke(ApacheHttpClient4Engine.java:283)
at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.invoke(ClientInvocation.java:436)
at org.jboss.resteasy.client.jaxrs.internal.ClientInvocationBuilder.get(ClientInvocationBuilder.java:159)
if it is relevant I am running on a wildfly server.
What could cause this kind of behaviour?

If you look at the source of InetAddress (e.g. http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/net/InetAddress.java) everywhere that UnknownHostException is instantiated it is instantiated with either a host string or a clearly non-empty string as the constructor parameter.
However, the exception in the stacktrace appears to have an empty message. So I infer that the code is doing a lookup on a hostname that is an empty string.
I would add some code to print the value of uri when this problem occurs. I suspect that is (sporadically) bogus in some interesting way.

Java caches DNS entries. The default is "forever" as defined in $JAVA_HOME/jre/lib/security/java.security under the entry networkaddress.cache.ttl.
I would encourage you to consider reducing the "positive" cache to a small amount, perhaps 10 seconds or so. You can modify the java.security file or have code like:
java.security.Security.setProperty("networkaddress.cache.ttl", "10");
In your code. There is also a "negative" cache that caches entries that have failed. It may take some experimentation in your environment with the positive and negative caches to find the right setting.

Amazon sqs throwing exception connection reset

I am using AmazonSQSAsyncClient to connect with Amazon SQS, but sometimes I see the following execution in the logs:
INFO [AmazonHttpClient:444] Unable to execute HTTP request: Connection reset
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:755)
at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doReceiveResponse(SdkHttpRequestExecutor.java:66)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:712)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:517)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:380)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:229)
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2169)
at com.amazonaws.services.sqs.AmazonSQSClient.getQueueUrl(AmazonSQSClient.java:468)
at com.amazonaws.services.sqs.AmazonSQSClient.getQueueUrl(AmazonSQSClient.java:1476)
I am using the AmazonSQSAsyncClient through out the application as singleton.
Code snippet is below.
static{
if(sqsObj == null){
sqsObj = new AmazonSQSAsyncClient(new ClasspathPropertiesFileCredentialsProvider("app.properties"));
sqsObj.setRegion(Region.getRegion(Regions.valueOf("sample region"));
}
}
By using sqsobj, I am doing operations like create queue, send message and receive messages. It works fine but some time throws the exception as above. After restarting the application it works fine for sometime.
I am using aws-java-sdk-1.7.1.
Please suggest on this.

The "connection reset" means that the other party (i.e. the server) closed the connection.
Internally the SDK builds and reuses a http connection to not have to open a connection every time you make a request.
The log message tells you that the connection was reset, but normally this is not something to worry about. The SDK will retry automatically in cases like this. So if you are only seeing this sporadically in the logs but everything works as expected you should probably not worry about it

How to Debug HttpRequestExecutor.execute(request, connection, context);.?

The below lines of code
HttpRequestExecutor httpexecutor = new HttpRequestExecutor();
HttpResponse response2 = httpexecutor.execute(request2, conn, context);
log.info("Status Line "+response2.getStatusLine());
while executing the second statement my request/response is blocked by proxy. How to debug this.?
While the same statement executing from my local environment where no proxy is configured, am able to see
the log returns the below code
"Status Line HTTP/1.1 200 OK"

I think what you are looking for is to set a timeout on the request - basically telling the executor to give up if no connection can be established within a certain period of time.
This SO article covers this: Java HTTP Client Request with defined timeout
the accepted answer has important information about the fact that Apache's implementation has two timeouts - be sure to read that.
Note that it won't be possible for you to actually distinguish between a connection failure that was caused by some other problem - all you will know is that your attempt timed out. In your response to the user, you'll probably wind up saying "Unable to connect to host XXXXX. Possible causes are that your Internet connection is down, or that you have a proxy server that prevents outbound connections to that host."

Glassfish throws com.sun.xml.ws.client.ClientTransportException: The server sent HTTP status code 500: Internal Server Error

I have deployed jax-ws web service into glassfish 3.1.My client request for the service method which returns 5000 to 10000 list of objects.In between processing server throws ClientTransportException with following stack-trace.
com.sun.xml.ws.client.ClientTransportException: The server sent HTTP status code 500: Internal Server Error
at com.sun.xml.ws.transport.http.client.HttpTransportPipe.createResponsePacket(HttpTransportPipe.java:314)
at com.sun.xml.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:265)
at com.sun.xml.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:184)
at com.sun.xml.ws.transport.DeferredTransportPipe.processRequest(DeferredTransportPipe.java:109)
at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:641)
at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:600)
at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:585)
at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:482)
at com.sun.xml.ws.client.Stub.process(Stub.java:323)
at com.sun.xml.ws.client.sei.SEIStub.doProcess(SEIStub.java:161)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:113)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:93)
at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:144)
at $Proxy190.webservicemethodcall(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
I try to monitor the glassfish request but it show errorcount 1 in request statistics but it don't provide me any proper reason of errorcount.
It has been observed in multiple test,i got the client Transport at client but at server the method thread separately working properly up to last line.It don't aware of broken connection.
I think that the connection is broken so thread can not return the response at last.
Note : If return response is small like up to 3000 objects it works fine.But i don't thing it is matter of size.It is matter of timeout.My request connection is broken before creating responce
Please help me

A HTTP 500 means Internal Server Error, which is no fault of your client. Something about your request is failing on the server. You should look there for more info. Your client side stack trace isn't going to help.

You can try any combination of the following:
While your request is running, run a continuous ping from the client. You should see a break in the pinging (or an increase in TTL at least to confirm the theory)
Set the following JVM property for a dump of the HTTP message exchange between server and client
-Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true
Try TCPMon
Implement a JAX-WS SOAP Handler to capture the exact moment when the pipe runs dry. This might have the extra benefit of throwing a meaningful exception when the handler attempts to log a message and gets burnt by the absent message

If your logging policy is not clearly defined, your exception might be silently swallowed on server side.
I would try to add a try/catch block to detect where this happens (and eventually remove it later if you improve your logging strategy)
public returnType yourMethod(){
try {
.... all your code
}catch (final Throwable t) {
log.error("Failed to wait for device update: " + t.getMessage());
//eventually re-throw the error
}
}