Develop Reference

Is there a way to simulate Socket and Connection timeout?

I have a certain piece of code that integrates with a third party using HTTP connection, which handles socket timeout and connection timeout differently.
I have been trying to simulate and test all the scenarios which could arise from the third party. was able to test connection timeout by connecting to a port which is blocked by the servers firewall e.g. port 81.
However I'm unable to simulate a socket timeout. If my understanding is not wrong socket timeout is associated with continuous packet flow, and the connection dropping in between. Is there a way I can simulate this?

So we are talking about to kinds of timeouts here, one is to connect to the server (connect timeout), the other timeout will happen when no data is send or received via the socket for a while (idle timeout).
Node sockets have a socket timeout, that can be used to synthesize both the connect and the idle timeout. This can be done by setting the socket timeout to the connect timeout and then when connected, setting it to the idle timeout.
example:
const request = http.request(url, {
timeout: connectTimeout,
});
request.setTimeout(idleTimeout);
This works because the timeout in the options is set immediately when creating the socket, the setTimeout function is run on the socket when connected!
Anyway, the question was about how to test the connect timeout. Ok so let's first park the idle timeout. We can simply test that by not sending any data for some time, that would cause the timeout. Check!
The connect timeout is a bit harder to test, the first thing that comes to mind is that we need a place to connect to that will not error, but also not connect. This would cause a timeout. But how the hell do we simulate that, in node?
If we think a little bit outside the box then we might figure out that this timeout is about the time it takes to connect. It does not matter why the connection takes as long as it does. We simply need to delay the time it takes to connect. This is not necessarily a server thing, we could also do it on the client. After all this is the part connecting, if we can delay it there, we can test the timeout.
So how could we delay the connection on the client side? Well, we can use the DNS lookup for that. Before the connection is made, a DNS lookup is done. If we simply delay that by 5 seconds or so we can test for the connect timeout very easily.
This is what the code could look like:
import * as dns from "dns";
import * as http from "http";
const url = new URL("http://localhost:8080");
const request = http.request(url, {
timeout: 3 * 1000, // connect timeout
lookup(hostname, options, callback) {
setTimeout(
() => dns.lookup(hostname, options, callback),
5 * 1000,
);
},
});
request.setTimeout(10 * 1000); // idle timeout
request.addListener("timeout", () => {
const message = !request.socket || request.socket.connecting ?
`connect timeout while connecting to ${url.href}` :
`idle timeout while connected to ${url.href}`;
request.destroy(new Error(message));
});
In my projects I usually use an agent that I inject. The agent then has the delayed lookup. Like this:
import * as dns from "dns";
import * as http from "http";
const url = new URL("http://localhost:8080");
const agent = new http.Agent({
lookup(hostname, options, callback) {
setTimeout(
() => dns.lookup(hostname, options, callback),
5 * 1000,
);
},
});
const request = http.request(url, {
timeout: 3 * 1000, // connect timeout
agent,
});
request.setTimeout(10 * 1000); // idle timeout
request.addListener("timeout", () => {
const message = !request.socket || request.socket.connecting ?
`connect timeout while connecting to ${url.href}` :
`idle timeout while connected to ${url.href}`;
request.destroy(new Error(message));
});
Happy coding!

"Connection timeout" determines how long it may take for a TCP connection to be established and this all happens before any HTTP related data is send over the line. By connecting to a blocked port, you have only partially tested the connection timeout since no connection was being made. Typically, a TCP connection on your local network is created (established) very fast. But when connecting to a server on the other side of the world, establishing a TCP connection can take seconds.
"Socket timeout" is a somewhat misleading name - it just determines how long you (the client) will wait for an answer (data) from the server. In other words, how long the Socket.read() function will block while waiting for data.
Properly testing these functions involves creating a server socket or a (HTTP) web-server that you can modify to be very slow. Describing how to create and use a server socket for connection timeout testing (if that is possible) is too much to answer here, but socket timeout testing is a common question - see for example here (I just googled "mock web server for testing timeouts") which leads to a tool like MockWebServer. "MockWebServer" might have an option for testing connection timeouts as well (I have not used "MockWebServer"), but if not, another tool might have.
On a final note: it is good you are testing your usage of the third-party HTTP library with respect to timeout settings, even if this takes some effort. The worst that can happen is that a socket timeout setting in your code is somehow not used by the library and the default socket timeout of "wait forever" is used. That can result in your application doing absolutely nothing ("hanging") for no apparent reason.

Is there any way to distinguish a connection timeout from a socket timeout?

The Apache HttpClient framework allows you to specify a connection timeout and a socket timeout, like so:
final HttpParams httpParams = new BasicHttpParams();
if (connectionTimeout > 0) {
HttpConnectionParams.setConnectionTimeout(httpParams, connectionTimeout);
}
if (socketTimeout > 0) {
HttpConnectionParams.setSoTimeout(httpParams, socketTimeout);
}
HttpClient client = new DefaultHttpClient(httpParams);
I would like to fine-tune my connection timeouts. This question explains the difference between the settings nicely, but provides no help for debugging the exceptions that get thrown.
My question is, what exception gets thrown for each type of timeout? I have a java.net.SocketTimeoutException; is this from a connection timeout or a socket timeout? I suspect it's the latter, especially since the message says "Read timed out". What exception will be thrown for a connection timeout? TimeoutException? ConnectException? Why aren't the relationships between timeout settings and exceptions thrown captured in the Apache Http Docs?
The direct parent classes for SocketTimeoutException are InterruptedIoException and IOException; the former has only the SocketTimeoutException for a subclass, but the IOException has many subclasses. Is there another reference that covers what all flavors of exceptions one can expect to receive from an HttpClient execute method? It only declares that it can throw an IOException, which is really very broad.

I haven't gone thru the relationship between classes in Java API, but I think what you need is
java.net.ConnectException : Packet loss due to wrong network, network overload, too many request to server, firewall.
java.net.SocketTimeoutException : The socket timeout is the amount of time to keep the server socket open while data is being transferred back to the caller. This could even be the server is still processing and writing back data but it's taking rather long and the client has just timed out waiting for it.

How to Debug HttpRequestExecutor.execute(request, connection, context);.?

The below lines of code
HttpRequestExecutor httpexecutor = new HttpRequestExecutor();
HttpResponse response2 = httpexecutor.execute(request2, conn, context);
log.info("Status Line "+response2.getStatusLine());
while executing the second statement my request/response is blocked by proxy. How to debug this.?
While the same statement executing from my local environment where no proxy is configured, am able to see
the log returns the below code
"Status Line HTTP/1.1 200 OK"

I think what you are looking for is to set a timeout on the request - basically telling the executor to give up if no connection can be established within a certain period of time.
This SO article covers this: Java HTTP Client Request with defined timeout
the accepted answer has important information about the fact that Apache's implementation has two timeouts - be sure to read that.
Note that it won't be possible for you to actually distinguish between a connection failure that was caused by some other problem - all you will know is that your attempt timed out. In your response to the user, you'll probably wind up saying "Unable to connect to host XXXXX. Possible causes are that your Internet connection is down, or that you have a proxy server that prevents outbound connections to that host."

How to debug SocketTimeoutException?

We are getting a java.net.SocketTimeoutException on server B when client A connects to server B. No idea why. The client is sending data to the server and the server then throws this exception. How would one troubleshoot this issue?
Note currently this has happened only once. Not sure if this is reproduceable. Attempting to setup the test again..

I had same problems, when my users used 3G or 2G network. It means, that you send request to server, and can't estabilish connection, because of weak internet signal. You can increase timeouts on your connection
URLConnection connection;
int timeout = 30 * 1000;
connection.setConnectTimeout(timeout);
connection.setReadTimeout(timeout);
But if you have weaaak weeeaaaak internet connection, timeouts does not help you.
I'm just created 1 testFunction in WebService (or you can use one of yours) for testing connection with server before calling another required functions, and if I get SockectTimeoutException calling this function - just report to user notification "Weak internet connection!".

No data arrived at the receiver within the timeout period. That's all it means. Debugging it means finding out why the data you think was sent wasn't sent. A missing flush() for example.

java.net.SocketException Connection timed out error

I am getting below error when I am trying to connect to a TCP server. My programs tries to open around 300-400 connections using diffferent threads and this is happening during 250th thread. Each thread uses its own connection to send and receive data.
java.net.SocketException: Connection timed out:could be due to invalid address
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:372)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:233)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:220)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:385)
Here is the code I have that a thread uses to get socket:
socket = new Socket(my_hostName, my_port);
Is there any default limit on number of connections that a TCP server can have at one time? If not how to solve this type of problems?

You could be getting a connection timeout if the server has a ServerSocket bound to the port you are connecting to, but is not accepting the connection.
If it always happens with the 250th connection, maybe the server is set up to only accept 250 connections. Someone has to disconnect so you can connect. Or you can increase the timeout; instead of creating the socket like that, create the socket with the empty constructor and then use the connect() method:
Socket s = new Socket();
s.connect(new InetSocketAddress(my_hostName, my_port), 90000);
Default connection timeout is 30 seconds; the code above waits 90 seconds to connect, then throws the exception if the connection cannot be established.
You could also set a lower connection timeout and do something else when you catch that exception...

Why all the connections? Is this a test program? In which case be aware that opening large numbers of connections from a single client stresses the client in ways that aren't exercised by real systems with large numbers of different client hosts, so test results from that kind of client aren't all that valid. You could be running out of client ports, or some other client resource.
If it isn't a test program, same question. Why all the connections? You'd be better off running a connection pool and reusing a much smaller number of connections serially. The network only has so much bandwidth after all; dividing it by 400 isn't very useful.