Develop Reference

What is the maximum connection timeout to any server?

I have this simple Spring boot based web app that downloads data from several APIs. Some of them don't respond in time, since my connectionTimeout is set to somewhat 4 seconds.
As soon as I get rid of connectionTimeout setting, I'm getting an exceptions after 20 or so seconds.
So, my question is, for how long am I able to try to connect to an API and what does it depend on? Where do those 20 seconds come from? What if an API responds after 40 minutes of time and I won't be able to catch that specific moment and just gonna lose data. I don't want that to happen. What are my options?
Here's the code to set the connection. Nothing special.
HttpComponentsClientHttpRequestFactory clientHttpRequestFactory = new HttpComponentsClientHttpRequestFactory(HttpClientBuilder.create().build());
clientHttpRequestFactory.setConnectTimeout(4000);
RestTemplate restTemplate = new RestTemplate(clientHttpRequestFactory);
Then I retrieve the values via:
myObject.setJsonString(restTemplate.getForObject(url, String.class));

Try increasing your timeout. 4 seconds is too little.
It will need to connect, formulate data and return. So 4 seconds is just for connecting, by the time it attempts to return anything, your application has already disconnected.
Set it to 20 seconds to test it. You can set it to much longer to give the API enough time to complete. This does not mean you app will use up all of the connection timeout time. It will finish as soon as a result is returned. Also API's are not designed to take long. They will perform the task and return the result as fast as possible

Connection timeout means that your program couldn't connect to the server at all within the time specified.
The timeout can be configured, as, like you say, some systems may take a longer time to connect to, and if this is known in advance, it can be allowed for. Otherwise the timeout serves as a guard to prevent the application from waiting forever, which in most cases doesn't really give a good user experience.
A separate timeout can normally be configured for reading data (socket timeout). They are not inclusive of each other.
To solve your problem:
Check that the server is running and accepting incoming connections.
You might want to use curl or depending on what it is simply your browser to try and connect.
If one tool can connect, but the other can't, check your firewall settings and ensure that outgoing connections from your Java program are permitted. The easiest way to test whether this is a problem is to disable anti virus and firewall tools temporarily. If this allows the connection, you'll either need to leave the FW off, or better add a corresponding exception.
Leave the timeout on a higher setting (or try setting it to 0, which is interpreted as infinite) while testing. Once you have it working, you can consider tweaking it to reflect your server spec and usability requirements.
Edit:
I realised that this doesn't necessarily help, as you did ultimately connect. I'll leave the above standing as general info.
for how long am I able to try to connect to an API and what does it depend on?
Most likely the server that the API is hosted on. If it is overloaded, response time may lengthen.
Where do those 20 seconds come from?
Again this depends on the API server. It might be random, or it may be processing each request for a fixed period of time before finding itself in an error state. In this case that might take 20 seconds each time.
What if an API responds after 40 minutes of time and I won't be able to catch that specific moment and just gonna lose data. I don't want that to happen. What are my options?
Use a more reliable API, possibly paying for a service guarantee.
Tweak your connection and socket timeouts to allow for the capabilities of the server side, if known in advance.
If the response is really 40 minutes, it is a really poor service, but moving on with that assumption - if the dataset is that large, explore whether the API offers a streaming callback, whereby you pass in an OutputStream into the API's library methods, to which it will (asynchronously) write the response when it is ready.
Keep in mind that connection and socket timeout are separate things. Once you have connected, the connection timeout becomes irrelevant (socket is established). As long as you begin to receive and continue to receive data (packet to packet) within the socket timeout, the socket timeout won't be triggered either.
Use infinite timeouts (set to 0), but this could lead to poor usability within your applications, as well as resource leaks if a server is in fact offline and will never respond. In that case you will be left with dangling connections.

The default and maximum has nothing to do with the the server. It depends on the client platform, but it is around a minute. You can decrease it, but not increase it. Four seconds is far too short. It should be measured in tens of seconds in most circumstances.
And absent or longer connection timeouts do not cause server errors of any kind. You are barking up the wrong tree here.

Unable to make a non blocking http request in minecraft mod

I'm creating a mod that needs to call a GET request to an endpoint.
I don't care about any result, I just want the request to be sent.
Right now I'm using
HttpClient httpClient = HttpClientBuilder.create().build();
HttpGet request = new HttpGet(url);
And it will block. Because the api takes some time to respond that's not good.
I saw that there's a library called async-http-client but I can't add libraries to my project.
I guess I have to create threads in my mod but that doesn't look like the best solution to me as minecraft mods shouldn't make new threads.
Is there any java package that won't care about the response?

Sending network traffic will always block until it's completed - there's no way around that. In this case it should be perfectly fine to create a new thread to do that actual work - the thread will just block (and not waste CPU resources) for most of the time.
Note that async-http-client will just create it's own threads to do it's work, so it won't help get around this restriction.

High lag with zeromq

I am facing a strange issue with ZMQ, which I'm just not able to debug. These are the components:
Java ZMQ Server - Almost an exact copy of this example. There are a hundred worker threads.
PHP Client - Simple request reply with a REQ socket. This is the request flow:
$zcontext = new ZMQContext();
$socket = new ZMQSocket($zcontext, ZMQ::SOCKET_REQ);
$socket->connect(<address>);
$startTime = microtime(true);
$socket->send(<request>);
$result = $socket->recv();
$totalTime = microtime(true) - $startTime;
The ZMQ sockets use TCP and both the server and client are on the same machine.
The PHP script is served by apache and I am load testing using apache benchmark. I make 5000 requests with a concurrency of 200. On the PHP client I log the time it takes for the request reply ($totalTime). In most of the cases, this time is very low (sub 500ms), but occasionally it takes a really long time - sometimes even 60 secs (for send + receive).
I added some extra logging to find out where the issue is happening, and it turns out that whenever it takes really long, almost all the time is between PHP's send and Java's receive - so packets are taking really long to reach the server.
I'm not setting any special ZMQ settings, or otherwise doing anything unusual so I don't know what is causing the issue. It should also be noted that the issue persists even at lower concurrencies (I tested at 100 and 150 too), but the max request times are lower.
Sorry if the question seems vague - I'll provide any other details that may be needed.

Apache HttpComponents code causing thread to block

I am currently running a program that will download the source code from a website using Apache HttpComponents. I will be downloading a lot (10,000s) and so am using multiple threads to do this.
Sometimes all threads die (join) and sometimes they don't. Through debugging I have determined that the line
CloseableHttpResponse response = httpClient.execute(httpget,context);
is the problem. Does anybody know how I can set a timeout for the this line, or why this line is blocking thread execution?

There can be various reasons for threads getting stuck in an i/o operation, incorrect timeout settings being the most likely cause. One can set desired timeout values using RequestConfig class. However if all threads get blocked at once inside #execute method connection leak (connection pool depletion) would be more likely. Make sure that you always close CloseableHttpResponse instances even if do not care about the response or its content. You can find out more details about request execution by turning on wire / context logging as described in the logging guide

I use the following timeout settings in HttpConnectionParams in my code (HttpParams are given to the HttpClient constructor):
org.apache.http.params.HttpConnectionParams.setConnectionTimeout(HttpParams, int)
org.apache.http.params.HttpConnectionParams.setSoTimeout(HttpParams, int)
A problem which I discovered when connecting to the same host with multiple threads, that blocking/timeouts occur when the maxPerRoute setting is lower than the number of threads. Have a look at PoolingClientConnectionManager:
org.apache.http.impl.conn.PoolingClientConnectionManager.setDefaultMaxPerRoute(int)

How to make work HttpResponse faster?

I am working on a project in which "I have to get 4096 bytes of data to server" from "server" every "between 1-millisecond to 10-millisecond".But it's "taking too much time" i.e "around 300ms - 700ms" which causes my application to lose data.
I am using below snippet
HttpClient client = new DefaultHttpClient();
HttpPost request = new HttpPost("http://192.168.1.40/ping");
HttpResponse response = client.execute(request);
The HttpResponse is only taking too much time i.e around 300ms - 700ms.
How I can get response faster ?
Instead of this what else I can use to get a response from sever faster then this?
Please let me know any solution or way to solve it.
I have done google, gone through other ways like DataOutputStream and ByteOutputStream but no use of this, it also taking too much time then HttpResponse.
Help will be appreciated.

Before you can make the responses faster, you are going to need to investigate and understand why they are currently taking a long time. Roughly speaking, it could be:
the client side taking a long time to create the request and/or preocess the result (seems unlikely ...)
a slow android network protocol stack
a problem with your local networking (e.g. WiFi) or your telecoms provider
a congested / overloaded server or server-side network, or
something pessimal in the server implementation.
Do things like:
try the request from a web browser on a conventional PC and use the browser's web-developer stuff to try to tease out whether/why the request is taking a long time ...
look in the server-side logs and/or monitoring for request load and timing information
other suggestions please
Implementing SPDY might help, but it is unlikely to change response times in the order of 500ms to a couple of tens of milliseconds. The problem seems more fundamental than "HTTP is old and slow". And the same reasoning applies to all of the other suggestions that people have made.

This is not possible. You are recreating a connection every time.
You need to hold a persistent connection with the server. Try creating a persistent http connection.
If that doesn't work you can try sending raw udp packets (or anything else). It will be harder but it will take less time.

#sheldonCooper answer is right if the server enables SPDY. Also you can add Gzip compression. It has been added to all requests after GingerBread but you could add it for former SDK versions : http://android-developers.blogspot.fr/2011/09/androids-http-clients.html

Use SPDY protocol. This would improve your response time.
I think in your case you can use websockets so that you would not have to create a connection each time and the live connection is available every time.