I am using Apache HttpClient with Failsafe java library. Below is how the (pseudo) code looks like:
RetryPolicy<CloseableHttpResponse> policy = new RetryPolicy<>()
.handleResultIf(/* Response code is 404 */)
.withMaxRetries(5)
.withDelay(Duration.ofSeconds(10));
CloseableHttpResponse response = Failsafe.with(policy).get(() -> httpClient.execute(myRequest));
It's calling a test endpoint at localhost and I have mocked it to do the following:
Return 404 for the first 3 requests
Return 200 for the 4th request
Now, when I execute the above code, I see the following behavior:
HttpClient sends get request, it results in 404
As the response is 404, retry policy kicks in and retries the request
Retried request fails with 400 without actually reaching the proxy
All the subsequent retries fail with 400. The response doesn't have any body
I expect the request in step 2 to hit my mock, however, it fails without hitting it. Does HttpClient cache the response or tries to prevent the subsequent retries?
Apparently, I was setting headers in the request with addHeader method before calling execute for httpClient. This resulted in requests with duplicate Content-Type and Authorization headers.
As these header values are certainly invalid, the requests resulted in 400 error without hitting the url.
Related
I'm new to Java and found a confusing behaviour related with RestTemplate.
It happened with an API returning large body (~5MB) over a quite slow network condition. The code is like below
ResponseEntity<MyEntity[]> result = restTemplate.exchange(url, HttpMethod.GET, entity, MyEntity[].class);
And also a ClientHttpRequestInterceptor is set to log before and after the request.
The confusing thing is that the after request log is logged only a while after remote server giving the response, and the HTTP Status code can be print in the log.
But the above statement took much more time to finally receive the data. Look inside the thread stack, it was reading data from socket.
I also look inside the resttemplate class and found:
response = request.execute();
handleResponse(url, method, response);
if (responseExtractor != null) {
return responseExtractor.extractData(response);
}
It seems to extractData after the execute().
My doubt is:
How does the client side know the status code even before get all the data? It just extracts necessary fields from the top packets?
Since the server has already sent out the response, where the response data is stored during the process?
It stores the data that it receives from the underlying HTTP in memory.
Client side can know what's the status code because with HTTP you get the headers and status code first before the response body. But this doesn't matter with RestTemplate as it promises to give you an object of ResponseEntity in the end, which contains everything from the http response be it status codex headers or body.
RestTemplate is an abstraction over an HttpClient, most client's give you the option to implement callbacks for separate events like onHeadersReceived(), onStatusReceived() etc. But if you are using RestTemplate this means you don't require such fine grained control.
I have a rest service and is consumed using Spring's RestTemplate with Apache HttpClient as,
#Autowired
public ClientImpl(#Value("${base-uri}") final String baseUrl,
#Qualifier("restOperations") RestOperations restTemplate) {
serviceUrl = baseUrl;
restTemplate = restTemplate;
}
private List<ResponseDetails> processRequest(CustomRequest request) throws Exception {
ResponseEntity<ResponseDetails[]> responseEntity = restTemplate.exchange(serviceUrl, HttpMethod.POST, entity, ResponseDetails[].class);
if (responseEntity.getStatusCode().value() == 204) {
return Collections.<ResponseDetails>emptyList();
}
ResponseDetails[] response = responseEntity.getBody();
return response != null ? Lists.newArrayList(response) : Collections.<ResponseDetails>emptyList();
}
When the webservice returns 204 response, then the second service call after 204 response, fails with read timeout.
Spring-web : 4.3.5
I cannot figure out the cause. Any help?
EDIT:
From debug logs,
org.apache.http.impl.conn.DefaultHttpResponseParser;Garbage in
response: ÿþ{"id":0}HTTP/1.1 204 Could not find
Response in server logs by httpclient:
<204 No Content,{Cache-Control=[no-cache], Pragma=[no-cache],
Content-Type=[application/json; charset=utf-16], Expires=[-1],
Server=[some], X-AspNet-Version=[someversion], X-Powered-By=[ASP.NET],
Date=[somedate]}>
HTTP 204 is status code for "No Content", but there seems to be garbage content in the response. This can be seen on your logs:
ÿþ{"id":0}
This is the cause of problems you have.
HTTP client is not expecting anything in the body content of 204 response so does not read it, hence response handler does not see that there is any garbage. However since there is garbage which is not yet consumed the connection stays open until it is read -> the next connection, which tries to reuse the connection, gets hit with a read timeout.
There is a separate thread about a similar problem, where the problem is worked around with a custom HTTP request executor. Using such executor, you could call getBody() to obtain the garbage response body and then next request would not have any issues.
I'm using jersey http client to send requests to some remote API. I need to measure how much time does it take to send request to the server and wait until it gets processed and server returns me some status code. Is there a way how I can do it with jersey?
Here is my code of post method:
public Response post(String targetUrl, Entity entity)
{
return client.target(targetUrl)
.request()
.accept(MediaType.APPLICATION_JSON_TYPE)
.header(SERVER_AUTH, true)
.post(entity);
}
Actually, it was my fault. By default, this client is synchronized so it blocks thread until response is received. But my problem was that URL was incorrect and code immediately returned status 'Resouce not found.'
In a java web-app I write to my HttpServletResponse:
httpResponse.getWriter().write(someJsonString);
httpResponse.getWriter().flush();
The client (apache jmeter in this case) gets the response with the json in the body and status 200 as expected.
If I decide to change the response status:
httpResponse.getWriter().write(someJsonString);
httpResponse.setStatus(Response.Status.NO_CONTENT.getStatusCode());
httpResponse.getWriter().flush();
My client gets the response with the right status (204 in this case) but an empty body for some reason.
What can cause this?
When you send response as 204, it means there is no body.
See w3c rfc
The 204 response MUST NOT include a message-body, and thus is always terminated by the first empty line after the header fields.
It means while sending response either container is not considering body or your client is discarding after reading status in response.
One way could be to check this response in web-browser if possible. With tools like fire-bug or similar in Chrome you could actual check response.
I have an URL and i want only to check the response code of the page and not the complete page source as fetching the the complete page source is quite slow. what is right way to go ?
does getResponseCode() in HttpUrlConnection feches the complete page source or only the header ?
Straight from the docs, HttpUrlConnection#getResponseCode()
Gets the status code from an HTTP response message. For example, in the case of the following status lines:
HTTP/1.0 200 OK
HTTP/1.0 401 Unauthorized
It will return 200 and 401 respectively. Returns -1 if no code can be discerned from the response (i.e., the response is not valid HTTP).
Depends what you're motivation is in making the request. If the request is normally just a GET request for a resource and doesn't have any side effects.
You can perform a HTTP HEAD request instead, which if implemented correctly should get you the same status codes but not the body. (i.e. setRequestMethod(HEAD))
Here method may be PostMethod or GetMethod
you can get status code from an HTTP response message by getStatusCode()
example:-
int statuscode=method.getStatusCode();