I have a rest service and is consumed using Spring's RestTemplate with Apache HttpClient as,
#Autowired
public ClientImpl(#Value("${base-uri}") final String baseUrl,
#Qualifier("restOperations") RestOperations restTemplate) {
serviceUrl = baseUrl;
restTemplate = restTemplate;
}
private List<ResponseDetails> processRequest(CustomRequest request) throws Exception {
ResponseEntity<ResponseDetails[]> responseEntity = restTemplate.exchange(serviceUrl, HttpMethod.POST, entity, ResponseDetails[].class);
if (responseEntity.getStatusCode().value() == 204) {
return Collections.<ResponseDetails>emptyList();
}
ResponseDetails[] response = responseEntity.getBody();
return response != null ? Lists.newArrayList(response) : Collections.<ResponseDetails>emptyList();
}
When the webservice returns 204 response, then the second service call after 204 response, fails with read timeout.
Spring-web : 4.3.5
I cannot figure out the cause. Any help?
EDIT:
From debug logs,
org.apache.http.impl.conn.DefaultHttpResponseParser;Garbage in
response: ÿþ{"id":0}HTTP/1.1 204 Could not find
Response in server logs by httpclient:
<204 No Content,{Cache-Control=[no-cache], Pragma=[no-cache],
Content-Type=[application/json; charset=utf-16], Expires=[-1],
Server=[some], X-AspNet-Version=[someversion], X-Powered-By=[ASP.NET],
Date=[somedate]}>
HTTP 204 is status code for "No Content", but there seems to be garbage content in the response. This can be seen on your logs:
ÿþ{"id":0}
This is the cause of problems you have.
HTTP client is not expecting anything in the body content of 204 response so does not read it, hence response handler does not see that there is any garbage. However since there is garbage which is not yet consumed the connection stays open until it is read -> the next connection, which tries to reuse the connection, gets hit with a read timeout.
There is a separate thread about a similar problem, where the problem is worked around with a custom HTTP request executor. Using such executor, you could call getBody() to obtain the garbage response body and then next request would not have any issues.
Related
I'm trying to create a GET request to retrieve commercial flights from latam. But I only get the answer through insomnia/postman...
I'm making the request through RestTemplate in Java, as shown below:
public class LatamRequest {
public void consumerAPILatam(){
RestTemplate template = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
//h.ttps://www.latamairlines.com/bff/air-offers/offers/search
UriComponents uri = UriComponentsBuilder.newInstance()
.scheme("https")
.host("www.latamairlines.com")
.path("bff/air-offers/offers/search")
.queryParam("sort","RECOMMENDED")
.queryParam("cabinType","Economy")
.queryParam("origin","GRU")
.queryParam("destination","BSB")
.queryParam("inFlightDate","null")
.queryParam("inFrom","null")
.queryParam("inOfferId","null")
.queryParam("outFlightDate","null")
.queryParam("outFrom","2022-11-15T15%3A00%3A00.000Z")
.queryParam("outOfferId","null")
.queryParam("adult","1")
.queryParam("child","0")
.queryParam("infant","0")
.queryParam("redemption","true")
.build();
headers.set("User-Agent", "test");
headers.set("Accept", "*/*");
headers.set("Content-Type", "application/json");
headers.set("X-latam-App-Session-Id", "84196897-1687-4d8c-8e63-083091ac204f");
headers.set("X-latam-Action-Name", "search-result.flightselection.offers-search");
headers.set("X-latam-Application-Name", "web-air-offers");
headers.set("X-latam-Client-Name", "web-air-offers");
headers.set("X-latam-Track-Id", "3a4ae189-e218-4606-bd9e-8b17efc93463");
headers.set("X-latam-Request-Id", "ff44ef24-e6d0-4cb0-984c-df1db18cee19");
headers.set("X-latam-Application-Country", "BR");
headers.set("X-latam-Application-Oc", "br");
headers.set("X-latam-Application-Lang", "pt");
HttpEntity<String> httpEntity = new HttpEntity<>(headers);
ResponseEntity<String> response = template.exchange(uri.toUriString(), HttpMethod.GET, httpEntity, String.class); //todo: No response, no error...
System.out.println(response);
}
}
I don't get any response or status after executing the above block. OBS: With the same parameters and headers I get status 200 in postman or insomnia.
I've tried several ways, but I can't get a response. Does anyone have any ideas for a more effective debug?
The reason that you don't get any response and no exception is because the server side gets your request and keeps holding it and doesn't respond. So you are still in a waiting mode. I ran your request from java code using different http client but used exact your params and I noticed that response never returns. I waited for over 10 minutes and I saw that the program was still running. So I modified the code and added connection timeout for 5 sec and read timeout for 30 seconds. When I ran it it I got read timeout exception after 30 seconds. So the code manages to connect to server side, but server side just doesn't respond. So you are in endless wait. So, I don't know why it works from postman. May be some headers values issues.
I am using Apache HttpClient with Failsafe java library. Below is how the (pseudo) code looks like:
RetryPolicy<CloseableHttpResponse> policy = new RetryPolicy<>()
.handleResultIf(/* Response code is 404 */)
.withMaxRetries(5)
.withDelay(Duration.ofSeconds(10));
CloseableHttpResponse response = Failsafe.with(policy).get(() -> httpClient.execute(myRequest));
It's calling a test endpoint at localhost and I have mocked it to do the following:
Return 404 for the first 3 requests
Return 200 for the 4th request
Now, when I execute the above code, I see the following behavior:
HttpClient sends get request, it results in 404
As the response is 404, retry policy kicks in and retries the request
Retried request fails with 400 without actually reaching the proxy
All the subsequent retries fail with 400. The response doesn't have any body
I expect the request in step 2 to hit my mock, however, it fails without hitting it. Does HttpClient cache the response or tries to prevent the subsequent retries?
Apparently, I was setting headers in the request with addHeader method before calling execute for httpClient. This resulted in requests with duplicate Content-Type and Authorization headers.
As these header values are certainly invalid, the requests resulted in 400 error without hitting the url.
From the docs,
immediately after sending the 101
(Switching Protocols) response, the server is expected to continue
responding to the original request as if it had received its
equivalent within the new protocol (i.e., the server still has an
outstanding request to satisfy after the protocol has been changed,
and is expected to do so without requiring the request to be
repeated).
If the Upgrade header field is received in a GET request
and the server decides to switch protocols, it first responds with a
101 (Switching Protocols) message in HTTP/1.1 and then immediately
follows that with the new protocol's equivalent of a response to a
GET on the target resource. This allows a connection to be upgraded
to protocols with the same semantics as HTTP without the latency cost
of an additional round trip.
I have made my interceptors(Using OkHttp) as follows
public class H2cUpgradeRequestInterceptor implements Interceptor {
private static final Log logger = LogFactory.getLog(H2cUpgradeRequestInterceptor.class);
#Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
Request upgradeRequest = request.newBuilder().addHeader("Connection", "Upgrade, HTTP2-Settings")
.addHeader("Upgrade", "h2c").addHeader("HTTP2-Settings", "AAMAAABkAARAAAAAAAIAAAAA").build();
Response upgradeResponse = chain.proceed(upgradeRequest);
if (upgradeResponse != null && upgradeResponse.code() == HttpStatus.SC_SWITCHING_PROTOCOLS) {
logger.debug("Switching Protocols success"); // Success. Got 101 in reply.
}
upgradeResponse.body(); // returns null
// I am clueless on how to read the response hereafter.
// As per the docs, the 101 status code reply will be followed by a data stream. How can I access this?
return upgradeResponse;
}
}
So basically, for a single request. I will receive 101 as response first if the upgrade is successful, then followed by another response as per the upgraded protocol(if my understanding is right?). Is there anyway to achieve this with OkHttp? Or, Any other client also would be helpful.
Thanks!
I'm new to Java and found a confusing behaviour related with RestTemplate.
It happened with an API returning large body (~5MB) over a quite slow network condition. The code is like below
ResponseEntity<MyEntity[]> result = restTemplate.exchange(url, HttpMethod.GET, entity, MyEntity[].class);
And also a ClientHttpRequestInterceptor is set to log before and after the request.
The confusing thing is that the after request log is logged only a while after remote server giving the response, and the HTTP Status code can be print in the log.
But the above statement took much more time to finally receive the data. Look inside the thread stack, it was reading data from socket.
I also look inside the resttemplate class and found:
response = request.execute();
handleResponse(url, method, response);
if (responseExtractor != null) {
return responseExtractor.extractData(response);
}
It seems to extractData after the execute().
My doubt is:
How does the client side know the status code even before get all the data? It just extracts necessary fields from the top packets?
Since the server has already sent out the response, where the response data is stored during the process?
It stores the data that it receives from the underlying HTTP in memory.
Client side can know what's the status code because with HTTP you get the headers and status code first before the response body. But this doesn't matter with RestTemplate as it promises to give you an object of ResponseEntity in the end, which contains everything from the http response be it status codex headers or body.
RestTemplate is an abstraction over an HttpClient, most client's give you the option to implement callbacks for separate events like onHeadersReceived(), onStatusReceived() etc. But if you are using RestTemplate this means you don't require such fine grained control.
I'm using jersey http client to send requests to some remote API. I need to measure how much time does it take to send request to the server and wait until it gets processed and server returns me some status code. Is there a way how I can do it with jersey?
Here is my code of post method:
public Response post(String targetUrl, Entity entity)
{
return client.target(targetUrl)
.request()
.accept(MediaType.APPLICATION_JSON_TYPE)
.header(SERVER_AUTH, true)
.post(entity);
}
Actually, it was my fault. By default, this client is synchronized so it blocks thread until response is received. But my problem was that URL was incorrect and code immediately returned status 'Resouce not found.'