Develop Reference

How to make an HTTP request in Java and ignore the body

To be more specific, I mean specifically to just consume the HTTP headers over the network and stop the communication before the client receives the response body.
Example
Client makes a request
GET / HTTP/1.1
Host: example.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/UNAVAILABLE (Java/1.8.0_262)
Accept-Encoding: gzip,deflate
Then the response over the network is just
HTTP/1.1 200 OK
Date: Wed, 23 Sep 2020 22:41:21 GMT
Server: Apache
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: public, max-age=10800
Content-Language: en
Vary: Cookie,Accept-Encoding
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Age: 1865
grace: none
Content-Length: 9213
Connection: keep-alive
Accept-Ranges: bytes

Http protocol has six method, one of the methods is 'HEAD'. You can try use HEAD method instead of GET method.
And another stupid way : declare a web interface, and return null string.Like this:
// a web interface
String result = "";
return result;

spring ResponseEntity limitations?

I have following code (it's a method in a controller) to return search results.
#RequestMapping(value="/findproducts", method = RequestMethod.GET)
#ResponseStatus(HttpStatus.OK)
public #ResponseBody ResponseEntity<FindResponse> findproducts(
#RequestParam(value = "regionId", required=true) String regionId,....)
ResponseEntity<FindResponse> responseEntity = new ResponseEntity<FindResponse>(result, headers, HttpStatus.OK);
return responseEntity;
}
Response:
HTTP/1.1 200 OK Cache-Control: no-cache Cache-Control: no-store Date: Wed, 22 Oct 2014 17:45:27
GMT Pragma: no-cache Transfer-Encoding: chunked Content-Type: application/json;charset=UTF-8
Expires: Thu, 01 Jan 1970 00:00:00 GMT X-Powered-By: Servlet/2.5 JSP/2.1
I am executing the service via SOAP UI.
Service returns results if the search finds 2K results.
AFterwards, response it empty or truncated.
In the RAW console, I am seeing improper JSON.
How to send larger payload?
Any compression is possible?
I am using spring 3.2 release

Apache HttpClient response content length returns -1

Why does the following Code returns -1? Seems that the request failed.
public static void main(String[] args)
{
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet("http://www.google.de");
HttpResponse response;
try
{
response = httpClient.execute(httpGet);
HttpEntity entity = response.getEntity();
EntityUtils.consume(entity);
// Prints -1
System.out.println(entity.getContentLength());
}
catch (ClientProtocolException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
httpGet.releaseConnection();
}
}
And is it possible to get the response as String?

Try running
Header[] headers = response.getAllHeaders();
for (Header header : headers) {
System.out.println(header);
}
It will print
Date: Tue, 10 Sep 2013 19:10:04 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=dad7e2356ddb3b7a:FF=0:TM=1378840204:LM=1378840204:S=vQcLzVPbOOTxfvL4; expires=Thu, 10-Sep-2015 19:10:04 GMT; path=/; domain=.google.de
Set-Cookie: NID=67=S11HcqAV454IGRGMRo-AJpxAPxClJeRs4DRkAJQ5vI3YBh4anN3qS0EVeiYX_4XDTGN-mY86xTBoJ3Ncca7eNSdtGjcaG31pbCOuqsZEQMWwKn-7-6Dnizx395snehdA; expires=Wed, 12-Mar-2014 19:10:04 GMT; path=/; domain=.google.de; HttpOnly
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 80:quic
Transfer-Encoding: chunked
This is not a problem, the page you requested simply doesn't provide a Content-Length header in its response. As such, the HttpEntity#getContentLength() returns -1.
EntityUtils has a number of methods, some of which return a String.
Running curl more recently produces
> curl --head http://www.google.de
HTTP/1.1 200 OK
Date: Fri, 03 Apr 2020 15:38:18 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2020-04-03-15; expires=Sun, 03-May-2020 15:38:18 GMT; path=/; domain=.google.de; Secure
Set-Cookie: NID=201=H8GdKY8_vE5Ehy6qSkmQru13HqdGEj2tvZUFqvTDAVBxFoL4POI0swPtfI45v1TBjrJuAAfbcNMUddniIf9HHituCAFwUqmUFMDwxDYK5qUlcWiB1A64OcGp6PTT6LKur2r_3z-ToSvLf8RZhKWdny6E8SaArMpkaOqUEWp4aoQ; expires=Sat, 03-Oct-2020 15:38:18 GMT; path=/; domain=.google.de; HttpOnly
Transfer-Encoding: chunked
Accept-Ranges: none
Vary: Accept-Encoding
The headers contain a Transfer-Encoding value of chunked. With chunked, the response contains "chunks" preceded by their length. An HTTP client uses those to read the entire response.
The HTTP Specification states that the Content-Length header should not be present when Transfer-Encoding has a value of chunked and MUST be ignored if it is.

Please notice that response header name Transfer-Encoding. Its value is chunked which means data is deliveryed block by block. Transfer-Encoding: chunked and Content-Length does not turn out at the same time.
There are two reason.
Server does not want sent content length.
Or server do not know the content length when it flush a big size data whose size is large than server's buffer.
So when there is no content length header, you can find the size of each chunked block before body of content. For example:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=8A7461DDA53B4C4DD0E89D73219CB5F8; Path=/
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Date: Wed, 18 Mar 2015 07:10:05 GMT
11
helloworld!
3
123
0
Above headers and content tell us, there are two block data. The size of first block is 11. the size of second block is 3. So the content length is 14 at all.
regards,
Xici

If you really want to get the content length without caring about the content, you can do this.
EntityUtils.toByteArray(httpResponse.getEntity()).length

Unable to cache images served by Spring MVC

I am trying to serve some assets using a Spring MVC controller. My assets are database managed and thus have to be served this way. The service looks up the metadata of the asset from the database, reads the file from file system and builds the response.
Here is how my controller looks like.
#Controller
#RequestMapping("/assets")
public class AssetController {
#Autowired
private AssetService assetService;
#RequestMapping("/{assetName:.+}")
public ResponseEntity<byte[]> getAsset(#PathVariable("assetName") String assetName) throws FileNotFoundException, IOException {
Asset asset = assetService.findByName(assetName);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.valueOf(asset.getContentType()));
headers.setCacheControl("max-age=1209600");
headers.setLastModified(asset.getModifiedOn().getTime()); // always in the past
return new ResponseEntity<byte[]>(assetService.toBytes(asset), headers, OK);
}
}
Seems simple and straightforward enough? One would hope to see the browser caching the images. But despite trying all combinations of Cache-Control, Expires, Last-Modified-On and ETag, I have had no success.
Below are the HTTP headers (irrelevant headers removed) spit out during two successive requests.
GET /adarshr-web/assets/Acer.png HTTP/1.1
Host: localhost:8080
Pragma: no-cache
Cache-Control: no-cache
HTTP/1.1 200 OK
Cache-Control: max-age=1209600
Last-Modified: Sun, 21 Jul 2013 11:56:32 GMT
Content-Type: image/png
Date: Tue, 23 Jul 2013 21:22:58 GMT
----------------------------------------------------------
GET /adarshr-web/assets/Acer.png HTTP/1.1
Host: localhost:8080
If-Modified-Since: Sun, 21 Jul 2013 11:56:32 GMT
Cache-Control: max-age=0
HTTP/1.1 200 OK <-- Why not 304 Not Modified?
Cache-Control: max-age=1209600
Last-Modified: Sun, 21 Jul 2013 11:56:32 GMT
Content-Type: image/png
Date: Tue, 23 Jul 2013 21:23:03 GMT
However, when I try the same sequence (Ctrl + F5 for first request and F5 for subsequent ones) on URLs such as
http://www.google.co.uk/images/srpr/logo4w.png (Google's logo)
http://fbstatic-a.akamaihd.net/rsrc.php/v2/yI/r/0PsXdTWc41M.png (Facebook's mobile image)
I see the headers such as these (shown for the Facebook URL) which indicate that the response is being cached by the browser.
GET /rsrc.php/v2/yI/r/0PsXdTWc41M.png HTTP/1.1
Host: fbstatic-a.akamaihd.net
Pragma: no-cache
Cache-Control: no-cache
HTTP/1.1 200 OK
Content-Type: image/png
Last-Modified: Sat, 15 Jun 2013 00:48:42 GMT
Cache-Control: public, max-age=31535893
Expires: Wed, 23 Jul 2014 21:27:47 GMT
Date: Tue, 23 Jul 2013 21:29:34 GMT
----------------------------------------------------------
GET /rsrc.php/v2/yI/r/0PsXdTWc41M.png HTTP/1.1
Host: fbstatic-a.akamaihd.net
If-Modified-Since: Sat, 15 Jun 2013 00:48:42 GMT
Cache-Control: max-age=0
HTTP/1.1 304 Not Modified <-- Note this
Content-Type: image/png
Last-Modified: Sat, 15 Jun 2013 00:48:42 GMT
Cache-Control: public, max-age=31535892
Expires: Wed, 23 Jul 2014 21:27:47 GMT
Date: Tue, 23 Jul 2013 21:29:35 GMT
Notes:
I don't have an <mvc:resources /> section in my Spring config since I am doing exactly the same in my controller. Even adding it doesn't make any difference.
I don't have a org.springframework.web.servlet.mvc.WebContentInterceptor defined in the Spring config again for the reasons above. I have tried adding one with no gain.
I have tried all methods explained in https://developers.google.com/speed/docs/best-practices/caching.
I can replicate this across all browsers.

You'll have to implement the check of the last modified, fortunately Spring makes that pretty easy.
From the Spring Framework Reference
#RequestMapping
public String myHandleMethod(WebRequest webRequest, Model model) {
long lastModified = // 1. application-specific calculation
if (request.checkNotModified(lastModified)) {
// 2. shortcut exit - no further processing necessary
return null;
}
// 3. or otherwise further request processing, actually preparing content
model.addAttribute(...);
return "myViewName";
}

Jersey Expires Header not working

I'm using Jersey bundle 1.11 to provide some RESTful web service.
Each time I browse a REST resource with Chrome, I notice that there's an HTTP Header Expires set to Thu, 01 Jan 1970 01:00:00 CET.
I tried to edit the Response adding:
return Response.ok( myObject ).expires(new Date(System.currentTimeMillis() + 3000)).build();
Unfortunately, this adds another HTTP Header Expires instead of replacing the old one.
What is the problem?

FWIW, I am seeing the exact same behaviour. The container here is JBoss 4.2.3. This is a PUT method with BASIC authentication. My response is generated thus:
Date exp = new Date(System.currentTimeMillis() + lifetime);
return Response.noContent().expires(exp).build();
When invoked with cURL, these are the returned headers:
< HTTP/1.1 204 No Content
< Server: Apache-Coyote/1.1
< Pragma: No-cache
< Cache-Control: no-cache
< Expires: Thu, 01 Jan 1970 01:00:00 CET
< X-Powered-By: Servlet 2.4; JBoss-4.2.3.GA (...
< Expires: Tue, 13 Mar 2012 11:08:54 GMT
< Date: Tue, 13 Mar 2012 11:08:24 GMT
<

This is to prevent your browser from caching of the requested resource.
The date itself is the timestamp with zero seconds, the begin of the UNIX era.

I found that my app server (In this case JBoss 4.2.3.GA) would not allow Jersey to overwrite the header this way.
To workaround:
Inject the response object into the method using a parameter:
#Context javax.servlet.http.HttpServletResponse response
Set the header on the response object rather than using .expires() :
response.setDateHeader("Expires", System.currentTimeMillis() + 14400000);
I used #2 before I called .build() on the ResponseBuilder, not sure if it makes a difference or not when you do this.

I have the same issue. My workaround is:
Inject the response
#Context javax.servlet.http.HttpServletResponse response
Reset the response object
response.reset();
Use the ResponseBuilder to set the headers.
return Response
.ok(icon.getData())
.type(icon.getContentType())
.expires(cal.getTime())
.build();