I have following code (it's a method in a controller) to return search results.
#RequestMapping(value="/findproducts", method = RequestMethod.GET)
#ResponseStatus(HttpStatus.OK)
public #ResponseBody ResponseEntity<FindResponse> findproducts(
#RequestParam(value = "regionId", required=true) String regionId,....)
ResponseEntity<FindResponse> responseEntity = new ResponseEntity<FindResponse>(result, headers, HttpStatus.OK);
return responseEntity;
}
Response:
HTTP/1.1 200 OK Cache-Control: no-cache Cache-Control: no-store Date: Wed, 22 Oct 2014 17:45:27
GMT Pragma: no-cache Transfer-Encoding: chunked Content-Type: application/json;charset=UTF-8
Expires: Thu, 01 Jan 1970 00:00:00 GMT X-Powered-By: Servlet/2.5 JSP/2.1
I am executing the service via SOAP UI.
Service returns results if the search finds 2K results.
AFterwards, response it empty or truncated.
In the RAW console, I am seeing improper JSON.
How to send larger payload?
Any compression is possible?
I am using spring 3.2 release
Related
To be more specific, I mean specifically to just consume the HTTP headers over the network and stop the communication before the client receives the response body.
Example
Client makes a request
GET / HTTP/1.1
Host: example.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/UNAVAILABLE (Java/1.8.0_262)
Accept-Encoding: gzip,deflate
Then the response over the network is just
HTTP/1.1 200 OK
Date: Wed, 23 Sep 2020 22:41:21 GMT
Server: Apache
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: public, max-age=10800
Content-Language: en
Vary: Cookie,Accept-Encoding
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Age: 1865
grace: none
Content-Length: 9213
Connection: keep-alive
Accept-Ranges: bytes
Http protocol has six method, one of the methods is 'HEAD'. You can try use HEAD method instead of GET method.
And another stupid way : declare a web interface, and return null string.Like this:
// a web interface
String result = "";
return result;
I am using the Dank Memer imgen api to manipulate images.
To perform the GET request, i used the Unirest-Java. (https://kong.github.io/unirest-java/)
The code looks like this:
HttpResponse<JsonNode> response = Unirest.get("https://dankmemer.services/api/changemymind")
.header("Authorization", "tokenhere")
.queryString("text", "I am a human")
.asJson();
If i request the Headers with response.getHeaders() it returns this:
Date: Mon, 02 Dec 2019 17:07:08 GMT
Content-Type: image/jpeg
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=dd737927432f802b76c89b2fa8ee72e171575306428; expires=Wed, 01-Jan-20 17:07:08 GMT; path=/; domain=.dankmemer.services; HttpOnly; Secure
Cache-Control: public, max-age=43200
Expires: Tue, 03 Dec 2019 05:07:08 GMT
X-RateLimit-Limit: 5
X-RateLimit-Remaining: 4
X-RateLimit-Reset: 1575306429645
X-Global-RateLimit-Limit: 300
X-Global-RateLimit-Remaining: 299
X-Global-RateLimit-Reset: 1575306488643
CF-Cache-Status: DYNAMIC
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 53eee53a48a5d709-FRA
However i need to get the response body which returns null when i request it.
response.getBody();
// returns null.
// Also returns null when i put: response.getBody.toString();
How do i display the body?
I tried getting the response as a file and store it.
This worked perfectly.
HttpResponse<File> response = Unirest.get("https://dankmemer.services/api/changemymind")
.header("Authorization", "tokenhere")
.queryString("text", "i am a human" )
.asFile("C:\\Users\\PC\\Some Folder\\changemymind.jpg");
I use java (and http client, or other lib). I Want doing a GET request to a web server (sample http://MyDomain:8080/login.jsp).
Now, I have <html> .... </html> and any headers.
How find any informations (Tomcat 7.0.75, Linux 2016.09 v2.5.0, Java 8.0.233.3...) of target Web server by java httpClient?
My java code is:
public class GetWebServerInfo {
public static void main(String[] args) throws ClientProtocolException,
IOException {
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet("http://www.mydomain...");
HttpResponse response = client.execute(request);
System.out.println(response);
Header[] headers = response.getAllHeaders();
for (Header header : headers) {
System.out.println("" + header);
}
}
}
My response in console is:
Date: Sun, 12 Feb 2017 00:11:41 GMT
Server: Apache-Coyote/1.1
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Application-Context: application:prod:8080
Last-Modified: Sat, 11 Feb 2017 23:36:56 GMT
Accept-Ranges: bytes
Content-Type: text/html;charset=UTF-8
Content-Length: 3426
Keep-Alive: timeout=60, max=100
Connection: Keep-Alive
I do not have Tomcat 8, ....
Different servers may or may not return headers telling you about themselves. eg.
Server:Microsoft-IIS/8.5
X-Powered-By:ARR/3.0, ASP.NET
or
server:Oracle-Application-Server-11g
x-powered-by:Servlet/2.5 JSP/2.1
You can't demand information and I don't think many servers will be telling you about their operating system. "Why is that any of your business?"
I recommend you print out all the returned headers and make requests to different servers and see what you get, and what can be of use to you.
How i can i display all the HTTP Headers when using the DefaultHTTPClient?
Why does the following Code returns -1? Seems that the request failed.
public static void main(String[] args)
{
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet("http://www.google.de");
HttpResponse response;
try
{
response = httpClient.execute(httpGet);
HttpEntity entity = response.getEntity();
EntityUtils.consume(entity);
// Prints -1
System.out.println(entity.getContentLength());
}
catch (ClientProtocolException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
httpGet.releaseConnection();
}
}
And is it possible to get the response as String?
Try running
Header[] headers = response.getAllHeaders();
for (Header header : headers) {
System.out.println(header);
}
It will print
Date: Tue, 10 Sep 2013 19:10:04 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=dad7e2356ddb3b7a:FF=0:TM=1378840204:LM=1378840204:S=vQcLzVPbOOTxfvL4; expires=Thu, 10-Sep-2015 19:10:04 GMT; path=/; domain=.google.de
Set-Cookie: NID=67=S11HcqAV454IGRGMRo-AJpxAPxClJeRs4DRkAJQ5vI3YBh4anN3qS0EVeiYX_4XDTGN-mY86xTBoJ3Ncca7eNSdtGjcaG31pbCOuqsZEQMWwKn-7-6Dnizx395snehdA; expires=Wed, 12-Mar-2014 19:10:04 GMT; path=/; domain=.google.de; HttpOnly
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 80:quic
Transfer-Encoding: chunked
This is not a problem, the page you requested simply doesn't provide a Content-Length header in its response. As such, the HttpEntity#getContentLength() returns -1.
EntityUtils has a number of methods, some of which return a String.
Running curl more recently produces
> curl --head http://www.google.de
HTTP/1.1 200 OK
Date: Fri, 03 Apr 2020 15:38:18 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2020-04-03-15; expires=Sun, 03-May-2020 15:38:18 GMT; path=/; domain=.google.de; Secure
Set-Cookie: NID=201=H8GdKY8_vE5Ehy6qSkmQru13HqdGEj2tvZUFqvTDAVBxFoL4POI0swPtfI45v1TBjrJuAAfbcNMUddniIf9HHituCAFwUqmUFMDwxDYK5qUlcWiB1A64OcGp6PTT6LKur2r_3z-ToSvLf8RZhKWdny6E8SaArMpkaOqUEWp4aoQ; expires=Sat, 03-Oct-2020 15:38:18 GMT; path=/; domain=.google.de; HttpOnly
Transfer-Encoding: chunked
Accept-Ranges: none
Vary: Accept-Encoding
The headers contain a Transfer-Encoding value of chunked. With chunked, the response contains "chunks" preceded by their length. An HTTP client uses those to read the entire response.
The HTTP Specification states that the Content-Length header should not be present when Transfer-Encoding has a value of chunked and MUST be ignored if it is.
Please notice that response header name Transfer-Encoding. Its value is chunked which means data is deliveryed block by block. Transfer-Encoding: chunked and Content-Length does not turn out at the same time.
There are two reason.
Server does not want sent content length.
Or server do not know the content length when it flush a big size data whose size is large than server's buffer.
So when there is no content length header, you can find the size of each chunked block before body of content. For example:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=8A7461DDA53B4C4DD0E89D73219CB5F8; Path=/
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Date: Wed, 18 Mar 2015 07:10:05 GMT
11
helloworld!
3
123
0
Above headers and content tell us, there are two block data. The size of first block is 11. the size of second block is 3. So the content length is 14 at all.
regards,
Xici
If you really want to get the content length without caring about the content, you can do this.
EntityUtils.toByteArray(httpResponse.getEntity()).length
I am trying to serve some assets using a Spring MVC controller. My assets are database managed and thus have to be served this way. The service looks up the metadata of the asset from the database, reads the file from file system and builds the response.
Here is how my controller looks like.
#Controller
#RequestMapping("/assets")
public class AssetController {
#Autowired
private AssetService assetService;
#RequestMapping("/{assetName:.+}")
public ResponseEntity<byte[]> getAsset(#PathVariable("assetName") String assetName) throws FileNotFoundException, IOException {
Asset asset = assetService.findByName(assetName);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.valueOf(asset.getContentType()));
headers.setCacheControl("max-age=1209600");
headers.setLastModified(asset.getModifiedOn().getTime()); // always in the past
return new ResponseEntity<byte[]>(assetService.toBytes(asset), headers, OK);
}
}
Seems simple and straightforward enough? One would hope to see the browser caching the images. But despite trying all combinations of Cache-Control, Expires, Last-Modified-On and ETag, I have had no success.
Below are the HTTP headers (irrelevant headers removed) spit out during two successive requests.
GET /adarshr-web/assets/Acer.png HTTP/1.1
Host: localhost:8080
Pragma: no-cache
Cache-Control: no-cache
HTTP/1.1 200 OK
Cache-Control: max-age=1209600
Last-Modified: Sun, 21 Jul 2013 11:56:32 GMT
Content-Type: image/png
Date: Tue, 23 Jul 2013 21:22:58 GMT
----------------------------------------------------------
GET /adarshr-web/assets/Acer.png HTTP/1.1
Host: localhost:8080
If-Modified-Since: Sun, 21 Jul 2013 11:56:32 GMT
Cache-Control: max-age=0
HTTP/1.1 200 OK <-- Why not 304 Not Modified?
Cache-Control: max-age=1209600
Last-Modified: Sun, 21 Jul 2013 11:56:32 GMT
Content-Type: image/png
Date: Tue, 23 Jul 2013 21:23:03 GMT
However, when I try the same sequence (Ctrl + F5 for first request and F5 for subsequent ones) on URLs such as
http://www.google.co.uk/images/srpr/logo4w.png (Google's logo)
http://fbstatic-a.akamaihd.net/rsrc.php/v2/yI/r/0PsXdTWc41M.png (Facebook's mobile image)
I see the headers such as these (shown for the Facebook URL) which indicate that the response is being cached by the browser.
GET /rsrc.php/v2/yI/r/0PsXdTWc41M.png HTTP/1.1
Host: fbstatic-a.akamaihd.net
Pragma: no-cache
Cache-Control: no-cache
HTTP/1.1 200 OK
Content-Type: image/png
Last-Modified: Sat, 15 Jun 2013 00:48:42 GMT
Cache-Control: public, max-age=31535893
Expires: Wed, 23 Jul 2014 21:27:47 GMT
Date: Tue, 23 Jul 2013 21:29:34 GMT
----------------------------------------------------------
GET /rsrc.php/v2/yI/r/0PsXdTWc41M.png HTTP/1.1
Host: fbstatic-a.akamaihd.net
If-Modified-Since: Sat, 15 Jun 2013 00:48:42 GMT
Cache-Control: max-age=0
HTTP/1.1 304 Not Modified <-- Note this
Content-Type: image/png
Last-Modified: Sat, 15 Jun 2013 00:48:42 GMT
Cache-Control: public, max-age=31535892
Expires: Wed, 23 Jul 2014 21:27:47 GMT
Date: Tue, 23 Jul 2013 21:29:35 GMT
Notes:
I don't have an <mvc:resources /> section in my Spring config since I am doing exactly the same in my controller. Even adding it doesn't make any difference.
I don't have a org.springframework.web.servlet.mvc.WebContentInterceptor defined in the Spring config again for the reasons above. I have tried adding one with no gain.
I have tried all methods explained in https://developers.google.com/speed/docs/best-practices/caching.
I can replicate this across all browsers.
You'll have to implement the check of the last modified, fortunately Spring makes that pretty easy.
From the Spring Framework Reference
#RequestMapping
public String myHandleMethod(WebRequest webRequest, Model model) {
long lastModified = // 1. application-specific calculation
if (request.checkNotModified(lastModified)) {
// 2. shortcut exit - no further processing necessary
return null;
}
// 3. or otherwise further request processing, actually preparing content
model.addAttribute(...);
return "myViewName";
}