I'm downloading a JAR file, and would like to utilize If-Modified-Since so I don't get the whole file if I don't need it, but for some reason my vanilla Apache (afaik) isn't returning the 304 correctly.
This is from wireshark:
GET /whatever.jar HTTP/1.1
If-Modified-Since: Sat, 04 Jan 2014 21:46:26 GMT
User-Agent: Jakarta Commons-HttpClient/3.1
Host: example.com
HTTP/1.1 200 OK
Date: Sat, 04 Jan 2014 20:32:31 GMT
Server: Apache/2.2.4 (Unix) mod_ssl/2.2.4 OpenSSL/0.9.8e DAV/2 mod_jk/1.2.26 PHP/5.3.6 SVN/1.4.4
Last-Modified: Sat, 04 Jan 2014 19:13:14 GMT
ETag: "b6c037-1ddad9f-d17a6680"
Accept-Ranges: bytes
Content-Length: 31305119
Vary: User-Agent
Content-Type: text/plain
... [bunch of bytes] ...
There aren't other headers I need to specify, is there? Am I missing a module that Apache needs in order to read this header correctly?
Any other thoughts or suggestions?
Here is my Java code, for reference:
File jarFile = new File(filePath);
GetMethod get = new GetMethod(downloadUrl);
Date lastModified = new Date(jarFile.lastModified());
get.setRequestHeader("If-Modified-Since", DateUtil.formatDate(lastModified));
HttpClient client = new HttpClient();
int code = client.executeMethod(get);
UPDATE: Solution
The If-Modified-Date needed to exactly match the server, and I achieved this by explicitly setting the lastModifiedDate on the downloaded file:
String serverModified = get.getResponseHeader("Last-Modified").getValue();
jarFile.setLastModified(DateUtil.parseDate(serverModified).getTime());
After doing this, subsequent calls would not download the file.
In order to use the "If-Modified-Since" header, you must send an identical header value as the "Last-Modified" header, that is Sat, 04 Jan 2014 19:13:14 GMT != Sat, 04 Jan 2014 21:46:26 GMT. Apache cannot guarantee the file wasn't modified and given a past time on purpose (perhaps through a version control roll-back).
If you want, you may check the "Last-Modified" header on the client side, by using a HeadMethod first to avoid "getting" the resource if it hasn't been modified. Then you would use a "GetMethod" if it has been modified.
See RFC2616 - Section 9, "HTTP/1.1: Method Definitions" for more.
Related
Hey I am trying to build an HTTP client, and I can't seem to figure out why I am not getting the code "304, not modified". The last modified date for the file is in 2007 and I am accessing it 10 yrs later.
Here is the output
HTTP/1.1 200 OK
Date: Tue, 03 Oct 2017 21:50:33 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.2k-fips
Last-Modified: Fri, 31 Aug 2007 04:21:06 GMT
ETag: "c12-438f726ceb080"
Accept-Ranges: bytes
Content-Length: 3090
Content-Type: image/gif
and the java code for reference, I am using sockets.
Date d = new Date();
outputStream.print("HEAD "+ "/" + pathName + " HTTP/1.1\r\n");
outputStream.print("If-Modified-Since: " + d.toString() + "\r\n");
outputStream.print("Host: " + hostString+"\r\n");
outputStream.print("\r\n");
outputStream.flush();
any help would be appreciated, I am new to HTTP clients.
You might need to format the date you're using for the "If-Modified-Since" header. Here's the syntax (taken from here):
If-Modified-Since: day-name, day month year hour:minute:second GMT
Example:
If-Modified-Since: Wed, 21 Oct 2015 07:28:00 GMT
I am downloading a file with okhttp and things work fine - now I want to show the progress and hit a road-bump. The returned content-length is -1.
It comes back correctly from the server:
⋊> ~ curl -i http://ipfs.io/ipfs/QmRMHb4Vhv8LtYqw8RkDgkdZYxJHfrfFeQaHbNUqJYmdF2 13:38:11
HTTP/1.1 200 OK
Date: Tue, 14 Jun 2016 11:38:16 GMT
Content-Type: application/octet-stream
Content-Length: 27865948
I traced the problem down to OkHeaders.java here:
public static long contentLength(Headers headers) {
return stringToLong(headers.get("Content-Length"));
}
I see all the other headers here in headers - but not Content-Length - so headers.get("Content-Length") returns null. Anyone has a clue how this can get lost?
Interestingly if I change the url to "http://google.com" I get a content-length from okhttp - but with curl both look same Content-Length wise - this really confuses me
Update: it seems to correlate with he size of the file. If I use smaller content from the same server I get a Content-Length with okhttp. The problem only happens when the file is big
It looks like above a certain size the server uses chunked encoding and you won't get a content length.
HTTP/1.1 200 OK
Date: Tue, 14 Jun 2016 14:30:07 GMT
Content-Type: application/octet-stream
Transfer-Encoding: chunked
I am trying to serve some assets using a Spring MVC controller. My assets are database managed and thus have to be served this way. The service looks up the metadata of the asset from the database, reads the file from file system and builds the response.
Here is how my controller looks like.
#Controller
#RequestMapping("/assets")
public class AssetController {
#Autowired
private AssetService assetService;
#RequestMapping("/{assetName:.+}")
public ResponseEntity<byte[]> getAsset(#PathVariable("assetName") String assetName) throws FileNotFoundException, IOException {
Asset asset = assetService.findByName(assetName);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.valueOf(asset.getContentType()));
headers.setCacheControl("max-age=1209600");
headers.setLastModified(asset.getModifiedOn().getTime()); // always in the past
return new ResponseEntity<byte[]>(assetService.toBytes(asset), headers, OK);
}
}
Seems simple and straightforward enough? One would hope to see the browser caching the images. But despite trying all combinations of Cache-Control, Expires, Last-Modified-On and ETag, I have had no success.
Below are the HTTP headers (irrelevant headers removed) spit out during two successive requests.
GET /adarshr-web/assets/Acer.png HTTP/1.1
Host: localhost:8080
Pragma: no-cache
Cache-Control: no-cache
HTTP/1.1 200 OK
Cache-Control: max-age=1209600
Last-Modified: Sun, 21 Jul 2013 11:56:32 GMT
Content-Type: image/png
Date: Tue, 23 Jul 2013 21:22:58 GMT
----------------------------------------------------------
GET /adarshr-web/assets/Acer.png HTTP/1.1
Host: localhost:8080
If-Modified-Since: Sun, 21 Jul 2013 11:56:32 GMT
Cache-Control: max-age=0
HTTP/1.1 200 OK <-- Why not 304 Not Modified?
Cache-Control: max-age=1209600
Last-Modified: Sun, 21 Jul 2013 11:56:32 GMT
Content-Type: image/png
Date: Tue, 23 Jul 2013 21:23:03 GMT
However, when I try the same sequence (Ctrl + F5 for first request and F5 for subsequent ones) on URLs such as
http://www.google.co.uk/images/srpr/logo4w.png (Google's logo)
http://fbstatic-a.akamaihd.net/rsrc.php/v2/yI/r/0PsXdTWc41M.png (Facebook's mobile image)
I see the headers such as these (shown for the Facebook URL) which indicate that the response is being cached by the browser.
GET /rsrc.php/v2/yI/r/0PsXdTWc41M.png HTTP/1.1
Host: fbstatic-a.akamaihd.net
Pragma: no-cache
Cache-Control: no-cache
HTTP/1.1 200 OK
Content-Type: image/png
Last-Modified: Sat, 15 Jun 2013 00:48:42 GMT
Cache-Control: public, max-age=31535893
Expires: Wed, 23 Jul 2014 21:27:47 GMT
Date: Tue, 23 Jul 2013 21:29:34 GMT
----------------------------------------------------------
GET /rsrc.php/v2/yI/r/0PsXdTWc41M.png HTTP/1.1
Host: fbstatic-a.akamaihd.net
If-Modified-Since: Sat, 15 Jun 2013 00:48:42 GMT
Cache-Control: max-age=0
HTTP/1.1 304 Not Modified <-- Note this
Content-Type: image/png
Last-Modified: Sat, 15 Jun 2013 00:48:42 GMT
Cache-Control: public, max-age=31535892
Expires: Wed, 23 Jul 2014 21:27:47 GMT
Date: Tue, 23 Jul 2013 21:29:35 GMT
Notes:
I don't have an <mvc:resources /> section in my Spring config since I am doing exactly the same in my controller. Even adding it doesn't make any difference.
I don't have a org.springframework.web.servlet.mvc.WebContentInterceptor defined in the Spring config again for the reasons above. I have tried adding one with no gain.
I have tried all methods explained in https://developers.google.com/speed/docs/best-practices/caching.
I can replicate this across all browsers.
You'll have to implement the check of the last modified, fortunately Spring makes that pretty easy.
From the Spring Framework Reference
#RequestMapping
public String myHandleMethod(WebRequest webRequest, Model model) {
long lastModified = // 1. application-specific calculation
if (request.checkNotModified(lastModified)) {
// 2. shortcut exit - no further processing necessary
return null;
}
// 3. or otherwise further request processing, actually preparing content
model.addAttribute(...);
return "myViewName";
}
When creating the HTTP Response manually, how can one get Server and ETag
* HTTP/1.1 200 OK
* Date: Mon, 23 Apr 2012 23:44:52 GMT
* Server: Apache/2.2.3 (Red Hat) <-----
* Last-Modified: Fri, 16 Sep 2005 18:08:50 GMT
* ETag: "421142-2f-400e77c517080" <-----
* Accept-Ranges: bytes
* Content-Length: 47
* Content-Type: text/plain
* Connection: close
"Server" is whatever your HTTP server wants to name/identify itself. I.e. "Zumgto Surver 4.5".
"ETag" identifies "version" of particular item, so as long as your server can reasonable say "this ETag corresponds to current version" you can send pretty much anything. I.e. "v3345", or hash of the item... Totally optional if you don't support "If-None-Match" header in requests.
Neither is required. You can make up your own sever tag using the same format above. Omit the eTag or just generate your own. You could use the current timestamp or a constant. The following formats should work.
Server: Program/version (O/S)
ETag: "Timestamp"
I'm using Jersey bundle 1.11 to provide some RESTful web service.
Each time I browse a REST resource with Chrome, I notice that there's an HTTP Header Expires set to Thu, 01 Jan 1970 01:00:00 CET.
I tried to edit the Response adding:
return Response.ok( myObject ).expires(new Date(System.currentTimeMillis() + 3000)).build();
Unfortunately, this adds another HTTP Header Expires instead of replacing the old one.
What is the problem?
FWIW, I am seeing the exact same behaviour. The container here is JBoss 4.2.3. This is a PUT method with BASIC authentication. My response is generated thus:
Date exp = new Date(System.currentTimeMillis() + lifetime);
return Response.noContent().expires(exp).build();
When invoked with cURL, these are the returned headers:
< HTTP/1.1 204 No Content
< Server: Apache-Coyote/1.1
< Pragma: No-cache
< Cache-Control: no-cache
< Expires: Thu, 01 Jan 1970 01:00:00 CET
< X-Powered-By: Servlet 2.4; JBoss-4.2.3.GA (...
< Expires: Tue, 13 Mar 2012 11:08:54 GMT
< Date: Tue, 13 Mar 2012 11:08:24 GMT
<
This is to prevent your browser from caching of the requested resource.
The date itself is the timestamp with zero seconds, the begin of the UNIX era.
I found that my app server (In this case JBoss 4.2.3.GA) would not allow Jersey to overwrite the header this way.
To workaround:
Inject the response object into the method using a parameter:
#Context javax.servlet.http.HttpServletResponse response
Set the header on the response object rather than using .expires() :
response.setDateHeader("Expires", System.currentTimeMillis() + 14400000);
I used #2 before I called .build() on the ResponseBuilder, not sure if it makes a difference or not when you do this.
I have the same issue. My workaround is:
Inject the response
#Context javax.servlet.http.HttpServletResponse response
Reset the response object
response.reset();
Use the ResponseBuilder to set the headers.
return Response
.ok(icon.getData())
.type(icon.getContentType())
.expires(cal.getTime())
.build();