Develop Reference

okhttp content-length is -1 with big files

I am downloading a file with okhttp and things work fine - now I want to show the progress and hit a road-bump. The returned content-length is -1.
It comes back correctly from the server:
⋊> ~ curl -i http://ipfs.io/ipfs/QmRMHb4Vhv8LtYqw8RkDgkdZYxJHfrfFeQaHbNUqJYmdF2 13:38:11
HTTP/1.1 200 OK
Date: Tue, 14 Jun 2016 11:38:16 GMT
Content-Type: application/octet-stream
Content-Length: 27865948
I traced the problem down to OkHeaders.java here:
public static long contentLength(Headers headers) {
return stringToLong(headers.get("Content-Length"));
}
I see all the other headers here in headers - but not Content-Length - so headers.get("Content-Length") returns null. Anyone has a clue how this can get lost?
Interestingly if I change the url to "http://google.com" I get a content-length from okhttp - but with curl both look same Content-Length wise - this really confuses me
Update: it seems to correlate with he size of the file. If I use smaller content from the same server I get a Content-Length with okhttp. The problem only happens when the file is big

It looks like above a certain size the server uses chunked encoding and you won't get a content length.
HTTP/1.1 200 OK
Date: Tue, 14 Jun 2016 14:30:07 GMT
Content-Type: application/octet-stream
Transfer-Encoding: chunked

Downloading from dropbox url ignores range

If i want to download a file from a dropbox url my http header range is ignored:
httpRequest = new HttpGet(url.toURI());
httpRequest.addHeader("Range", "bytes=" + startPos + "-" + dwnInfo.getStopRange());
httpRequest.addHeader("Accept-Encoding", "");
So instead of making my file download in x chunks of 5mb for ex, the connection ignores the specified range and it downloads x chunks of Y mb, where y is the full size of the file.
Downloading from an amazon storange link i don't have any problems.
Anyone else encountered this situation ? This only happens from some days ago. This wasn't a issue until now.
I tried to look on dropbox dev page but didn't see anything that specifies if they removed the accepted range on urls

The link you gave is to an HTML page (total size ~46KB), so even if range retrieval worked there, it wouldn't be very useful.
Per https://www.dropbox.com/help/201/en, you can turn a share link into a direct link to the file by changing the domain to dl.dropboxusercontent.com, so your link becomes https://dl.dropboxusercontent.com/s/5c7atlfmacjf3qn/02%20Armin%20Van%20Buuren%20-%20A%20State%20Of%20Trance%20Year%20Mix%202013%20%28Cd%202%29.mp3, and range retrieval works for that URL.
(Here I'm using httpie.)
$ http get https://dl.dropboxusercontent.com/s/5c7atlfmacjf3qn/02%20Armin%20Van%20Buuren%20-%20A%20State%20Of%20Trance%20Year%20Mix%202013%20%28Cd%202%29.mp3 range:bytes=0-0
HTTP/1.1 206 PARTIAL CONTENT
Connection: keep-alive
Content-Length: 1
Content-Type: audio/mpeg
Date: Wed, 18 Jun 2014 14:53:32 GMT
Server: nginx
accept-ranges: bytes
cache-control: max-age=0
content-range: bytes 0-0/146014047
etag: 346n
pragma: public
set-cookie: uc_session=2cqmevWxG8lmGt743KMXebc23dRC5iuZEfm8Etx6V2VShWk60jmnUJajFnH1wRG4; Domain=dropboxusercontent.com; Path=/; secure; httponly
x-dropbox-request-id: 2f0c5986a62cf2f0b06af1704ece5bd7
x-server-response-time: 535
I

Apache not obeying If-Modified-Since

I'm downloading a JAR file, and would like to utilize If-Modified-Since so I don't get the whole file if I don't need it, but for some reason my vanilla Apache (afaik) isn't returning the 304 correctly.
This is from wireshark:
GET /whatever.jar HTTP/1.1
If-Modified-Since: Sat, 04 Jan 2014 21:46:26 GMT
User-Agent: Jakarta Commons-HttpClient/3.1
Host: example.com
HTTP/1.1 200 OK
Date: Sat, 04 Jan 2014 20:32:31 GMT
Server: Apache/2.2.4 (Unix) mod_ssl/2.2.4 OpenSSL/0.9.8e DAV/2 mod_jk/1.2.26 PHP/5.3.6 SVN/1.4.4
Last-Modified: Sat, 04 Jan 2014 19:13:14 GMT
ETag: "b6c037-1ddad9f-d17a6680"
Accept-Ranges: bytes
Content-Length: 31305119
Vary: User-Agent
Content-Type: text/plain
... [bunch of bytes] ...
There aren't other headers I need to specify, is there? Am I missing a module that Apache needs in order to read this header correctly?
Any other thoughts or suggestions?
Here is my Java code, for reference:
File jarFile = new File(filePath);
GetMethod get = new GetMethod(downloadUrl);
Date lastModified = new Date(jarFile.lastModified());
get.setRequestHeader("If-Modified-Since", DateUtil.formatDate(lastModified));
HttpClient client = new HttpClient();
int code = client.executeMethod(get);
UPDATE: Solution
The If-Modified-Date needed to exactly match the server, and I achieved this by explicitly setting the lastModifiedDate on the downloaded file:
String serverModified = get.getResponseHeader("Last-Modified").getValue();
jarFile.setLastModified(DateUtil.parseDate(serverModified).getTime());
After doing this, subsequent calls would not download the file.

In order to use the "If-Modified-Since" header, you must send an identical header value as the "Last-Modified" header, that is Sat, 04 Jan 2014 19:13:14 GMT != Sat, 04 Jan 2014 21:46:26 GMT. Apache cannot guarantee the file wasn't modified and given a past time on purpose (perhaps through a version control roll-back).
If you want, you may check the "Last-Modified" header on the client side, by using a HeadMethod first to avoid "getting" the resource if it hasn't been modified. Then you would use a "GetMethod" if it has been modified.
See RFC2616 - Section 9, "HTTP/1.1: Method Definitions" for more.

taking mail into text boxes

I've written a function in java to get this. On the reception of an e-mail I'm storing the contents of the mail into an external file
MIME-Version: 1.0 Received: by 10.216.237.25 with SMTP id x25mr3166819weq.30.1346350346502; Thu, 30 Aug 2012 11:12:26 -0700 (PDT)Received: by 10.216.123.197 with HTTP; Thu, 30 Aug 2012 11:12:26 -0700 (PDT)Date: Thu, 30 Aug 2012 20:12:26 +0200 Message-ID: <CAEE+wmOpUFC9Nb9wNAVKsCrFRX7GmDwhDya4brB=hvRHXY6q7Q#mail.gmail.com> Subject: ok, lets see ... From: ABCD DEFG <xyz#gmail.com> To: XYX <xyz#yahoo.com>, def#gmail.com Content-Type: multipart/alternative; boundary=000e0cd5f7142581b204c87f9ed5 X-Processed-By: mx3.mail.genotec.ch X-Spam-Checksum: a84a5db7738f685c28343edc93a6546a X-Greylist: Sender IP 209.85.212.179 not delayed X-Spam-Report: ---- Start der SpamAssassin Auswertung -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -0.1 SPF_PASS SPF: sender matches SPF record 0.0 HTML_MESSAGE BODY: HTML included in message
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
domain
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid
---- Ende der SpamAssassin Auswertung X-GIC-MailScanner-SpamScore: -1.199
X-GIC-MailScanner-SpamCheck: ham, hits=-1.199 required=10 scantime="4.74 seconds" origip=209.85.212.179 X-Scanned-By: "MGate 1.0" on 82.195.224.57
--000e0cd5f7142581b204c87f9ed5 Content-Type: text/plain; charset=ISO-8859-1
hello worldli ... --000e0cd5f7142581b204c87f9ed5 Content-Type: text/html; charset=ISO-8859-1 hello worldli ... --000e0cd5f7142581b204c87f9ed5--
As we can see it not aligned .. I'd like to sort the data according to SENDER, RECEIVER, ACTUAL CONTENTS OF AN EMAIL, DATE (in short I want only SENDER, RECEIVER, DATE, ACTUAL DATA that's all).... how can I achieve this using JAVASCRIPT. Other unnecessary things should be avoided.... and have to display the fetched data in text-boxes ..
Let's say four text-boxes for four fields...

You can use regular expression to finish this task.
You can write 4 different regular expressions for each of the field.

Jersey Expires Header not working

I'm using Jersey bundle 1.11 to provide some RESTful web service.
Each time I browse a REST resource with Chrome, I notice that there's an HTTP Header Expires set to Thu, 01 Jan 1970 01:00:00 CET.
I tried to edit the Response adding:
return Response.ok( myObject ).expires(new Date(System.currentTimeMillis() + 3000)).build();
Unfortunately, this adds another HTTP Header Expires instead of replacing the old one.
What is the problem?

FWIW, I am seeing the exact same behaviour. The container here is JBoss 4.2.3. This is a PUT method with BASIC authentication. My response is generated thus:
Date exp = new Date(System.currentTimeMillis() + lifetime);
return Response.noContent().expires(exp).build();
When invoked with cURL, these are the returned headers:
< HTTP/1.1 204 No Content
< Server: Apache-Coyote/1.1
< Pragma: No-cache
< Cache-Control: no-cache
< Expires: Thu, 01 Jan 1970 01:00:00 CET
< X-Powered-By: Servlet 2.4; JBoss-4.2.3.GA (...
< Expires: Tue, 13 Mar 2012 11:08:54 GMT
< Date: Tue, 13 Mar 2012 11:08:24 GMT
<

This is to prevent your browser from caching of the requested resource.
The date itself is the timestamp with zero seconds, the begin of the UNIX era.

I found that my app server (In this case JBoss 4.2.3.GA) would not allow Jersey to overwrite the header this way.
To workaround:
Inject the response object into the method using a parameter:
#Context javax.servlet.http.HttpServletResponse response
Set the header on the response object rather than using .expires() :
response.setDateHeader("Expires", System.currentTimeMillis() + 14400000);
I used #2 before I called .build() on the ResponseBuilder, not sure if it makes a difference or not when you do this.

I have the same issue. My workaround is:
Inject the response
#Context javax.servlet.http.HttpServletResponse response
Reset the response object
response.reset();
Use the ResponseBuilder to set the headers.
return Response
.ok(icon.getData())
.type(icon.getContentType())
.expires(cal.getTime())
.build();