I created cookies with HTTPOnly flag in Safari browser using java
See Response header below.
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Referer:http://anil.mlbextrabases.com/SafariIssue/
User-Agent:Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2
Query String Parametersview URL encoded
userName:servlet
userPass:servlet
flagRequired:true
Response Headersview source
Content-Length:331
Content-Type:text/html;charset=ISO-8859-1
Date:Wed, 08 Aug 2018 09:23:45 GMT
Server:Apache-Coyote/1.1
Set-Cookie:userInfo="username: servletpassword: servlet"; Version=1; Comment="creating cookie"; Domain=anil.mlbextrabases.com; Max-Age=31536000; Expires=Thu, 08-Aug-2019 09:23:45 GMT; Path=";Path=/;HttpOnly;", JSESSIONID=076942707A8D62AD1296102A1593F664; Path=/SafariIssue; HttpOnly
Strict-Transport-Security:max-age=7776000; includeSubdomains
But cookies are not passed in subsequent requests in safari browser. While passing in other browsers like Chrome, Firefox, IE, Edge
When i removed HTTPOnly flag cookies are passing good in Safari browser as well.
I did a lot of R&D over developer communities. But it not helped.
HTTPOnly cookies by definition are created to NOT allow access from javascript.
This is a security control to make sure that a hacker can not exploit your software by extracting valid cookie values (like session IDs)
HttpOnly is a flag the website can specify about a cookie. In other words, the webserver tells your browser “Hey, here is a cookie, and you should treat is as HttpOnly”.
An HttpOnly Cookie is not accessible by the JavaScript. Only the browser knows about it, and it doesn’t give it to the JavaScript code in the page. At first, it might sound like a limitation, and it is. However, the goal of that is that we cannot trust the JavaScript code. An attacker may use JavaScript to steal our authentication token stored in a cookie, and then access the website with our account. With HttpOnly cookies, this is not possible. This makes XSS attacks (the one we just described) harder to perform.
It is a recognized best practice to share any authentication data only with HttpOnly cookies. Using a standard cookie for authentication is a known vulnerability we should avoid in any case.
https://www.ictshore.com/ict-basics/httponly-cookie/
I'm trying to create a simple REST service to serve audio files (with seek support).
I'm using this example which is based on Jersey:
https://github.com/aruld/jersey-streaming/tree/jersey2
This is a quite simple example, it listens to GET and HEAD requests used by the browsers, look for the Range header and respond with 206 plus the archive slice requested (with byte ranges).
The catch here is that I'm re-writing this on spark java (a tiny framework with an embedded jetty server).
Every thing seems to be OK. The browser sends the GET and the server crates the response accordingly... though the player never loads nor plays anything. The request is made and the response header is perfect:
Request:
Host: localhost:4567
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0
Accept: audio/webm,audio/ogg,audio/wav,audio/*;q=0.9,application/ogg;q=0.7,video/*;q=0.6,*/*;q=0.5
Accept-Language: en-US,en;q=0.5
Range: bytes=0-
Connection: keep-alive
Response:
Accept-Ranges: bytes
Content-Length: 1048577
Content-Range: bytes 0-1048576/5563904
Content-Type: audio/mp3
Date: Sat, 20 Aug 2016 05:41:23 GMT
Last-Modified: Sat Aug 20 01:12:37 BRT 2016
Server: Jetty(9.3.6.v20151106)
sasd
One thing I noticed is that seems the transfer for this request never ends. When I close the server, the transfer ends at 0,03KB (always).
My proof of concept app code:
http://pastebin.com/xjkLne7E
Found an answer!
I did some more research and found that it is a Spark feature to be implemented:
https://github.com/perwendel/spark/issues/397
User tchoulihan already tried to implement such feature with success here:
https://github.com/tchoulihan/torrenttunes-client
Here is a sample of the spark get request that handles the upload:
https://github.com/tchoulihan/torrenttunes-client/blob/master/src/main/java/com/torrenttunes/client/webservice/Platform.java#L555
I can't paste a blob here since GPLv3 would clash with cc-wiki license. Hes work is inspired on the same resource I first found. Based on that I have coded a version of my own that works on android, mozilla and chrome.
TL;DR The problem was that mozilla doesn't understand 206 request properly and I wasn't closing and flushing the ByteOutputStream. Also I tried to use the StreamingOutput from JAX as a response instead of sending a simple raw http response.
so far, I have been able to use the HttpURLConnection class in java to make an app that can GET the form of my php website, put in the proper login details (username, password) and POST them back. I have double checked this with the response codes and am getting 200 for both GET and POST.
I'm having an issue now accessing the page that a successful login should redirect to. It is to my understanding that after a POST or GET, the connection is terminated once the response code is requested. My attempts to get the response cookies while logging in produce a "null" cookie.
The PHP site I am accessing does not seem to have any response cookies after a login when using "inspect element" in Chrome. Regardless of this, I have tried accessing the cookies all sorts of ways with no such luck. The request cookie header is there when I go the the website.
Am I missing something and the cookies are actually there? Or is it possible that the site does not use cookies to maintain a session? If that's the case, how would I access the page I want after logging in on my Android app?
Response Headers
Cache-Control:no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection:Keep-Alive
Content-Encoding:gzip
Content-Length:23030
Content-Type:text/html; charset=utf-8
Date:Mon, 10 Aug 2015 23:03:26 GMT
Expires:Thu, 19 Nov 1981 08:52:00 GMT
Keep-Alive:timeout=15, max=100
Pragma:no-cache
Server:Apache/2.2.22 (Debian)
Vary:Accept-Encoding
X-Powered-By:PHP/5.4.4-14+deb7u11
Request Header
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip, deflate
Accept-Language:en-US,en;q=0.8
Cache-Control:max-age=0
Connection:keep-alive
Content-Length:52
Content-Type:application/x-www-form-urlencoded
Cookie:__utma=83554121.1278939357.1435860313.1435944069.1438202297.3; __utmc=83554121; __utmz=83554121.1438202297.3.3.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); _ga=GA1.2.1278939357.1435860313; PHPSESSID=4q03j4ihb7trnm1pvvofc9f3f5
Host:WEBSITE
Origin:WEBSITE
Referer:WEBSITE
Upgrade-Insecure-Requests:1
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36
Just an update on what I have figured out so far.
I logged in to my website on my browser on my PC. I took this PHPSESSID and used it in my app and I can access the page that the login page redirects to just fine.
My new question is: Why can't I assign myself a random PHPSESSID and go back to it? In other words, if I give myself PHPSESSID=123456 for example and use that when posting my login details, why does using that specific PHPSESSID still bring me back to the login page instead of the redirected one?
I'm currently trying to read the request header "set-cookie" but am having trouble doing so. In addition, even putting in the wrong user and/or password gives a response code of 200. Suggestions on how to check if logging in was successful?
I'm trying to retrieve links from this page: http://www.seas.harvard.edu/academics/areas
There is a link named "Computer Science" in the middle of the page. Its underlying link is given as "/academics/areas/computer-science". I'm able to convert it to an absolute URL with the Java built-in URL class, obtaining "http://www.seas.harvard.edu/academics/areas/computer-science".
When I click the link in Chrome browser, however, the absolute URL changes to "http://www.seas.harvard.edu/computer-science".
So my question is two-fold:
How does the URL redirect work in this page?
Is there any library or method in Java that would help me obtain the URL after redirect?
I need to obtain the URL after redirect because I want to read the source code of the page but the URL before redirect doesn't work for me. I'm using the JSoup library to read from the URL so I suspect it might be a javascript-based redirect.
From curl --dump-header [file] [URL] the file looked like:
HTTP/1.1 301 Moved Permanently
Age: 0
Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0
Content-Type: text/html
Date: Tue, 13 Aug 2013 13:00:12 GMT
ETag: "1376398812"
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Last-Modified: Tue, 13 Aug 2013 13:00:12 GMT
Location: http://www.seas.harvard.edu/computer-science
Server: nginx
Vary: Accept-Encoding
Via: 1.1 varnish
X-AH-Environment: prod
X-Cache: MISS
X-Drupal-Cache: MISS
X-Redirect-ID: 44
X-Varnish: 2704315535
transfer-encoding: chunked
Connection: keep-alive
As you can see this is a 301 permanent redirect served from the server.
To obtain the data:
You can use HttpURLConnection to connect, but before connecting, call myConn.setInstanceFollowRedirects(true). Redirects are followed and you can get your output stream and read it.
To obtain the URL itself:
You can use HttpURLConnection to connect, but before connecting, call myConn.setInstanceFollowRedirects(false) to not follow redirects. This will save the actual URL in the right place.
The trick here is that for some odd reason, HttpURLConnection doesn't allow to retrieve a header by name unless you parse it as a date.
So, you will need to iterate an integer, calling getHeaderFieldKey after making the connection and checking if it equal to Location and if it is, getting getHeaderField with the same integer to get the location. Annoying, I know. But a location isn't a date and this is a JRE oversight.
I used Fiddler to investigate and the site return for link http://www.seas.harvard.edu/academics/areas/computer-science HTTP 301 response code, that performs the redirect.
I you want to get real URL. You should perform real request to harvard.edu web server and parse response. (Redirect URL is located in Location key in HTTP Header).
Sorry about your second question. I don't have skill in Java.
This SO question may help (httpclient-4-how-to-capture-last-redirect-url)
There is probably e.g. a .htaccess and mod_rewrite redirect. Using Firefox's Console I could see the requests. As you can see below the server is sending back a 301 Moved Permanently message. This tells the browser to redirect to the address returned in the Location header of the response.
The way you obtain the changed URL depends on the way you load the page:
If you use ready libraries & code to load the page to e.g. a DOM object, the you could use that ready HTTP system to load the response, this will probably result to it automatically redirecting -> you will get the URL from the URL of the loaded page. If it does not do that, then you must check for status code 301 or 302 and when those are received then the changed URL is in the Location header of the response.
If you have your own code written to load the response via TCP sockets, then you must just load the response as normal, but again check for the 301 and 302 status codes and do as described in the previous section.
I can only attempt to address Q1 since I'm not a Java programmer. The source code says they're using Drupal, so I speculate that they're using Drupal's global redirect module (SO discussion about Drupal redirect module here). Looking at the module's documentation might shed some light on how to obtain the correct url with Java.
There's also numerous ways within javascript to have url requests automatically redirect to some base page (e.g., CS homepage), while physically navigating the site allows the user to advance to new pages. This is standard practice in many single page web apps. If this is the case, then #hexafraction 's suggestion might be able to help you retrieve the desired url, though I'm unfamiliar with the Java methods (s)he is suggesting.
You can get the Redirect URL from the below code setting followRedirects to false.
You will get the source code of the redirected page if you set it to true and that's the default behavior of Jsoup
Connection con = Jsoup.connect("http://www.seas.harvard.edu/academics/areas/computer-science")
.userAgent("Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36")
.followRedirects(false);
System.out.println("Redirected Url : " + con.execute().header("Location")); //null if followRedirect is true
Document doc = con.get();
System.out.println(doc.html());
System.out.println("=================================================");
I recently built a simple web-app deployed over Tomcat. The app uses pretty standard session based security where a user who has logged in is given a session.
Sessions work fine in Firefox and Chrome, but require the use of jsessionid in the URL for IE (tested 7 & 8), set to medium privacy. In IE 8, I tried to override cookie handling, setting "Allow all 3rd party cookies" and "Allow all session cookies"- no dice. However, when I run Tomcat on my local machine, IE accepts the cookie, and sessions work just fine.
And now, for the HTTP headers.
From Chrome, a logged in user gets a session
GET http://devl:8080/testing/ HTTP/1.1
Host: devl:8080
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1036 Safari/532.5
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
P3P: CP="NON CURa ADMa DEVa TAIa OUR BUS IND UNI COM NAV INT STA"
Set-Cookie: JSESSIONID=9280023BCE2046F32B13C89130CBC397; Path=/testing
Content-Type: text/html;charset=UTF-8
Content-Language: en-US
Content-Length: 2450
Date: Fri, 26 Mar 2010 14:14:40 GMT
GET http://devl:8080/testing/logout HTTP/1.1
Host: devl:8080
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1036 Safari/532.5
Referer: http://devl:8080/testing/
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: JSESSIONID=9280023BCE2046F32B13C89130CBC397
...
From IE 8, with standard medium level security and privacy-
GET http://devl:8080/testing/ HTTP/1.1
Accept: application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, */*
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MDDC; Tablet PC 2.0)
UA-CPU: AMD64
Accept-Encoding: gzip, deflate
Host: devl:8080
Connection: Keep-Alive
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
P3P: CP="NON CURa ADMa DEVa TAIa OUR BUS IND UNI COM NAV INT STA"
Set-Cookie: JSESSIONID=192999F922D6E9C868314452726764BA; Path=/testing
Content-Type: text/html;charset=UTF-8
Content-Language: en-US
Content-Length: 2450
Date: Fri, 26 Mar 2010 14:32:34 GMT
GET http://devl:8080/testing/logout HTTP/1.1
Accept: application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, */*
Referer: http://devl:8080/testing/;jsessionid=6371A83EFE39A46997544F9146AA5CEA
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MDDC; Tablet PC 2.0)
UA-CPU: AMD64
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
Host: devl:8080
...
I thought it might be P3P, but on adding a compact policy, nothing changes. This is the standard Tomcat session, so I'm really surprised I haven't been able to find other people with the same problem so far. Anyone have any ideas?
EDIT 4/3/2010 -
Sorry if I didn't make this clear- I've tried from multiple other instances of IE - co-workers down the hall, etc.
EDIT 4/3/2010 -
I've also tried turning on prompting for all cookies, but I don't get a prompt. Setting the domain in the "Set-Cookie" header using Fiddler didn't make a difference, either.
I ran into this exact problem, dug around for a while, and found this:
http://forums.iis.net/p/1147938/1879164.aspx
which says that domain names that have underscores in them cause problems with Windows Server, tomcat and IE
not sure if this fixes your problem (and at this point, you probably don't care) but maybe the next person who comes along can gain some value from it.
Problem: IE8 refused to accept cookies on a site I had built, but Firefox and IE7 worked just fine and had done so for ages - this was stable code.
Solution (for me): My server is in a different time zone to the client machine. The STUPID, IDIOTIC IE8 tries to be clever and refuses to accept cookies (stored in the local client machine) with a 20 minute life. My PHP code was straight from the text book, thus:
setcookie($name,$value,time()+1200);
But it works fine if I change it to, for example -
setcookie($name,$value,time()+120000);
This still leaves me with the problem of making the cookie die after 20 minutes, but at least my users can now use my website with IE8. I pass on this information in case it may help someone else.
Have you checked that the server time is correct?
I have had similar problems recently with IE not accepting cookies properly. After a lot of head scratching it turned out to be because the time difference between the server and client machines was so big that IE refused to accept the cookie. This was in Apache however.
Try using the standard HTTP port (80). I've read about issues with port numbers in URLs regarding privacy/security in IE more than once but can't seem to find relevant links at this time.
I agree with Lexicore - the cookie protocol from the web server looks right, so there's something with IE. It would be easier to figure out how to address the issue if we understood better why IE is rejecting the cookie. Alternatively, ask a friend to hit the site for you in IE to help confirm its a server issue not a browser instance issue.
Here is some things to check to help debug with IE and cookies - unfortunately, there's a mess of options to check. Sorry if some of these items seem basic - I just don't wnat to make any assumptions. I'm following along in IE 8.0 for this.
First, browse to the target site (http://devl:8080/testing/) in IE. Then:
Confirm what zone IE classifies 'http://devl:8080/testing/'. (This could explain why its works with Tomcat on your local machine.) The zone is displayed in the bottom bar of the browser and it most likely says "Internet". If it instead says "Local intranet", "Trusted Site", or "Restricted Site", this may be part of the problem and you should update your question or figure out why it isn't classified as Internet.
Double-click on the zone indicator in the bottom bar (presumably "Internet") to open the Security dialog. Is the Security Level for Internet set to Medium-high? If it isn't, this could be part of the problem and you should probably reset it back to match your users.
Select the "Internet" zone and then click the "Custom level ..." button to open the Security Settings dialog. Confirm the "Userdata persistence" option is set to "Enable". The "Userdata persistence" option is in the bottom 1/4 of the list of options in the "Miscenllaneous" section (near the bottom of the section just above the next section "Scripting").
Click OK on each dialog to close both of them.
On the menubar (enable it if it is not enabled), click "Tools" > "Internet Options". Select the "Privacy" tab. I know you mentioned you tried some things here, but those changes may not affect your site if your site is not in the Internet zone or if your site in the "Per Site Privacy Actions" exception list, so its best to just confirm.
Is the privacy setting in the Privacy tab set to Medium? If not, you may want to reset to default.
Click the "Sites" button to open the Per Site Privacy Actions dialog. Is your dev1 site listed? If so, remove it. Click OK to dismiss the dialog. Alternatively, you could force your dev1 site to always Allow cookies.
Click the "Advanced" button. Is "Override automatic cookie handling checked? If so, you might want to uncheck it to match your users. Alternatively, try checking it and checking "Always allow session cookies."
Click OK on each dialog to close both of them.
Confirm the browser is still at target site ('http://devl:8080/testing/'). Click "View" > "Webpage Privacy Policy..." to view the Privacy Report dialog. Does the list include "http://dev1:8080/testing/"? Does the Cookie column indicate "Accepted" for "http://dev1:8080/testing/"?
Select "http://dev1:8080/testing/" from the list. Click Summary to see the Privacy Policy. If set one for the your site, you should see it here. Otherwise, you should get a message that a privacy policy was not found. Look at the bottom of the dialog to see how the site is set to use cookies (compare, always allow, or never allow).
Hope this helps or gives you some ideas to pursue.
Ref:
http://blogs.msdn.com/ieinternals/archive/2009/08/20/WinINET-IE-Cookie-Internals-FAQ.aspx
http://www.practicalmachinist.com/vb/general/how-manage-cookies-internet-explorer-181641/
http://support.microsoft.com/kb/283185
This forum concerning P3P seems relevant.
Also have you considered setting your domain and expiration date for the session cookie?
This has clearly nothing to do with Tomcat, since the cookie is being set - just not accepted by the IE. This must be security issue in IE then.
Maybe this MS article would help to tune it.
What security zone is the dev1 site part of? IE handles cookies and lots of other security differently depending on which zone (and how the zone is configured).
Try setting the dev1 site to explicitly be part of the Trusted Sites for example and see what happens.
Zones:
Internet
Local Intranet
Trusted Sites
Restricted Sites
Also, does the cookie have to be restricted to the /testing path? Try setting it for / and see if that makes a difference.
I would try using the fully qualified hostname of the server. MSIE treats hostname without domains as being in the "Local intranet" and handles security differently.
Specifically, instead of:
http://devl:8080/testing/
Try using something like:
http://devl.mydomain.com:8080/testing/
It seems from what you're saying that you've only seen this issue in IE and only using computers in your office. Is there any sort of "security suite" installed by IT on all office computers, and if so, can you temporarily disable it? Oftentimes, these types of applications hook into IE and muck with its HTTP stack. If you do have software like that installed, do you have a "clean" installation or non-company computer you can test with?
The time on our servers were off by 14 minutes (and in the correct time zone EST).
Once we set the time on the server to the correct time cookies starting working again.
Ed