How can I get a cookie from a web page using Java? I mean only Java not with Servlets or etc..
You can use java.net.URLConnection for this. It offers a getHeaderFields() method to get the response headers. The cookies are set by Set-Cookie header.
URLConnection connection = new URL("http://google.com").openConnection();
List<String> cookies = connection.getHeaderFields().get("Set-Cookie");
// ...
You can either get the cookies from the header,
or you can use Apache commons and use their functionality.
Related
How does one decode Client Cookies strings jax-rs? What I mean by this is how do you convert a String cookie value into a javax.ws.rs.core.Cookie?
Here is an example of how to do it in Netty:
Cookie myCookie = ClientCookieDecoder.LAX.decode("my cookie string here")
Does anyone know of a useful utility method equivalent of this for jax rs?
Here is the method that works:
Cookie cookie = RuntimeDelegate.getInstance().createHeaderDelegate(Cookie.class).fromString(cookieStringValue);
Note: Don't use
Cookie.valueOf(cookieStringValue)
because it is deprecated.
I have a problem when used HttpURLConnection in Java, The connection to rest server give client cookie after authenticated. I can save the cookies from response header (Set-Cookie) after requesting and use them for next request but cookies that has !httponly flag is not available in response header (Set-Cookie).
I need the session_id cookie with !httponly flaged to be sent to next request so the connection no need to be authenticated in next request after the app exiting.
How can I save all cookies (basic cookies, secure cookies, httponly cookies) in Java just like cURL does?
After searching for a long time, I found the solution: http://jaunt-api.com/
I realize this question is old, but it was the first answer that came up in an internet search.
To resolve, use CookieHandler.setDefault( new CookieManager( null, CookiePolicy.ACCEPT_ALL ) ); per here: Should HttpURLConnection with CookieManager automatically handle session cookies?
I need to get the HttpOnly cookies set as Session cookie in my java code.
To get non HttpOnly cookies I used Jsoup but now am stucked with HttpOnly cookies ?
Note :
Don't know if it matters but the site from which I want to get HttpCookies is developed in ASP.net.
I would expect Jsoup to make those available:
Connection connection = Jsoup.connect("http://example.com");
Connection.Response response = connection.execute();
...then use the cookies map on response.
If the HttpOnly cookies aren't there, you may have to read the contents of the URL directly (e.g., via URLConnection or similar) before passing them on to JSoup.parse, and look for the cookie headers in that stream.
Side note: To get a Document after connecting as with the above, without repeating the request, use Connection#parse:
Document doc = connection.parse();
I just mention this because you're likely to want the document as well as the cookies.
I access a webpage by passing the session id and url and output is a HTML response.
I want to use jSoup to parse this response and get the tag elements.
I see the examples in Jsoup takes a String for establishing connection. How do i proceed.
pseudo code:
I tried the above method and got this exception
java.io.IOException: 401 error loading URL http://www.abc.com/index
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:387)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:364)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:143)
at org.jsoup.helper.HttpConnection.get(HttpConnection.java:132)
Basically the entity.getContent() has the HTML response which has to be passed as a String to the connect method. But it doesn't work.
Apache Commons HttpClient and Jsoup do not share the same cookie store. You basically need to pass the very same cookies as HttpClient has retrieved back through Jsoup's Connection. You can find some concrete examples here:
Sending POST request with username and password and save session cookie
how to maintain variable cookies and sessions with jsoup?
Alternatively, you can also just continue using HttpClient for firing HTTP requests and maintaining the cookies and instead feeds its HttpResponse as String through Jsoup#parse().
So this should do:
HttpResponse httpResponse = httpclient1.execute(httpget, httpContext);
String html = EntityUtils.toString(httpResponse.getEntity());
Document doc = Jsoup.parse(html, testUrl);
// ...
By the way, you do not necessarily need to create a whole new HttpClient for a subsequent request. Just reuse httpclient which you already created. Also your way of obtaining the response as String is clumsy. The second line in the above example shows how to do it at simplest.
It shows an http error 401 which means
Similar to 403 Forbidden, but specifically for use when authentication is possible but has failed or not yet been provided.
Therefore, i think you need to login into the website using your java code or identify yourself by sending cookies through your code.
I am trying to simulate HTTP requests in Java with the URL class and the HttpURLConnection class, the GET requests are easy to simulate while some POST requests seem harder because most POST request need Cookie in the request header. Some cookies were set by the HTTP response in the Set-Cookie field and I can get them by the function provided by HttpURLConnection, but I found that other cookies may be set by JavaScript and I have no way to handle them, so I wonder is there any packaged tool to simulate HTTP requests in Java?
try Apache commons Httpclient:
http://hc.apache.org/httpclient-3.x/
Why do you need to generate HTTP requests? Do you want to perform some stress tests?
I'd advise using something like JMeter (you can find a brief tutorial here).
Hope this helps something, it's better to avoid reinventing the wheel (if you need something like this, but it wasn't clear for me from your question).
For the cookie set with Javascript, you could try to parse the HTTP response, extract the cookie information and set it for your next request
For example, lets say, the response has code which calls a setCookie() function (setCookie is user-defined javascript function),
...
//some javascript code
setCookie("username", "johndoe");
//some more javascript
...
then you would extract the line setCookie() and the parse it for the name and value