I have a method that creates a cookie for localhost:9090/application.
public Object makeCookie(String p) throws IOException{
URL myUrl = new URL("localhost:9090/application");
URLConnection urlConn = myUrl.openConnection();
urlConn.setRequestProperty("testCookie", p);
urlConn.connect();
return urlConn;
}
I have a method that should print the name and domain of the cookie I just set, but I am getting no results.
CookieManager cookieManager;
URL url;
URLConnection connection;
CookieStore cookieStore;
List<HttpCookie> cookieList;
public boolean checkone (String test1) throws ClassNotFoundException, IOException{
cookieManager = new CookieManager();
CookieHandler.setDefault(cookieManager);
url = new URL("localhost:9090/application/");
connection = url.openConnection();
connection.getContent();
cookieStore = cookieManager.getCookieStore();
cookieList = cookieStore.getCookies();
for (HttpCookie cookie: cookieList){
System.out.println("Domain: " + cookie.getDomain());
System.out.println("name of cookie: " + cookie.getName());
}
return true;
}
Am I missing something when creating the cookie?
I do not think you are getting the cookies the right way. The cookies are set in the headers of the request. I do not see that in your example. Take a look at this:
https://examples.javacodegeeks.com/core-java/net/urlconnection/get-cookies-from-http-connection/
Take a look at this also:
How to set Cookies at Http Get method using Java
Related
I'm trying to get an access token from https://www.reddit.com/api/v1/access_token. To do so I need to send a CLIENT_ID and a CLIENT_SECRET to the above URL. I did so using Postman:
As highlighted on the screenshot, I've sent a grant_type as a GET parameter with value client_credentials and an Authorization parameter with value Basic heregoestheencodedkeyandid. The reuest type was set as POST. It worked correctly - I got an access token in a JSON response.
However, when I try to do the same thing by means of Java, I receieve a Server returned HTTP response code: 411 error:
public class RedditExample {
private static String loginLink = "https://www.reddit.com/api/v1/access_token";
public static void main(String[] args) {
RedditExample redditExample = new RedditExample ();
redditExample.login();
}
public boolean login() {
try {
URL loginURL = new URL(loginLink + "?grant_type=client_credentials");
HttpURLConnection connection = (HttpURLConnection) loginURL.openConnection();
setupPOSTConnection(connection);
InputStream input = connection.getInputStream();
String inputString = new Scanner(input, "UTF-8").useDelimiter("\\Z").next();
System.out.println(inputString);
}
catch (Exception e) {
System.out.println(e.toString());
}
return true;
}
private static void setupPOSTConnection(HttpURLConnection connection) throws Exception {
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Authorization", "Basic heregoestheencodedkeyandid");
connection.connect();
}
}
I'm not sure what I'm doing different here, compared to Postman, so any help would be appreciated.
EDIT: Here is what I tried adding:
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setRequestProperty("Content-Length", "0");
connection.setRequestProperty("Content-Length", "10");
String userAgent = "test /u/someuser";
connection.setRequestProperty("User-Agent", userAgent);
Unfortunately, neither worked - the error is still the same.
Setting content-length explicitly is not taken by HttpUrlConnection. So just provide request body with no content.
StringBuilder postDataBuilder = new StringBuilder();
byte[] postData = postDataBuilder.toString().getBytes("UTF-8");
OutputStream out = conn.getOutputStream();
out.write(postData);
out.close();
So the method will be like this:-
private static void setupPOSTConnection(HttpURLConnection connection) throws Exception {
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Authorization", "Basic heregoestheencodedkeyandid");
StringBuilder postDataBuilder = new StringBuilder();
byte[] postData = postDataBuilder.toString().getBytes("UTF-8");
OutputStream out = conn.getOutputStream();
out.write(postData);
out.close();
connection.connect();
}
also i found another way of simply adding one line:-
private static void setupPOSTConnection(HttpURLConnection connection) throws Exception {
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Authorization", "Basic heregoestheencodedkeyandid");
conn.setFixedLengthStreamingMode(0);
connection.connect();
}
HTTP Status Code 411 (Length Required) is sent by the server as a response when it refuses to accept a message without a content-length header, for whatever reason.
A server simply may or may not accept a content without a Content-Length header, and it seems this API is picky. See this post detailing a working POST connection using Java.
I want to execute some post but this post required PHPSESSID and one parameter.
If i take this parameters from browser and execute in my code all work correct.
Unfortunately when i use my code to get page content and cookie for "PHPSESSID" and parameter and use this to execute post, it doesn't work.
Example
PHPSESSID and param value we can obtain when we go in browser to 'http://www.kreskowkazone.pl/odcinki-online_rick-i-morty-2013_7'
PHPSESSID we can get from cookie and value for param "o" is a long number in icon/link play for "openload.io" (code look like this 181774:bb19d7426f8eda85ba82265a01eda9c7 but it change).
Whan we use this param in the code all works fine we see iframe html
Code to execute post
public void sendPostTest(String param, String phpSESSID) throws IOException {
URL url = new URL("http://www.kreskowkazone.pl/odcinki_emb");
Map<String, Object> params = new LinkedHashMap<>();
params.put("o", param);
StringBuilder postData = new StringBuilder();
for (Map.Entry<String, Object> p : params.entrySet()) {
if (postData.length() != 0) postData.append('&');
postData.append(URLEncoder.encode(p.getKey(), "UTF-8"));
postData.append('=');
postData.append(URLEncoder.encode(String.valueOf(p.getValue()), "UTF-8"));
}
byte[] postDataBytes = postData.toString().getBytes("UTF-8");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setRequestProperty("Host", "www.kreskowkazone.pl");
conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0");
conn.setRequestProperty("Accept", "text/html, */*; q=0.01");
conn.setRequestProperty("Accept-Language", "pl,en-US;q=0.7,en;q=0.3");
conn.setRequestProperty("Accept-Encoding", "gzip, deflate");
conn.setRequestProperty("Referer", "http://www.kreskowkazone.pl/odcinki-online_rick-i-morty-2013_24");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("X-Requested-With", "XMLHttpRequest");
conn.setRequestProperty("Content-Length", String.valueOf(postDataBytes.length));
conn.setRequestProperty("Cookie", phpSESSID);
conn.setRequestProperty("DNT", "1");
conn.setRequestProperty("Connection", "keep-alive");
conn.setDoOutput(true);
conn.setDoInput(true);
conn.getOutputStream().write(postDataBytes);
Reader in = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));
for (int c; (c = in.read()) >= 0; ) {
System.out.print((char) c);
}
}
Unfortunately when i take this param by my code i don't get any error and i don't see iframe.
To get page content and PHPSESSID I use this code.
public class CustomCookieManager {
public static void main(String[] args) {
try {
getPageContent("http://www.kreskowkazone.pl/odcinki-online_rick-i-morty-2013_7");
} catch (IOException e) {
e.printStackTrace();
}
}
public static void getPageContent(String pageUrl) throws IOException {
CookieManager ckman = new java.net.CookieManager();
ckman.setCookiePolicy(CookiePolicy.ACCEPT_ALL);
CookieHandler.setDefault(ckman);
URL url = new URL(pageUrl);
URLConnection connection = url.openConnection();
connection.getContent();
String html = getHtml(connection);
CookieStore ckStore = ckman.getCookieStore();
List<HttpCookie> cks = ckStore.getCookies();
for (HttpCookie ck : cks) {
System.out.println("PHPSESSID=" + ck.getValue());
System.out.println(html);
}
}
public static String getHtml(URLConnection conn) {
StringBuilder sb = new StringBuilder();
try {
BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
while ((inputLine = br.readLine()) != null) {
sb.append(inputLine);
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
return sb.toString();
}
}
I know PHPSESSID is related with param value because when i take PHPSESSID from one browser and parameter value from other browser
post doesn't work (no error no iframe).
My question is, is it posible that my CookieManager return cookie and PHPSESSID not for page which i'm reading in CustomCookieManager?
I tried the same example from HttpUnit with no result.
Try only using the same PHPSESSID as the one in your browser for every request, i was supposed to comment this but i wasn't allowed to.
Dr. Wells
I could but it will be one-off solution because PHPSESSID or number which is sending as parameter expired after some time.
After about half hour PHPSESSID and parameter are invalid (i don't see iframe html).
I can't use PHPSESSID from browser and parameter value returned by my java code because it's doesn't work.
It looks like generating parameter value depending on PHPSESSID.
I start thinking that my CookieManager is not releated with page which content i take in CustomCookieManager.
Is it possible that CookieManager joust return some PHPSESSID which is not related with page i downloaded.
I start thinking that page have some javascript which change something so i use HttpUnit but with no result.
Maybe it's impossible maybe it's some kind of security for the page but this CookieManager bites me.
I am trying to do automation of find all the broken link of a page. I have gone through so many articles here but none helped. The real problem I am facing is I am not able to get(returing) the correct httpresponse code. Below is the code:
public static int getResponseCode(String urlString) {
try {
URL url = new URL(urlString);
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("GET");
connection.connect();
return connection.getResponseCode();
} catch (Exception e) {
}
return -1;
}
you cant get the response using this code alone using java.
you need the java selenium driver code for this to be implemented.
use the below code to get the right response:
private static int statusCode;
public static void main(String... args) throws IOException{
WebDriver driver = new FirefoxDriver();
driver.manage().window().maximize();
driver.get("https://www.google.com/");
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
List<WebElement> links = driver.findElements(By.tagName("a"));
for(int i = 0; i < links.size(); i++){
if(!(links.get(i).getAttribute("href") == null) && !(links.get(i).getAttribute("href").equals(""))){
if(links.get(i).getAttribute("href").contains("http")){
statusCode= intgetResponseCode(links.get(i).getAttribute("href").trim());
if(statusCode == 403){
System.out.println("HTTP 403 Forbidden # " + i + " " + links.get(i).getAttribute("href"));
}
}
}
}
}
public static int getResponseCode(String urlString) throws MalformedURLException, IOException{
URL url = new URL(urlString);
HttpURLConnection huc = (HttpURLConnection)url.openConnection();
huc.setRequestMethod("GET");
huc.connect();
return huc.getResponseCode();
}
Else you can do get the response by setting the response method as "HEAD" [if its a simple test].
hope it helps. Cheers!
The following code worked for me with given example URL:
public static int getResponseCode(String urlString) throws IOException {
// String url = "http://www.google.com/search?q=mkyong";
String url = "https://www.google.co.in/intl/en/about.html?fg=1";
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// optional default is GET
con.setRequestMethod("GET");
//add request header
con.setRequestProperty("User-Agent", "Mozilla/5.0");
con.connect();
int responseCode = con.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
}
Output I got:
Sending 'GET' request to URL : https://www.google.co.in/intl/en/about.html?fg=1
Response Code : 200
Getting unauthorize execption while connecting to share point rest web service:
URL myURL = new URL("http://test:2014/PWA/_api/ProjectData/Projects");
URLConnection uc = myURL.openConnection();
HttpURLConnection myURLConnection = (HttpURLConnection)myURL.openConnection();
String userCredentials = "admin:pasword";
String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary("password".getBytes());
uc.setRequestProperty ("Authorization", basicAuth);
InputStream in = uc.getInputStream();
Getting following errors while reading from url
java.io.IOException: Server returned HTTP response code: 401 for URL: http://test:2014/PWA/_api/ProjectData/Projects
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at com.jw.sharepoint.examples.XMLParser.getDocumentFromUrl(XMLParser.java:127)
at com.jw.sharepoint.examples.XMLParser.main(XMLParser.java:27)
java.lang.NullPointerException
at com.jw.sharepoint.examples.XMLParser.main(XMLParser.java:29)
Add request property and request method solved the problem.
InputStream getAuthenticatedResponse(final String urlStr, final String domain,final String userName, final String password) throws IOException {
Authenticator.setDefault(new Authenticator() {
#Override
public PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(
domain + "\\" + userName, password.toCharArray());
}
});
URL urlRequest = new URL(urlStr);
HttpURLConnection conn = (HttpURLConnection) urlRequest.openConnection();
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "*/*");
return conn.getInputStream();
}
http://127.0.0.1:8080/Mailchimp/Access#access_token=^^^^^^^^^^^^^^^^^^^^^^^^^&expires_in=0
This is my mailchimp url , how to extract the access_token through java
and there another way to get the access token but here getting response message - Bad Request
URL siteUrl = new URL("https://login.mailchimp.com/oauth2/token");
HttpURLConnection conn = (HttpURLConnection) siteUrl.openConnection();
conn.setRequestMethod("POST");
conn.setDoOutput(true);
conn.setDoInput(true);
DataOutputStream out = new DataOutputStream(conn.getOutputStream());
String content = "?grant_type=authorization_code&client_id=^^^^^^^^^^&client_secret=^^^^^^^^^^^^^^^^^^^^^^^^&code="+auth_code+"&redirect_uri=http://127.0.0.1:8080/Mailchimp/Access";
out.writeBytes(content);
System.out.println(content);
out.flush();
out.close();
System.out.println(conn.getResponseMessage());
The part after the # is called the "fragment" or "reference". This is a basic way to extract the relevant part:
public static String getAccessToken(String uriString) {
URI uri = URI.create(uriString);
String[] parameters = uri.getFragment().split("\\&");
for (String parameter : parameters) {
String[] parts = parameter.split("\\=");
if (parts[0].equals("access_token")) {
if (parts.length == 1) {
throw new RuntimeException("missing access token");
}
return parts[1];
}
}
throw new RuntimeException("no access token");
}