The location header is there, I can see it in browser:
I'm using org.apache.httpcomponents.httpclient to send http request with cookie:
```
URI uri = new URIBuilder().setScheme("https").setHost("api.weibo.com").setPath("/oauth2/authorize").setParameter("client_id","3099336849").setParameter("redirect_uri","http://45.78.24.83/authorize").setParameter("response_type", "code").build();
HttpGet req1 = new HttpGet(uri);
RequestConfig config = RequestConfig.custom().setRedirectsEnabled(false).build();
req1.setConfig(config);
req1.setHeader("Connection", "keep-alive");
req1.setHeader("Cookie", cookie);
req1.setHeader("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36");
response = httpclient.execute(req1);
```
I googled a lot and tried enable/disable auto redirect,but it doesn't seem to work for me.So can somebody tell me how to get the location header in response just like the browser did?
You cannot see 'location' header, because HttpClient followed that redirect immediately - even before giving you that response.
Try disabling redirect while setting up your HttpClient:
HttpClient instance = HttpClientBuilder.create().disableRedirectHandling().build();
Check this URL and you'll see the Location Header:
URI uri = new URIBuilder().setScheme("https").setHost("www.googlemail.com").setPath("/").build();
I found out my real question...I didn't pass the auth process in my code,so I keep getting oauth2 page.After I set all the headers in my request just like the browser did and finally I get the right response.
Related
Tried to request with postman which gives desired response.
The java code used for making the same request is as below fails with a 403 status.
String url = "https://steamcommunity.com/inventory/76561198865293952/440/2?l=english&count=5000";
String cookiesString = "sessionid=" + generateSessionId() + ";steamCountry=IN%7Ce744269b3c4e531facb33ecaff29eb44";
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest
.newBuilder()
.GET()
.header("User-Agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36")
.header("Accept", "*/*")
.header("Cookie", cookiesString)
.uri(URI.create(url))
.build();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.statusCode()); //prints 403
System.out.println(response.body()); // prints �+�� O��%
/**
* Generates a Steam Session Id.
* #return sessionId.
*/
public static String generateSessionId() {
return new BigInteger(96, new Random()).toString(16);
}
Postman request does not need any headers(Not even User-Agent, only Host is kept, No cookies are needed) at all to get the desired response. Even navigating to link in browser shows the json response.
PostMan request ScreenShot
Postman Request Image with cookies
Thank you. Have a great day.
I am pretty sure that the 2 cookies you have in postman and not in the java code are responsible for having the 403 response.
Postman is a google chrome plugin so when connecting w/ chrome, maybe you stored in the cache the cookies.
You also need to add them to the java code.
Firstly Thanks for all your answers and replies.
So I found that the inventory that I was trying to access was set visible to friends only, so response was 403. So it does work for inventories that are set to public.
I am sorry, I should have check it first.
What I still dont understand is how did the postman desktop client get the intended response? I understand that chrome was able to get the inventory because I have active steam login with cookies that would have been send but postman request did not have any cookies other than sessionid and country. If someone could explain how this happens it would really help me.
I got a problem with sending bearer token to the One Note API.
String returnUri = "https://login.live.com/oauth20_token.srf";
HttpClient client = HttpClientBuilder.create().build();
HttpPost tokenRequest = new HttpPost(returnUri);
tokenRequest.setHeader(HttpHeaders.CONTENT_TYPE, "application/x-www-form-urlencoded");
tokenRequest.setEntity(new UrlEncodedFormEntity(Connection.getParametersForURLBody(), Consts.UTF_8));
tokenRequest.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0");
HttpResponse tokenResponse = client.execute(tokenRequest);
HttpGet getTopFiveNotebooks = new HttpGet("https://www.onenote.com/api/v1.0/me/notes/notebooks?top=5");
getTopFiveNotebooks.setHeader(HttpHeaders.AUTHORIZATION, "Bearer " + Connection.getValueByKey("access_token", Connection.getTokenInJson(tokenResponse)));
I got the Bearer Token and the header of the HttpGet-Request looks like this, if I look at it in debug-mode:
But when I try to perform the get, the API gives me a 401 Unauthorized Error.
My Scope is scope=wl.basic+onedrive.readwrite, so the token should have all permissions it needs.
Update: If I login into https://apigee.com/onenote/embed/console/onenote/ with my microsoft-account and copy the access-token from there into this piece of code:
getTopFiveNotebooks.setHeader(HttpHeaders.AUTHORIZATION, "Bearer " + acces-key-from-the apigee-console)
it performs the get and give me Status 200 back instead of 401.
So is my permission scope wrong?
Edit: My Scope was false.
Yes, you don't have the right scopes.
https://msdn.microsoft.com/en-us/library/office/dn807159.aspx
You need at least "office.onenote" to be able to get the user's notebooks.
Btw, if you look at the body of the 401 response, you'll see which scopes are missing.
Here are some cases where error could happen:
Please pay attention that string of scopes must be encoded too, so
instead of + you should use %20.
Also make sure that this function you used, returns anything:
Connection.getTokenInJson(tokenResponse)
And try this permission scope which works fine for me:
"office.onenote%20office.onenote_create%20office.onenote_update_by_app%20office.onenote_update"
I have a java program that is trying to read any arbitrary file from URL. However, it return an 404 error when the file clearly exist, try it for yourself with the URL. What is wrong?
URL url = new URL("http://images.all-free-download.com/images/graphiclarge/blue_abstract_background_310971.jpg");
HttpURLConnection myHTTPConTest = null;
myHTTPConTest = (HttpURLConnection) url.openConnection();
int responseCode = myHTTPConTest.getResponseCode(); // Returns 404
Added user-agent, no change:
myHTTPConTest.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401");
I think you had a bad luck and you just hit the server restart time or something like that.
Because it returns 200 now.
This is my code:
URL url = new URL("http://superchillin.com/login2.php");
HttpURLConnection urlConnection = (HttpURLConnection)url.openConnection();
urlConnection.setUseCaches(false);
urlConnection.setRequestMethod("POST");
String data = "email="+URLEncoder.encode(name, "UTF-8")+"&password="+URLEncoder.encode(pass, "UTF-8");
urlConnection.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
urlConnection.setRequestProperty("Accept-Encoding", "gzip,deflate");
urlConnection.setRequestProperty("Accept-Language", "en-US,en;q=0.8,lt;q=0.6");
urlConnection.setRequestProperty("Cache-Control", "max-age=0");
urlConnection.setRequestProperty("Connection", "keep-alive");
urlConnection.setRequestProperty("Content-Length", Integer.toString(data.getBytes().length));
urlConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
urlConnection.addRequestProperty("Cookie", "place=1");
urlConnection.addRequestProperty("Cookie", "lvca_unique_user=1");
urlConnection.setRequestProperty("Host", "superchillin.com");
urlConnection.setRequestProperty("Origin", "http://superchillin.com");
urlConnection.setRequestProperty("Referer", "http://superchillin.com/login.php");
urlConnection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36");
urlConnection.setDoOutput(true);
urlConnection.setDoInput(true);
urlConnection.setInstanceFollowRedirects(true);
DataOutputStream wr = new DataOutputStream(urlConnection.getOutputStream());
wr.writeBytes(data);
wr.flush();
wr.close();
After that code I only read the response. It redirects me to "login.php" and is trying to set cookie "place=1"...
Connecting via browser works great. The reason for so many headers is I thought they may be the problem so I copied all headers from which I see when using a browser.
The response code is 200.
I also noticed that if password or email is incorrect, there's a message saying that in HTML which i retrieve.
When I use a browser I get redirected to index.php and cookie "auth" is set. So that's what I'm expecting from my program aswell. Curently I get redirected back to "login.php".
There is no universal answer to this question, I'm afraid. What you're asking is "why does the remote server not return an auth cookie when I send this exact request?" And that depends entirely on what the server's documentation says about those requests, whether it has any bugs in its implementation, etc.
If you don't have access to the server's own source and logs, then you'll likely have to get by with experimentation. Use something like Firebug or Chrome's Developer Tools to capture the exact requests sent by the browser with the login works successfully. Since these text strings are the only thing the remote server sees, if you replicate them exactly with your Java program you will(/should) get exactly the same responses.
If you think you're sending the same requests from Java and find that you're still not getting the expected responses, there must be some difference. Try recording the network traffic with something like Wireshark in order to see exactly what your app is sending - and then address the differences.
And if you get to the point where e.g. a redirect isn't being followed, and you're not sure how to do that with a URLConnection - then that's a good concrete question to ask.
Trying to download a file in Java from a Http server (nginx)
The exact same link java is attempting to download works in the browser and downloads, but java responds with:
java.io.IOException: <URL WOULD BE HERE> returned response code -1
at com.atlauncher.data.Downloadable.getConnection(Downloadable.java:149)
at com.atlauncher.data.Downloadable.getFilesize(Downloadable.java:85)
at com.atlauncher.workers.InstanceInstaller.configurePack(InstanceInstaller.java:1134)
at com.atlauncher.workers.InstanceInstaller.doInBackground(InstanceInstaller.java:1399)
at com.atlauncher.workers.InstanceInstaller.doInBackground(InstanceInstaller.java:59)
at javax.swing.SwingWorker$1.call(SwingWorker.java:296)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at javax.swing.SwingWorker.run(SwingWorker.java:335)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
The download code:
this.connection = (HttpURLConnection) new URL(this.url).openConnection();
this.connection.setUseCaches(false);
this.connection.setDefaultUseCaches(false);
this.connection.setConnectTimeout(9000);
//this.connection.setRequestProperty("Accept-Encoding", "gzip");
this.connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36");
this.connection.setRequestProperty("Cache-Control", "no-store,max-age=0,no-cache");
this.connection.setRequestProperty("Expires", "0");
this.connection.setRequestProperty("Pragma", "no-cache");
this.connection.connect();
if (this.connection.getResponseCode() / 100 != 2) {
System.out.println(this.connection);
throw new IOException(this.url + " returned response code "
+ this.connection.getResponseCode());
}
Any ideas why this is occurring? It's strange that the exact same url works in the browser. And the exact same code works downloading different files from the same server, and directory...
To find out, you must not throw the IOException and print the complete response. As according to the javadoc of the getResponseCode() method:
Returns -1 if no code can be discerned from the response (i.e., the
response is not valid HTTP).
You may have a response which is not HTTP at all.
This has happened to me on one occasion where I was using my own implementation of an HTTP server. When running in browsers, it would work fine but making an HttpURLConnection would give the Invalid Http response and response code would be -1.
The problem was that the HttpURLConnection looks for header strictly in the format
HTTP/1.1 200 OK
but my custom server was giving just an
HTTP 200 OK
As soon as I changed the version to 1.1, it worked. So check your response using cURL as follows:
curl -v -I URL_HERE
Hope it helps!