A piece of existing Java code is broken. It is used to pull a static file from Github in raw format. For some reason, the URL is redirecting to another one with a random token at the end. I've specified HttpURLConnection to follow the redirection, but it seems not working. Here is the code:
HttpURLConnection.setFollowRedirects(true);
URL urlClass = new URL(url);
HttpURLConnection uc = (HttpURLConnection)urlClass.openConnection();
String userpass = githubUserName + ":" + githubPassword;
String basicAuth = "Basic " + new String(new Base64().encode(userpass.getBytes()));
uc.setRequestProperty("Authorization", basicAuth);
uc.setInstanceFollowRedirects(true);
String fileContent = IOUtils.toString(uc.getInputStream());
The URL looks like this:
https://github.com/project/raw/master/filename.json
And after the redirection: https://github.com/raw/project/master/filename.json?token=somerandomlettershere
Interestingly the web browser can handle the redirection automatically and I can see the content there while the HttpURLConnection only returns 404.
Update:
It was resolved by refactoring original code with Github Java API(https://github.com/eclipse/egit-github/tree/master/org.eclipse.egit.github.core) in which way there is no need to access the file through the full URL, which led to the redirection issue, but rather providing the repository and the file name.
Related
I have a currently working program in Python and I am trying to convert it to Java.
webCmd = "http://192.168.1.xxx/rest/nodes/21 F1 DD 1/ST"
r = requests.get(webCmd, timeout=(0.1,2), auth=('username', 'password'))
I am new to Java and my code fails.
String authString = "username" + ":" + "password";
String encodedAuth = Base64.getEncoder().encodeToString(authString.getBytes());
String authHeader = "Basic " + encodedAuth;
webCmd = "http://192.168.1.xxx/rest/nodes/21 F1 DD 1/ST";
HttpURLConnection connection = (HttpURLConnection) new URL(webCmd).openConnection();
connection.setRequestProperty ("Authorization", authHeader);
connection.setRequestMethod("GET");
int responseCode = connection.getResponseCode();
If I run the above code, I get a 404 error. If I paste the webCmd in the browser, it prompts me for username, password and responds with the correct xml output. If I comment out the setRequestProperty line, then the response is 401 as expected.
Any ideas?
Check https://docs.oracle.com/javase/8/docs/api/java/net/URL.html
Specifically
The URL class does not itself encode or decode any URL components according to the escaping mechanism defined in RFC2396. It is the responsibility of the caller to encode any fields, which need to be escaped prior to calling URL, and also to decode any escaped fields, that are returned from URL. Furthermore, because URL has no knowledge of URL escaping, it does not recognise equivalence between the encoded or decoded form of the same URL. For example, the two URLs:
http://foo.com/hello world/ and http://foo.com/hello%20world
would be considered not equal to each other.
Your URL contains spaces, so you need to encode them explicitly. I guess Python does that automatically.
I am attempting to authenticate from the android app. It is an apache basic auth. It seems, through error logs, that the webpage is taking what is after the URL as the username for some reason? Any thoughts on this? Here is my code:
URL url = new URL("https://website.com/");
HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
connection.setSSLSocketFactory(context.getSocketFactory());
connection.setRequestProperty("Authorization", "Basic " +
Base64.encode("user:pass".getBytes(), Base64.NO_WRAP));
connection.setDoOutput(true);
connection.connect();
If i add a path the the website, it tells me that that path is not a user. Without a path it says that "/" is not a user.
I have also tried doing user:pass#website.com which didn't work either.
I'm trying to authenticate to a site that uses OAuth2 and store the token in my session object. My web app initially checks to see if there's a token already there, and if there isn't it redirects the user to the login page on the external site, where the user logs in and gets redirected back to my app. So far, so good, this works. My app directs me to the external site (Mendeley), I log in there, and then it redirects me back to the url in my app that I expect it to.
When it redirects back to my app, I expect a code and a state parameter on the request, and I do see these, so I assume I'm on the right track (stop me if I'm wrong). So then, if I understand correctly, I'm supposed to post the code back to the Mendeley service to get my authorization token, and that's where it all blows up.
URL url = new URL("https://api-oauth2.mendeley.com/oauth/token");
HttpsURLConnection connection = (HttpsURLConnection) url
.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded");
String authString = getClientId() + ":" + "[MY CLIENT SECRET]";
System.out.println("auth string: " + authString);
byte[] authEncBytes = Base64.getUrlEncoder().encode(
authString.getBytes());
String authStringEnc = new String(authEncBytes);
System.out.println("Base64 encoded auth string: " + authStringEnc);
connection.addRequestProperty("Authorization", "Basic "
+ authStringEnc);
connection.setDoOutput(true);
OutputStream os = connection.getOutputStream();
OutputStreamWriter writer = new OutputStreamWriter(os);
writer.write("scope=all&grant_type=authorization_code");
writer.write("&client_id=");
writer.write(getClientId());
writer.write("&code=");
writer.write(code);
writer.write("&redirect_uri=");
writer.write(getMendeleyRedirectUrl(request));
writer.write("&client_secret=");
writer.write("[MY CLIENT SECRET]");
writer.flush();
writer.close();
int responseCode = connection.getResponseCode();
BufferedReader reader = new BufferedReader(new InputStreamReader(
connection.getInputStream()));
The response code I get is 401. On that last line where it tries to get the inputStream from the connection it throws an exception, and that makes sense to me sense it returned a 401 and doesn't have one.
Yes, the redirect_uri is encoded. (I don't think the initial redirect to the login would work otherwise.)
My Spidey Sense tells me I'm overlooking something that should be obvious to me, but I've tried everything I could think of. Any help would be greatly appreciated.
Edit: changed how auth header is added, now getting response code 400.
You should check if you are creating the correct basic auth header. It should be something like this:
String user = "your app id";
String password = "your app secret";
String authValue = user + ":" + password;
Base64.Encoder encoder = Base64.getEncoder();
Bytes[] btyes = authValue.getBytes(StandardCharsets.UTF_8);
String authValueEncoded = encoder.encodeToString(bytes);
connection.addRequestProperty("Authorization",
"Basic "+authValueEncoded);
This values for user and password are specific for Mendeley. See step 4 of http://dev.mendeley.com/reference/topics/authorization_auth_code.html
Regarding the error 400, you might want to check the grant_type, code or redirect_uri. Remember that the code can only be used once.
from the docs:
Errors due to incorrect or missing values for grant_type, code and
redirect_uri result in a HTTP bad request response with a status of
400 Bad Request and a JSON format error code and message:
HTTP/1.1 400 Bad Request Content-Type: application/json
Content-Length: 82
{"error":"invalid_grant","error_description":"Invalid access code"}
Missing values generate a response with an invalid_request error code.
Invalid values (including previously used codes) generate a response
with an invalid_grant error code. Specifying a value other than
authorization_code (or refresh_token) generate a response with an
unsupported_grant_type error code.
So you might wan to look inside the response body to see what's wrong.
Can someone please explain how exactly the user credentials are passed to the server in the below code...
URL urlObj = new URL("https://javaguy.com");
HttpURLConnection conn = (HttpURLConnection) urlObj.openConnection(); conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "text/xml");
String userPassword = username + ":" + password;
byte[] authEncBytes = Base64.encodeBase64(userPassword.getBytes());
String authStringEnc = new String(authEncBytes);
conn.setRequestProperty("Authorization", "Basic " + authStringEnc);
Is it part of the HTTP header? Just curious.
Thanks in advance.
You've answered your own question, but yes. The "Authorization" is part of the header.
You can read more about basic authentication on the wikipedia.
Then the javadoc isn't super clear, but the setRequestProperty should add the new property to the request header.
As a side note, I would urge you to consider using a library like HttpClient if you're planning on doing any http requests in a production system. Working directly with URL and URLConnection directly can be tricky. HttpClient isn't super easy to work with either, but it is easier then URL/URLConnection.
There are two HttpClient libraries, make sure you're working with version 4 (which is the latest version at the time of this post) and not version 3.
I want to download the mp3 file from url : "http://upload13.music.qzone.soso.com/30671794.mp3", i always got java.io.IOException: Server returned HTTP response code: 403 for URL. But it's ok when open the url using browser. Below is part of my code:
BufferedInputStream bis = null;
BufferedOutputStream bos = null;
try {
URL url = new URL(link);
URLConnection urlConn = url.openConnection();
urlConn.addRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
String contentType = urlConn.getContentType();
System.out.println("contentType:" + contentType);
InputStream is = urlConn.getInputStream();
bis = new BufferedInputStream(is, 4 * 1024);
bos = new BufferedOutputStream(new FileOutputStream(
fileName.toString()));
Anyone could help me? Thanks in advance!
You can also use
System.setProperty("http.agent", "Chrome");
it worked for me.
//Update
Explanation
Because HttpURLConnection reads the property "http.agent" if set.
You can read it here: https://www.innovation.ch/java/HTTPClient/advanced_info.html
Or you can look it up in the source code of the HttpURLConnection Class:
String agent = java.security.AccessController.doPrivileged(new sun.security.action.GetPropertyAction("http.agent"));
Instead of using URLConnection in java, if you use HttpURLConnection you should beable to access the requested web page from java. Try the following code:
HttpURLConnection httpcon = (HttpURLConnection) url.openConnection();
httpcon.addRequestProperty("User-Agent", "Mozilla/4.76");
Normal java using urlConnection wont be accepted to access the internet. To access the browser it will need to perform a search without theexception HTTP response code : 403 for URL
EDIT (#Mordechai): No need to do the casting, just add the user agent.
When I access the URL with my browser I also get 403. Perhaps you're logged in to the site with your browser?
If that's the case you need to duplicate the cookie from your browser and send it along, perhaps even do more to replicate your browser's signature if the site does any extra checks.
You can set the cookie by adding:
urlConn.setRequestProperty("Cookie", "foo=bar");
Where foo=bar is the key-value pair you'll find when you locate the site's cookie in your browser.
The problem is given by the Status code. 403 means actually "Forbidden" and implies The request was denied for a reason the server does not want to (or has no means to) indicate to the client.
the problem lies at the server-side.
I would also check if the server were the resource is located has an ACL or similar in place, we just resolved a "java.io.IOException: 403" issue this way.
It happens that 403 errors are very generic and you cannot really be sure of the source as it can be just anything.