Develop Reference

HttpsURLConnection authenticating twice?

I establish a secure http connection and attempt to get the InputStream from it afterwards. The connection occurs, and I am able to get the data, but I am actually sending two authorization requests to the server?? Here is my code that is getting the connection and getting the input stream established:
someConnection = (HttpsURLConnection) url.openConnection();
String userPass = username + ":" + password;
String basicAuth = "Basic" + new String(new Base64().encode(userPass.getBytes()));
someConnection.setRequestProperty("Authorization", basicAuth);
if (header != null) someConnection.setRequestProperty(header, headerValue);
InputStream is = someConnection.getInputStream();
There is no traffic until the .getInputStream() method is called. Then I see two requests for authorization:
Any ideas why it is doing that? the first request appears to be failing for some reason.

The value of your header Authorization doesn't match with the expected format, it should be "Basic " followed by ${username}:${password} encoded with RFC2045-MIME variant of Base 64 (more details about Basic access authentication).
Here you forgot to add the trailing space after Basic such that authentication is never done properly which leads to this unexpected behavior.

There should be space between "Basic" and the base64 encoded data.
Without this the Authorization header is wrong. I would guess that you receive 401 on the first request and send the next with other credentials possibly obtained from different source (JAAS?).

OAuth2 requesting token returns 401

I'm trying to authenticate to a site that uses OAuth2 and store the token in my session object. My web app initially checks to see if there's a token already there, and if there isn't it redirects the user to the login page on the external site, where the user logs in and gets redirected back to my app. So far, so good, this works. My app directs me to the external site (Mendeley), I log in there, and then it redirects me back to the url in my app that I expect it to.
When it redirects back to my app, I expect a code and a state parameter on the request, and I do see these, so I assume I'm on the right track (stop me if I'm wrong). So then, if I understand correctly, I'm supposed to post the code back to the Mendeley service to get my authorization token, and that's where it all blows up.
URL url = new URL("https://api-oauth2.mendeley.com/oauth/token");
HttpsURLConnection connection = (HttpsURLConnection) url
.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded");
String authString = getClientId() + ":" + "[MY CLIENT SECRET]";
System.out.println("auth string: " + authString);
byte[] authEncBytes = Base64.getUrlEncoder().encode(
authString.getBytes());
String authStringEnc = new String(authEncBytes);
System.out.println("Base64 encoded auth string: " + authStringEnc);
connection.addRequestProperty("Authorization", "Basic "
+ authStringEnc);
connection.setDoOutput(true);
OutputStream os = connection.getOutputStream();
OutputStreamWriter writer = new OutputStreamWriter(os);
writer.write("scope=all&grant_type=authorization_code");
writer.write("&client_id=");
writer.write(getClientId());
writer.write("&code=");
writer.write(code);
writer.write("&redirect_uri=");
writer.write(getMendeleyRedirectUrl(request));
writer.write("&client_secret=");
writer.write("[MY CLIENT SECRET]");
writer.flush();
writer.close();
int responseCode = connection.getResponseCode();
BufferedReader reader = new BufferedReader(new InputStreamReader(
connection.getInputStream()));
The response code I get is 401. On that last line where it tries to get the inputStream from the connection it throws an exception, and that makes sense to me sense it returned a 401 and doesn't have one.
Yes, the redirect_uri is encoded. (I don't think the initial redirect to the login would work otherwise.)
My Spidey Sense tells me I'm overlooking something that should be obvious to me, but I've tried everything I could think of. Any help would be greatly appreciated.
Edit: changed how auth header is added, now getting response code 400.

You should check if you are creating the correct basic auth header. It should be something like this:
String user = "your app id";
String password = "your app secret";
String authValue = user + ":" + password;
Base64.Encoder encoder = Base64.getEncoder();
Bytes[] btyes = authValue.getBytes(StandardCharsets.UTF_8);
String authValueEncoded = encoder.encodeToString(bytes);
connection.addRequestProperty("Authorization",
"Basic "+authValueEncoded);
This values for user and password are specific for Mendeley. See step 4 of http://dev.mendeley.com/reference/topics/authorization_auth_code.html
Regarding the error 400, you might want to check the grant_type, code or redirect_uri. Remember that the code can only be used once.
from the docs:
Errors due to incorrect or missing values for grant_type, code and
redirect_uri result in a HTTP bad request response with a status of
400 Bad Request and a JSON format error code and message:
HTTP/1.1 400 Bad Request Content-Type: application/json
Content-Length: 82
{"error":"invalid_grant","error_description":"Invalid access code"}
Missing values generate a response with an invalid_request error code.
Invalid values (including previously used codes) generate a response
with an invalid_grant error code. Specifying a value other than
authorization_code (or refresh_token) generate a response with an
unsupported_grant_type error code.
So you might wan to look inside the response body to see what's wrong.

URL redirect for Github raw file

A piece of existing Java code is broken. It is used to pull a static file from Github in raw format. For some reason, the URL is redirecting to another one with a random token at the end. I've specified HttpURLConnection to follow the redirection, but it seems not working. Here is the code:
HttpURLConnection.setFollowRedirects(true);
URL urlClass = new URL(url);
HttpURLConnection uc = (HttpURLConnection)urlClass.openConnection();
String userpass = githubUserName + ":" + githubPassword;
String basicAuth = "Basic " + new String(new Base64().encode(userpass.getBytes()));
uc.setRequestProperty("Authorization", basicAuth);
uc.setInstanceFollowRedirects(true);
String fileContent = IOUtils.toString(uc.getInputStream());
The URL looks like this:
https://github.com/project/raw/master/filename.json
And after the redirection: https://github.com/raw/project/master/filename.json?token=somerandomlettershere
Interestingly the web browser can handle the redirection automatically and I can see the content there while the HttpURLConnection only returns 404.

Update:
It was resolved by refactoring original code with Github Java API(https://github.com/eclipse/egit-github/tree/master/org.eclipse.egit.github.core) in which way there is no need to access the file through the full URL, which led to the redirection issue, but rather providing the repository and the file name.

URL encoding of "&"

I am currently facing an issue with sending some encoded characters. My main aim is to send the text using a POST https request. The catch is that the back-end is not quite the best one, so special letters (such as æ) I have to send in a special (custom) way.
Giving a simple example, I have the text hjælp. The letter æ should become &aelig in order the back-end to understand that it's this specific letter.
My url looks like this:
https://example.com/back-end/sendText?user=admin&text=hj&aeliglp
Obviously, this wouldn't work, because the back-end would see 3 parameter keys: user, text and aeliglp.
Of course, in code, my url is an actual URL object. However, if I use URLEncoder.encode(value, "utf-8"); it would turn my & into %26 and the %26 itself to %2526.
On wikipedia I read about this:
Because the percent ("%") character serves as the indicator for
percent-encoded octets, it must be percent-encoded as "%25" for that
octet to be used as data within a URI.
Nevertheless, I must send it with a %26, but without encoding it to %2526. That is because I cannot change or ask for a change on the back-end.
In order to send the POST I use the most basic way:
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(); //url is my URL object
conn.setRequestMethod("POST");
int result = conn.getResponseCode();
Is there any way I can create an URL object without encoding it automatically?

Different behavior when sending POST Request through Java application and Chrome's POSTMAN Extension. How do I debug and find the mistake?

I am making a POST Request to a server with Content-Type set to application/x-www-form-urlencoded. One of the query parameter has a value which contains & in it. I replace the & with & before sending the request.
When I send this request using POSTMAN (Chrome Extension), the request goes fine and I receive the expected response. But when I send this request using a Java application, the server throws an error (unable to parse document). Here is the code that I'm using to send the request:
URL url = new URL(url); // url => "http://myserver.com/api/update"
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setConnectTimeout(60000);
urlConnection.setReadTimeout(60000);
urlConnection.setDoOutput(true);
urlConnection.setRequestMethod("POST");
urlConnection.setDoInput(true);
urlConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
PrintWriter printWriter = new PrintWriter(urlConnection.getOutputStream());
printWriter.print(params); // params => api_key=abcd123&update_data_id=123&update_data_value=Test&Value
printWriter.flush();
InputStream inputStream = urlConnection.getInputStream();
String contentType = urlConnection.getContentType();
FileUtil.closeWriter(printWriter);
// Parse response ...
Here, the problem occurs in the parameter update_data_value. If I remove the &, the request goes fine from both, POSTMAN as well as my application. But when the & is present, only the request through POSTMAN works fine.
What could be wrong ?
Thanks a lot for your help!

After a long conversation in chat, the problem was this:
It's about a XML string, where an ampersand is used. The ampersand needs to replaced with "&", according to the XML-Standard.
This XML string needs to be sent in a POST request, so the ampersand and likely the semicolon need to be escaped.
The final replacement string looks like this: "%26amp%3B".