Develop Reference

Getting an access token using OAuth, the response appears to be encoded

I'm pretty new to OAuth and am trying to refresh a token using a simple java client. Everything seems to go ok (or I think it does), the problem is I can't tell from the response if it's a good one. The http response code is 200, but when trying to parse the response for access_token, I get a null. Also difficult to troubleshoot is the "raw" response is garbled, or encoded in some way. I was thinking maybe it's byte but it doesn't seem to be. Here's the code:
private static String getClientCredentials() {
String postParams = "grant_type=refresh_token&refresh_token=1234567890";
Pattern pat = Pattern.compile(".*\"access_token\"\\s*:\\s*\"([^\"]+)\".*");
String clientId = "myClientID123";
String clientSecret = "myClientSecret123";
String tokenUrl = "https://www.host.com/oauth2/tenant/token";
String auth = clientId + ":" + clientSecret;
String authentication = Base64.getEncoder().encodeToString(auth.getBytes());
BufferedReader reader = null;
HttpsURLConnection connection = null;
String returnValue = "";
try {
URL url = new URL(tokenUrl);
connection = (HttpsURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Authorization", "Basic " + authentication);
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.setRequestProperty("Accept", "application/x-www-form-urlencoded");
connection.setRequestProperty("Accept-Encoding", "gzip, deflate, br");
connection.setRequestProperty("Connection", "keep-alive");
connection.setDoOutput(true);
OutputStream outStream = connection.getOutputStream();
outStream.write(postParams.getBytes());
outStream.flush();
outStream.close();
System.out.println("Resp code: " + connection.getResponseCode());
System.out.println("Resp message: " + connection.getResponseMessage());
reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line = null;
StringWriter out = new StringWriter(connection.getContentLength() > 0 ? connection.getContentLength() : 2048);
while ((line = reader.readLine()) != null) {
out.append(line);
}
String response = out.toString();
Matcher matcher = pat.matcher(response);
if (matcher.matches() && matcher.groupCount() > 0) {
returnValue = matcher.group(1);
}
System.out.println("response: " + response);
System.out.println("returnValue: " + returnValue);
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException e) {
}
}
connection.disconnect();
}
return returnValue;
}
Here's the console output, sorry, I had to screenshot it because the characters didn't paste right:
Response Screenshot
Am I trying something that isn't allowed or is there a way to decode the response so I can run the pattern match to extract only the access_token? Or am I going about it all wrong? Any help is greatly appreciated in advance!

Microsoft IdentityModel Tokens AudienceUriValidationFailedException when trying to access Sharepoint using REST API

I am struggling to understand the whole new idea of accessing my organizatioon's Sharepoint content using Sahrepoint REST API and I am trying to implementing it in java. My aim is to read all the files in "abc" folder which is in Documents folder. Steps I did.
Register the app:
Click Generate Client ID,
Click Generate Client Secret,
Gave Title,
Gave Appdomain as companyname.onmicrosoft, and
Gave Request URI as https://companyname.sharepoint.com/Shared%20Documents/Forms/AllItems.aspx
Got the app registered. I have client id, client secret, and tenant id.
I used the below code to generate the access token
public String getSpToken(String shp_clientId, String shp_tenantId, String shp_clientSecret) {
String accessToken = "";
try {
// AccessToken url
String wsURL = "https://accounts.accesscontrol.windows.net/" + shp_tenantId + "/tokens/OAuth/2";
URL url = new URL(wsURL);
URLConnection connection = url.openConnection();
HttpURLConnection httpConn = (HttpURLConnection) connection;
// Set header
httpConn.setRequestProperty("Content-Type", " ");
httpConn.setDoOutput(true);
httpConn.setDoInput(true);
httpConn.setRequestMethod("POST");
// Prepare RequestData
String jsonParam = "grant_type=client_credentials"
+ "&client_id=" + shp_clientId + "#" + shp_tenantId
+ "&client_secret=" + shp_clientSecret
+ "&resource=00000003-0000-0ff1-ce00-000000000000/www.companyname.sharepoint.com#" + shp_tenantId;
// Send Request
DataOutputStream wr = new DataOutputStream(httpConn.getOutputStream());
wr.writeBytes(jsonParam);
wr.flush();
wr.close();
// Read the response.
InputStreamReader isr = null;
if (httpConn.getResponseCode() == 200) {
isr = new InputStreamReader(httpConn.getInputStream());
} else {
isr = new InputStreamReader(httpConn.getErrorStream());
}
BufferedReader in = new BufferedReader(isr);
String responseString = "";
String outputString = "";
// Write response to a String.
while ((responseString = in.readLine()) != null) {
outputString = outputString + responseString;
}
//Printing the response to the console
System.out.println("Output from the REST" + outputString);
// Extracting accessToken from string, here response (outputString)is a Json format string
if (outputString.indexOf("access_token\":\"") > -1) {
int i1 = outputString.indexOf("access_token\":\"");
String str1 = outputString.substring(i1 + 15);
int i2 = str1.indexOf("\"}");
String str2 = str1.substring(0, i2);
accessToken = str2;
}
//Printing the access token
System.out.println("Access token is " + accessToken);
} catch (Exception e) {
accessToken = "Error: " + e.getMessage();
}
return accessToken;
}
Now that I have the access token in the String variable "accessToken", I used the following code to read the filenames inside the folder "abc" in Documents folder using readFiles() method
public void readFiles(String accessToken) {
try {
//Frame SharePoint siteURL
String siteURL = "https://companyname.sharepoint.com";
//Frame SharePoint URL to retrieve the name of all of the files in a folder
String wsUrl = siteURL + "/_api/web/GetFolderByServerRelativeUrl('Shared%20Documents/abc')/Files";
//Create HttpURLConnection
URL url = new URL(wsUrl);
URLConnection connection = url.openConnection();
HttpURLConnection httpConn = (HttpURLConnection) connection;
//Set Header
httpConn.setRequestMethod("GET");
httpConn.setRequestProperty("Authorization", "Bearer " + accessToken);
httpConn.setRequestProperty("accept", "application/json;odata=verbose"); //To get response in JSON
//httpConn.setRequestProperty("AllowAppOnlyPolicy", "true");
//httpConn.setRequestProperty("Scope", "http://sharepoint/content/sitecollection/web\" Right=\"FullControl");
//Read the response
String httpResponseStr = "";
InputStreamReader isr = null;
System.out.println(httpConn.getResponseCode());
if (httpConn.getResponseCode() == 200) {
isr = new InputStreamReader(httpConn.getInputStream());
} else {
isr = new InputStreamReader(httpConn.getErrorStream());
}
BufferedReader in = new BufferedReader(isr);
String strLine = "";
while ((strLine = in.readLine()) != null) {
httpResponseStr = httpResponseStr + strLine;
}
//Print response
System.out.println(httpResponseStr);
} catch (Exception e) {
System.out.println("Error while reading file: " + e.getMessage());
}
}
}
When I execute the above code I am getting {"error_description":"Exception of type 'Microsoft.IdentityModel.Tokens.AudienceUriValidationFailedException' was thrown."}. Could someone please help me to figure out what I am doing wrong? I have been sitting on this for days and not able to resolve it.
Please help!

HttpURLConnection sending JSON POST request to Apache/PHP

I'm struggling with HttpURLConnection and OutputStreamWriter.
The code actually reaches the server, as I do get a valid error
response back. A POST request is made, but no data is received
server-side.
Any hints to proper usage of this thingy is highly appreciated.
The code is in an AsyncTask
protected JSONObject doInBackground(Void... params) {
try {
url = new URL(destination);
client = (HttpURLConnection) url.openConnection();
client.setDoOutput(true);
client.setDoInput(true);
client.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
client.setRequestMethod("POST");
//client.setFixedLengthStreamingMode(request.toString().getBytes("UTF-8").length);
client.connect();
Log.d("doInBackground(Request)", request.toString());
OutputStreamWriter writer = new OutputStreamWriter(client.getOutputStream());
String output = request.toString();
writer.write(output);
writer.flush();
writer.close();
InputStream input = client.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
StringBuilder result = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
result.append(line);
}
Log.d("doInBackground(Resp)", result.toString());
response = new JSONObject(result.toString());
} catch (JSONException e){
this.e = e;
} catch (IOException e) {
this.e = e;
} finally {
client.disconnect();
}
return response;
}
The JSON I'm trying to send:
JSONObject request = {
"action":"login",
"user":"mogens",
"auth":"b96f704fbe702f5b11a31524bfe5f136efea8bf7",
"location":{
"accuracy":25,
"provider":"network",
"longitude":120.254944,
"latitude":14.847808
}
};
And the response I get from the server:
JSONObject response = {
"success":false,
"response":"Unknown or Missing action.",
"request":null
};
And the response I should have had:
JSONObject response = {
"success":true,
"response":"Welcome Mogens Burapa",
"request":"login"
};
The server-side PHP script:
<?php
$json = file_get_contents('php://input');
$request = json_decode($json, true);
error_log("JSON: $json");
error_log('DEBUG request.php: ' . implode(', ',$request));
error_log("============ JSON Array ===============");
foreach ($request as $key => $val) {
error_log("$key => $val");
}
switch($request['action'])
{
case "register":
break;
case "login":
$response = array(
'success' => true,
'message' => 'Welcome ' . $request['user'],
'request' => $request['action']
);
break;
case "location":
break;
case "nearby":
break;
default:
$response = array(
'success' => false,
'response' => 'Unknown or Missing action.',
'request' => $request['action']
);
break;
}
echo json_encode($response);
exit;
?>
And the logcat output in Android Studio:
D/doInBackground(Request)﹕ {"action":"login","location":{"accuracy":25,"provider":"network","longitude":120.254944,"latitude":14.847808},"user":"mogens","auth":"b96f704fbe702f5b11a31524bfe5f136efea8bf7"}
D/doInBackground(Resp)﹕ {"success":false,"response":"Unknown or Missing action.","request":null}
If I append ?action=login to the URL I can get a success response from the server. But only the action parameter registers server-side.
{"success":true,"message":"Welcome ","request":"login"}
The conclusion must be that no data is transferred by URLConnection.write(output.getBytes("UTF-8"));
Well, data get transferred after all.
Solution offered by #greenaps does the trick:
$json = file_get_contents('php://input');
$request = json_decode($json, true);
PHP script above updated to show the solution.

echo (file_get_contents('php://input'));
Will show you the json text. Work with it like:
$jsonString = file_get_contents('php://input');
$jsonObj = json_decode($jsonString, true);

try to use DataOutputStream instead of OutputStreamWriter.
DataOutputStream out = new DataOutputStream(_conn.getOutputStream());
out.writeBytes(your json serialized string);
out.close();

I've made server tell me what it got from me.
Request Headers and POST Body
<?php
$requestHeaders = apache_request_headers();
print_r($requestHeaders);
print_r("\n -= POST Body =- \n");
echo file_get_contents( 'php://input' );
?>
Works like a charm)

The code actually reaches the server, as I do get a valid error
response back. A POST request is made, but no data is received
server-side.
got this same situation, and come to #greenapps answer.
you should know what server recieved from 'post request'
what i do first on the server side :
echo (file_get_contents('php://input'));
then print/Toast/show message response on the client side. make sure its correct form, like :
{"username": "yourusername", "password" : "yourpassword"}
if the response like this (because you post the request with yourHashMap.toString()) :
{username=yourusername,password=yourpassword}
instead using .toString(), use this method instead to turn HashMap into String :
private String getPostDataString(HashMap<String, String> postDataParams) {
StringBuilder result = new StringBuilder();
boolean first = true;
for (Map.Entry<String,String> entry : postDataParams.entrySet()){
if(first){
first = false;
}else{
result.append(",");
}
result.append("\"");
result.append(entry.getKey());
result.append("\":\"");
result.append(entry.getValue());
result.append("\"");
}
return "{" + result.toString() + "}";
}

setting up mutual authentication client using HttpURLConnection

I have a simple http service with JSON payload I want to test with a Java test harness.
Initially I set up a client using Basic Auth which works fine; server certificate is in the trustStore and I'm supplying username/password in the code. I send the request, I get the correct response.
public static void main(String[] args) {
// TODO Auto-generated method stub
String xxxURL = new String("https://www.xxx.yyy/zzz/AdminServlet?data=");
String username = new String("username");
String password = new String("password");
String authString = new String (username+":"+password);
String apiList = new String("{\"apiVersion\":\"1.4\",\"method\":\"api.list\",\"params\":{}}"); // Create JSON string. A bit ugly
try
{
System.setProperty("javax.net.ssl.trustStore","C:\\workspace\\http_client_test\\security\\cacerts");
String jsonStr = apiList;
URL url = new URL(xxxURL + URLEncoder.encode(jsonStr, "UTF-8") );
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection(); // create connection object
String encoded = Base64.encodeBase64String(authString.getBytes());
httpConn.setRequestProperty("Authorization", "Basic "+encoded);
httpConn.setRequestMethod("GET");
httpConn.connect(); // open connection
BufferedReader in = new BufferedReader(new InputStreamReader(httpConn.getInputStream()));
String temp = null;
StringBuilder sb = new StringBuilder();
while((temp = in.readLine()) != null)
{
sb.append(temp).append(" ");
}
String result = sb.toString();
System.out.println("result = " + result);
}
catch(Exception e)
{
e.printStackTrace();
}
}
I want to do the same test using mutual authentication. I have set up the keystore and truststore on both server and client, and imported the necessary certificates on each.
My problem is I cannot find out how to tell the HttpURLConnection that I want mutual certificate authentication.
I tried :
public class Test3 {
public static void main(String[] args) {
// TODO Auto-generated method stub
String xxxURL = new String("https://www.xxx.yyy/zzz/AdminServlet?data=");
String apiList = new String("{\"apiVersion\":\"1.4\",\"method\":\"api.list\",\"params\":{}}"); // Create JSON string. A bit ugly
try
{
System.setProperty("javax.net.ssl.trustStore","C:\\workspace\\http_client_test\\security\\cacerts");
System.setProperty("javax.net.ssl.trustStorePassword","changeit");
System.setProperty("javax.net.ssl.keyStore","C:\\workspace\\http_client_test\\security\\keystore.jks");
System.setProperty("javax.net.ssl.keyStorePassword","password");
String jsonStr = apiList;
URL url = new URL(netThingsURL + URLEncoder.encode(jsonStr, "UTF-8") );
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection(); // create connection object
String encoded = Base64.encodeBase64String(authString.getBytes());
httpConn.setRequestProperty("Authorization", "?????????"); // ????????
httpConn.setRequestMethod("GET");
httpConn.connect(); // open connection
BufferedReader in = new BufferedReader(new InputStreamReader(httpConn.getInputStream()));
String temp = null;
StringBuilder sb = new StringBuilder();
while((temp = in.readLine()) != null)
{
sb.append(temp).append(" ");
}
String result = sb.toString();
System.out.println("result = " + result);
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
What should I have here : httpConn.setRequestProperty("Authorization", "?????????"); I realise I may need more than just this 1 line. I tried various resources to find appropriate values but drew a blank. I tried various intuitive arguments but get a '401' error or NoSuchAlgorithException.
Any help, code, links to resources is greatly appreciated.

My problem is I cannot find out how to tell the HttpURLConnection that I want mutual certificate authentication.
You can't. You have to configure the server to ask for the client certificate. All you can do at the client is specify where the client certificate is. You can't force it to be sent. Only the server can do that.

Processing a website by using POST data and cookies

I try to access an ASPX-website where subsequent pages are returned based on
post data. Unfortunately all my attempts to get the following pages fail.
Hopefully, someone here has an idea where to find the error!
In step one I read the session ID from the cookie as well as the value of the
viewstate variable in the returned html page. Step two intends to send it
back to the server to get the desired page.
Sniffing the data in the webbrowser gives
Host=www.geocaching.com
User-Agent=Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100618
Iceweasel/3.5.9 (like Firefox/3.5.9)
Accept=text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language=en-us,en;q=0.5
Accept-Encoding=gzip,deflate
Accept-Charset=ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive=300
Connection=keep-alive
Referer=http://www.geocaching.com/seek/nearest.aspx?state_id=149
Cookie=Send2GPS=garmin; BMItemsPerPage=200; maprefreshlock=true; ASP.
NET_SessionId=c4jgygfvu1e4ft55dqjapj45
Content-Type=application/x-www-form-urlencoded
Content-Length=4099
POSTDATA=__EVENTTARGET=ctl00%24ContentBody%24pgrBottom%
24lbGoToPage_3&__EVENTARGUMENT=&__VIEWSTATE=%2FwEPD[...]2Xg%3D%
3D&language=on&logcount=on&gpx=on
Currently, my script looks like this
import java.net.*;
import java.io.*;
import java.util.*;
import java.security.*;
import java.net.*;
public class test1 {
public static void main(String args[]) {
// String loginWebsite="http://geocaching.com/login/default.aspx";
final String loginWebsite = "http://www.geocaching.com/seek/nearest.aspx?state_id=159";
final String POST_CONTENT_TYPE = "application/x-www-form-urlencoded";
// step 1: get session ID from cookie
String sessionId = "";
String viewstate = "";
try {
URL url = new URL(loginWebsite);
String key = "";
URLConnection urlConnection = url.openConnection();
if (urlConnection != null) {
for (int i = 1; (key = urlConnection.getHeaderFieldKey(i)) != null; i++) {
// get ASP.NET_SessionId from cookie
// System.out.println(urlConnection.getHeaderField(key));
if (key.equalsIgnoreCase("set-cookie")) {
sessionId = urlConnection.getHeaderField(key);
sessionId = sessionId.substring(0, sessionId.indexOf(";"));
}
}
BufferedReader in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
// get the viewstate parameter
String aLine;
while ((aLine = in.readLine()) != null) {
// System.out.println(aLine);
if (aLine.lastIndexOf("id=\"__VIEWSTATE\"") > 0) {
viewstate = aLine.substring(aLine.lastIndexOf("value=\"") + 7, aLine.lastIndexOf("\" "));
}
}
}
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(sessionId);
System.out.println("\n");
System.out.println(viewstate);
System.out.println("\n");
// String goToPage="3";
// step2: post data to site
StringBuilder htmlResult = new StringBuilder();
try {
String encoded = "__EVENTTARGET=ctl00$ContentBody$pgrBottom$lbGoToPage_3" + "&" + "__EVENTARGUMENT=" + "&"
+ "__VIEWSTATE=" + viewstate;
URL url = new URL(loginWebsite);
URLConnection urlConnection = url.openConnection();
urlConnection = url.openConnection();
// Specifying that we intend to use this connection for input
urlConnection.setDoInput(true);
// Specifying that we intend to use this connection for output
urlConnection.setDoOutput(true);
// Specifying the content type of our post
urlConnection.setRequestProperty("Content-Type", POST_CONTENT_TYPE);
// urlConnection.setRequestMethod("POST");
urlConnection.setRequestProperty("Cookie", sessionId);
urlConnection.setRequestProperty("Content-Type", "text/html");
DataOutputStream out = new DataOutputStream(urlConnection.getOutputStream());
out.writeBytes(encoded);
out.flush();
out.close();
BufferedReader in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String aLine;
while ((aLine = in.readLine()) != null) {
System.out.println(aLine);
}
} catch (MalformedURLException e) {
// Print out the exception that occurred
System.err.println("Invalid URL " + e.getMessage());
} catch (IOException e) {
// Print out the exception that occurred
System.err.println("Unable to execute " + e.getMessage());
}
}
}
Any idea what's wrong? Any help is very appreciated!
Update
Thank you for the fast reply!
I switched to use the HttpURLConnection instead of the URLConnection which implements the setRequestMethod(). I also corrected the minor mistakes you mentioned, e.g. removed the obsolete first setRequestProperty call.
Unfortunately this doesn’t change anything... I think I set all relevant parameters but still get the first page of the list, only. It seems that the "__EVENTTARGET=ctl00$ContentBody$pgrBottom$lbGoToPage_3" is ignored. I don't have any clues why.
Internally, the form on the website looks like this:
<form name="aspnetForm" method="post" action="nearest.aspx?state_id=159" id="aspnetForm">
It is called by the following javascript:
<script type="text/javascript">
//<![CDATA[
var theForm = document.forms['aspnetForm'];
if (!theForm) {
theForm = document.aspnetForm;
}
function __doPostBack(eventTarget, eventArgument) {
if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
theForm.__EVENTTARGET.value = eventTarget;
theForm.__EVENTARGUMENT.value = eventArgument;
theForm.submit();
}
}
//]]>
</script>
Hopefully, this helps to find a solution?
Greetings
maik.

Do you actually want to GET or POST? If you want to POST, then you may need the setRequestMethd() line.
You're setting Content-Type twice -- I think you may need to combine these into one line.
Then, don't close the output stream before you try and read from the input stream.
Other than that, is there any more logging you can put in/clues you can give as to what way it's going wrong in?

Hey use following code
String userAgent = "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0";
org.jsoup.nodes.Document jsoupDoc = Jsoup.connect(url).timeout(15000).userAgent(userAgent).referrer("http://calendar.legis.ga.gov/Calendar/?chamber=House").ignoreContentType(true)
.data("__EVENTTARGET", eventtarget).data("__EVENTARGUMENT", eventarg).data("__VIEWSTATE", viewState).data("__VIEWSTATEGENERATOR", viewStateGenarator)
.data("__EVENTVALIDATION", viewStateValidation).parser(Parser.xmlParser()).post();