Develop Reference

HttpURLConnection PUT to Google Cloud Storage giving error 403

I tried to upload a file to Google Cloud Storage using XML API. I have the right GoogleAccessId, expiry date and signature generated for each upload. The strange thing is that I can PUT file using Postman (application for Chrome), so I'm sure that the URL is ok. I just cannot PUT it using my Android Java program (it returns to me 403 error). The source code performing upload is here (it base on this one: https://cloud.google.com/storage/docs/access-control#Signing-Strings):
URL url;
HttpURLConnection connection;
try {
url = new URL("http://google-testbucket.storage.googleapis.com/testdata.txt?GoogleAccessId=1234567890123#developer.gserviceaccount.com&Expires=1331155464&Signature=BClz9e4UA2MRRDX62TPd8sNpUCxVsqUDG3YGPWvPcwN%2BmWBPqwgUYcOSszCPlgWREeF7oPGowkeKk7J4WApzkzxERdOQmAdrvshKSzUHg8Jqp1lw9tbiJfE2ExdOOIoJVmGLoDeAGnfzCd4fTsWcLbal9sFpqXsQI8IQi1493mw%3D");
connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("PUT");
OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream());
out.write("Test");
out.close();
Log.i("TAG", "PUT Response code: " + connection.getResponseCode());
} catch (MalformedURLException e) {
e.printStackTrace();
Log.e("TAG", "MalformedURLException");
} catch (ProtocolException e) {
e.printStackTrace();
Log.e("TAG", "ProtocolException");
} catch (IOException e) {
e.printStackTrace();
Log.e("TAG", "IOException");
}
Documentation for PUT Object: https://cloud.google.com/storage/docs/xml-api/put-object-upload
Can anybody look into this problem and give me hints what might went wrong with this one?

I just figured out that HttpURLConnection adds Content-Type header with value application/x-www-form-urlencoded by itself. I've done it using HTTP sniffer on my android emulator.
This auto-added header caused signature mismatch. After I changed the code on the server-side to allow requests with Content-Type: application/x-www-form-urlencoded it generates the right signature and it works fine.
Thank you #morpheus05 for your commitment.

Please set your Content-Type like this.
connection.setRequestProperty("Content-Type"," ");
Because HttpsUrlConnection automatically generate Content-Type as
"Content-Type: application/x-www-form-urlencoded"
this will cause a signature mismatch.

Build and use Cookie for token based authentication to gain access to api through openAM

Please excuse me if any of this sounds very stupid or inexperienced, however I have looked everywhere else and haven't been able to find a simple explanation as to how to properly implement this.
So far I have made a restful call to a server running on openAm; the call sends my user name and password credentials and returns to me a secure token. I then need to make another restful call to request certain json files in their api.
I understand that in my second restful call I need to somehow embed the token with it so the server knows that I am allowed to access the requested data. My question is what is the proper way to go about this. I have found/heard of multiple possibilities such as passing it in the header, parameters, or as a cookie, but each time my request is redirected to the log in url instead of returning my request.
From my understanding it appears the cookie method works best (if I'm wrong then please post a different method). So for openAm authentication, how do I properly build a cookie with my token. Once the cookie is built how do I embed that into the connection. Do I need to make a whole new connection or can I redirect my original connection with the cookie? Any help or advice is greatly appreciated.
Some of my code, using HttpURLConnection:
//takes url and builds our connection
String url = "http://some.url.net/openam/json/authenticate";
HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("X-OpenAM-Username", name);
connection.setRequestProperty("X-OpenAM-Password", pass);
connection.setRequestProperty("Content-Type", "application/json");
//takes in the connections response
BufferedReader in = new BufferedReader(new InputStreamReader(response, "UTF-8"));
String output = in.readLine();
//this is to cut the token out of the response
int i = 14;
while(true){
if (output.charAt(i)=='"'){
break;
}
i++;
}
String token = output.substring(14,i);
//build our new connection and second call
url = "https://other.url.net/api/v1/resource/attributes";
HttpURLConnection request_conn = (HttpURLConnection) new URL(url).openConnection();
/*
request_conn.setRequestProperty("iPlanetDirectoryPro", token);
request_conn.setRequestMethod("POST");
request_conn.connect();
*/ //Tried to put the token through the header, doesnt work
/*
Cookie cookie;
cookie = new Cookie("iPlanetDirectoryPro", token);
cookie.setDomain(".di2e.net");
cookie.setPath("/");
cookie.setSecure(true);
request_conn.addCookie(cookie);//addCookie() doesnt work for a urlConection?
*/ //Tried building the cookie and adding it to the new conection

Trying to get html texts from google url, but error 401

I am trying to retrieve some html texts from a list of google returned pages. most of them work fine, but for urls such as https://www.google.com/patents/US6034687 always gives 401 error see below
Server returned HTTP response code: 401 for URL: https://www.google.com/patents/US6034687
I am using java and I did look up on this error code, it seems authentication related, but this kind of URL can be accessed from any browsers without asking for login. So I am confused, how come only this kind of URL does not work for me.
here is my code for retrieving html
URL u=new URL(url);
StringBuilder html =new StringBuilder();
HttpURLConnection conn = (HttpURLConnection) u.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "text/html");
BufferedReader br;
try {
br = new BufferedReader(new InputStreamReader((conn.getInputStream())));
String out="";
while ((out= br.readLine()) != null) {
// System.out.println(out);
html.append(out+"\n");
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Any idea?
thanks

Try sending a User-Agent header in the request. That 401 status is misleading. Some servers do not allow requests from non-browser clients.
conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 5.2; rv:21.0) Gecko/20100101 Firefox/21.0");
BTW, when you do openConnection() for an https scheme, the return value is HttpsURLConnection, which extends HttpURLConnection.

The request requires user authentication. The response MUST include a WWW-Authenticate header field containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field. If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity might include relevant diagnostic information. HTTP access authentication is explained in "HTTP Authentication: Basic and Digest Access Authentication

SignUp using Google Android Auth 2.0 HTTP post request after getting authorization code

https://developers.google.com/accounts/docs/OAuth2InstalledApp
I am giving user to signup with Google account in webview with the following link
webview.loadUrl("https://accounts.google.com/o/oauth2/auth?scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile&state=%2F&response_type=code&redirect_uri=urn:ietf:wg:oauth:2.0:oob&client_id=706645665586.apps.googleusercontent.com");
which contain my client id and redirect URI as per given by Google API console i.e Choosing a Redirect URI
https://developers.google.com/accounts/docs/OAuth2InstalledApp
and finally a get the authorization code that is returned in the title bar of the browser using view.getTitle()
Afterwards Another Request is required to send
The actual request might look like:
POST /o/oauth2/token HTTP/1.1
Host: accounts.google.com
Content-Type: application/x-www-form-urlencoded
code=4/y_jtre05wvb6QSPo0Tkx5AbLfWB
client_id=706645665586.apps.googleusercontent.com
client_secret={client_secret}&
redirect_uri=urn:ietf:wg:oauth:2.0:oob
grant_type=authorization_code
So Now while making HTTP POST request ..
DefaultHttpClient httpcl = new DefaultHttpClient();
HttpPost httpp = new HttpPost("https://accounts.google.com/o/oauth2/auth");
List<NameValuePair> a = new ArrayList<NameValuePair>();
a.add(new BasicNameValuePair("code", "4/y_jtre05wvb6QSPo0Tkx5AbLfWB"));
a.add(new BasicNameValuePair("client_id", "706645665586.apps.googleusercontent.com"));
try {
StringEntity mEntity = new StringEntity("");
mEntity.setContentType(" application/x-www-form-urlencoded");
httpp.setEntity(mEntity);
httpp.setEntity(new UrlEncodedFormEntity(a));
HttpResponse response1 = httpcl.execute(httpp);
String response = EntityUtils.toString(response1.getEntity());
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ParseException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
So I am getting Bad Token response ... I am try this since yesterday and suggestion and help would be appreciated .. my main aim is to get user info using gmail account in android

I think you are mixing the different flows here a bit:
The Client-side flow does not require a client secret, but is primarily meant for Javascript applications.
The Installed Applications flow does require a client secret, though:
The client_id and client_secret obtained during registration are embedded in the source code of your application. In this context, the client_secret is obviously not treated as a secret.
You probably generated a client ID in the API console for Installed application -> Android, so you only got a client ID and had to specify your application's certificate fingerprint. This type of client ID is meant for use with the recently released and recommended (because it's more secure) Google Play Services.
If you want to use the Installed Applications flow manually, you have to generate a client ID for Installed application -> Other, where you also get a client secret. When exchanging the authorization code for an access token, you are then required to specify all five parameters:
code The authorization code returned from the initial request
client_id The client_id obtained during application registration
client_secret The client secret obtained during application registration
redirect_uri The URI registered with the application
grant_type As defined in the OAuth 2.0 specification, this field must contain a value of authorization_code

I finally found the working sample
You can use Google+ developers API. Look at this project on github and this article.here

Cookies turned off with Java URLConnection

I am trying to make a request to a webpage that requires cookies. I'm using HTTPUrlConnection, but the response always comes back saying
<div class="body"><p>Your browser's cookie functionality is turned off. Please turn it on.
How can I make the request such that the queried server thinks I have cookies turned on. My code goes something like this.
private String readPage(String page) throws MalformedURLException {
try {
URL url = new URL(page);
HttpURLConnection uc = (HttpURLConnection) url.openConnection();
uc.connect();
InputStream in = uc.getInputStream();
int v;
while( (v = in.read()) != -1){
sb.append((char)v);
}
in.close();
uc.disconnect();
} catch (IOException e){
e.printStackTrace();
}
return sb.toString();
}

You need to add a CookieHandler to the system for it handle cookie. Before Java 6, there is no CookieHandler implementation in the JRE, you have to write your own. If you are on Java 6, you can do this,
CookieHandler.setDefault(new CookieManager());
URLConnection's cookie handling is really weak. It barely works. It doesn't handle all the cookie rules correctly. You should use Apache HttpClient if you are dealing with sensitive cookies like authentication.

I think server can't determine at the first request that a client does not support cookies. So, probably server sends redirects. Try to disable redirects:
uc.setInstanceFollowRedirects(false);
Then you will be able to get cookies from response and use them (if you need) on the next request.

uc.getHeaderFields()
// get cookie (set-cookie) here
URLConnection conn = url.openConnection();
conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 6.0; pl; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2");
conn.addRequestProperty("Referer", "http://xxxx");
conn.addRequestProperty("Cookie", "...");

If you're trying to scrape large volumes of data after a login, you may even be better off with a scripted web scraper like WebHarvest (http://web-harvest.sourceforge.net/) I've used it to great success in some of my own projects.