Develop Reference

Java : Google AppEngine : OAuth : Google Drive API to access user's Google Drive contents created by Web Application

Scenario: I am working on a Web Application on Google AppEngine where users have to access files from their own Google Drive. I have looked for online help and this is what I have figured out.
I have used this link for help that worked fine while testing with local machine
https://github.com/gsuitedevs/java-samples/blob/master/drive/quickstart/src/main/java/DriveQuickstart.java
Step 1 (Seemed simple): Enable Google Drive API and Setup necessary credential, i.e. OAuth 2.0 Client ID for Web Application. (This is done while enabling IAP on Google AppEngine, so I did not do it again. Whenever anyone opens the web application, he/she is asked to authenticate via iap.googleapis.com and details saved. This works fine). Also, I have added Google Drive Scopes to the OAuth consent screen (../auth/drive.appdata & ../auth/drive.file) that don't need verification by Google.
Step 2: Downloaded the credentials.json from OAuth Client ID and stored inside "resources" folder created in the root of application package (inside main folder, next to java and webapp folders)
Step 3: I have created a testing class (GoogleDriveServiceTest.class) that includes following code:
String USER_ID1 = UserServiceFactory.getUserService().getCurrentUser().getUserId();
List<String> SCOPES = Collections.singletonList(DriveScopes.DRIVE_METADATA_READONLY);
NetHttpTransport httpTransport = new NetHttpTransport();
JsonFactory jsonFactory = new JacksonFactory();
InputStream inputStream =
GoogleDriveServiceTest.class.getResourceAsStream("/credentials.json");
if (inputStream == null)
throw new FileNotFoundException("Required credentials file not found");
GoogleClientSecrets googleClientSecrets =
GoogleClientSecrets.load(jsonFactory, new InputStreamReader(inputStream));
AppEngineDataStoreFactory appEngineDataStoreFactory =
AppEngineDataStoreFactory.getDefaultInstance();
//IS SOMETHING MISSING HERE???
GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow
.Builder(httpTransport, jsonFactory, googleClientSecrets, SCOPES)
.setDataStoreFactory(appEngineDataStoreFactory)
.setAccessType("offline")
.build();
Now I am trying to create the credential to be used for accessing Google Drive with this line:
Credential credential = flow.loadCredential(USER_ID1);
that is returning null.
In my opinion I am missing to assign the credentials to AppEngineDataStoreFactory based on what I have seen in the example from the github link above. However, I am not sure if this is the issue, and if it is, how do I resolve it.
Is there a straight forward way to assign credentials using logged in userID obtained from
UserServiceFactory.getUserService().getCurrentUser().getUserId() ? Or should I be obtaining accetoken and create the credential? if so, how?
(I don't want to use javascript as the same does not seem suitable for web application)
Any help would be great!!!!
PS: I also wanted to add a point that user needs to access only files added by the same web application either via web or from android
Update #1 responding to #Aerials:
Here is the code I was trying with to get the TokenResponse:
VerificationCodeReceiver receiver = new GooglePromptReceiver();
(I know above one is not the right option, but I am not able to find any other)
AuthorizationCodeRequestUrl authorizationUrl =
flow.newAuthorizationUrl().setRedirectUri(receiver.getRedirectUri());
String code = receiver.waitForCode();
(Above line returns: java.util.NoSuchElementException: No line found)
TokenResponse tokenResponse =
flow.newTokenRequest(code).setRedirectUri(redirectUri).execute();
Update #2 Code that worked in getting the TokenResponse and rest of the tasks of creating a Credential and connect to Google Drive successfully:
GenericUrl genericUrl = new GenericUrl(request.getRequestURL().toString());
genericUrl.setRawPath("/googleDriveTest");
String redirectUri = genericUrl.build();
(redirectUri should match with authorised redirect URI inside OAuth ClientID under GCP API Credentials. If you added it now, you need to redownload the credentials.json file)
String redirectUrl = authorizationCodeFlow
.newAuthorizationUrl()
.setRedirectUri(redirectUri)
.build();
String authorizationCode = request.getParameter("code");
if (authorizationCode == null || authorizationCode.isEmpty())
response.sendRedirect(redirectUrl);
TokenResponse tokenResponse = authorizationCodeFlow
.newTokenRequest(authorizationCode)
.setRedirectUri(redirectUri).execute();
authorizationCodeFlow.createAndStoreCredential(tokenResponse, USER_ID1);
Credential credential = authorizationCodeFlow.loadCredential(USER_ID1);
Drive service = new Drive.Builder(httpTransport, jsonFactory, credential)
.setApplicationName("myapplicationname")
.build();

You need to first create and store the credentials in the flow's credential store:
createAndStoreCredential(TokenResponse response, String userId)
to be able to load them with loadCredential(String userId)
The loadCredential() method will return credential found in the credential store of the given user ID or null for none found.

TokenResponseException: 401 Unauthorized

I want to implement Google Sheet API request using service account. I created this code:
HttpTransport httpTransport = new NetHttpTransport();
JacksonFactory JSON_FACTORY = new JacksonFactory();
ClassLoader classLoader = this.getClass().getClassLoader();
java.io.File path = new java.io.File(classLoader.getResource("i-6dc0c917ee63.p12").getFile());
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId("test221#sonora-project.iam.gserviceaccount.com")
.setServiceAccountPrivateKeyFromP12File(path)
.setServiceAccountScopes(Arrays.asList(SheetsScopes.SPREADSHEETS))
.setServiceAccountUser("sonoraw#gmail.com")
.build();
Sheets service = new Sheets.Builder(httpTransport, JSON_FACTORY, null)
.setApplicationName("project")
.setHttpRequestInitializer(credential).build();
Sheets.Spreadsheets spreadsheets = service.spreadsheets();
Spreadsheet includeGridData = spreadsheets.get(spreadsheetId).execute();
But I get this error:
com.google.api.client.auth.oauth2.TokenResponseException: 401 Unauthorized
at this method .execute();
Do you know how I can fix this issue?

I was having the same issue and it was driving me insane. I had 2 different copies of the same app, using the same credentials.json, but I'd get a successful response in one, but a 401 unauthorized in the other. I finally solved it by deleting the StoredCredential. So next time I ran my app I was taken to Google's authentication page, and it worked.
TLDR:
delete StoredCredential
One way to find it is running find . -name 'StoredCredential' from the root of your application.

Based from Standard Error Responses, an error 401 is due to invalidCredentials and this indicates that the auth token is invalid or has expired.
Recommended Action:
Do not retry without fixing the problem. You need to get a new auth token.
With this, you may want to check Token Expiration which mentioned that you must write your code to anticipate the possibility that a granted token might no longer work. It also gives these possible reasons why a token might stop working:
The user has revoked access.
The token has not been used for six months.
The user changed passwords and the token contains Gmail scopes.
The user account has exceeded a certain number of token requests.
Hope that helps!

OAuth 2.0 access with Google Plus API in java

I've been trying to write a web application using Google Plus API and i need to set up OAuth access with java , I searched a lot and found google java starter and other examples and they were very confusing, I can't figure out what the code that I should write to get the token
I hope if there is someone who can tell me how to get the OAuth access with java in straight forward steps, I saw other questions on stackoverflow.com but they weren't very helpful for me
so any help would be very appreciated :)

The latest Google+ Java Quickstart is pretty straightforward, perhaps you found an older project when searching? Also, the documentation for getting started on Google+ with Java should help to get you going.
The following snippet shows you the relevant code for exchanging the authorization code for an access token when using the hybrid client/server flow:
GoogleTokenResponse tokenResponse =
new GoogleAuthorizationCodeTokenRequest(TRANSPORT, JSON_FACTORY,
CLIENT_ID, CLIENT_SECRET, code, "postmessage").execute();
// Create a credential representation of the token data.
GoogleCredential credential = new GoogleCredential.Builder()
.setJsonFactory(JSON_FACTORY)
.setTransport(TRANSPORT)
.setClientSecrets(CLIENT_ID, CLIENT_SECRET).build()
.setFromTokenResponse(tokenResponse);
I'm removing the lines performing the requisite checks discussed in this thread for simplicity.
// Store the token in the session for later use.
request.session().attribute("token", tokenResponse.toString());
It's worth noting here that you want to persist these credentials unless the user disconnects your app. The sample is using a session because in production environments, the session can be DB-backed and will be restored after the server restarts.
After you have the access / refresh token and expiration time, build the credentials for the OAuth v2 token and then the library will internally refresh the access token appropriately. The following code shows how this is done on the quickstart by retrieving the token data from the user's session and also includes an API call performed by the client, proving the server's Java client is working:
// Build credential from stored token data.
GoogleCredential credential = new GoogleCredential.Builder()
.setJsonFactory(JSON_FACTORY)
.setTransport(TRANSPORT)
.setClientSecrets(CLIENT_ID, CLIENT_SECRET).build()
.setFromTokenResponse(JSON_FACTORY.fromString(
tokenData, GoogleTokenResponse.class));
// Create a new authorized API client.
Plus service = new Plus.Builder(TRANSPORT, JSON_FACTORY, credential)
.setApplicationName(APPLICATION_NAME)
.build();
// Get a list of people that this user has shared with this app.
PeopleFeed people = service.people().list("me", "visible").execute();
If you wanted to do this differently, you could explicitly construct the tokenData object from the access token, refresh token, and so on, before constructing the Plus service object.

Google Analytics 3.0 auth flow

EDIT: Originally this question asked how I could authenticate with the Google Analytics API using only my API key. As vlatko pointed out, this isn't possible. Now I'm just focused on getting OAuth2 to work. I will be trying vlatko's suggestions when I get a chance and will update the question. In the meantime, feel free to contribute answers with anything you think I'm missing.
ORIGINAL QUESTION:
I'm trying to make requests to the Google Analytics API. I'm walking through the Hello Analytics tutorial trying to replicate the steps. Whatever I try, I can't seem to authenticate succesfully.
The tutorial says the following:
Open the file you created named HelloAnalyticsApi.java and add the
following method:
private static Analytics initializeAnalytics() throws Exception {
// Authorization.
Credential credential = OAuth2Native.authorize(
HTTP_TRANSPORT, JSON_FACTORY, new LocalServerReceiver(),
Arrays.asList(AnalyticsScopes.ANALYTICS_READONLY));
// Set up and return Google Analytics API client.
return Analytics.builder(HTTP_TRANSPORT, JSON_FACTORY)
.setApplicationName("Google-Analytics-Hello-Analytics-API-Sample")
.setHttpRequestInitializer(credential)
.build();
}
When a user encounters this script, the application will attempt to
open the default browser and navigate the user to a URL hosted on
google.com. At this point, the user will be prompted to login and
grant the application access to their data. Once granted, the
application will attempt to read a code from the browser window, then
close the window.
The difference is that I'm trying to do this with a servlet application, and I want to use simple API access with an API key (rather than an OAuth 2.0 client ID). I know that OAuth 2.0 is recommended, but I only need to access data that I own and want to simplify the technical requirements. I based this decision on this page, which says:
An API key is a unique key that you generate using the Console. When
your application needs to call an API that's enabled in this project,
the application passes this key into all API requests as a key={API_key}
parameter. Use of this key does not require any user action or
consent, does not grant access to any account information, and is not
used for authorization.
If you are only calling APIs that do not require user data, such as
the Google Custom Search API, then API keys may be simpler to
implement. However, if your application already uses an OAuth 2.0
access token, then there is no need to generate an API key as well. In
fact, Google ignores passed API keys if an OAuth 2.0 access token is
already associated with the corresponding project.
I can't find many code examples of auth flow just using the API key - most everything I've found shows using the client ID with the downloaded .p12 file, for example the GoogleCredential javadoc. The one example application I could find was Google's Books Sample app. Anyway, here's what I tried (mimicking the first request in the tutorial, which gets a list of the accounts from the management API):
Analytics analytics =
new Analytics.Builder(httpTransport, jsonFactory, null)
.setApplicationName("Dev API Access")
.build();
Management.Accounts.List list =
analytics.management().accounts().list().setKey(apiKey);
Accounts accounts = list.execute();
Where "Dev API Access" is the "Name" field in my API console dashboard. The API key is a server key restricted to my IP address. This fails with the following response:
401 Unauthorized
{
"code": 401,
"errors": [
{
"domain": "global",
"location": "Authorization",
"locationType": "header",
"message": "Login Required",
"reason": "required"
}
],
"message": "Login Required"
}
I also tried this:
Analytics analytics =
new Analytics.Builder(httpTransport, jsonFactory, null)
.setApplicationName("Dev API Access")
.setGoogleClientRequestInitializer(new AnalyticsRequestInitializer(apiKey))
.build();
Management.Accounts.List list = analytics.management().accounts().list();
Accounts accounts = list.execute();
Which shows the same error. What am I doing wrong here? Is OAuth2 required for analytics calls? If so, why does just using the API key work in the Books Sample app?
Moving on, I went ahead and tried OAuth2 anyway - I created a client ID and downloaded the .p12 private key file. But I couldn't get that working either. Here's what I tried:
Credential credential =
new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountId(serviceAccountId)
.setServiceAccountScopes(AnalyticsScopes.ANALYTICS_READONLY)
.setServiceAccountPrivateKeyFromP12File(new File(p12FilePath))
.setServiceAccountUser(serviceAccountUser)
.build();
Analytics analytics =
new Analytics.Builder(httpTransport, jsonFactory, credential)
.setApplicationName("Dev API Access")
.build();
Management.Accounts.List list = analytics.management().accounts().list();
Accounts accounts = list.execute();
Where serviceAccountId is the email address of the Google account owning the project and serviceAccountUser is the email address listed on the generated client ID. This fails with the following:
400 Bad Request
{
"error": "invalid_grant"
}
What does "invalid grant" mean, and how do I successfully authenticate (ideally without OAuth2)?

To answer your first question: in general, OAuth2.0 is used for authorized access to user's private data, so getting user consent and obtaining an access token is required. In the case with Google Books API, however, if you're accessing public data, there is no need for end user consent so an API key is sufficient. If you try accessing non public data with the Books API, you'll still need an OAuth2 token.
The good news for your case is that even with OAuth2, you can bypass user involvement and streamline your flow with Service Accounts - assuming your application has access to the API. There is a way to set that up for the Analytics API, explained here (check the steps in the Service Accounts section). I think you are on the right track with your Credential builder, but I don't think you need to set the service account user in there, since you are not doing any user impersonation.

vlatko's answer got me on the right track. The issue turned out to be that I was confusing the owner email address with the service account email address. For example, I was doing the following:
Credential credential =
new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountId("owneremail#gmail.com") //wrong
.setServiceAccountScopes(AnalyticsScopes.ANALYTICS_READONLY)
.setServiceAccountPrivateKeyFromP12File(new File(p12FilePath))
.setServiceAccountUser("xxx#developer.gserviceaccount.com") //unnecessary
.build();
When I needed to do this:
Credential credential =
new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountId("xxx#developer.gserviceaccount.com")
.setServiceAccountScopes(AnalyticsScopes.ANALYTICS_READONLY)
.setServiceAccountPrivateKeyFromP12File(new File(p12FilePath))
.build();
Also, I had added owneremail#gmail.com as a user on my Analytics application - this similarly needed to be the service account email instead.

Authenticate programmatically to Google with OAuth2

How can I authenticate programmatically to Google?
Now that ClientLogin (https://developers.google.com/accounts/docs/AuthForInstalledApps)
is deprecated, how can we perform a programmatic authentication to Google with OAuth2?
With ClientLogin we could perform a post to
https://www.google.com/accounts/ClientLogin
with email and password parameters and obtain the authentication token.
With OAuth2 i can't find a solution!
#
My app is a java background process.
I saw, following this link: developers.google.com/accounts/docs/OAuth2InstalledApp#refresh, how to obtain a new access token using a refreshed token.
The problem is that I can't find a java example about how to instantiate an Analytics object (for example) to perform a query when I have a new valid access token
This is my code that returns a 401 Invalid credentials when invoke the "execute()":
public class Test {
static final String client_id = "MY_CLIENT_ID";
static final String client_secret = "MY_SECRET";
static final String appName = "MY_APP";
private static final HttpTransport HTTP_TRANSPORT = new NetHttpTransport();
private static final JsonFactory JSON_FACTORY = new JacksonFactory();
static String access_token = "xxxx";
static String refreshToken = "yyyyy";
public static void main (String args[]){
try {
GoogleCredential credential =
new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setClientSecrets(client_id, client_secret).build();
credential.setAccessToken(access_token);
credential.setRefreshToken(refreshToken);
//GoogleCredential
Analytics analytics = Analytics.builder(HTTP_TRANSPORT, JSON_FACTORY)
.setApplicationName(appName)
.setHttpRequestInitializer(credential)
.build();
Accounts accounts = analytics.management().accounts().list().execute();
} catch (Exception e) {
e.printStackTrace();
}
}
What is the problem?

Check the OAuth 2 flow for Installed Application:
https://developers.google.com/accounts/docs/OAuth2InstalledApp
It still requires the user to authenticate with a browser the first time, but then you can store the refresh token and use it for subsequent requests.
For alternative solutions, check the Device flow or Service Accounts, they are explained in the same documentation set.

I found the Google Java client to be overly complex and poorly documented. Here's plain and simple Servlet example with Google Oauth2. For a background process you'll need to request access_type=offline. As others have mentioned you need the user to do a one time authorization. After that you can request refresh tokens as google tokens expire in an hour.

Although I appreciate that the OP was originally targeting the OAuth2InstalledApp approach, I would like to point out a working solution using the OAuth2WebServer approach. They don't differ significantly and this worked for me. I have found the google OAuth library to be pretty good as it will handle most of the OAuth dance for you and it makes it easy to refresh the access token. The solution below depends on using a pre-obtained refresh token.
As the accepted answer states, to get OAuth authentication working (even for a Java background process) where the request relies upon access to user data
requires the user to authenticate with a browser the first time, but then you can store the refresh token and use it for subsequent requests.
From previous comments by the OP I see the following
So I followed OAuth2 for Web Server Applications (here offline access is documented) but I have still problems.
1) I perform the first request via browser and I obtain autenticaton code for offline access
2) I perform a java post of the authentication code and obtain acces token and refresh token
The approach I used is more like
1) I perform the first request via a browser and obtain the refresh token for offline access
2) In java I provide the refresh token to the library and the library will obtain the access token etc
specifically, using the google-api-java-client library the code is quite straightforward and note that I haven't set an access token as the OP did, as I am calling credential.refreshToken(); elsewhere. (I check if I have a valid access token already and if not call refresh prior to the API call)
private Credential generateCredentialWithUserApprovedToken() throws IOException,
GeneralSecurityException {
JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
InputStreamReader inputStreamReader =
new InputStreamReader(jsonFileResourceForClient.getInputStream());
GoogleClientSecrets clientSecrets = GoogleClientSecrets.load(jsonFactory, inputStreamReader);
return new GoogleCredential.Builder().setTransport(httpTransport).setJsonFactory(jsonFactory)
.setClientSecrets(clientSecrets).build().setRefreshToken(REFRESH_TOKEN);
}
Note this covers step 2 of my approach, and the REFRESH_TOKEN mentioned in step 1 can be obtained as explained below.
First there is a prior set up of a web app creating an OAuth 2.0 client ID on the Google console for Credentials where you end up with a downloaded json file which will be read into the GoogleClientSecrets object.
i.e.
Make sure you add the Google playground callback uri into Authorized redirect URIs
Then you have your client id and the client secret ready for the playground and you can also download the json which you can pull into your Java code.
The REFRESH_TOKEN is obtained by sending a request to the google oauth playground with the following configuration. Note that prior to Step 1 and selecting your scope you should go to settings to check that you are providing you own credentials and add your client id and secret just below that
Note that the Access type is Offline, which corresponds to this.
There is also a nice explanation on grabbing the refresh token here https://www.youtube.com/watch?v=hfWe1gPCnzc
That is enough to get going and is a one time set up!
Regarding refresh tokens you should be aware of their lifecycle as discussed in the docs here
In the oauthplayground you will see this
but on point 4 of the docs here it says this
Hmmm.
Also for reference see How do I authorise an app (web or installed) without user intervention? (canonical ?)

For applications that authenticate on behalf of themselves (i.e., to another application, traditionally by signing into a role account using a shared password), the OAuth2 alternative to ClientLogin offered by Google is Service Accounts:
https://developers.google.com/accounts/docs/OAuth2ServiceAccount