Develop Reference

When using service account impersonation, when calling export on Google Docs using v3 API, viewedByMeTime timestamp is updated

I am using a service account to access google doc files of users in my enterprise google account using impersonation.
See:
https://developers.google.com/drive/api/v3/about-auth#OAuth2Authorizing
So far so good.
Then, I need to download contents of Google Docs.
When calling Google Drive API to download the contents of a Google Doc, the documentation says to run the following:
https://developers.google.com/drive/api/v3/manage-downloads
Here is a java program that should reproduce the problem:
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.http.HttpRequestInitializer;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.util.SecurityUtils;
import com.google.api.services.drive.Drive;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.util.Arrays;
import java.util.List;
public class FetchGoogleDocContentsWithServiceAccount {
static int readTimeout = 60000;
static int connectTimeout = 60000;
static String serviceAccountId = "";
static String serviceAccountEmail = "";
static String serviceAccountPrivateKeyFile = "";
static String serviceAccountPrivateKeyFilePassword = "";
static String fileId = "";
static JacksonFactory jacksonFactory = new JacksonFactory();
static NetHttpTransport httpTransport = new NetHttpTransport();
static List<String> googleScopeList = Arrays.asList("https://www.googleapis.com/auth/drive.readonly",
"https://www.googleapis.com/auth/admin.directory.group.readonly",
"https://www.googleapis.com/auth/admin.directory.user.alias.readonly",
"https://www.googleapis.com/auth/admin.directory.group", "https://www.googleapis.com/auth/admin.directory.user",
"https://www.googleapis.com/auth/drive");
public static void main(String[] args) throws Exception {
Drive drive = (new Drive.Builder(httpTransport,
jacksonFactory,
getRequestInitializer(getGoogleCredentials())))
.setApplicationName("Sample app").build();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
drive.files().export(fileId, "application/vnd.google-apps.document")
.executeMediaAndDownloadTo(baos);
System.out.println(baos.toString("UTF-8"));
}
public static HttpRequestInitializer getRequestInitializer(final GoogleCredential requestInitializer) {
return httpRequest -> {
requestInitializer.initialize(httpRequest);
httpRequest.setConnectTimeout(readTimeout);
httpRequest.setReadTimeout(connectTimeout);
};
}
public static GoogleCredential getGoogleCredentials() {
GoogleCredential credential;
try {
GoogleCredential.Builder b = new GoogleCredential.Builder().setTransport(httpTransport)
.setJsonFactory(jacksonFactory).setServiceAccountId(serviceAccountId)
.setServiceAccountPrivateKey(SecurityUtils.loadPrivateKeyFromKeyStore(SecurityUtils.getPkcs12KeyStore(),
new FileInputStream(new File(serviceAccountPrivateKeyFile)), serviceAccountPrivateKeyFilePassword,
"privatekey", serviceAccountPrivateKeyFilePassword))
.setServiceAccountScopes(googleScopeList);
if (serviceAccountEmail != null) {
b = b.setServiceAccountUser(serviceAccountEmail);
}
credential = b.build();
} catch (IOException | GeneralSecurityException e1) {
throw new RuntimeException("Could not build client secrets", e1);
}
return credential;
}
}
When I have performed this operation, we are seeing that the viewedByMeTime field is actually being updated as the impersonated user.
This is not good, because now people think someone might have stolen access to their account. They are going to open tickets with the security team.
Is this expected? How can I make this stop? Is there another method in the API I can call to download the google docs without updating this timestamp?
Also opened a ticket on the github for the google drive java sdk: https://github.com/googleapis/google-api-java-client-services/issues/3160

Updating the viewedByMeTime field upon calling the endpoint is indeed intended behaviour. Any action performed through the API is considered the same way as if the user did that action manually (i.e. that field would be updated too when the user visits the document through the UI).
By using domain-wise delegation (or "user impersonation"), you have no way to avoid this issue.
The only workaround would be to give the service account access to this file, and let it export the file without domain-wide delegation. The viewedByMeTime field will be updated only for the service account itself, but not for the original owner of that file (or any other user having access to it).

Google Sheets use API key instead of client_secret.json

In the QuickStart.java example on Java Quickstart they use OAuth client ID to identify the application, and this pops up a windows asking for Google credentials to use the application. You have to download a client_secret.json to modify a Google Sheet.
My question is: Can you evade the popping up window asking for Google credentials using an API Key or something else? And, if it's possible, how do you change the Java code in order to do that?

An API key could only work when accessing the resources owned by the project that created the key.
For resources like spreadsheets, you're typically accessing resources owned by a user. It would be pretty awful if you got access to my private sheets simply by having an API key.
So no, I wouldn't expect there to be any way to avoid getting authorization to work with a user's documents. However, you should be able to use the Java OAuth library to retain the auth token so you can avoid needing to ask for it more than once. (Unless the user revokes access, of course.)
As DalmTo says, you can use service account credentials if you're trying to access resources owned by the project (or which the project can be granted access to). Note that if you're running on AppEngine, Google Kubernetes Engine or Google Compute Engine, the service account credentials for that environment should be available automatically.

The popup window you are seeing is the Oauth2 consent screen. In order to access private user data you need to have consent of the user in order to access their data.
There is another option its called a service account. If the sheet you are trying to access is one that you as the developer have control of then you can create service account credeitals take the service account email address and grant the service account access to the sheet.
The best example for service account access with java that i am aware of is the one for Google Analytics you will have to alter it for Google sheets i may be able to help with that if you have any issues. hello analytics service account.
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.services.analytics.Analytics;
import com.google.api.services.analytics.AnalyticsScopes;
import com.google.api.services.analytics.model.Accounts;
import com.google.api.services.analytics.model.GaData;
import com.google.api.services.analytics.model.Profiles;
import com.google.api.services.analytics.model.Webproperties;
import java.io.FileInputStream;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.io.IOException;
/**
* A simple example of how to access the Google Analytics API using a service
* account.
*/
public class HelloAnalytics {
private static final String APPLICATION_NAME = "Hello Analytics";
private static final JsonFactory JSON_FACTORY = GsonFactory.getDefaultInstance();
private static final String KEY_FILE_LOCATION = "<REPLACE_WITH_JSON_FILE>";
public static void main(String[] args) {
try {
Analytics analytics = initializeAnalytics();
String profile = getFirstProfileId(analytics);
System.out.println("First Profile Id: "+ profile);
printResults(getResults(analytics, profile));
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* Initializes an Analytics service object.
*
* #return An authorized Analytics service object.
* #throws IOException
* #throws GeneralSecurityException
*/
private static AnalyticsReporting initializeAnalytic() throws GeneralSecurityException, IOException {
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
GoogleCredential credential = GoogleCredential
.fromStream(new FileInputStream(KEY_FILE_LOCATION))
.createScoped(AnalyticsScopes.all());
// Construct the Analytics service object.
return new Analytics.Builder(httpTransport, JSON_FACTORY, credential)
.setApplicationName(APPLICATION_NAME).build();
}
private static String getFirstProfileId(Analytics analytics) throws IOException {
// Get the first view (profile) ID for the authorized user.
String profileId = null;
// Query for the list of all accounts associated with the service account.
Accounts accounts = analytics.management().accounts().list().execute();
if (accounts.getItems().isEmpty()) {
System.err.println("No accounts found");
} else {
String firstAccountId = accounts.getItems().get(0).getId();
// Query for the list of properties associated with the first account.
Webproperties properties = analytics.management().webproperties()
.list(firstAccountId).execute();
if (properties.getItems().isEmpty()) {
System.err.println("No Webproperties found");
} else {
String firstWebpropertyId = properties.getItems().get(0).getId();
// Query for the list views (profiles) associated with the property.
Profiles profiles = analytics.management().profiles()
.list(firstAccountId, firstWebpropertyId).execute();
if (profiles.getItems().isEmpty()) {
System.err.println("No views (profiles) found");
} else {
// Return the first (view) profile associated with the property.
profileId = profiles.getItems().get(0).getId();
}
}
}
return profileId;
}
private static GaData getResults(Analytics analytics, String profileId) throws IOException {
// Query the Core Reporting API for the number of sessions
// in the past seven days.
return analytics.data().ga()
.get("ga:" + profileId, "7daysAgo", "today", "ga:sessions")
.execute();
}
private static void printResults(GaData results) {
// Parse the response from the Core Reporting API for
// the profile name and number of sessions.
if (results != null && !results.getRows().isEmpty()) {
System.out.println("View (Profile) Name: "
+ results.getProfileInfo().getProfileName());
System.out.println("Total Sessions: " + results.getRows().get(0).get(0));
} else {
System.out.println("No results found");
}
}
}

How to authenticate a service account without download for the Google Cloud Java SDK (not on App Engine)

I am trying to create a service account key programmatically in Java with the Google Cloud SDK, for an application not running on App/Compute engine. This question is similar to mine, but it is running on App engine, so I cannot use the same code as it uses classes from the App Engine API.
The relevant code is below. My issue is that AppIdentityCredential is part of the AppEngine API and thus cannot be used here. What can I pass in as a parameter instead? The third parameter in the new Builder() method takes in an HttpRequestInitializer, but I don't understand what implementation of this interface I should pass in. Any help is appreciated.
import com.google.api.services.iam.v1.Iam;
import com.google.api.services.iam.v1.model.CreateServiceAccountKeyRequest;
import com.google.api.services.iam.v1.model.ServiceAccountKey;
AppIdentityCredential credential = new AppIdentityCredential(
Arrays.asList("https://www.googleapis.com/auth/cloud-platform"));
Iam iam = new Iam.Builder(httpTransport, JSON_FACTORY,credential)
.setApplicationName(APPLICATION_NAME).build();
ServiceAccountKey key = iam.projects().serviceAccounts().keys()
.create(SERVICE_ACCOUNT_RESOURCE_NAME, new CreateServiceAccountKeyRequest()).execute();

You can use Application Default Credentials which will allow you to use the same code to fetch the credentials based on the environment where the application is running.
For example, it lets you use your gcloud account credentials when you're developing on your system. When the code runs on Google Compute Engine or Google App Engine, the code will automatically use the associated service account credentials for authentication in the APIs. You can also override it using GOOGLE_APPLICATION_CREDENTIALS environment variable if required to load the credentials from a JSON file instead.
Here is a working example which creates a new key for an existing service account and prints it.
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.iam.v1.Iam;
import com.google.api.services.iam.v1.IamScopes;
import com.google.api.services.iam.v1.model.CreateServiceAccountKeyRequest;
import com.google.api.services.iam.v1.model.ServiceAccountKey;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.util.ArrayList;
import java.util.List;
public class IamDemo {
/** Name of the application. */
private static final String APPLICATION_NAME = "IamDemoJava";
/** Project Name. */
private static final String PROJECT_NAME = "MY_PROJECT_NAME";
/** Name of the service account to create a new key for. */
private static final String SERVICE_ACCOUNT_NAME = "dummy-sa";
/** Full email address of the service account. */
private static final String SERVICE_ACCOUNT_EMAIL =
SERVICE_ACCOUNT_NAME + "#" + PROJECT_NAME + ".iam.gserviceaccount.com";
/** Full service account resource string expected by the IAM API. */
private static final String SERVICE_ACCOUNT_RESOURCE_NAME =
"projects/" + PROJECT_NAME + "/serviceAccounts/" + SERVICE_ACCOUNT_EMAIL;
/** Global instance of the HTTP transport. */
private static HttpTransport httpTransport;
/** Global instance of the JSON factory. */
private static final JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
public static void main() throws IOException, GeneralSecurityException {
Iam iam = initIam();
ServiceAccountKey key = createServiceAccountKey(iam);
// Print the key
System.out.println(key.toString());
}
private static Iam initIam() throws IOException, GeneralSecurityException {
httpTransport = GoogleNetHttpTransport.newTrustedTransport();
// Authenticate using Google Application Default Credentials.
GoogleCredential credential = GoogleCredential.getApplicationDefault();
if (credential.createScopedRequired()) {
List<String> scopes = new ArrayList<>();
// Enable full Cloud Platform scope.
scopes.add(IamScopes.CLOUD_PLATFORM);
credential = credential.createScoped(scopes);
}
// Create IAM API object associated with the authenticated transport.
return new Iam.Builder(httpTransport, JSON_FACTORY, credential)
.setApplicationName(APPLICATION_NAME)
.build();
}
private static ServiceAccountKey createServiceAccountKey(Iam iam)
throws IOException, GeneralSecurityException {
CreateServiceAccountKeyRequest request = new CreateServiceAccountKeyRequest();
// Customize the request parameters if needed
return iam.projects()
.serviceAccounts()
.keys()
.create(SERVICE_ACCOUNT_RESOURCE_NAME, request)
.execute();
}
}

Collect Google Sheets Information Without Authorization

How would I go about collecting information correctly off of Google Sheets from a public Google Sheet document without having the user to authenticate anything?
So far, I have the following code I found that looks like it should've done the trick but I get stuck with the
"com.google.gdata.util.AuthenticationException: Error authenticating (check service name)"
The following code below generates it, the user login and passwords are correct
package streamupdater.util;
import com.google.gdata.client.spreadsheet.SpreadsheetService;
import com.google.gdata.data.spreadsheet.ListEntry;
import com.google.gdata.data.spreadsheet.ListFeed;
import com.google.gdata.data.spreadsheet.SpreadsheetEntry;
import com.google.gdata.util.ServiceException;
import java.io.IOException;
import java.net.URL;
public class readData {
public static final String GOOGLE_ACCOUNT_USERNAME = "abc";
public static final String GOOGLE_ACCOUNT_PASSWORD = "xxx";
public static final String SPREADSHEET_URL = "https://docs.google.com/spreadsheets/d/1fS0d19BOExTPdWqIYTvD9IskGIYEskqPY0WL9i7FByk/edit#gid=0";
public String getData(){
String status="";
try{
/** Our view of Google Spreadsheets as an authenticated Google user. */
SpreadsheetService service = new SpreadsheetService("Print Google Spreadsheet Demo");
// Login and prompt the user to pick a sheet to use.
service.setUserCredentials(GOOGLE_ACCOUNT_USERNAME,
GOOGLE_ACCOUNT_PASSWORD);
// Load sheet
URL metafeedUrl = new URL(SPREADSHEET_URL);
SpreadsheetEntry spreadsheet = service.getEntry(metafeedUrl,SpreadsheetEntry.class);
URL listFeedUrl = spreadsheet.getWorksheets().get(0).getListFeedUrl();
// Print entries
ListFeed feed = service.getFeed(listFeedUrl, ListFeed.class);
for (ListEntry entry : feed.getEntries()) {
System.out.println("new row");
for (String tag : entry.getCustomElements().getTags()) {
System.out.println(" " + tag + ": "
+ entry.getCustomElements().getValue(tag));
status=entry.getCustomElements().getValue(tag);
}
}
}catch(Exception e){
System.out.println(e);
}
System.out.println(status);
return(status);
}
public static void main(String[] args) {
readData rd = new readData();
rd.getData();
}
}

Using a login and password to access private Google data is called client login. Google Shut down the client login servers in May 2015. There for your code will not work. You will need to use Open Authentication.
Suggestions:
Switch to Google Sheets v4
Try and authenticate with using an API key. It should give you access to a public Google sheet. Another less risky thing would be to use a Service account to access a private Google sheet.

Making own queries Google Analytics (Java)

I am trying to implement the google analytics library in a website in order to make my own queries and obtain some kind of data to manage them afterwards.
I have followed the example and everything works fine. However, I am only able to code the example query (visitors in last week). I've read many information and documentation about that and I'm still having the same issue.
I'm sure there must be a way to accomplish that, but actually I'm not able to code anything to make my own queries.
The code is (I'm using maven):
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.services.analytics.Analytics;
import com.google.api.services.analytics.AnalyticsScopes;
import com.google.api.services.analytics.model.Accounts;
import com.google.api.services.analytics.model.GaData;
import com.google.api.services.analytics.model.Profiles;
import com.google.api.services.analytics.model.Webproperties;
import java.io.File;
import java.io.IOException;
/**
* A simple example of how to access the Google Analytics API using a service
* account.
*/
public class test {
private static final String APPLICATION_NAME = "example";
private static final JsonFactory JSON_FACTORY = GsonFactory.getDefaultInstance();
private static final String KEY_FILE_LOCATION = //route to p.12 file;
private static final String SERVICE_ACCOUNT_EMAIL = //mail example;
public static void main(String[] args) {
try {
Analytics analytics = initializeAnalytics();
String profile = getFirstProfileId(analytics);
System.out.println("First Profile Id: " + profile);
printResults(getResults(analytics, profile));
} catch (Exception e) {
e.printStackTrace();
}
}
private static Analytics initializeAnalytics() throws Exception {
// Initializes an authorized analytics service object.
// Construct a GoogleCredential object with the service account email
// and p12 file downloaded from the developer console.
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(SERVICE_ACCOUNT_EMAIL)
.setServiceAccountPrivateKeyFromP12File(new File(KEY_FILE_LOCATION))
.setServiceAccountScopes(AnalyticsScopes.all())
.build();
// Construct the Analytics service object.
return new Analytics.Builder(httpTransport, JSON_FACTORY, credential)
.setApplicationName(APPLICATION_NAME).build();
}
private static String getFirstProfileId(Analytics analytics) throws IOException {
// Get the first view (profile) ID for the authorized user.
String profileId = null;
// Query for the list of all accounts associated with the service account.
Accounts accounts = analytics.management().accounts().list().execute();
if (accounts.getItems().isEmpty()) {
System.err.println("No accounts found");
} else {
String firstAccountId = accounts.getItems().get(0).getId();
// Query for the list of properties associated with the first account.
Webproperties properties = analytics.management().webproperties()
.list(firstAccountId).execute();
if (properties.getItems().isEmpty()) {
System.err.println("No Webproperties found");
} else {
String firstWebpropertyId = properties.getItems().get(0).getId();
// Query for the list views (profiles) associated with the property.
Profiles profiles = analytics.management().profiles()
.list(firstAccountId, firstWebpropertyId).execute();
if (profiles.getItems().isEmpty()) {
System.err.println("No views (profiles) found");
} else {
// Return the first (view) profile associated with the property.
profileId = profiles.getItems().get(0).getId();
}
}
}
return profileId;
}
private static GaData getResults(Analytics analytics, String profileId) throws IOException {
// Query the Core Reporting API for the number of sessions
// in the past seven days.
return analytics.data().ga()
.get("ga:" + profileId, "7daysAgo", "today", "ga:sessions")
.execute();
}
private static void printResults(GaData results) {
// Parse the response from the Core Reporting API for
// the profile name and number of sessions.
if (results != null && !results.getRows().isEmpty()) {
System.out.println("View (Profile) Name: "
+ results.getProfileInfo().getProfileName());
System.out.println("Total Sessions: " + results.getRows().get(0).get(0));
} else {
System.out.println("No results found");
}
}
}
Note that the code that makes the queries is:
private static GaData getResults(Analytics analytics, String profileId) throws IOException {
// Query the Core Reporting API for the number of sessions
// in the past seven days.
return analytics.data().ga()
.get("ga:" + profileId, "7daysAgo", "today", "ga:sessions")
.execute();
}
The problem is: how I set dimensions of my queries using this code?
This query works (without dimensions):
private static GaData getMobileTraffic(Analytics analytics, String profileId) throws IOException {
return analytics.data().ga()
.get("ga:" + profileId, "30daysAgo", "today", "ga:sessions, ga:pageviews, ga:sessionDuration")
.execute();
}
This one doesn't work (with dimensions):
private static GaData getMobileTraffic(Analytics analytics, String profileId) throws IOException {
return analytics.data().ga()
.get("ga:" + profileId, "30daysAgo", "today", "ga:sessions, ga:pageviews, ga:sessionDuration",**"ga:userType"**)
.execute();
}
I would appreciate any help. Thanks a lot!

Well, seems that I've finally solved my issue. I answer myself in order to help someone with similar problems and hope it works for them too.
The answer is simply, the documentation about Google Analytics is a bit confusing, but if someone wants to add dimensions, metrics (or edit the example query from Google analytics) just need to add the code in the following link:
https://developers.google.com/analytics/devguides/reporting/core/v3/coreDevguide
Just adding this code just following the query example (if you want to add dimensions):
setDimensions("ga:userType")
So, the final code will be:
private static GaData getMobileTraffic(Analytics analytics, String profileId) throws IOException {
return analytics.data().ga()
.get("ga:" + profileId, "30daysAgo", "today", "ga:sessions, ga:pageviews, ga:sessionDuration").setDimensions("ga:userType")
.execute();
}
Note that in the main code, you need to add the print function, just like this:
public static void main(String[] args) {
try {
Analytics analytics = initializeAnalytics();
printMobileTraffic(getMobileTraffic(analytics, profile));
} catch (Exception e) {
e.printStackTrace();
}
}