Develop Reference

Having Issue with error refreshing access token

Hi I am trying to connect to Big Query and I am using a google service account with a JSON key. I am getting the below error. This is in my java batch program.
Insert operation not performed
com.google.cloud.bigquery.BigQueryException: Unexpected error refreshing access token

This fixed my issue. more here https://cloud.google.com/bigquery/docs/authorization
import com.google.auth.Credentials;
import com.google.auth.oauth2.ServiceAccountJwtAccessCredentials;
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.Dataset;
import java.io.FileInputStream;
import java.net.URI;
public class Example {
public static void main(String... args) throws Exception {
String projectId = "myproject";
// Load JSON file that contains service account keys and create ServiceAccountJwtAccessCredentials object.
String credentialsPath = "/path/to/key.json";
URI audience = URI.create("https://bigquery.googleapis.com/");
Credentials credentials = null;
try (FileInputStream is = new FileInputStream(credentialsPath)) {
credentials = ServiceAccountJwtAccessCredentials.fromStream(is, audience);
}
// Instantiate BigQuery client with the credentials object.
BigQuery bigquery =
BigQueryOptions.newBuilder().setCredentials(credentials).build().getService();
// Use the client to list BigQuery datasets.
System.out.println("Datasets:");
bigquery
.listDatasets(projectId)
.iterateAll()
.forEach(dataset -> System.out.printf("%s%n", dataset.getDatasetId().getDataset()));
}
}

When using service account impersonation, when calling export on Google Docs using v3 API, viewedByMeTime timestamp is updated

I am using a service account to access google doc files of users in my enterprise google account using impersonation.
See:
https://developers.google.com/drive/api/v3/about-auth#OAuth2Authorizing
So far so good.
Then, I need to download contents of Google Docs.
When calling Google Drive API to download the contents of a Google Doc, the documentation says to run the following:
https://developers.google.com/drive/api/v3/manage-downloads
Here is a java program that should reproduce the problem:
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.http.HttpRequestInitializer;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.util.SecurityUtils;
import com.google.api.services.drive.Drive;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.util.Arrays;
import java.util.List;
public class FetchGoogleDocContentsWithServiceAccount {
static int readTimeout = 60000;
static int connectTimeout = 60000;
static String serviceAccountId = "";
static String serviceAccountEmail = "";
static String serviceAccountPrivateKeyFile = "";
static String serviceAccountPrivateKeyFilePassword = "";
static String fileId = "";
static JacksonFactory jacksonFactory = new JacksonFactory();
static NetHttpTransport httpTransport = new NetHttpTransport();
static List<String> googleScopeList = Arrays.asList("https://www.googleapis.com/auth/drive.readonly",
"https://www.googleapis.com/auth/admin.directory.group.readonly",
"https://www.googleapis.com/auth/admin.directory.user.alias.readonly",
"https://www.googleapis.com/auth/admin.directory.group", "https://www.googleapis.com/auth/admin.directory.user",
"https://www.googleapis.com/auth/drive");
public static void main(String[] args) throws Exception {
Drive drive = (new Drive.Builder(httpTransport,
jacksonFactory,
getRequestInitializer(getGoogleCredentials())))
.setApplicationName("Sample app").build();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
drive.files().export(fileId, "application/vnd.google-apps.document")
.executeMediaAndDownloadTo(baos);
System.out.println(baos.toString("UTF-8"));
}
public static HttpRequestInitializer getRequestInitializer(final GoogleCredential requestInitializer) {
return httpRequest -> {
requestInitializer.initialize(httpRequest);
httpRequest.setConnectTimeout(readTimeout);
httpRequest.setReadTimeout(connectTimeout);
};
}
public static GoogleCredential getGoogleCredentials() {
GoogleCredential credential;
try {
GoogleCredential.Builder b = new GoogleCredential.Builder().setTransport(httpTransport)
.setJsonFactory(jacksonFactory).setServiceAccountId(serviceAccountId)
.setServiceAccountPrivateKey(SecurityUtils.loadPrivateKeyFromKeyStore(SecurityUtils.getPkcs12KeyStore(),
new FileInputStream(new File(serviceAccountPrivateKeyFile)), serviceAccountPrivateKeyFilePassword,
"privatekey", serviceAccountPrivateKeyFilePassword))
.setServiceAccountScopes(googleScopeList);
if (serviceAccountEmail != null) {
b = b.setServiceAccountUser(serviceAccountEmail);
}
credential = b.build();
} catch (IOException | GeneralSecurityException e1) {
throw new RuntimeException("Could not build client secrets", e1);
}
return credential;
}
}
When I have performed this operation, we are seeing that the viewedByMeTime field is actually being updated as the impersonated user.
This is not good, because now people think someone might have stolen access to their account. They are going to open tickets with the security team.
Is this expected? How can I make this stop? Is there another method in the API I can call to download the google docs without updating this timestamp?
Also opened a ticket on the github for the google drive java sdk: https://github.com/googleapis/google-api-java-client-services/issues/3160

Updating the viewedByMeTime field upon calling the endpoint is indeed intended behaviour. Any action performed through the API is considered the same way as if the user did that action manually (i.e. that field would be updated too when the user visits the document through the UI).
By using domain-wise delegation (or "user impersonation"), you have no way to avoid this issue.
The only workaround would be to give the service account access to this file, and let it export the file without domain-wide delegation. The viewedByMeTime field will be updated only for the service account itself, but not for the original owner of that file (or any other user having access to it).

Google Sheets use API key instead of client_secret.json

In the QuickStart.java example on Java Quickstart they use OAuth client ID to identify the application, and this pops up a windows asking for Google credentials to use the application. You have to download a client_secret.json to modify a Google Sheet.
My question is: Can you evade the popping up window asking for Google credentials using an API Key or something else? And, if it's possible, how do you change the Java code in order to do that?

An API key could only work when accessing the resources owned by the project that created the key.
For resources like spreadsheets, you're typically accessing resources owned by a user. It would be pretty awful if you got access to my private sheets simply by having an API key.
So no, I wouldn't expect there to be any way to avoid getting authorization to work with a user's documents. However, you should be able to use the Java OAuth library to retain the auth token so you can avoid needing to ask for it more than once. (Unless the user revokes access, of course.)
As DalmTo says, you can use service account credentials if you're trying to access resources owned by the project (or which the project can be granted access to). Note that if you're running on AppEngine, Google Kubernetes Engine or Google Compute Engine, the service account credentials for that environment should be available automatically.

The popup window you are seeing is the Oauth2 consent screen. In order to access private user data you need to have consent of the user in order to access their data.
There is another option its called a service account. If the sheet you are trying to access is one that you as the developer have control of then you can create service account credeitals take the service account email address and grant the service account access to the sheet.
The best example for service account access with java that i am aware of is the one for Google Analytics you will have to alter it for Google sheets i may be able to help with that if you have any issues. hello analytics service account.
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.services.analytics.Analytics;
import com.google.api.services.analytics.AnalyticsScopes;
import com.google.api.services.analytics.model.Accounts;
import com.google.api.services.analytics.model.GaData;
import com.google.api.services.analytics.model.Profiles;
import com.google.api.services.analytics.model.Webproperties;
import java.io.FileInputStream;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.io.IOException;
/**
* A simple example of how to access the Google Analytics API using a service
* account.
*/
public class HelloAnalytics {
private static final String APPLICATION_NAME = "Hello Analytics";
private static final JsonFactory JSON_FACTORY = GsonFactory.getDefaultInstance();
private static final String KEY_FILE_LOCATION = "<REPLACE_WITH_JSON_FILE>";
public static void main(String[] args) {
try {
Analytics analytics = initializeAnalytics();
String profile = getFirstProfileId(analytics);
System.out.println("First Profile Id: "+ profile);
printResults(getResults(analytics, profile));
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* Initializes an Analytics service object.
*
* #return An authorized Analytics service object.
* #throws IOException
* #throws GeneralSecurityException
*/
private static AnalyticsReporting initializeAnalytic() throws GeneralSecurityException, IOException {
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
GoogleCredential credential = GoogleCredential
.fromStream(new FileInputStream(KEY_FILE_LOCATION))
.createScoped(AnalyticsScopes.all());
// Construct the Analytics service object.
return new Analytics.Builder(httpTransport, JSON_FACTORY, credential)
.setApplicationName(APPLICATION_NAME).build();
}
private static String getFirstProfileId(Analytics analytics) throws IOException {
// Get the first view (profile) ID for the authorized user.
String profileId = null;
// Query for the list of all accounts associated with the service account.
Accounts accounts = analytics.management().accounts().list().execute();
if (accounts.getItems().isEmpty()) {
System.err.println("No accounts found");
} else {
String firstAccountId = accounts.getItems().get(0).getId();
// Query for the list of properties associated with the first account.
Webproperties properties = analytics.management().webproperties()
.list(firstAccountId).execute();
if (properties.getItems().isEmpty()) {
System.err.println("No Webproperties found");
} else {
String firstWebpropertyId = properties.getItems().get(0).getId();
// Query for the list views (profiles) associated with the property.
Profiles profiles = analytics.management().profiles()
.list(firstAccountId, firstWebpropertyId).execute();
if (profiles.getItems().isEmpty()) {
System.err.println("No views (profiles) found");
} else {
// Return the first (view) profile associated with the property.
profileId = profiles.getItems().get(0).getId();
}
}
}
return profileId;
}
private static GaData getResults(Analytics analytics, String profileId) throws IOException {
// Query the Core Reporting API for the number of sessions
// in the past seven days.
return analytics.data().ga()
.get("ga:" + profileId, "7daysAgo", "today", "ga:sessions")
.execute();
}
private static void printResults(GaData results) {
// Parse the response from the Core Reporting API for
// the profile name and number of sessions.
if (results != null && !results.getRows().isEmpty()) {
System.out.println("View (Profile) Name: "
+ results.getProfileInfo().getProfileName());
System.out.println("Total Sessions: " + results.getRows().get(0).get(0));
} else {
System.out.println("No results found");
}
}
}

Collect Google Sheets Information Without Authorization

How would I go about collecting information correctly off of Google Sheets from a public Google Sheet document without having the user to authenticate anything?
So far, I have the following code I found that looks like it should've done the trick but I get stuck with the
"com.google.gdata.util.AuthenticationException: Error authenticating (check service name)"
The following code below generates it, the user login and passwords are correct
package streamupdater.util;
import com.google.gdata.client.spreadsheet.SpreadsheetService;
import com.google.gdata.data.spreadsheet.ListEntry;
import com.google.gdata.data.spreadsheet.ListFeed;
import com.google.gdata.data.spreadsheet.SpreadsheetEntry;
import com.google.gdata.util.ServiceException;
import java.io.IOException;
import java.net.URL;
public class readData {
public static final String GOOGLE_ACCOUNT_USERNAME = "abc";
public static final String GOOGLE_ACCOUNT_PASSWORD = "xxx";
public static final String SPREADSHEET_URL = "https://docs.google.com/spreadsheets/d/1fS0d19BOExTPdWqIYTvD9IskGIYEskqPY0WL9i7FByk/edit#gid=0";
public String getData(){
String status="";
try{
/** Our view of Google Spreadsheets as an authenticated Google user. */
SpreadsheetService service = new SpreadsheetService("Print Google Spreadsheet Demo");
// Login and prompt the user to pick a sheet to use.
service.setUserCredentials(GOOGLE_ACCOUNT_USERNAME,
GOOGLE_ACCOUNT_PASSWORD);
// Load sheet
URL metafeedUrl = new URL(SPREADSHEET_URL);
SpreadsheetEntry spreadsheet = service.getEntry(metafeedUrl,SpreadsheetEntry.class);
URL listFeedUrl = spreadsheet.getWorksheets().get(0).getListFeedUrl();
// Print entries
ListFeed feed = service.getFeed(listFeedUrl, ListFeed.class);
for (ListEntry entry : feed.getEntries()) {
System.out.println("new row");
for (String tag : entry.getCustomElements().getTags()) {
System.out.println(" " + tag + ": "
+ entry.getCustomElements().getValue(tag));
status=entry.getCustomElements().getValue(tag);
}
}
}catch(Exception e){
System.out.println(e);
}
System.out.println(status);
return(status);
}
public static void main(String[] args) {
readData rd = new readData();
rd.getData();
}
}

Using a login and password to access private Google data is called client login. Google Shut down the client login servers in May 2015. There for your code will not work. You will need to use Open Authentication.
Suggestions:
Switch to Google Sheets v4
Try and authenticate with using an API key. It should give you access to a public Google sheet. Another less risky thing would be to use a Service account to access a private Google sheet.

Java Google Spreadsheet API: Updating Public Worksheet 'java.lang.UnsupportedOperationException: Entry cannot be updated'

I am quite new to Java and new to the Google Data API's. I am trying to change the size of a worksheet in a publicly shared spreadsheet. I'm using the following code:
import com.google.gdata.client.authn.oauth.*;
import com.google.gdata.client.spreadsheet.*;
import com.google.gdata.data.*;
import com.google.gdata.data.batch.*;
import com.google.gdata.data.spreadsheet.*;
import com.google.gdata.util.*;
import java.io.IOException;
import java.net.*;
import java.util.*;
static void writeToGoogleSpreadsheet(String spreadsheetKey) throws IOException, ServiceException {
SpreadsheetService service = new SpreadsheetService("com.example");
String urlString = "https://spreadsheets.google.com/feeds/worksheets/" + spreadsheetKey + "/public/basic";
URL url = new URL(urlString);
try {
WorksheetFeed worksheetFeed = service.getFeed(url, WorksheetFeed.class);
List<WorksheetEntry> worksheets = worksheetFeed.getEntries();
WorksheetEntry worksheet = worksheets.get(0);
System.out.println(worksheet.getTitle().getPlainText());
System.out.println(worksheet.getCanEdit());
// Update the local representation of the worksheet.
worksheet.setTitle(new PlainTextConstruct("Updated Worksheet"));
worksheet.setColCount(40);
worksheet.setRowCount(40);
// Send the local representation of the worksheet to the API for
// modification.
worksheet.update();
} finally{}
}
The console displays the correct worksheet title and size, so I'm pretty sure I am accessing the right worksheet. However, worksheet.update() throws the following exception:
Exception in thread "main" java.lang.UnsupportedOperationException: Entry cannot be updated
at com.google.gdata.data.BaseEntry.update(BaseEntry.java:635)
atcom.example.GoogleSpreadsheetCommunicator.writeToGoogleSpreadsheet(GoogleSpreadsheetCommunicator.java:119)
at com.example.Main.guildSim(Main.java:114)
at com.example.Main.main(Main.java:73)
Does anyone know what I am doing wrong?
Thanks for reading and kind regards,
Karel

You can't edit a public feed,
change urlString to use .../private/...
and you will need to use one of the 3 ways to authenticate you can find on https://developers.google.com/google-apps/spreadsheets/#authorizing_requests_with_clientlogin