I am trying to use Google Cloud Translation API, but I can not authenticate to Google Cloud.
TranslationServiceClient client = TranslationServiceClient.create()
This line fires the error below:
W/System.err: java.io.IOException: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
I have created the project on Google Cloud, downloaded the json file when I created the Service Account as required by the API and I exported the parameters using command line but still not working:
export GOOGLE_APPLICATION_CREDENTIALS="/Users/admin/Desktop/franzosischlernen-d8db6-9457de40bc93.json"
Related
I have installed google datastore emulator in my local machine along with it written a sample spring boot application .
I can't connection datastore emulator
This is my application.properties config
spring.cloud.gcp.datastore.project-id=project-id
spring.cloud.gcp.datastore.emulator.enabled=true
spring.cloud.gcp.datastore.emulator-host=http://localhost:8081
by this config , I will throw Exception
The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
When using the Datastore emulator, you don't need credentials for running the application, so it might be that the library doesn't know that.
However, if you want to try it providing credentials, once you have a service account created, then run in the shell the following:
export GOOGLE_APPLICATION_CREDENTIALS="KEY_PATH"
KEY_PATH you have to replace it with the path of the JSON file that contains your service account key. You can find more information here.
The application
Simple REST API registration service in Spring, after sending proper POST request new user is created in database and Amazon SES sends an email with registration link to verify.
The problem
Locally after setting local variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION) in my OS (Windows) app works just fine, but the problem starts after deploying it. I have an EC2 Instance with Amazon Linux AMI on AWS:
created user at AWS Identity and Access Management (IAM), granted AmazonSESFullAccess role
logging by CMD using shh with the private key file *.pem
Tomcat8 service started
MySQL service started
application *.war file deployed
created environment variables using 'export' command and I checked 'printenv' just to be sure everything is fine
after sending POST request I got exception (below) which means that user has been created but Amazon SES didn't send an email confirmation because couldn't authenticate
{
"timestamp": "2020-04-26T15:44:44.010+0000",
"message": "Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: To use assume role profiles the aws-java-sdk-sts module must be on the class path., com.amazonaws.auth.profile.ProfileCredentialsProvider#23fac1a3: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper#68aa5a98: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]"
}
I checked again local environment variables on my EC2 instance and it was looking fine but to be sure I re-configured it using 'aws configure' command
exception keeps showing, somehow application cannot get environment variables, I'm fighting with that for over 5 hours now so hopefully someone will come here to rescue me...
Piece of code (works fine locally):
AmazonSimpleEmailService client =
AmazonSimpleEmailServiceClientBuilder
.standard()
.withCredentials(DefaultAWSCredentialsProviderChain.getInstance())
.withRegion(Regions.EU_CENTRAL_1)
.build();
I am total Linux noob, having problems with simple commands so please be gentle with solutions requiring some console commands.
If you're running app on EC2, don't use IAM user.
Instead create IAM role with same permissions and assign that role to the instance. If app uses AWS SDK it will be able to pick up credentials without any problems.
In your case problem is probably app's environment being different from yours, if you export credentials in your bash session it will not pass to app if it's loaded under different user or bash session.
The DefaultAWSCredentialsProvider has multiple places it will look for credentials. Instead of setting up your credentials as an environment variable, you can set up a credentials profile. See this documentation: Working with AWS Credentials.
Make sure you have the AWS CLI installed, then you can run the following command to configure your profile: aws configure
Click here for the documentation on the aws configure command.
If you have already configured your aws profile and it still does not work, you have most likely configured the profile for the wrong linux user. For example, if a linux user named tomcat8 is the user who is running your tomcat instance, then you need to set up a credentials profile at /home/tomcat8/.aws/credentials/
I am both locally, as well as within our Kubernetes pods, authenticated into Google cloud. On both I can get correct response with gcloud info.
However, when I want to access GoogleDrive, I need to use GoogleCredential as follows:
GoogleCredential.Builder()
.setTransport(transport)
.setJsonFactory(jsonFactory)
.setServiceAccountPrivateKey(privateKey)
.setServiceAccountId(serviceAccount)
.setServiceAccountScopes(scopes.toList()).build()
Meaning - I need to specifically set privateKey and serviceAccount. Is there a way to force it to use the locally authenticated account?
When using Google buckets this can be done quite easily:
StorageOptions.getDefaultInstance().service.options.credentials
I cannot find the same way for Google Drive.
As #DazWilkin indicated, many GCP client libraries (such as GCS in your case) know how to automatically detect "Application Default Credentials" available.
These ADC credentials currently work only on Google Cloud Platform APIs (Google Drive predates that). You can read Google Drive Java quickstart to learn how to retrieve credentials: https://developers.google.com/drive/api/v3/quickstart/java
For refernce, GCP client libraries will look for ADCs by:
GOOGLE_APPLICATION_CREDENTIALS environment variable, if set, pointing to JSON key file of the service account.
Find %APPDATA%/gcloud/application_default_credentials.json (Windows) or $HOME/.config/gcloud/application_default_credentials.json (other) if the user has executed gcloud auth application-default login command.
On Google App Engine 1st gen (not GAE Flex) use appengine.AccessToken API.
On GCE, GKE or GAE 2nd gen environments, it calls the GCE Metadata API (an url like http://metadata.google.internal or http://169.254.169.254) to retrieve a short-lived access_token.
In your case, your GKE pods are using method #4 to retrieve a token for GCS bucket operations; but not for Drive API.
I am using google app engine and google datastore.
I am using the google library
com.google.cloud.datastore.Datastore
My sample code is:
Datastore datastore = DatastoreOptions.getDefaultInstance().getService();
Key taskKey = datastore.newKeyFactory().setKind(entityName).newKey(id);
//populate some fields....
datastore.put(task);
I am using spring-boot and jetty as a container.
On local, it is working properly and the data updated in the google datastore.
The issue is when im deploying the app to the google-app-engine, im getting the Below exception when i get to datastore.put method.
com.google.cloud.datastore.DatastoreException: Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:129)
com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.commit(HttpDatastoreRpc.java:155)
com.google.cloud.datastore.DatastoreImpl$4.call(DatastoreImpl.java:485)
Note: in both, on local environment and google-app-engine, i defined the environment variable GOOGLE_APPLICATION_CREDENTIALS that point to json file with all required credentials generated by google API.
According to the documentation for connecting to Datastore from App Engine in Java, there are several options available, so you can either go with Objectify (third party library), Datastore API or Datastore Client Library.
With the usage of Client Libraries, you must know that they make use of the Application Default Credentials, in such a way that, as documented, if the environment variable GOOGLE_APPLICATION_CREDENTIALS, ADC uses the default service account that App Engine provides for applications running over that service. So in your case, I think you should not define the environment variable, so that App Engine uses its default Service Account.
if you still struggling with com.google.cloud.datastore.DatastoreException: Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. you can set credentials explicitly, like it is shown in the example below:
Resource credentialsCyberpower = resourceLoader.getResource("classpath:yourservice-datastore-access.json");
GoogleCredentials credentials = GoogleCredentials.fromStream(credentialsCyberpower.getInputStream())
.createScoped(Lists.newArrayList("https://www.googleapis.com/auth/cloud-platform"));
DatastoreOptions options =
DatastoreOptions.newBuilder().setProjectId("XXXXXX").setCredentials(credentials).build();
Datastore datastore = options.getService();
ObjectifyService.init(new ObjectifyFactory(datastore));
generate yourservice-datastore-access.json in IAM service accounts. working with Objectify 6.0.5
If you are using <url-stream-handler>urlfetch</url-stream-handler> with Java8 and Objectify 6 you will have to switch into native and enable billing.
I was hit by this issue recently and spend a lot of time on fixing the problem, more info can be found here
I've created my java web application on a tomcat server which will start another instance using the AWS Java SDK, on windows i just place the credentials in my user. Im now trying to host my application on an AWS EC2 Instance and hence i am trying to place my credentials on the Linux EC2 i've follow some steps on the AWS SDK - http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-setup.html as per the link but im still thrown the same error upon calling the method -
Cannot load the credentials from the credential profiles file. Please
make sure that your credentials file is at the correct location
(~/.aws/credentials), and is in valid format.
I've created a .aws folder in my home directory an placed the credential file within it, i've also added the export codes within the .bashrc file but it doesnt seem to work.
At Wits end here :(
Check what user tomcat runs as on the other machine.
When you store the credential for your user, they are stored at ~/.aws/credentials.
That's ok for you, but Tomcat may not be running as you.
So ensure a copy is present also at /home/{whateveryourtomcatuseris}/.aws/credentials.