As my question title says, is Memcache supposed to play well with Google cloud endpoints?
Locally it does, I can store a key / value pair using JCache in my application and read it from within a Google Cloud Endpoints API method.
When I upload my application and run it on the cloud it returns null exactly as the way it happens when I've found out that we can't access sessions from inside of cloud endpoints...
And I doing something wrong or Google cloud endpoints isn't supposed to access the cache as well?
I really need to share some tokens safely between my application and cloud endpoints and I don't want to write / read from the Datastore (those are volatile tokens...). Any ideas?
Endpoints definitely works with memcache, using the built-in API. I just tried the following trivial snippet within an API method and saw the incrementing values as expected:
String key = "key";
Integer cached = 0;
MemcacheService memcacheService = MemcacheServiceFactory.getMemcacheService();
memcacheService.setErrorHandler(new StrictErrorHandler());
cached = (Integer) memcacheService.get(key);
if (cached == null) {
memcacheService.put(key, 0);
} else {
memcacheService.put(key, cached + 1);
}
This should work for you, unless you have a specific requirement for JCache.
Related
I have the following rule in Firestore for read if true I recieve email that point anybody can read and my requests can be drained. I need some approach read action to be allowed only from my app.I don't have authentication in the app is accessible without registration.What can I do to restrict the access only from the app.I read can be set SHA1 key but not quite sure how to implement it.
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
match /{document=**} {
allow read, write: if request.auth != null;
allow read: if true;
}
}
}
There is no way to do that but there is a way around just add Anonymously authentication with firebase Auth Ui this way users will not have to create an email and they will be authenticated
another way is: try to make some parts which doesn't require security public for anyone to read .
I‘m afraid there’s actually nothing you can do.
Since you are using a backend as a service provider you will have to use some sort of authentication if you want to prevent certain people to execute certain actions.
This could be classic username/password authentication or a more modern means like oauth.
If you only want your application code to be able to access the database, consider setting up Firebase App Check for your project.
I have deployed an application to GAE that uses the Cloud Datastore (Cloud Firestore in Datastore mode). It is a fairly simple API that creates, updates, and gets a pretty simple record.
When I POST to the API sometimes multiple of the same record (different keys) are inserted into the database. Sometimes one record, sometimes multiple. I have put in some log statements to verify the code is only executed once. I can see the multiple records using the online Datastore UI/query tool.
I'm not sure how much of this matters, but here are some other points of interest:
Micronaut
Compiled natively using GraalVM
App Engine Standard
Is not reproducible locally using the native binary
Same behavior with add() or put()
Creating the KeyFactory in the class itself doesn't matter (but probably better form).
Keys are different on all duplicate records.
The duplicate records are returned in a list operation.
Here is the code that is doing the insert:
public Find create(Find find) {
LOGGER.info("creating find with description {}", find.getDescription());
IncompleteKey key = keyFactory.setKind(COLLECTION).newKey();
FullEntity<IncompleteKey> e = FullEntity.newBuilder(key)
.set("dateFound", find.getDateFound())
.set("description", find.getDescription())
.set("latitude", find.getLatitude())
.set("longitude", find.getLongitude())
.set("userId", find.getUserId())
.set("updated", Timestamp.now())
.build();
Entity result = datastore.add(e);
LOGGER.info("creating find with description {} id {}", find.getDescription(), result.getKey().getId());
return fromEntity(result);
}
datastore and keyFactory are both singletons injected into this class.
They are built in a #Factory class:
#Singleton
public Datastore datastore() {
Datastore ds = DatastoreOptions
.getDefaultInstance()
.getService();
return ds;
}
#Singleton
public KeyFactory keyFactory(Datastore datastore) {
KeyFactory keyFactory = datastore.newKeyFactory();
return keyFactory;
}
Any ideas what might be going on? I'm only expecting one record to be inserted per request.
UPDATE
It appears to be related to the native image. If I deploy a jar I do not see this problem. Anybody know who/where I can send information on this to help me figure it out?
I have a service in Google Cloud Compute that is part of a multi-tenant application. We use vault to refresh credentials every 4-5 hours through a transient account in this location: /dev/shm/gcp_credentials.json.
When creating a GCS storage object, I want to figure out what's the best way to update those credentials.
I noticed there's a method to build the storage object that allows me to pass a credentials object. I'm using google-auth-credentials-library:0.16.2The credentials object has a method credentials.refresh() as per the doc reference: https://javadoc.io/doc/com.google.auth/google-auth-library-credentials/0.16.2/com/google/auth/Credentials.html and for the storage object, I'm using google-cloud-storage:1.83.0
Storage refreshStorage() {
return StorageOptions.newBuilder().setCredentials(credentials)
.setProjectId("project_id").build().getService();
}
What I want to understand is, if I call credentials.refresh(), will this guarantee that when I call storage.create(blobInfo, records); it re-authenticates, or do I have to call again the above method to pass a new storage object with credentials refreshed?
The credentials.refresh() will remove the cached state of your credentials so the proper process will be:
refreshing your credentials > calling credentials.refresh() to avoid
any error reading the cache credential > working with your storage
I am trying to retrieve all the instances running in my AWS account (say instance id, etc). I use the following code. I am not able to print the instance ids. When I debug, I am just getting null values. But I have three instances running on AWS. Can someone point out what I am doing wrong here?
DescribeInstancesResult result = ec2.describeInstances();
List<Reservation> reservations = result.getReservations();
for (Reservation reservation : reservations) {
List<Instance> instances = reservation.getInstances();
for (Instance instance : instances) {
System.out.println(instance.getInstanceId());
}
}
The most common cause for issues like this is a missing region specification when initializing the client, see section To create and initialize an Amazon EC2 client within Create an Amazon EC2 Client for details:
Specifically, step 2 only creates an EC2 client without specifying the region explicitly:
2) Use the AWSCredentials object to create a new AmazonEC2Client instance, as follows:
amazonEC2Client = new AmazonEC2Client(credentials);
This yields a client talking to us-east-1 - surprisingly, the AWS SDKs and the AWS Management Console use different defaults even as outlined in step 3, which also shows how to specify a different endpoint:
3) By default, the service endpoint is ec2.us-east-1.amazonaws.com. To specify a different endpoint, use the setEndpoint method. For example:
amazonEC2Client.setEndpoint("ec2.us-west-2.amazonaws.com");
The AWS SDK for Java uses US East (N. Virginia) as the default region
if you do not specify a region in your code. However, the AWS
Management Console uses US West (Oregon) as its default. Therefore,
when using the AWS Management Console in conjunction with your
development, be sure to specify the same region in both your code and
the console. [emphasis mine]
The differing defaults are easy to trip over, and the respective default in the AWS Management Console has in fact changed over time - as so often in software development, I recommend to always be explicit about this in your code to avoid such subtle error sources.
So I have a java program running within an Amazon EC2 instance. Is there a way to programatically get its own tags? I have tried instantiating a new AmazonEC2Client to us the describeTags() function but it only gives me null. Any help would be appreciated thank you.
Edit: To make things clearer, the instances are going to be unmanned worker machines spun up to solely do some computations
This should help you get started...
String instanceId = EC2MetadataUtils.getInstanceId();
AmazonEC2 client = AmazonEC2ClientBuilder.standard()
.withCredentials(new DefaultAWSCredentialsProviderChain())
.build();
DescribeTagsRequest req = new DescribeTagsRequest()
.withFilters(new Filter("resource-id", Collections.singletonList(instanceId)));
DescribeTagsResult describeTagsResult = client.describeTags(req);
List<TagDescription> tags = describeTagsResult.getTags()
You should be able to get the current instance id by sending a request to: http://169.254.169.254/latest/meta-data/instance-id. This only works within ec2. With this you can access quite a bit of information about the instance. However, tags do not appear to be included.
You should be able to take the instance id along with the correct authentication to get the instance tags. If you are going to run this on an instance, you may want to provide an IAM user with limited access instead of a user which has access to everything in case the instance is compromised.
While using user-data may be the simplest solution, the OP was asking specifically about the tagging, and unfortunately amazon hasn't made this as easy as it could be. However, It can be done. You want to use a combination of 2 amazon services.
First you need to retrieve the Instance ID. This can be achieved by hitting the URL from within your instance:
http://169.254.169.254/latest/meta-data/instance-id
Once you have the resource ID, you'll want to use Amazon's EC2 API to access the tags. Since you said you're using Java, I would suggest the Using the AWS SDK amazon makes available. Within this SDK you'll find a method called describeTags (documentation). You can use a Resource ID as one of the filters to get the specific tags to your instance. Supported filters are
tag key
resource-id
resource-type
I suggest doing this retrieval at boot using something like cloud-init and caching the tags on your server for use later if necessary.