I have an application in which I want to overwrite an individual entity. This is how I originally create entity logs:
Entity log = new Entity("Log", "Logkey");
String property1 = req.getParameter("property1");
String property2 = req.getParameter("property2");
log.setProperty("property1", property1);
log.setProperty("property2", property2);
datastore.put(log);
Here is how entity logs are retrieved to be overwritten:
Query query = new Query("Log", "Logkey")
.setFilter(timeStampFilter);
List<Entity> logs = datastore.prepare(query).asList(FetchOptions.Builder.withLimit(1));
request.setAttribute("logs", logs);
and sent to a jsp form page as value="${log.properties.property1}" where they should be overwritten. This entry is then sent to a second servlet with the POST method and requested as parameters as in the earlier code but saved as a new entity with the same kind:
Entity edit_log = new Entity("Log", "Logkey");
String property1 = req.getParameter("property1");
String property2 = req.getParameter("property2");
edit_log.setProperty("property1", property1);
For rewriting and existing entity, after retreiving a specific log by timestamp, you can get the key of this log using getKey() method and then create an entity with this key and the new details. Now when you put this new entity to the datastore it will replace the earlier one with the same key
With the code you've written, you only have a single Log entity in your datastore with the key "Logkey" that you are constantly overwriting.
If you're using some other code to retrieve entities and rewrite them, then you'll need to show that other code. Otherwise, this question is poorly written, because the code given is already describing what you want to do (always overwrite the same entity).
If you have code elsewhere creating/saving entities, it's best to show that too.
Edit: It looks like you end up creating a nested entity with the data from the old entity in a new entity with the same key. It's far easier just to reuse the entity you received from the query.
log = logs.get(0)
log.setProperty("property1", req.getParameter("property1");
log.setProperty("property2", req.getParameter("property2");
datastore.put(log);
Furthermore, since you actually know the key ("Logkey"), you don't need to issue a datastore query, you can just fetch the entity by key - which is good because you get around eventual-consistency behavior.
If your new entity has the same key as your original one, then when you store it it will override the old entity.
Related
I have written a code to fetch data from Google Datastore in my Google Cloud Dataflow program. I am able to fetch all fields of the entity except Id field which is autogenerated field. I have tried to use entity.getKey() but I am getting null.
Below is my code snippet,
Datastore datastore = DataflowDatastoreService.getDatastoreObject(null, null, null);
Query.Builder queryBuilder = Query.newBuilder();
Filter filter1 = Filter.newBuilder()
.setPropertyFilter(PropertyFilter.newBuilder() .setProperty(PropertyReference.newBuilder().setName("cId"))
.setOp(PropertyFilter.Operator.EQUAL)
.setValue(Value.newBuilder().setIntegerValue(1059438885900008L).build()).build()).build();
Filter filter2 = Filter.newBuilder()
.setPropertyFilter(PropertyFilter.newBuilder()
.setProperty(PropertyReference.newBuilder().setName("active"))
.setOp(PropertyFilter.Operator.EQUAL)
.setValue(Value.newBuilder().setBooleanValue(Boolean.TRUE).build()).build()).build();
Filter composeFilter = Filter.newBuilder().setCompositeFilter(CompositeFilter.newBuilder()
.addFilters(filter1).setOp(Operator.AND).addFilters(filter2).build()).build();
queryBuilder.addKind(KindExpression.newBuilder().setName("MyMaster").build());
queryBuilder.setFilter(composeFilter).build();
RunQueryRequest request = DataflowDatastoreService.makeRequest(queryBuilder.build(), null);
RunQueryResponse response = datastore.runQuery(request);
QueryResultBatch batch = response.getBatch();
List<EntityResult> entityResutls = batch.getEntityResultsList();
List<Entity> myEntities = new ArrayList<>();
Map<String, Value> entityMap = myEntities(0).getPropertiesMap();
In my code I am able to get all fields in entityMap key but I am not getting key, is there any other way through which I can fetch all the fields with Id.
Note: I'm not a java user, answer based on python experience
Indeed, entities returned in a regular query result do not contain the entity key/ID. Attempting to obtain that from the entity is rather inefficient - you need to reach to the datastore for each individual entity (not even looking at why that doesn't appear to be working for you).
If I need the entity keys/IDs I'd instead use keys-only queries - obtaining the keys, from which I can easily get:
the key IDs, locally, without making actual datastore calls (in python via key.id(), I don't know the java equivalent)
the entities via direct key lookup, which can be batched for efficiency.
entity.getKey().getPathList().get(0).getId()
This help me to achieve the result. Getting entity Id through getKey method.
Which of these is the correct / preferred way to create Datastore Entity:
// First, create a fake email for our fake user
final String email = "me#fake.com";
// Now, create a 'key' for that user using the email
final Key userKey =
datastore.newKeyFactory().kind("user").newKey(email);
// Now create a entity using that key adn add some fields to it
final Entity newUser =
Entity
.builder(userKey)
.set("name", "Faker")
.set("email", email)
.build();
or like it's done in DatastoreWordCount example ?
Entity from the first example is com.google.cloud.datastore.Entity.
Entity from the second example is com.google.datastore.v1.Entity.
I have code using com.google.cloud.datastore.Entity and I don't know how to store it Datastore as a part of Dataflow pipeline, since all code examples I found online suggest:
com.google.cloud.dataflow.sdk.io.datastore.DatastoreIO.v1().write()
but it's only working with com.google.datastore.v1.Entity.
I'm using com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:1.7.0.
You have analyzed the situation well - Dataflow's DatastoreIO.v1() works with com.google.datastore.v1.Entity.
AppEngine Datastore uses EmbeddedEntity for nested entities. But I do not understand the purpose of Key in the EmbeddedEntity? When it can be used?
For example:
Entity book = new Entity("Book");
book.setProperty("name", "Hollow Man");
book.setProperty("year", 2000);
EmbeddedEntity author = new EmbeddedEntity();
author.setKey(KeyFactory.createKey("AUTHOR", 123));
author.setProperty("name", "Den Simons");
book.setProperty("author", author);
datastore.put(book);
Later I can fetch the book and to get the author from it. The author will have the key set, meaning the key is persisted. But is it possible to query with this key? The EmbeddedEntity javadoc says:
It is not queryable when stored in the datastore.
So what would be the purpose of Key in EmbeddedEntity?
The example at https://cloud.google.com/appengine/docs/java/datastore/entities#Java_Embedded_entities shows how you can optionally set, as your embedded entity's key, the key to another entity of the same type that's not embedded, so that later you can "recover the original entity from the embedded entity". Not a frequent need, I guess, but handy when you do need it, perhaps!
The relevant code snippet from there is:
// Entity contactInfo = /*...*/;
EmbeddedEntity embeddedContactInfo = new EmbeddedEntity();
Key infoKey;
infoKey = contactInfo.getKey();
embeddedContactInfo.setKey(infoKey);
Folks,
I am using Play 1.2.5. The database is Oracle 10g and I am using an existing table for my application.
I am generating a unique key like this:
#GeneratedValue(strategy=GenerationType.SEQUENCE)
public int transactionId;
When I use the below code, the transactionId is generated and saved successfully ib the database:
transactionDetails.save();
But I am not able to get the uniquely generated transactionId once the save operation performs successfully. The save method returns a type <JPABase>. Then how can I retrieve the transactionId after a successful save operation from the <JPABase> ?
Note: I don't want to make another DB hit for fetching the transactionId because I believe that there might be some way to retrieve it for a successful save operation.
Please let me know about this.
Thanks,
You should be able to write
TransactionDetails savedDetails = transactionDetails.save();
save() method actually is not returning JPABase, it is declared as so you will get the saved entity
I notice strange behavior when querying the GAE datastore. Under certain circumstances Filter does not work for integer fields. The following java code reproduces the problem:
log.info("start experiment");
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
int val = 777;
// create and store the first entity.
Entity testEntity1 = new Entity(KeyFactory.createKey("Test", "entity1"));
Object value = new Integer(val);
testEntity1.setProperty("field", value);
datastore.put(testEntity1);
// create the second entity by using BeanUtils.
Test test2 = new Test(); // just a regular bean with an int field
test2.setField(val);
Entity testEntity2 = new Entity(KeyFactory.createKey("Test", "entity2"));
Map<String, Object> description = BeanUtilsBean.getInstance().describe(test2);
for(Entry<String,Object> entry:description.entrySet()){
testEntity2.setProperty(entry.getKey(), entry.getValue());
}
datastore.put(testEntity2);
// now try to retrieve the entities from the database...
Filter equalFilter = new FilterPredicate("field", FilterOperator.EQUAL, val);
Query q = new Query("Test").setFilter(equalFilter);
Iterator<Entity> iter = datastore.prepare(q).asIterator();
while (iter.hasNext()) {
log.info("found entity: " + iter.next().getKey());
}
log.info("experiment finished");
the log looks like this:
INFO: start experiment
INFO: found entity: Test("entity1")
INFO: experiment finished
For some reason it only finds the first entity even though both entities are actually stored in the datastore and both 'field' values are 777 (I see it in the Datastore Viewer)! Why does it matter how the entity is created? I would like to use BeanUtils, because it is convenient.
The same problem occurs on the local devserver and when deployed to GAE.
Ok I found out what is going on. The "problem" is that for some reason BeanUtils transforms integers into strings. A string looks exactly the same in the datastore viewer but it is of course not the same. This pretty much fooled me. I should have studied the apache BeanUtils manual or something.
Have you given the datastore 1 second after writing before you query the data? Sometimes you don't have to (ancestor queries, perhaps) but other times you do. The GAE/J documentation will give full details.
The fact that the entities are created with BeanUtils is completely irrelevant. If the entities are in the datastore (you can see them in the viewer) and the field value is indexed (it does not show "unindexed" next to value in datastore viewer) then you can query for them using a filter. This works... its is the basic functionality of the datastore.
Given the entities are created and indexed, I suggest that Ian Marshalls suggestion is probably correct. To test this, go to the preferences for App Engine and un-tick "Enable local HRD support". This will ensure that when you write an Entity you can query for it immediately.
It is not important if you store an Integer or int or any other numeric value - they are all stored as a long value internally and when you read your value back you will get a Long (despite storing an Integer)