AppEngine Datastore uses EmbeddedEntity for nested entities. But I do not understand the purpose of Key in the EmbeddedEntity? When it can be used?
For example:
Entity book = new Entity("Book");
book.setProperty("name", "Hollow Man");
book.setProperty("year", 2000);
EmbeddedEntity author = new EmbeddedEntity();
author.setKey(KeyFactory.createKey("AUTHOR", 123));
author.setProperty("name", "Den Simons");
book.setProperty("author", author);
datastore.put(book);
Later I can fetch the book and to get the author from it. The author will have the key set, meaning the key is persisted. But is it possible to query with this key? The EmbeddedEntity javadoc says:
It is not queryable when stored in the datastore.
So what would be the purpose of Key in EmbeddedEntity?
The example at https://cloud.google.com/appengine/docs/java/datastore/entities#Java_Embedded_entities shows how you can optionally set, as your embedded entity's key, the key to another entity of the same type that's not embedded, so that later you can "recover the original entity from the embedded entity". Not a frequent need, I guess, but handy when you do need it, perhaps!
The relevant code snippet from there is:
// Entity contactInfo = /*...*/;
EmbeddedEntity embeddedContactInfo = new EmbeddedEntity();
Key infoKey;
infoKey = contactInfo.getKey();
embeddedContactInfo.setKey(infoKey);
Related
I've been trying to use insert...returning in MySQL with the DSL-based table definition (I'm not using the code generation) and my returned record is always null. Based on reading, I need to specify the identify column in the table definition, but I have no idea how!
Record recordKey = create.insertInto(table("modulerecords"),
field("id"),
field("module_id"),
field("created_date"),
field("created_by"),
field("state"),
field("tag_id"),
field("start_time",Timestamp.class),
field("kill_time", Timestamp.class),
field("feed_guid")
)
.values(null, moduleId, currentTimestamp(),
userId, state, tagId,
new Timestamp(startTime),
new Timestamp(killTime), feedGuid)
.returning(field("id"))
.fetchOne();
The field "id" is auto_increment primary key in the database, but recordKey is always null.
As of jOOQ 3.14, this is possible by specifying the field's datatype as being an identity, which can be done using SQLDataType.INTEGER.identity(true).
So for example, if you had a table with an auto-generating integer id and a string name, you would call:
int id = DSL.using(connection, MYSQL_5_7)
.insertInto(
table("myTable"),
field("name", String.class))
.values("John Smith")
.returning(field("id", SQLDataType.INTEGER.identity(true)))
.fetchAny(field("id", Integer.class))
So for your example, you would do
Record recordKey = create.insertInto(table("modulerecords"),
field("id"),
field("module_id"),
field("created_date"),
field("created_by"),
field("state"),
field("tag_id"),
field("start_time",Timestamp.class),
field("kill_time", Timestamp.class),
field("feed_guid")
)
.values(null, moduleId, currentTimestamp(),
userId, state, tagId,
new Timestamp(startTime),
new Timestamp(killTime), feedGuid)
.returning(field("id", SQLDataType.INTEGER.identity(true)))
.fetchOne();
See this Github comment for more background.
It is highly recommended you use the code generator to provide all the meta information to the DSL API. You can, of course, not use the code generator and still use the internal APIs that the code generator would otherwise use. Instead of creaating your table and field references using the plain SQL API, you'd have to create a TableImpl subclass and override / implement all the relevant methods.
Or, you just use the code generator.
I have written a code to fetch data from Google Datastore in my Google Cloud Dataflow program. I am able to fetch all fields of the entity except Id field which is autogenerated field. I have tried to use entity.getKey() but I am getting null.
Below is my code snippet,
Datastore datastore = DataflowDatastoreService.getDatastoreObject(null, null, null);
Query.Builder queryBuilder = Query.newBuilder();
Filter filter1 = Filter.newBuilder()
.setPropertyFilter(PropertyFilter.newBuilder() .setProperty(PropertyReference.newBuilder().setName("cId"))
.setOp(PropertyFilter.Operator.EQUAL)
.setValue(Value.newBuilder().setIntegerValue(1059438885900008L).build()).build()).build();
Filter filter2 = Filter.newBuilder()
.setPropertyFilter(PropertyFilter.newBuilder()
.setProperty(PropertyReference.newBuilder().setName("active"))
.setOp(PropertyFilter.Operator.EQUAL)
.setValue(Value.newBuilder().setBooleanValue(Boolean.TRUE).build()).build()).build();
Filter composeFilter = Filter.newBuilder().setCompositeFilter(CompositeFilter.newBuilder()
.addFilters(filter1).setOp(Operator.AND).addFilters(filter2).build()).build();
queryBuilder.addKind(KindExpression.newBuilder().setName("MyMaster").build());
queryBuilder.setFilter(composeFilter).build();
RunQueryRequest request = DataflowDatastoreService.makeRequest(queryBuilder.build(), null);
RunQueryResponse response = datastore.runQuery(request);
QueryResultBatch batch = response.getBatch();
List<EntityResult> entityResutls = batch.getEntityResultsList();
List<Entity> myEntities = new ArrayList<>();
Map<String, Value> entityMap = myEntities(0).getPropertiesMap();
In my code I am able to get all fields in entityMap key but I am not getting key, is there any other way through which I can fetch all the fields with Id.
Note: I'm not a java user, answer based on python experience
Indeed, entities returned in a regular query result do not contain the entity key/ID. Attempting to obtain that from the entity is rather inefficient - you need to reach to the datastore for each individual entity (not even looking at why that doesn't appear to be working for you).
If I need the entity keys/IDs I'd instead use keys-only queries - obtaining the keys, from which I can easily get:
the key IDs, locally, without making actual datastore calls (in python via key.id(), I don't know the java equivalent)
the entities via direct key lookup, which can be batched for efficiency.
entity.getKey().getPathList().get(0).getId()
This help me to achieve the result. Getting entity Id through getKey method.
I have an application in which I want to overwrite an individual entity. This is how I originally create entity logs:
Entity log = new Entity("Log", "Logkey");
String property1 = req.getParameter("property1");
String property2 = req.getParameter("property2");
log.setProperty("property1", property1);
log.setProperty("property2", property2);
datastore.put(log);
Here is how entity logs are retrieved to be overwritten:
Query query = new Query("Log", "Logkey")
.setFilter(timeStampFilter);
List<Entity> logs = datastore.prepare(query).asList(FetchOptions.Builder.withLimit(1));
request.setAttribute("logs", logs);
and sent to a jsp form page as value="${log.properties.property1}" where they should be overwritten. This entry is then sent to a second servlet with the POST method and requested as parameters as in the earlier code but saved as a new entity with the same kind:
Entity edit_log = new Entity("Log", "Logkey");
String property1 = req.getParameter("property1");
String property2 = req.getParameter("property2");
edit_log.setProperty("property1", property1);
For rewriting and existing entity, after retreiving a specific log by timestamp, you can get the key of this log using getKey() method and then create an entity with this key and the new details. Now when you put this new entity to the datastore it will replace the earlier one with the same key
With the code you've written, you only have a single Log entity in your datastore with the key "Logkey" that you are constantly overwriting.
If you're using some other code to retrieve entities and rewrite them, then you'll need to show that other code. Otherwise, this question is poorly written, because the code given is already describing what you want to do (always overwrite the same entity).
If you have code elsewhere creating/saving entities, it's best to show that too.
Edit: It looks like you end up creating a nested entity with the data from the old entity in a new entity with the same key. It's far easier just to reuse the entity you received from the query.
log = logs.get(0)
log.setProperty("property1", req.getParameter("property1");
log.setProperty("property2", req.getParameter("property2");
datastore.put(log);
Furthermore, since you actually know the key ("Logkey"), you don't need to issue a datastore query, you can just fetch the entity by key - which is good because you get around eventual-consistency behavior.
If your new entity has the same key as your original one, then when you store it it will override the old entity.
I'm working with the low-level datastore API. I've created an entity like this:
Entity entity = new Entity("Error");
entity.setProperty("description", "foo");
In the datastore viewer, I ses this:
Key Write Ops ID/Name description
----------------------------------------------
ahN0c... 4 259 foo
So the ID/Name field will be generated for me automatically since I'm not supplying anything in the Entity constructor. It generates an "ID" instead of a "Name", which is a number rather than an opaque string (like the "Key" value).
Is there a way to have the datastore generate a random "Name" instead of an "ID" for the Entity's "ID/Name" field?
I ask because if I share this ID with third parties, they could start to figure out roughly how many Error instances I have in my system. I'd rather give them an opaque string for the lookup ID, similar to what's in the auto-generated "Key" field. But I don't see a way to do this.
Thanks
For a similar task I used UUID to create a random string.
String uuid = UUID.randomUUID().toString();
You can use com.google.appengine.api.datastore.KeyFactory, combining the answer from #Devolus, it would look like
final Key key = KeyFactory.createKey("Error", UUID.randomUUID().toString());
final Entity e = new Entity(key);
You could even pass around the String representation of your Entitie's key via KeyFactory.keyToString(key) , may be after an encrypting depending on your security needs.
I notice strange behavior when querying the GAE datastore. Under certain circumstances Filter does not work for integer fields. The following java code reproduces the problem:
log.info("start experiment");
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
int val = 777;
// create and store the first entity.
Entity testEntity1 = new Entity(KeyFactory.createKey("Test", "entity1"));
Object value = new Integer(val);
testEntity1.setProperty("field", value);
datastore.put(testEntity1);
// create the second entity by using BeanUtils.
Test test2 = new Test(); // just a regular bean with an int field
test2.setField(val);
Entity testEntity2 = new Entity(KeyFactory.createKey("Test", "entity2"));
Map<String, Object> description = BeanUtilsBean.getInstance().describe(test2);
for(Entry<String,Object> entry:description.entrySet()){
testEntity2.setProperty(entry.getKey(), entry.getValue());
}
datastore.put(testEntity2);
// now try to retrieve the entities from the database...
Filter equalFilter = new FilterPredicate("field", FilterOperator.EQUAL, val);
Query q = new Query("Test").setFilter(equalFilter);
Iterator<Entity> iter = datastore.prepare(q).asIterator();
while (iter.hasNext()) {
log.info("found entity: " + iter.next().getKey());
}
log.info("experiment finished");
the log looks like this:
INFO: start experiment
INFO: found entity: Test("entity1")
INFO: experiment finished
For some reason it only finds the first entity even though both entities are actually stored in the datastore and both 'field' values are 777 (I see it in the Datastore Viewer)! Why does it matter how the entity is created? I would like to use BeanUtils, because it is convenient.
The same problem occurs on the local devserver and when deployed to GAE.
Ok I found out what is going on. The "problem" is that for some reason BeanUtils transforms integers into strings. A string looks exactly the same in the datastore viewer but it is of course not the same. This pretty much fooled me. I should have studied the apache BeanUtils manual or something.
Have you given the datastore 1 second after writing before you query the data? Sometimes you don't have to (ancestor queries, perhaps) but other times you do. The GAE/J documentation will give full details.
The fact that the entities are created with BeanUtils is completely irrelevant. If the entities are in the datastore (you can see them in the viewer) and the field value is indexed (it does not show "unindexed" next to value in datastore viewer) then you can query for them using a filter. This works... its is the basic functionality of the datastore.
Given the entities are created and indexed, I suggest that Ian Marshalls suggestion is probably correct. To test this, go to the preferences for App Engine and un-tick "Enable local HRD support". This will ensure that when you write an Entity you can query for it immediately.
It is not important if you store an Integer or int or any other numeric value - they are all stored as a long value internally and when you read your value back you will get a Long (despite storing an Integer)