Develop Reference

How to get Datastore entity id from com.google.datastore.v1.Entity

I have written a code to fetch data from Google Datastore in my Google Cloud Dataflow program. I am able to fetch all fields of the entity except Id field which is autogenerated field. I have tried to use entity.getKey() but I am getting null.
Below is my code snippet,
Datastore datastore = DataflowDatastoreService.getDatastoreObject(null, null, null);
Query.Builder queryBuilder = Query.newBuilder();
Filter filter1 = Filter.newBuilder()
.setPropertyFilter(PropertyFilter.newBuilder() .setProperty(PropertyReference.newBuilder().setName("cId"))
.setOp(PropertyFilter.Operator.EQUAL)
.setValue(Value.newBuilder().setIntegerValue(1059438885900008L).build()).build()).build();
Filter filter2 = Filter.newBuilder()
.setPropertyFilter(PropertyFilter.newBuilder()
.setProperty(PropertyReference.newBuilder().setName("active"))
.setOp(PropertyFilter.Operator.EQUAL)
.setValue(Value.newBuilder().setBooleanValue(Boolean.TRUE).build()).build()).build();
Filter composeFilter = Filter.newBuilder().setCompositeFilter(CompositeFilter.newBuilder()
.addFilters(filter1).setOp(Operator.AND).addFilters(filter2).build()).build();
queryBuilder.addKind(KindExpression.newBuilder().setName("MyMaster").build());
queryBuilder.setFilter(composeFilter).build();
RunQueryRequest request = DataflowDatastoreService.makeRequest(queryBuilder.build(), null);
RunQueryResponse response = datastore.runQuery(request);
QueryResultBatch batch = response.getBatch();
List<EntityResult> entityResutls = batch.getEntityResultsList();
List<Entity> myEntities = new ArrayList<>();
Map<String, Value> entityMap = myEntities(0).getPropertiesMap();
In my code I am able to get all fields in entityMap key but I am not getting key, is there any other way through which I can fetch all the fields with Id.

Note: I'm not a java user, answer based on python experience
Indeed, entities returned in a regular query result do not contain the entity key/ID. Attempting to obtain that from the entity is rather inefficient - you need to reach to the datastore for each individual entity (not even looking at why that doesn't appear to be working for you).
If I need the entity keys/IDs I'd instead use keys-only queries - obtaining the keys, from which I can easily get:
the key IDs, locally, without making actual datastore calls (in python via key.id(), I don't know the java equivalent)
the entities via direct key lookup, which can be batched for efficiency.

entity.getKey().getPathList().get(0).getId()
This help me to achieve the result. Getting entity Id through getKey method.

Java Couchbase Querying to find a document's ID?

I'm new to couchbase. I'm using Java for this. I'm trying to remove a document from a bucket by looking up its ID with query parameters(assuming the ID is unknown).
Lets say I have a bucket called test-data. In that bucked I have a document with ID of 555 and Content of {"name":"bob","num":"10"}
I want to be able to remove that document by querying using 'name' and 'num'.
So far I have this (hardcoded):
String statement = "SELECT META(`test-data`).id from `test-data` WHERE name = \"bob\" and num = \"10\"";
N1qlQuery query = N1qlQuery.simple(statement);
N1qlQueryResult result = bucket.query(query);
List<N1qlQueryRow> row = result.allRows();
N1qlQueryRow res1 = row.get(0);
System.out.println(res1);
//output: {"id":"555"}
So I'm getting a json that has the document's ID in it. What would be the best way to extract that ID so that I can then remove the queryed document from the bucket using its ID? Am I doing to many steps? Is there a better way to extract the document's ID?
bucket.remove(docID)
Ideally I'd like to use something like a N1q1QueryResult to get this going but I'm not sure how to set that up.
N1qlQueryResult result = bucket.query(select("META.id").fromCurrentBucket().where((x("num").eq("\""+num+"\"")).and(x("name").eq("\""+name+"\""))));
But that isn't working at the moment.
Any help or direction would be appreciated. Thanks.

There might be a better way which is running this kind of query:
delete from `test-data` use keys '00000874a09e749ab6f199c0622c5cb0' returning raw META(`test-data`).id
or if your fields has index:
delete from `test-data` where name='bob' and num='10' returning raw META(`test-data`).id
This query deletes the specified document with given document key (which is meta.id) and returns document id of deleted document if it deletes any document. Returns empty if no documents deleted.
You can implement this query with couchbase sdk as follows:
Statement statement = deleteFrom("test-data")
.where(x("name").eq(s("bob")).and(x("num").eq(s("10"))))
.returningRaw(meta(i("test-data")).get("id"));
You can make this statement parameterized or just execute like that.

JPA CriteriaBuilder value like column

I'm trying to create a query using CriteriaBuilder where I need to have a predicate where the value of the predicate is like the value in the database.
Basically, I need to be able to do the following:
WHERE myTestValue LIKE columnValue
In native queries, it is an option to do that, but using the CriteriaBuilder or NamedQueries, it does not seem to work.
String myValue = "foo#bar.com";
cb.where(cb.like(myValue, root.get(Entity_.email));
Is there an option in JPA to do it like this? Or should I fall back to native queries?
EDIT
I need to be able to check if a given value matches a wildcard entry in database. So the database has a record %#bar.com%, and I need to check if my given value foo#bar.com matches to that record.

I think your params should be other way round:
cb.where(cb.like(root.get(Entity_.email),myValue);
Aditionally you may need to use add this to the second param:
cb.where(cb.like(root.get(Entity_.email),"%"+myValue+"%");

Chris found the answer. First I need to "generate" a parameter.
ParameterExpression<String> senderEmailParameter = cb.parameter(String.class, "senderEmailParameter");
Path<String> senderEmailPath = root.get(Entity_.senderEmail);
Predicate predEmail = cb.like(senderEmailParameter, senderEmailPath);
And then I need to fill the parameter in the query.
q.where(predEmail);
return em.createQuery(q).setParameter("senderEmailParameter", senderEmail).getSingleResult();
This works! Thanks Chris!

GAE datastore querying integer fields

I notice strange behavior when querying the GAE datastore. Under certain circumstances Filter does not work for integer fields. The following java code reproduces the problem:
log.info("start experiment");
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
int val = 777;
// create and store the first entity.
Entity testEntity1 = new Entity(KeyFactory.createKey("Test", "entity1"));
Object value = new Integer(val);
testEntity1.setProperty("field", value);
datastore.put(testEntity1);
// create the second entity by using BeanUtils.
Test test2 = new Test(); // just a regular bean with an int field
test2.setField(val);
Entity testEntity2 = new Entity(KeyFactory.createKey("Test", "entity2"));
Map<String, Object> description = BeanUtilsBean.getInstance().describe(test2);
for(Entry<String,Object> entry:description.entrySet()){
testEntity2.setProperty(entry.getKey(), entry.getValue());
}
datastore.put(testEntity2);
// now try to retrieve the entities from the database...
Filter equalFilter = new FilterPredicate("field", FilterOperator.EQUAL, val);
Query q = new Query("Test").setFilter(equalFilter);
Iterator<Entity> iter = datastore.prepare(q).asIterator();
while (iter.hasNext()) {
log.info("found entity: " + iter.next().getKey());
}
log.info("experiment finished");
the log looks like this:
INFO: start experiment
INFO: found entity: Test("entity1")
INFO: experiment finished
For some reason it only finds the first entity even though both entities are actually stored in the datastore and both 'field' values are 777 (I see it in the Datastore Viewer)! Why does it matter how the entity is created? I would like to use BeanUtils, because it is convenient.
The same problem occurs on the local devserver and when deployed to GAE.

Ok I found out what is going on. The "problem" is that for some reason BeanUtils transforms integers into strings. A string looks exactly the same in the datastore viewer but it is of course not the same. This pretty much fooled me. I should have studied the apache BeanUtils manual or something.

Have you given the datastore 1 second after writing before you query the data? Sometimes you don't have to (ancestor queries, perhaps) but other times you do. The GAE/J documentation will give full details.

The fact that the entities are created with BeanUtils is completely irrelevant. If the entities are in the datastore (you can see them in the viewer) and the field value is indexed (it does not show "unindexed" next to value in datastore viewer) then you can query for them using a filter. This works... its is the basic functionality of the datastore.
Given the entities are created and indexed, I suggest that Ian Marshalls suggestion is probably correct. To test this, go to the preferences for App Engine and un-tick "Enable local HRD support". This will ensure that when you write an Entity you can query for it immediately.
It is not important if you store an Integer or int or any other numeric value - they are all stored as a long value internally and when you read your value back you will get a Long (despite storing an Integer)

Order by another table with Liferay's DynamicQuery

I have a problem while I'm making a Dynamic Query in Liferay 6. I'm trying to make a query to order JournalArticles based on their view count. The view count is specified in another table (AssetEntry).
I'm stuck with this:
DynamicQuery query = DynamicQueryFactoryUtil.forClass(
JournalArticle.class, "articleParent", PortalClassLoaderUtil.getClassLoader());
//adding criterions
query.add(...);
DynamicQuery dq0 = DynamicQueryFactoryUtil.forClass(AssetEntry.class, "asset",
PortalClassLoaderUtil.getClassLoader())
.setProjection(ProjectionFactoryUtil.property("asset.classPK"))
.add(PropertyFactoryUtil.forName("asset.companyId")
.eqProperty("articleParent.companyId"))
.add(PropertyFactoryUtil.forName("asset.groupId")
.eqProperty("articleParent.groupId"));
query.add(PropertyFactoryUtil.forName("articleParent.resourcePrimKey").in(dq0))
.addOrder(OrderFactoryUtil.desc("asset.viewCount"));
With this I get an error message saying: could not resolve property: asset of: com.liferay.portlet.journal.model.impl.JournalArticleImpl.
If I remove the addOrder-call, this error disappears. How should I add the order statement so the main query is aware of asset.viewCount?

AssetEntryQuery assetEntryQuery = new AssetEntryQuery();
assetEntryQuery.setClassName(JournalArticle.class.getName());
assetEntryQuery.setXXX //adding criterions
assetEntryQuery.setOrderByCol1("viewCount");
List<AssetEntry> assetEntries = AssetEntryServiceUtil.getEntries(assetEntryQuery);

I am afraid that there is no direct way to do this with the DynamicQuery API.
I think you would need to use Service builder Finders i.e. Custom Query with Service builder.

You can't use dynamic query because there is no direct reference from JournalArticle entity to AssetEntry entity.
One possibility is to retrieve ordered ids of articles from the AssetEntry table (basically you dq0), then do another query and do the sorting programatically (you have the ids ordered).
Finally I think that this line
query.add(PropertyFactoryUtil.forName("articleParent.resourcePrimKey").in(dq0))
doesn't do what you think it does. resoucePrimKey is reference to resource table for permissions. You need to use id column.