Develop Reference

Query annotation not working with and / or operators

I'm trying to create custom MongoDB queries in Java with Spring
A simple query, like this, works fine
#Query("{'status' : ?0}")
Page<Project> filterProjects(String status, Pageable pageable);
But when I try a more complicated query with $and and $or, I don't get back any results
#Query(value = "{ '$and' : [ { '$or' : [ { 'project_title': {$regex:?0,$options:'i'}}, { 'project_description': {$regex:?0,$options:'i'}}, { 'project_short_name': {$regex:?0,$options:'i'}}]}, { 'status' : ?1}, { 'assignee' : ?2} ]}")
Page<Project> filterProjects(String search, String status, String assignee, Pageable pageable);
The raw mongodb query version of the above works fine
db.project.find( {
$and: [
{ $or: [ { 'project_title': {$regex: <search> ,$options:'i'}}, { 'project_description': {$regex: <search>,$options:'i'}}, { 'project_short_name': {$regex:<search>,$options:'i'}}]},
{ 'status' : <status>},
{ 'assignee' : <assignee>}
]
} )
Is there something wrong with the query in #Query or are these operations not supported at all, in #Query?

have you tried out removing the single quotes ('') around $and/$or in the value of your #Query annotation ?

Mongo Projection is not working for SubFields

I have a json object something like this. I want to exclude the field "placeOfBirth" from the response. For that I am using projection. But somehow it is working only for fileds but not for the subfields. So placeofBirth is never excluded but status is removed in the response.
Here is my code
Projection projectionExclude = Projection.of().exclude("subObject.placeOfBirth").exclude("status");
MorphiaCursor<T> cursor = datastore.aggregate(T.class)
.match(Filters.eq("about", id))
.project(projectionExclude).execute(T.class);
if(cursor != null && cursor.hasNext()){
result = cursor.toList().get(0);
}
Json data
{
"about: " "testing/123",
"subObject" : [
{
"about" : "subobject/123",
"placeOfBirth": {
"birth": ["Lisbon"]
}
}
],
"status" : "approved"
}
How can make this work? Is there some other way to achieve this?

This actually works for me on 2.2. Here's the test I'm running:
MongoCollection<Document> collection = getDocumentCollection(User.class);
collection.insertOne(parse("{'about': 'testing/123', 'subObject' : [ {'about' : 'subobject/123', 'placeOfBirth': {'birth': " +
"['Lisbon']}}],'status' : 'approved'}"));
Document next = getDs().aggregate(User.class)
.match(eq("about", "testing/123"))
.project(project()
.exclude("subObject.placeOfBirth")
.exclude("status"))
.execute(Document.class)
.next();
assertFalse(next.toJson().contains("placeOfBirth"));
assertFalse(next.toJson().contains("status"));

Improving mongo query Performance(using Sub query )

Two documents can have same IMAGE_CONTENT_INSTANCE_HANDLE and state can be BOOKED or RELEASED
but I want all image instances handles which are only RELEASED state,
Currently I am doing this by firing two queries it introduced performance issues.
{
"state" : "RELEASED"
}
with projection { "imageContentInstance.handle" : 1}
i am iterating through the result which is coming out from this query
and firing another query as below and excluding the handles which are also in BOOKED state from adding to the list.So i gets handles only in the RELEASED state
while (cursor.hasNext()) {
ICI ici = objectMapper.readValue(result, ICI_COLLECTION_TYPE_REF);
String result = JSON.serialize(cursor.next());
try {
queryDocument = { "imageContentInstance.handle" : ici.getImageContentInstance().getHandle() , "state" : "BOOKED"}
Document bookedDoc = iciDAO.findOne(queryDocument);
if (null != bookedDoc)
LOGGER.debug("Calling continue and skipping booked state ");
continue;
}
iciHandles.add(ici.getImageContentInstance().getHandle().toString());
LOGGER.debug("ImageInstanceHandle is added to the lisr and the lise is "+iciHandles.size());
}
I want to achieve this in a single mongo query as an example query written in sql to increase performance .I really appreciate your comments .
SELECT *
FROM ici i
WHERE i.state = 'RELEASED'
AND NOT EXISTS
(SELECT * FROM ici ic WHERE ic.handle = i.handle AND ic.state = 'BOOKED'
);
example :
Suppose the documents are as below
{
"_id" : ObjectId("58c9f524fa8cd6a517cf5ddf"),
"imageContentInstance" : {
"handle" : "ICI:1234",
"key" : null,
}
"instanceHandle" : "LCI:RNBM12",
"state" : "BOOKED",
}
{
"_id" : ObjectId("58c9f524fa8cd6a517cf5ddf"),
"imageContentInstance" : {
"handle" : "ICI:1234",
"key" : null,
}
"instanceHandle" : "LCI:RNBM13",
"state" : "RELEASED",
}
{
"_id" : ObjectId("58c9f524fa8cd6a517cf5ddf"),
"imageContentInstance" : {
"handle" : "ICI:456",
"key" : null,
}
"instanceHandle" : "LCI:RNBM14",
"state" : "RELEASED"
}
My query should return the handle of the last document alone .ie, document with the status only with the RELEASED status .i am stuck, I really appreciate your ideas to improve this.

From Your question,i understand that you want all state ='Released' ans state!= 'BOOKED' which i think you have written little incorrect.
MongoDB query:
db.inventory.find({'state' : 'RELEASED'}})
Also go through mognodb docs
I hope it will help.I am also new to mongodb,if there is an error please make it correct.

How to enable query logging in Spring-data-elasticsearch

I use spring-data-elasticsearch framework to get query result from elasticsearch server, the java code like this:
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery()).withSearchType(SearchType.COUNT)
.addAggregation(new MinBuilder("min_createDate").field("createDate"))
.build();
List<Entity> list = template.queryForList(searchQuery, Entity.class);
While how can I know the raw http query sent to elasticssearch server?
How can I enable the logging, I tried add log4j, but it seems the spring-data-elasticsearch doesn't log the query.

After digging through the spring data code i found this helpful little logger called "tracer" (name not very unique)
By setting the following in application.properties
logging.level.tracer=TRACE
It will print out a full curl statement for the request along with full JSON the response from Elasticsearch.

This one is quite old, but I'd still like to share the solution that worked for me. To log Spring Data Elasticsearch queries executed through the Repository, you need to enable DEBUG logging for the package org.springframework.data.elasticsearch.core.*, e.g. as follows:
logging:
level:
org:
springframework:
data:
elasticsearch:
core: DEBUG
After that, queries will appear in logs:
{
"from" : 0,
"size" : 1,
"query" : {
"bool" : {
"should" : [ {
"query_string" : {
"query" : "John Doe",
"fields" : [ "entityName" ],
"default_operator" : "and"
}
}, {
"query_string" : {
"query" : "John Doe",
"fields" : [ "alias" ],
"default_operator" : "and"
}
} ]
}
},
"post_filter" : {
"bool" : { }
}
}
One would expect an elegant solution similar to JPA, but it seems that it doesn't simply exist.
Tested with Spring Boot 1.4.0 and Spring Data Elasticsearch 1.7.3.

If you are using spring boot you can set the following in your application.properties:
logging.level.org.elasticsearch.index.search.slowlog.query=INFO
spring.data.elasticsearch.properties.index.search.slowlog.threshold.query.info=1ms

I don't have an answer for Spring Data Elasticsearch, but in ES itself you can bump up the default settings for slow query logging and see all the queries in the slow log. More details about slow log here.
As to how to change the thresholds, a command like this should be used:
PUT /_settings
{
"index.search.slowlog.threshold.query.info": "1ms"
}
1ms is kindof the smallest value you can set.

This works on Spring Boot 2.3.3.RELEASE
logging.level.org.springframework.data.elasticsearch.client.WIRE=trace

I encountered the same problem, In the ElasticsearchTemplate only a few method have log debug level, E.g:
public <T> Page<T> queryForPage(CriteriaQuery criteriaQuery, Class<T> clazz) {
QueryBuilder elasticsearchQuery = new CriteriaQueryProcessor().createQueryFromCriteria(criteriaQuery.getCriteria());
QueryBuilder elasticsearchFilter = new CriteriaFilterProcessor().createFilterFromCriteria(criteriaQuery.getCriteria());
SearchRequestBuilder searchRequestBuilder = prepareSearch(criteriaQuery, clazz);
if (elasticsearchQuery != null) {
searchRequestBuilder.setQuery(elasticsearchQuery);
} else {
searchRequestBuilder.setQuery(QueryBuilders.matchAllQuery());
}
if (criteriaQuery.getMinScore() > 0) {
searchRequestBuilder.setMinScore(criteriaQuery.getMinScore());
}
if (elasticsearchFilter != null)
searchRequestBuilder.setPostFilter(elasticsearchFilter);
if (logger.isDebugEnabled()) {
logger.debug("doSearch query:\n" + searchRequestBuilder.toString());
}
SearchResponse response = getSearchResponse(searchRequestBuilder
.execute());
return resultsMapper.mapResults(response, clazz, criteriaQuery.getPageable());
}

#2280258 is correct, and here comes the official doc:
https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/index.html#elasticsearch.clients.logging
<logger name="org.springframework.data.elasticsearch.client.WIRE" level="trace"/>
Here is the reason: in org.springframework.data.elasticsearch.client.ClientLogger, spring data elasticsearch creates a logger named "org.springframework.data.elasticsearch.client.WIRE"：
private static final Logger WIRE_LOGGER = LoggerFactory
.getLogger("org.springframework.data.elasticsearch.client.WIRE");

Just to add my two cents to #AndreiStefan: Now you can set 0ms instead of 1ms. It seems that some very fast queries can be captured using this method.
Simply do:
PUT /_settings
{
"index.search.slowlog.threshold.query.info": "0ms"
}

Spring data mongodb removes a positional operator from "$unset" update query part

I have a collection of users:
> db.users.find().pretty()
{
"_id" : ObjectId("544ab933e4b099c3cfb62e12"),
"token" : "8c9f8cf4-1689-48ab-bf53-ee071a377f60",
"categories" : [
DBRef("cue_categories", ObjectId("544ab933e4b099c3cfb62e10")),
DBRef("cue_categories", ObjectId("544ab933e4b099c3cfb62e11"))
]
}
I want to find all users who have (let's say) ObjectId("544ab933e4b099c3cfb62e10") category and remove it (because this category was deleted and I don't want users to refer to it anymore).
The valid query to do it in JSON format would be:
db.users.update({
categories:{
$in:[
DBRef("cue_categories", ObjectId("544ab933e4b099c3cfb62e10"))
]
}
},
{
$unset:{
"categories.$":true
}
})
Here's a Spring mongodb query:
Query query = new Query();
query.addCriteria(Criteria.where("categories.$id").in(categoryIds));
Update update = new Update();
update.unset("categories.$");
operations.updateMulti(query, update, User.class);
In order to make an appropriate DB reference I have to provide a list of category IDs, each category ID (in categoryIds) is an instance of org.bson.types.ObjectId.
The problem is that the result query turns out to be without a positional operator:
DEBUG o.s.data.mongodb.core.MongoTemplate - Calling update using
query: { "categories.$id" : { "$in" : [ { "$oid" :
"544ab933e4b099c3cfb62e10"}]}} and update: { "$unset" : { "categories"
: 1}} in collection: users
So the update part must be { "$unset" : { "categories.$" : 1}}
P.S.
I managed to get around by falling back to the plain Java driver use
DBObject query = new BasicDBObject("categories.$id", new BasicDBObject("$in", categoryIds));
DBObject update = new BasicDBObject("$unset", new BasicDBObject("categories.$", true));
operations.getCollection("users").updateMulti(query, update);
But my question still remains open!
P.S.S.
My case is very similar to Update Array Field Using Positional Operator ($) Does Not Work bug and looks like it was fixed for versions 1.4.1 and 1.5. That being said I use spring-data-mongodb version 1.5.1. And I'm confused. Does anybody have a clue?

You can not use positional $ operator with unset as per MongoDB documentation. It will set the value as Null. https://docs.mongodb.com/manual/reference/operator/update/positional/