Develop Reference

Spring Data MongoDb elemMatch criteria matching all search strings

I'm having an issue with custom Spring Data queries with MongoDb and Java. I'm attempting to implement a flexible search functionality against most of the fields of the document.
This document represents a person, and it contains a set of addresses embedded in it; the address has a field that is a set of strings that are the 'street address lines'.
I started with Query By Example, and this works for the single fields. but doesn't work for other types - such as this set of strings. For these, I'm building custom criteria.
The search criteria includes a set of street lines that I would like to match against the document's lines. If every line in the search is found in the document, the criteria should be considered matching.
I've tried using elemMatch, but this doesn't quite work like I want:
addressCriteriaList.add(Criteria.where("streetAddressLines").elemMatch(new Criteria().in(addressSearch.getStreetAddressLines())));
This seems to match if only ONE line in the document matches the search. If I have the following document:
"streetAddressLines": [ "123 Main Street", "Apt 1" ]
and the search looks like this:
"streetAddressLines": [ "123 Main Street", "Apt 2" ]
the elemMatch succeeds, but that's not what i want.
I've also tried looping through each of the search lines, trying an elemMatch to see if each is in the document:
var addressLinesCriteriaList = new Array<Criteria>();
var streetAddressLines = address.getStreetAddressLines();
streetAddressLines.forEach(l -> addressLinesCriteriaList.add(Criteria.where("streetAddressLines").elemMatch(new Criteria().is(l))))
var matchCriteria = new Criteria.andOperator(addressLinesCriteriaList);
This doesn't seem to work. I have done some experimenting, and it may be that this doesn't seem to work: new Criteria().is(l)
I tried this, and this DOES seem to work, but I would think that it's really inefficient to create a collection for each search line:
streetAddressLines.forEach(l ->
{
var list = new ArrayList<String>();
list.add(l);
addressCriteriaList.add(Criteria.where("streetAddressLines").elemMatch(new Criteria().in(l)));
});
So I don't know exactly what's going on - does anyone have any ideas of what I'm doing wrong? Thanks in advance.

You need to use the $all operator or the all method of Criteria class. Something along these lines:
addressCriteriaList.add(Criteria.where("streetAddressLines").all(addressSearch.getStreetAddressLines()));
If addressSearch.getStreetAddressLines returns a list, try this:
addressCriteriaList.add(Criteria.where("streetAddressLines").all(addressSearch.getStreetAddressLines().toArray(new String[0])));

How can I add a new Field (key with value) below another Field (key with value)?

Hello I am trying to set a new key below or above another key (field) just to make the documents a little more organized but when I try to use $addField or $set in Aggregations in MongoDB Compass it adds the new key at the end of the document, for example:
I want to add the key "amazing" above the key "something" with value "0",
I use that:
{
"name": "$name",
"city": "$city",
"amazing": "0",
"something": "$something",
"something2": "$something2",
}
but when I try to do that the preview of the compass shows me that the key "amazing" will be added but at the end of the document, below "something2".
If there is a way to do that in Java it would be cool too, please, thanks in advance.

As #Takis mentioned with the answer here, you can use projections to specify the fields and the order of them. I verified it works very well in Mongosh.
As for java implementation, you may refer to this post which has a an example to help you.
https://www.mongodb.com/docs/drivers/java/sync/current/fundamentals/builders/projections/
Code snippet:
Bson filter = Filters.empty();
Bson projection = include("year", "type");
collection.find(filter).projection(projection).forEach(doc -> System.out.println(doc.toJson()));

Java Mongo - findAndReplace returns document dates in subkeys

I have a slight problem with return data from Mongo find() vs a findOneAndReplace().
First, to know I'm working on an API that queries Mongo and returns data in JSON format.
Problem I'm having is that if I do a findOneAndReplace() to update and return the modified document, much like so:
// javaDocument() is just a org.bson.Document
var modDoc = jobsCollection.findOneAndReplace( javaDocument({"_id": jobid}), javaDocument(jobData), foarOptions );
if(isDefined("modDoc")) {
sReturn.DATA = deserializeJSON(modDoc.toJSON());
}
the dates set in my document returns within subkeys named "$date", which I do not want:
It really should read:
"created_at": "2022-02-03T10:15:01.634Z"
Doing a simple jobsCollection.find() however, seems to return me the date appropriately, like so:
var data= [];
jobsCollection.find( javaDocument({}) ).into(data);
sReturn.DATA = data;
What am I missing here ? I could simple copy the "$date" key and fix the structure, but I don't always know where in the structure I'll have dates.... Is there a way I could have this properly returned with FindOneAndReplace() ? I'm thinking in might have something to do with the modDoc.toJSON() but haven't quite found my answer yet...
Thanks for your time. Cheers! Pat

How to index new document in elasticsearch using term_vector?

I'm trying to implement application that get document from MongoDB and insert it to ElasticSearch. Here is a piece of code that should insert document to the ElasticSearch index:
final Document o = (Document) document.get("o"); // this is where object lives
client.prepareIndex(index, mapping, id.toString())
.setSource(o.toJson())
.execute().actionGet();
And finally I get this error:
java.lang.IllegalArgumentException: Mapper for [title] conflicts with existing mapping in other types:
[mapper [title] has different [store_term_vector] values, mapper [title] has different [store_term_vector_offsets] values, mapper [title] has different [store_term_vector_positions] values, mapper [title] has different [store_term_vector_payloads] values]
at org.elasticsearch.index.mapper.FieldTypeLookup.checkCompatibility(FieldTypeLookup.java:117)
at org.elasticsearch.index.mapper.MapperService.checkNewMappersCompatibility(MapperService.java:368)
at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:319)
I'v tried to remove index completely using XDELETE and recreate using XPUT and error remain.
Here is how my index settings look like:
{
"msg": {
"mappings": {
"Message": {
"properties": {
"title": {
"type": "string",
"term_vector": "with_positions_offsets_payloads",
"analyzer": "russian"
}
}
}
}
}
}
However if I remove this term_vector part from index settings the code is inserts new document successfully.
Can someone explain me what is the problem? The same problem occur when I'm trying to use mongo-connector. If settings contain term_vector part for title field => mongo-connector fails with same Exception. And it works well without term_vector.

Are you sure you are using the correct term_vector value? I am only aware of five valid values for that attribute, as listed in the documentation:
Possible values are no, yes, with_offsets, with_positions, with_positions_offsets. Defaults to no.
I would suggest trying a different term_vector such as with_positions_offsets to see if you get the results you're expecting.

I hope my answer will help someone else.
The problem was that I have another mapping in the same index that also have field title. You have to update all other mappings to use same settings.

Elasticsearch similar documents in Java

I'm doing a website (an auction website) using java. I have one page to show the product in auction and I want to show 10 similar products.
To perform the search I'm using elasticsearch (by using the elasticsearch java implementation dadoonet).
One requirement I have is to show only the 10 similar documents that has date > now.
I say the elasticsearch documentation and I found the query "More like this" but first I'm not getting this to work using:
new MoreLikeThisRequest("auction").searchSize(size).id(productId + "").fields(new String[] { "name", "description", "brand" }).type("string");
Because is always showing the error:
org.elasticsearch.index.engine.DocumentMissingException: [_na][_na] [string][2]: document missing
And I'm not find the way to filter the date.
Someone can point me on the right way to do this?
thks

My best bet would be that you have the wrong id and I also see that you are missing the type. To use more like this, you have to provide the document to use. This is defined by the combination of index,type and id. If you do not specify the document right, elasticsearch cannot find the document and that is most probably why you get the document missing message.
In java I would do something like this:
FilteredQueryBuilder queryBuilder =
new FilteredQueryBuilder(
QueryBuilders.matchAllQuery(),
FilterBuilders.rangeFilter("datefield").lte("now")
);
SearchSourceBuilder query = SearchSourceBuilder.searchSource().query(queryBuilder);
client.prepareMoreLikeThis("index","type","id")
.setField("field1","field2")
.setSearchSource(query)
.execute().actionGet();

So after strugling a little bit I found someone with the same problem. So his suggestion was to set the min_term_freq to 1.
So the code now looks like this:
FilteredQueryBuilder queryBuilder = new FilteredQueryBuilder(QueryBuilders.matchAllQuery(), FilterBuilders.rangeFilter("finish_date").lt("now"));
SearchSourceBuilder query = SearchSourceBuilder.searchSource().query(queryBuilder);
SearchResponse response = esClient.prepareMoreLikeThis("auction", "product", productId + "").setField("name.name", "description", "brand").setPercentTermsToMatch(0.3f)
.setMinTermFreq(1).setSearchSource(query).execute().actionGet();
But I dont know what this MinTermFreq does and if the value 1 is the right value. Someone know what is this field?
Thks for all the help!
Once again, Thank you for all the help and sorry for all the trouble!