Develop Reference

ES - Convert Legacy ElasticSearch from Rest to High level Client

I'm wondering what is the equivalent for the following query with elastic 5 - 7 (doesn't matter for me)
I know that this query has deprecated but actually i'm trying to use legacy 1.7.5 cluster work with High level ES cluster.
I did some tests and although the documentation points that it isn't support i tried and most of the simple actions work. What is left is convert some queries like in the example below
{
"size" : 3000,
"query" : {
"filtered" : {
"filter" : {
"bool" : {
"must" : [ {
"terms" : {
"source" : [ "o365mail" ]
}
}, {
"range" : {
"bckdate" : {
"from" : "1549360021398l",
"to" : null,
"include_lower" : true,
"include_upper" : true
}
}
} ]
}
}
}
},
"fields" : "*"
}
What i've tried so far is with 7.9.3:
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.9/java-rest-high.html
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
boolQueryBuilder
.must(QueryBuilders.termsQuery(IndexFields.SOURCE.getIndexName(),Arrays.asList(Source.O365MAIL.toString().toLowerCase())))
.must(QueryBuilders.rangeQuery("bckdate").gte(1549360021398l).lte(null));
sourceBuilder.query(boolQueryBuilder);
SearchRequest sr = new SearchRequest();
sr.source(sourceBuilder);
SearchResponse searchResponse2 = client.search(sr, RequestOptions.DEFAULT);
The query from debugging is:
{
"bool" : {
"must" : [
{
"terms" : {
"source" : [
"o365mail"
],
"boost" : 1.0
}
},
{
"range" : {
"bckdate" : {
"from" : 1549360021398,
"to" : null,
"include_lower" : true,
"include_upper" : true,
"boost" : 1.0
}
}
}
],
"adjust_pure_negative" : true,
"boost" : 1.0
}
}
Im wondering if it is the same regarding the filters of the legacy code, cause the responded data is pretty the same .
I need not to break the logic with all Filters like in legacy query...
thanks for help

Elasticsearch query from Java REST High level client returns different/undesirable results compared to execution of Query DSL on Kibana

I'm implementing an elasticsearch search feature using Java High-level REST client, which queries the ES index residing on one of the clusters hosted on the cloud. My intended query JSON DSL looks like this,
{
"query" :{
"bool": {
"should" :[
{
"query_string":{
"query":"cla-180",
"default_field": "product_title",
"boost" : 3
}
},
{
"match" : {
"product_title" : {
"query" : "cla-180",
"fuzziness" : "AUTO"
}
}
}
]
}
}
}
Corresponding to this, I have written the code to be executed using the Java High-level REST client, which performs the same functionality as the DSL above.
BoolQueryBuilder boolQueryBuilder = buildBoolQuery();
boolQueryBuilder.should(QueryBuilders.queryStringQuery("cla-180").defaultField("product_title")).boost(3);
boolQueryBuilder.should(QueryBuilders.matchQuery("product_title", "cla-180").fuzziness(Fuzziness.AUTO));
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(boolQueryBuilder);
What I'm noticing is that the search results for the java methods are different to the results when I execute the DSL on kibana directly. I find records that have no relation to the search content that I have given above when executed from Java. I consider this weird because I guess I have implemented the Java code to match that of the JSON query DSL given above.
When I try to print the generated JSON from the Java side, its output is like this below,
{
"query": {
"bool" : {
"should" : [
{
"query_string" : {
"query" : "cla-180",
"default_field" : "product_title",
"fields" : [ ],
"type" : "best_fields",
"default_operator" : "or",
"max_determinized_states" : 10000,
"enable_position_increments" : true,
"fuzziness" : "AUTO",
"fuzzy_prefix_length" : 0,
"fuzzy_max_expansions" : 50,
"phrase_slop" : 0,
"escape" : false,
"auto_generate_synonyms_phrase_query" : true,
"fuzzy_transpositions" : true,
"boost" : 1.0
}
},
{
"match" : {
"product_title" : {
"query" : "cla-180",
"operator" : "OR",
"fuzziness" : "AUTO",
"prefix_length" : 0,
"max_expansions" : 50,
"fuzzy_transpositions" : true,
"lenient" : false,
"zero_terms_query" : "NONE",
"auto_generate_synonyms_phrase_query" : true,
"boost" : 1.0
}
}
}
],
"adjust_pure_negative" : true,
"minimum_should_match" : "1",
"boost" : 3.0
}
}
}
Am I missing something in my java code, that makes the search results to be returned in an undesirable fashion? Or what could be the reason for this mismatch in records that are being returned in these 2 methods?
Thanks in advance!

How do I return full documents containing a distinct value using Spring Framework Aggregation?

Say I have some documents like this:
{
"name" : "Wendy",
"phone" : "123",
"GroupId" : 1
},
{
"name" : "Tom",
"phone" : "234",
"GroupId" : 1
},
{
"name" : "Sally",
"phone" : "345",
"GroupId" : 3
},
{
"name" : "Bob",
"phone" : "456",
"GroupId" : 3
},
{
"name" : "Cortana",
"phone" : "567",
"GroupId" : 7
}
I'd like to return a list of full-data documents that contains the first occurrence of each distinct GroupId. I'm thinking Aggregation is the best route for a task like this. Here is what I have so far:
MatchOperation matchStage = Aggregation.match(new Criteria());
GroupOperation groupStage = Aggregation.group("GroupId").first("$$CURRENT").as("??");
// I know the above line is semi-nonsensical
Aggregation aggregation = Aggregation.newAggregation(matchStage, groupStage);
// I only need help creating the aggregation object, beyond this is just a MongoOperations aggregate call
It should be noted that I don't necessarily need to use aggregation so if there is a way to achieve this using a simple "find" then I'm okay with that.
I'm a MongoDb noob, sorry if my "have tried" section is not very useful. However, this is what I would want back:
{
"name" : "Wendy",
"phone" : "123",
"GroupId" : 1
},
{
"name" : "Sally",
"phone" : "345",
"GroupId" : 3
},
{
"name" : "Cortana",
"phone" : "567",
"GroupId" : 7
}

Try this. $first helps to get the first occurrence of the data
Aggregation aggregation = Aggregation.newAggregation(
group("GroupId")
.first("name").as("name")
.first("GroupId").as("GroupId")
.first("phone").as("pnone"),
project().andExclude("_id")
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(YOUR_COLLECTION_NAME.class), Object.class).getMappedResults();
Working Mongo playground

Why I'm getting documents not included in the projection?

I have these 2 documents in my collection:
{
"_id" : ObjectId("5722042f8648ba1d04c65dad"),
"companyId" : ObjectId("570269639caabe24e4e4043e"),
"applicationId" : ObjectId("5710e3994df37620e84808a8"),
"steps" : [
{
"id" : NumberLong(0),
"responsiveUser" : "57206f9362d0260fd0af59b6",
"stepOnRejection" : NumberLong(0),
"notification" : "test"
},
{
"id" : NumberLong(1),
"responsiveUser" : "57206fd562d0261034075f70",
"stepOnRejection" : NumberLong(1),
"notification" : "test1"
}
]
}
{
"_id" : ObjectId("5728f317a8f9ba14187b84f8"),
"companyId" : ObjectId("570269639caabe24e4e4043e"),
"applicationId" : ObjectId("5710e3994df37620e84808a8"),
"steps" : [
{
"id" : NumberLong(0),
"responsiveUser" : "57206f9362d0260fd0af59b6",
"stepOnRejection" : NumberLong(0),
"notification" : "erter"
},
{
"id" : NumberLong(1),
"responsiveUser" : "57206f9362d0260fd0af59b6",
"stepOnRejection" : NumberLong(1),
"notification" : "3232"
}
]
}
Now I'm trying to get the document with the max _id and the id that equals 0 from a document inside of the steps array. I also have a projection that is supposed to show only the id of the matched element and nothing else.
Here is my query:
collection
.find(new Document("companyId", companyId)
.append("applicationId", applicationId)
.append("steps",
new Document("$elemMatch",
new Document("id", 0))))
.sort(new Document("_id", 1))
.limit(1)
.projection(new Document("steps.id", 1)
.append("_id", 0));
And it returns:
Document{{steps=[Document{{id=0}}, Document{{id=1}}]}}
Why is it returning 2 documents instead of 1?
The result should be looking like:
Document{{id=0}}
What am I missing here? I know that is something basic, but I really can't spot my mistake here.

Your query document tells Mongo to return those documents where in the 'steps' array they have a document where id: 0. You are NOT telling Mongo to return ONLY that field. You can use $elemMatch inside the projection document to get what you want (I'm writing this in the Mongo shell syntax because I'm not too familiar with the Java syntax):
{ steps: { $elemMatch: { id: 0 } },
'steps.id': 1,
_id: 0
}

google distance matrix api strange

I am facing a strange problem here,
when I run below url from web browser or from java command line
http://maps.googleapis.com/maps/api/distancematrix/json?origins=416%2063,Sweden&destinations=424%2069,Stor%C3%A5s%20Industrigatan%2020,Angered,G%C3%B6teborg&sensor=false
I get below results.
{
"destination_addresses" : [ "Storås Industrigata 20, 424 69
Angered, Sweden" ],
"origin_addresses" : [ "Gothenburg, Sweden" ],
"rows" : [
{
"elements" : [
{
"distance" : {
"text" : "10.4 km",
"value" : 10388
},
"duration" : {
"text" : "15 mins",
"value" : 924
},
"status" : "OK"
}
]
}
],
"status" : "OK"
}
but when I run the same url from glassfish server I mean sending an
http request from
form submit I get below strange response
{ "destination_addresses" : [ "" ],
"origin_addresses" : [ "Gothenburg, Sweden" ],
"rows" : [
{
"elements" : [
{
"status" : "NOT_FOUND"
}
]
}
],
"status" : "OK"
}
please not destination_addresses is empty in this case and status in
NOT_FOUND.
Java code I used to get the response is
private String getResponse(String URL) throws Exception {
InputStream stream = new URL(URL).openStream();
byte[] array = new byte[stream.available()];
stream.read(array);
return new String(array);
}
please guide me to resolve this issue,
thanks....

Use https
unless google will not let your software to connect.
and
get a api key.
read this:
https://developers.google.com/maps/documentation/distance-matrix/start
https://maps.googleapis.com/maps/api/distancematrix/json?origins=416%2063,Sweden&destinations=424%2069,Stor%C3%A5s%20Industrigatan%2020,Angered,G%C3%B6teborg&sensor=false&key=YOUR_API_KEY
be sure to replace YOUR_API_KEY with your actual API key

I had a similar problem when using the Distance Matrix API in Java. Setting the language in my request seems to make it work:
DistanceMatrixElement distanceMatrixElement;
DistanceMatrix matrix;
try
{
matrix = distanceMatrixApiRequest
.origins(...)
.destinations(...)
.language("en")
.await();
}