Develop Reference

DynamoDb querying objects based on nested attribute

How can I query all the objects from a given table in Amazon DynamoDb using sub-object id? I will explain with example:
Orders table data:
{
"id": "76ds6ds76",
"publishDate": "2022-09-20",
"returnDate": null,
"book": {
"bookId": "327a7cdfeececd",
"name": "Hello world"
}
},
{
"id": "838ds990",
"publishDate": "2022-09-30",
"returnDate": null,
"book": {
"bookId": "327a7cdfeececd",
"name": "Hello world"
}
}
I want to get all the Orders with bookId == "327a7cdfeececd". Is there any possibility to query that?

The best way you can achieve that is by making bookid a top level attribute and creating a GSI based on it.
The other option you have is using a Scan and FilterExpression. This will read all the items in the table in order to find the one you need, while it would work well for small tables, it may not be advisable to do it with large tables due to poor performance and cost.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Scan.html#Scan.FilterExpression

How to include kql in dsl query/java high level rest client query

Is there a way to include KQL (nearly the syntax) into a DSL bool query for example. I have an aggregation whose buckets make up a dataset, which is "flattened" into a table view. Within the UI we allow KQL filtering, but upon a given action I get the set of data included in a row, to use as boolean query WHERE clause like values for getting the document instances that we included in that set.
So I construct a bool query which is great and works fine from those properties:
{
"query": {
"bool": {
"must": [{
"match": {
"color": "red"
},{
"shape": "circle"
}
}]
}
}
}
but I also get the KQL string the user used to filter on the dataset within the UI
item.available: true
Is there a way to include this within the boolean query I have already constructed through the java high level rest client / or just generic dsl for that matter?
I have tried query-string as it seems to be the closest in the documentation that I can find
{
"query": {
"bool": {
"must": [{
"match": {
"color": "red"
},{
"shape": "circle"
}
}]
},
"query_string" : {
"query" : "item.available: true"
}
}
}
but this doesn't seem to work in parallel with the bool query? any ideas? thanks as always

KQL is only available/exposed in the Kibana UI. Whenever Kibana communicates with Elasticsearch it is using Elasticsearch's query DSL. One notable exception is Lucene expressions (or KQL expressions translated into Lucene syntax) which will end up as search-strings in Elasticsearch query-string-query, as you observed.
The query-string-query is a "standard" Elasticsearch-query and can be used wherever any other query (e.g. match-query) can be used. So yes, it should be possible to use a query-string-query within a bool-query, also when using language clients.

Elasticsearch universal search query

I have a string "Jhon Abraham 18". I want to create search query that will search by divided by spaces words from the string in an index. This search have to be set to all fields of the index and you don't know what meaning have to be mapped(set) to a field.
So, I have a document:
{
"_index": "recipient",
"_type": "recipient",
"_id": "37a15258d9",
"_version": 1,
"_score": 1,
"_source": {
"name": "Jhon ",
"surname": "Abraham",
"age": "18 ",
}
and I don't know to what fields of index meanings Jhon, Abraham and 18 correspond. I just have a string and by this string I want to search in all fields of the index documents. I can divide it by separete words by spaces but I don't know exact mapping fields for search. Also, I want to do it at Java.
I'll be appreciate for help.

I think you should use query_string in elasticsearch.
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
This will solve your problem.

You can use multi match query, writing all fields or wildcards.
Multi Match Query

How to get index ids from an index in elasticsearch

I am having trouble getting the index ids from an index using the java api for elasticsearch.
When creating an IndexResponse I can get the IndexResponse id from the "IndexResponse" object. When creating an index I do not specify the id so I let elasticsearch handle this. How can I get a listing of the ids for a specific index?
I would then iterate through the ids to submit other requests (i.e. GET, DELETE).
I am using the java api and not spring-data. The version is 1.7 for those interested.

Retrieving all of the IDs from an index is generally a terrible idea, which gets more terrible depending on how large your index is. If you really need it, consider using a scroll query to achieve what you want.
https://www.elastic.co/guide/en/elasticsearch/guide/master/scroll.html#CO33-1
The guide has is written for Elasticsearch 2.x, but it works for Elasticsearch 5.x if you're using that.
Essentially how it works is this:
Create a scroll window of size x, return the first 1000 results without the overhead of scoring, analysis, etc. The resources are allocated by Elasticsearch for a time of y. The first response returns not only the first x documents, but returns a _scroll_id that can be used to fetch the next x documents.
GET http://yourhost:9200/old_index/_search?scroll=1m
{
"query": { "match_all": {}},
"sort" : ["_doc"],
"size": 1000
}
Say the response to the above query is something like...
{
"_scroll_id": "abcdefghijklmnopqrstuvwxyz",
"took": 15,
"timed_out": false,
"terminated_early": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1027,
"max_score": null,
"hits": [
{
...
You would then use the _scroll_id like so to fetch the next x results.
GET http://yourhost:9200/_search/scroll
{
"scroll": "1m",
"scroll_id" : "abcdefghijklmnopqrstuvwxyz"
}
It returns a response similar to the above. Ensure you rip out the _scroll_id from each request response and use it in the next. Using all of these responses, you can iterate through the hits and rip out the IDs.

ElasticSearch : search more like this in java

Let's say I've indexed a document like this :
{
"_index": "indexapm",
"_type": "membres",
"_id": "3708",
"_score": 1,
"_source": {
"firstname": "John",
"lastname": "GUERET-TALON"
}
}
I want to retrieve this document when searching for "GUER", "GUERET", "TAL" for example.
I have a Java application and I tried this :
MoreLikeThisQueryBuilder qb = QueryBuilders.moreLikeThisQuery(
"firstname^3",
"lastname^3")
.likeText("GUER");
SearchResponse response = client.prepareSearch("myindex")
.setTypes("mytype")
.setSearchType(SearchType.DFS_QUERY_AND_FETCH)
.setQuery(qb) // Query
.setFrom(0)
.setSize(limit)
.setExplain(true)
.execute()
.actionGet();
But this search doen't retrieve my document. Of course if I try an exact match query and search for "GUERET", it works.
Does anyone know what kind of query I have to use and how to make it work with the Java library? Thanks!

The More Like This Query isn't the best choice in this case.
If, as you described, you're looking for documents using the first letters of words, you should use a Prefix Query instead, but they are limited to one field. For a search on more than one field, use the MultiMatch Query (providing the PHRASE_PREFIX type). I would try something like:
QueryBuilders.multiMatchQuery("GUER", "firstname", "lastname")
.type(MatchQueryBuilder.Type.PHRASE_PREFIX);

QueryBuilders.boolQuery().should(QueryBuilders.wildcardQuery("lastname", "*GUER*"));
I got the result using WildcardQueryBuilder it generates following:
{"bool":{"should":[{"wildcard":{"firstname":"*GUER*"}}]}}