Develop Reference

ElasticSearch / Java - Dynamic Templates aggregation with null values included

I'm having a diffuculties with aggregations over dynamic templates. I have values stored like this.
[
{
"country": "CZ",
"countryName": {
"en": "Czech Republic",
"es": "Republica checa",
"de": "Tschechische Republik"
},
"ownerName": "..."
},
{
"ownerName": "..."
}
]
Country field is classic keyword, mapping for country name is indexed as dynamic template according to the fact that I want to extend with with another languages when I need to.
{
"dynamic_templates": [
{
"countryName_lsi_object_template": {
"path_match": "countryName.*",
"mapping": {
"type": "keyword"
}
}
}
]
}
countryName and country are not mandatory parameters - when the document is not assigned to any country, I can't have countryName filled either. However I need to do a sorted aggregation over the country names with according to chosen key and also need to include buckets with null countries. Is there any way to do that?
Previously, I used TermsValuesSourceBuilder with order on "country" field, but I need data sorted according to specifix language and name and that can't be done over country codes.
(I'm using elasticsearch 7.7.1 and java 8 and recreation of index / changing data structure is not my option.)
I tried to use missing bucket option, but the response does not include buckets with "countryName" missing at all.
TermsValuesSourceBuilder("countryName").field("countryName.en").missingBucket(true);

Sort the search result in ascending order of a multivalued field in Solr

I'm using Solr of version 6.6.0. I have a schema of title (text_general), description(text_general), id(integer). When I search for a keyword to list the results in ascending order of the title my code returns an error can not sort on multivalued field: title.
I have tried to set the sort using the following 3 methods
SolrQuery query = new SolrQuery();
1. query.setSort("title", SolrQuery.ORDER order);
2. query.addSort("title", SolrQuery.ORDER order);
3. SortClause ab = new SolrQuery.SortClause("title", SolrQuery.ORDER.asc);
query.addSort(ab);
but all of these returns the same error
I found a solution by referring to this answer
It says to use min/max functions.
query.setSort(field("pageTitle",min), ORDER.asc);
this what I'm trying to set as the query, I didn't understand what are the arguments used here.
This is the maven dependency that I'm using
<dependency>
<groupId>org.apache.solr</groupId>
<artifactId>solr-solrj</artifactId>
<version>6.5.1</version>
</dependency>

Unless title actually is multiValued - can your post have multiple titles - you should define it as multiValued="false" in your schema. However, there's a second issue - a field of the default type text_general isn't suited for sorting, as it'll generate multiple tokens, one for each word in the title. This is useful for searching, but will give weird and non-intuitive results when sorting.
So instead, define a title_sort field and use a field type with a KeywordTokenizer and LowerCaseFilter attached (if you want case insensitive sort), or if you want case sensitive sort, use the already defined string field type for the title_sort field.

The first thing to check is do you really need that title field to be multivalued, or do your documents really have multiple titles ? If not, you just need to fix the field definition by setting multivalued="false".
That said, sorting on a multivalued field doesn't make sense unless determining which one of these multiple values should be used to sort on, or how to combine them into one.
Let' say we need to sort a given resultset by title (alphabetically), first using a single-valued title field :
# Unsorted
"docs": [
{ "id": "1", "title": "One" },
{ "id": "2", "title": "Two" },
{ "id": "3", "title": "Three" },
]
# Sorted
"docs": [
{ "id": "1", "title": "One" },
{ "id": "3", "title": "Three" },
{ "id": "2", "title": "Two" },
]
# -> ok no problem here
Now applying the same logic with a multi-valued field is not possible as is, you would necessarily need to determine which title to use in each document to properly sort them :
# Unorted
"docs": [
{ "id": "1", "title": ["One", "z-One", "a-One"] },
{ "id": "2", "title": ["Two", "z-Two", "a-Two"] },
{ "id": "3", "title": ["Three", "z-Three", "a-Three"] }
]
Hopefully, Solr allows to sort results by the output of a function, meaning you can use any from Solr's function queries to "get" a single value per title field. The answer you referred to is a good example even though it may not work for you (because title would need docValues enabled - depends on field definition - and knowing that max/min functions should be used only with numeric values), just to get the idea :
# here the 2nd argument is a callback to max(), used precisely to get a single value from title
sort=field(title,max) asc

Using two different structures in one Firebase database

I'm currently working on a project where I need two different types of non-related datastructures in my Java Android app. One being users, the other being types of food.
Users are set up like this:
users
userid
name
age
gender
weight
height
But, I also need one that looks like this, which must be searchable:
foods
name
carbohydrate
fat
protein
Is it possible to user the same database (preferrable Firebase, as I'm already using that), or do I need to add another database to the app I'm making?
Edit: I ended up exporting the JSON, rewriting it according to the good answers given here, and then importing it again. It works flawlessly. Thanks for your answers!
{
"foods" : {
"name" : {
"carbohydrates" : "5",
"fats" : "5",
"proteins" : "5"
}
},
"users" : {
"FjtMNTcDrOP2wcaPAa0E0Cc1jRz2" : {
"activity" : "Moderate Exercise (3–5 days/week)",
"age" : "40",
"gender" : "Male",
"height" : "180",
"name" : "Flex",
"weight" : "86"
}
}
}

Yes, it is possible to model multiple entity types (such as your users and foods) in the Firebase Realtime Database. While it doesn't have the concept of a table, it's a hierarchy of JSON values and you can model any structure you want in that.
For example, you could express you data model with this JSON:
{
"users": {
"userid": {
"name": "value",
"age": 42,
"gender: "value",
"weight": 190,
"height": 172
}
},
"foods": {
"name": {
"carbohydrate": 42
"fat": 11,
"protein": 8
}
}
}
In relational terms, the above model defines two "tables": users and foods. In Android code you can define separate references to each of these with:
DatabaseReference rootReference = FirebaseDatabase.getInstance().getReference();
DatabaseReference usersReference = rootReference.getChild("users");
DatabaseReference foodsReference = rootReference.getChild("foods");

Yes, you can use the same Firebase Realtime Database to store that data.
The RTDB can be simplified down to being just a JSON tree. So for your desired implementation, you would have two keys at the root of your database (such as "users" and "foods").
{
"users": {
"userid1": {
"name": "somestring",
"age": "somenum",
"gender": "somestring",
"height": "somenum",
"weight": "somenum",
...
},
...
},
"foods": {
"food1": {
"name": "somename",
"carbs": "somenum",
"fat": "somepercent",
"protein": "somepercent",
...
},
...
}
}
You can also add or remove more root keys as you wish and your project takes shape.
However,
As #Tamir Abutbul suggests in their answer, I would use Cloud Firestore for this project over the RTDB.
The reason for this is that based on your data, you are likely going to need to filter results by a number of different values at a time in the future. Cloud Firestore supports these types of queries natively (docs) whereas you'd need to write a custom solution for the RTDB.
Getting Started with Cloud Firestore

You can use Firebase with Cloud Firestore.
Create "users" collections with your wanted data structure and another collection called "foods" with its own data structure.
The next step is just to decide when to use any one of those collections(according to your app logic).

Good way to store many nested JSON objects and arrays as data structure in Android SQLite?

Considering the below JSON, what would be the best way to store this into SQLite?
I am already parsing this with Gson, but wondering what would be a pain-free way to store this into SQLite and be able to retrieve it with no parsing issues.
I am already storing the desc, deposit objects as a HashMaps. My issue is the lease object. What would be an elegant way to store the leasees array?
Should I just create another Leasee object? And then serialize the ArrayList into a Blob for storage into the database?
{
"name": "1",
"desc": {
"country": "1",
"city": "1",
"postal": "1",
"street": "1",
"substreet": "1",
"year": 1,
"sqm": 1
},
"owner": [
"1"
],
"manager": [
"1"
],
"lease": {
"leasee": [
{
"userId": "1",
"start": {
"$date": 1420070400000
},
"end": {
"$date": 1420070400000
}
}
],
"expire": {
"$date": 1420070400000
},
"percentIncrease": 1,
"dueDate": 1
},
"deposit": {
"bank": "China Construction Bank",
"description": "Personal Bank Account USA"
}
}

Storing everything in a BLOB ignores the benefit that a DB provides.
You have much of a relational database structure already described (however loosely) in the JSON:
Properties table with location and description info.
Persons table with names and contacts.
Roles table relating Properties and Persons (residents, managers, owners, service providers).
Leases table with terms related to Properties and Persons.
Payments table with payment info related to Leases.
You can manually write in the primary keys in your JSON, taking care to match those relationships between tables, then insert the resulting rows by processing that modified JSON. Here's a link to SQLite doc on using INSERT with auto increment.

Freemarker, Maps and Mongo. Getting a value out a nested structure

I have a Mongo Database and I am making a small web application to display the values from the database. I'm having a little problem with getting a value from a nested structure.
The database structure I have is:
{
"_id": ObjectId("4e244ddcb1633b25c38f2155"),
"id": "2173",
"type": "P",
"title": "Test title",
"tag": {
"0": {
"_id": ObjectId("4e244ddcb1633b25c38f1fc1"),
"name": "Education",
"id": "6"
}
}
}
What I do is, I get the database from the database and put this in a map and this map I put together with several other maps into one map (root) and put that into the .ftl page.
I can then read the value in .ftl like: ${root.title} which will ofcourse print 'Test title' in the above example.
I now what to get the value of name in tage so that it can print 'Education'
I don't know how to get this. Any ideas.

You can access the value using Freemarker's built-in's for hashes. Similar to below:
<#assign tagKeys = root.tag?keys>
<#list tagKeys as tagKey>
${root.tag[tagKey].name}
</#list>
I have not tested this yet but generally speaking this should work. You might also want to read some previous answers on SO:
Freemarker iterating over hashmap keys