Develop Reference

Vespa.ai: How to make an array of floats handle null values?

I am trying to make a simple Vespa application, where one of my data fields are an Array. However the array contains some null values. For instance an array like: [2.0,1.4,null,5.6,...].
What can I use instead of float to represent elements in the array?

Seems like you want to use a sparse tensor field instead since some addresses does not have a value. x{} denotes a sparse tensor, x[128] is an example of a dense tensor. See https://docs.vespa.ai/documentation/tensor-user-guide.html for an intro to vespa tensor fields.
field stuff type tensor<float>(x{}) {
indexing: summary |attribute
}
[
{ "put": "id:example:example::0", "fields": {
"stuff" : {
"cells": [
{ "address" : { "x" : "0" }, "value": 2.0 },
{ "address" : { "x" : "1" }, "value": 1.4 },
{ "address" : { "x" : "3" }, "value": 5.6 },
]
}
}
}
]

Issue when trying to query a MongoDB data

I have a document in a collection that has the following attributes:
nodeid : long
type: string
bagid: long
So, nodes can be on a bag, and be of different types.
I need to find,
all nodes of type A, or nodes of type B in a given list of nodes, or, nodes of type C in a given bag.
How can I design that query in MongoDB? I had all IN clauses but it is the works way to go performance wise. Could you please point me into the right direction? I could not find an aggregation or reduce that would help me make this simpler.
I tried also doing a text search, using the three elements, but the or in the text search, for instance "type: A \"type: B node:X\" \"type: B node: Y\" and so on, does not work.
Thanks
Edit, adding samples:
{ "_id" : BinData(3,"NJUuYHEBAAAdCda3V+kXvg=="),
"type" : "question", "bagid" : NumberLong(1067),
"topics" : [ NumberLong(33), NumberLong(67), NumberLong(203), NumberLong(217) ],
"nodeid" : NumberLong(15855),
"creationDate" : ISODate("2020-04-09T18:23:17.812Z"),
"_class" : "com.test.NodeEvent" }
{ "_id" : BinData(3,"NJUuYHEBAAAdCda3V+kXvg=="),
"type" : "comment", "bagid" : NumberLong(1067),
"topics" : [ NumberLong(33), NumberLong(67), NumberLong(203), NumberLong(217) ],
"nodeid" : NumberLong(15857),
"creationDate" : ISODate("2020-04-09T18:23:17.812Z"),
"_class" : "com.test.NodeEvent" }
{ "_id" : BinData(3,"NJUuYHEBAAAdCda3V+kXvg=="),
"type" : "question", "bagid" : NumberLong(1069),
"topics" : [ NumberLong(33), NumberLong(67) ],
"nodeid" : NumberLong(15859), "creationDate" : ISODate("2020-04-09T18:23:17.812Z"),
"_class" : "com.test.NodeEvent" }

You can build your queries using the MongoDB Query Language. The query is written using the db.collection.find method. It uses various query operators like, $or, $in, and $and.
I need to find, all nodes of type A, or nodes of type B in a given
list of nodes, or, nodes of type C in a given bag. How can I design
that query in mongo?
db.collection.find( { $or: [
{ type: "A" },
{ type: "B", node: { $in: [ "A", "C" ] },
{ type: "C", bag: 130 }
]
} );
In the above query the condition { type: "C", bag: 130 } is equivalent to { $and: [ { type: "C" }, { bag: 130 } ] } }. This is also the case with the condition { type: "B", node: { $in: [ "A", "C" ] }. But, using the $and is optional, in this case (see the documentation for details).
The query would be something like, given bag 1067 give me all nodes
with topics: 33 or 203
db.collection.find( { bag: 1067, topics: { $in: [ 33, 203 ] } } )
The output depends upon the data in the collection. The find method returns a cursor, and you can apply various cursor methods on the retrned documents (for example, you can sort them by a field).

mongo java for finding and upserting a nested array element

I have a mongo collection element with the following structure
{
"_id" : ObjectId("568eaba5e4b0fa3cdf9aaf85"),
"type" : "X",
"textId" : "568eaba5e4b0fa3cdf9aaf84",
"createDate" : "2016-01-07 18:17:09",
"likes" : [
{
"creatorName" : "ABCD",
"creationDate" : "2016-01-10 19:48:37",
"likeId" : "56932615569aae9d1459eb9b"
}
]
}
My need is as follows:
When a new like is posted by a creatorName for a particular element identified by textId, I should
a) check if the like is already present for the creatorName for the specific textId object. If present, do nothing
b) if the like is not present, I should insert a new element into the likes array
How to do this effectively in java? I see that we have an update method and a findAndModify method. Which one to use and how to handle this in the most effective manner?

Unique indexes
You can create a unique index as follows:
db.collection.createIndex( { "likes.creatorName": 1 }, { unique: true } )
As official documentation says:
A unique index causes MongoDB to reject all documents that contain a
duplicate value for the indexed field.
if the creatorName already exists, write operation will fail as follows:
WriteResult({
"nInserted" : 0,
"writeError" : {
"code" : 11000,
"errmsg" : "E11000 duplicate key error index: test.collection.$likes.creatorName_1 dup key: { : 1.0 }"
}
})
Then, you don't need to worry about if creatorName already exists or not.
On the other hand, if you need to know if the creatorName already exists, you need to have a look to the WriteResult returned value (0 or 1).

You may insert likes by update function with query condition:
db.collection.update({
"textId" : "568eaba5e4b0fa3cdf9aaf84",
"likes.likeId": {"$nin": ["56932615569aae9d1459eb10"]},
"likes.creatorName" : {"$nin": ["ABCE"]}
},
{
"$push" : {
"likes" : {
"creatorName" : "ABCE",
"creationDate" : "2016-01-10 21:48:37",
"likeId" : "56932615569aae9d1459eb10"
}
}
}
)
Updated 1 existing record(s)
If you try to add new like for the same creatorName, it will not update.
db.collection.update({
"textId" : "568eaba5e4b0fa3cdf9aaf84",
"likes.likeId": {"$nin": ["56932615569aae9d1459eb11"]},
"likes.creatorName" : {"$nin": ["ABCD"]}
},
{
"$push" : {
"likes" : {
"creatorName" : "ABCD",
"creationDate" : "2016-01-11 19:48:37",
"likeId" : "56932615569aae9d1459eb11"
}
}
}
)
Updated 0 record(s)
Java driver does not give any performance advantage.
If you add index depending on query condition, it will improve the performance. (Needs only once)
db.colecction.createIndex({"textId":1, "likes.likeId":1, "likes.creatorName":1})

MongoDB search in embedded array

I am trying to store all my written java files in a MongoDB and so far I've applied a schema like this (incomplete entry):
{
"_id" : ObjectId("52b861c230044fd08d6c27c4"),
"interfaces" : [
{
"methodInterfaces" : [
{
"name" : "add",
"name_lc" : "add",
"returnType" : "Integer",
"returnType_lc" : "integer",
"parameterTypes" : [
"Integer",
"Integer"
],
"parameterTypes_lc" : [
"integer",
"integer"
]
},
{
"name" : "isValid",
"name_lc" : "isvalid",
"returnType" : "Boolean",
"returnType_lc" : "boolean",
"parameterTypes" : [
"Integer",
"Double"
],
"parameterTypes_lc" : [
"integer",
"double"
]
}
],
"name" : "Calculator",
"name_lc" : "calculator",
"filename" : "Calculator.java",
"filename_lc" : "calculator.java"
}
],
"name" : "Calculator",
"name_lc" : "calculator",
"filename" : "Calculator",
"filename_lc" : "calculator",
"path" : "/xyz/Calculator.java",
"md5" : "6dec7e62c5e4f9060c7612c252cd741",
"lastModification" : ""
}
So far I am able to query a class that contains a method name, but I am not able to query a class with a certain name (let interfaces.name_lc="calculator") that must contain two methods with particular names (let's say "add" and "divide") which themselves should have two integer, resp. an integer and a double as parameters and both return an integer (don't question whether this is reasonable or not -- just for illustration purposes).
This is just one example; it can be more complex, of course.
I don't know how I can query for a particular class with method and specified parameters. I need to describe it sharp and want sharp results.
I am not able to construct a query, that would only return files like Calculator ( add(integer,integer):integer; divide(integer,double):integer; ). I get, e.g., OtherClass ( add():void; method(integer):integer; ), which is not what I want. I am trying this for days now, and maybe one can enlighten me, how to solve this in MongoDB. Thanks a lot in advance!

I'm not sure you'll be able to do this in MongoDB with your document structure. The issue I ran into is around the parameters - I'm assuming you care about the order of the parameters (i.e. doSomething(String, int) is not the same as doSomething(int, String)), and the query operators to check all the values in an array treat the array as a set, so a) order agnostic and b) eliminates duplicates (so doSomething(String, String) matches doSomething(String)) (this is because I was using the $all keyword, see the documentation, especially the note at the bottom).
I managed to get a large part of the query you wanted, however, which might point you in the right direction.
{ "$and" : [ //putting these in an "and" query means all parts have to match
{ "interfaces.methodInterfaces" :
{ "$elemMatch" : { "name" : "add"}} //this bit finds documents where the method name is "add"
} ,
{ "interfaces.methodInterfaces" :
{ "$elemMatch" : { "returnType" : "Integer"}} // This bit matches the return type
} ,
{ "interfaces.methodInterfaces.parameterTypes" :
{ "$all" : [ "Integer" , "Integer"]} //This *should* find you all documents where the parameter types matches this array. But it doesn't, as it treats it as a set
}
]}
If you're querying via the Java driver, this looks like:
BasicDBObject findMethodByName = new BasicDBObject("interfaces.methodInterfaces",
new BasicDBObject("$elemMatch", new BasicDBObject("name", "add")));
BasicDBObject findMethodByReturn = new BasicDBObject("interfaces.methodInterfaces",
new BasicDBObject("$elemMatch", new BasicDBObject("returnType", "Integer")));
BasicDBObject findMethodByParams = new BasicDBObject("interfaces.methodInterfaces.parameterTypes",
new BasicDBObject("$all", asList("Integer", "Integer")));
BasicDBObject query = new BasicDBObject("$and", asList(findMethodByName, findMethodByReturn, findMethodByParams));
DBCursor found = collection.find(query);
I haven't included matching the class name, as this didn't seem to be the tricky part - just build up another simple query for that an add it into the "$and".
Since the arrays to store parameter types are not giving what you want, I suggest you think about something a little more structured, although it's a bit unwieldy. Instead of
"parameterTypes" : [
"Integer",
"Integer"
]
Consider something like
"parameterTypes" : {
"param1" : "Integer",
"param2" : "Integer"
}
Then you won't be doing set operations, you can query each parameter individually. This means you'll also get them in the correct order.

How to add where clause in elasticsearch

I am very new to elasticsearch. I have an elastic search query which returns all users present in database based on keyword but here I need to get only the users whose active status is true but now I am getting all list of users.
I have built a search query like
{
"from" : 0,
"size" : 30,
"query" : {
"query_string" : {
"query" : "*jhon*",
"default_field" : "_all"
}
},
"sort" : [
{
"id" : {
"order" : "desc"
}
}
]
}
I am using a query like *jhon*, but I need to get result like users with name jhon and his active status should be true, so I have tried *"+jhon+"\*"#activated~true" but I am not getting the desired result.
I don't know what I am doing wrong.

It's best to add a filter to your query. Filters are faster since they don't involve any scoring and are cached. There are different filters that you can use, the elasticsearch query DSL is really flexible. The kind of filters depends on the way you indexed data, which depends on your data. I'll assume the easiest possbile solution: a term filter. Have a look at the following example.
{
"from" : 0,
"size" : 30,
"query" : {
"query_string" : {
"query" : "*jhon*",
"default_field" : "_all"
}
},
"filter" : {
"term" : {
"activated" : "true"
}
},
"sort" : [
{
"id" : {
"order" : "desc"
}
}
]
}
Beware that there are different ways to apply filters when searching. In my example I applied a top level filter, which will only be applied to the search results, and not to the facets. If you want it to be applied to the facets too, you should have a look at the filtered query; instead of adding a filter you would wrap your current query into new filtered query, which can also contain the same term filter.