I'm need run queries with 1000 objects. Using /batch endpoint I can get this to work but is too slow (30 seconds with 300 items).
So I'm trying the same approach as said in this docs page: http://docs.neo4j.org/chunked/2.0.1/rest-api-cypher.html#rest-api-create-mutiple-nodes-with-properties
POST this JSON to http://localhost:7474/db/data/cypher
{
"params": {
"props": [
{
"_user_id": "177032492760",
"_user_name": "John"
},
{
"_user_id": "177032492760",
"_user_name": "Mike"
},
{
"_user_id": "100007496328",
"_user_name": "Wilber"
}
]
},
"query": "MERGE (user:People {id:{_user_id}}) SET user.id = {_user_id}, user.name = {_user_name} "
}
The problem is I'm getting this error:
{ message: 'Expected a parameter named _user_id',
exception: 'ParameterNotFoundException',
fullname: 'org.neo4j.cypher.ParameterNotFoundException',
stacktrace:
...
Maybe this works only with CREATE queries, as showing in the docs page?
Use FOREACH and MERGE with ON CREATE SET:
FOREACH (p in {props} |
MERGE (user:People {id:{p._user_id}})
ON CREATE user.name = {p._user_name})
POST this JSON to http://localhost:7474/db/data/cypher
{
"params": {
"props": [
{
"_user_id": "177032492760",
"_user_name": "John"
},
{
"_user_id": "177032492760",
"_user_name": "Mike"
},
{
"_user_id": "100007496328",
"_user_name": "Wilber"
}
]
},
"query": "FOREACH (p in {props} | MERGE (user:People {id:{p._user_id}}) ON CREATE user.name = {p._user_name}) "
}
Actually, the equivalent to the example in the doc would be:
{
"params": {
"props": [
{
"id": "177032492760",
"name": "John"
},
{
"id": "177032492760",
"name": "Mike"
},
{
"id": "100007496328",
"name": "Wilber"
}
]
},
"query": "CREATE (user:People {props})"
}
It might be legal to replace CREATE to MERGE, but the query may not do what you expect.
For example, if a node with the id "177032492760" already exists, but it does not have the name "John", then the MERGE will create a new node; and you'd end up with 2 nodes with the same id (but different names).
Yes, a CREATE statement can take an array of maps and implicitly convert it to several statements with one map each, but you can't use arrays of maps that way outside of simple create statements. In fact you can't use literal maps the same way either when you use MERGE and MATCH. You can CREATE ({map}) but you have to MATCH/MERGE ({prop:{map}.val} i.e.
// {props:{name:Fred, age:2}}
MERGE (a {name:{props}.name})
ON CREATE SET a = {props}
For your purposes either send individual parameter maps with a query like above or for an array of maps iterate through it with FOREACH
FOREACH (p IN props |
MERGE (user:People {id:p._user_id})
ON CREATE SET user = p)
Related
am using opensearch 2.4 and I have an index with some fields while creating , later i started saving new field to the index , now when i query on the newly created field am not getting any results
ex : query 1
POST abc/_search
{
"query": {
"bool": {
"must": [
{
"terms": {
"name": [
"john"
]
}
}
]
}
}
}
above works fine because name fields exists since creation of index
query 2 :
POST abc/_search
{
"query": {
"bool": {
"must": [
{
"terms": {
"lastname": [
"William"
]
}
}
]
}
}
}
above query doesnt work though i have some documents with lastname william
When you index a new field without previously declaring it in the mapping, opensearch/elastic will generate text type and type keyword.
There are two ways for you to get results with the Term Query. First remember that Term query works with exact terms.
The first option is to use the keyword field.
{
"terms": {
"lastname.keyword": [
"William"
]
}
}
The second option is to search in the text field, but remember that when indexing the default parser is applied, then the lowecase filter leaves the token like this: william.
In this case, the query should be:
{
"terms": {
"lastname": [
"william"
]
}
}
When you use "terms" there must be an exact match (including casing).
So make sure your document contains William and not william or Williams
If you want more tolerance you can explore the match query:
https://opensearch.org/docs/latest/opensearch/query-dsl/full-text/#match
I've a collection Users which looks something like this-
{ "post": "teacher",
"salary": 4500
},
{ "post": "teacher",
"salary": 9000
},
{ "post": "principal",
"salary": 7000
},
{ "post": "teacher",
"salary": 4500
}
I want to calculate the total salary of all the teachers together and the same way for principals. So I want something which looks like
"teachers_salary": 18000
"principals_salary": 7000
I want to use aggregation but I'm not getting the desired output. It'd be really helpful if anyone of you can help me get a solution to this.
Here is an example of aggregating using spring-data-mongodb.
Assuming a results model object of PostTotalSalary.class:
public class PostTotalSalary {
private String post; // "teacher" or "principal"
private Integer totalSalary;
}
We will create a GroupOperation to gather all documents with the same value with a key of "post"
GroupOperation groupByPostAndSumSalary = group("post")
.sum("salary")
.as("totalSalary");
We can now use spring-data-mongodb to create an aggregation and then map the results to your results model object (assuming a collection name of "posts"):
Aggregation aggregation = Aggregation.newAggregation(groupByPostAndSumSalary);
AggregationResults<PostTotalSalary> groupResults = mongoTemplate.aggregate(aggregation, "posts", PostTotalSalary.class);
If you then want a list, AggregationResults has the getMappedResults method to do this:
List<PostTotalSalary> mappedResults = groupResults.getMappedResults();
The end result will be
[
{
"post" : "teacher",
"totalSalary" : "18000" // the total of all salaries for every "teacher" document
},
{
"post" : "principal",
"totalSalary" : "7000" // the total of all salaries for every "principal" document
}
]
On the mongoshell you can try this
db.collection_name.aggregate([ {
$group: {
_id: "$post",
totalsalary: { $sum: "$salary" }
}
} ] )
https://docs.mongodb.com/manual/reference/operator/aggregation/sum/#mongodb-group-grp.-sum
I am using couchbase Community Edition 5.0.1 and java-client 2.7.4. I want to store the following nested json object into couchbase. If I want to update the same object without affecting the other fields.
Eg:
If I want to add one more player object under players object
array
If I want to add One more group say 'Z Group' under group object array
How can I Achieve this without affecting other fields.
{
"doctype": "config:sample",
"group": [{
"name": "X Group",
"id": 1,
"players": [{
"name": "Roger Federer",
"number": 3286,
"keyword": "tennies"
},
{
"name": "P. V. Sindhu",
"number": 4723,
"keyword": "badminton"
}
]
},
{
"name": "Y Group",
"id": "2",
"players": [{
"name": "Jimmy Connors",
"number": 5623,
"keyword": "tennies"
},
{
"name": "Sachin",
"number": 8756,
"keyword": "Cricket"
}
]
}
]
}
N1QL has a huge variety of functions to operate on arrays:
https://docs.couchbase.com/server/current/n1ql/n1ql-language-reference/arrayfun.html
In your case, you could simply use ARRAY_INSERT or ARRAY_PREPEND
Check out update/update-for syntax (last example) https://docs.couchbase.com/server/current/n1ql/n1ql-language-reference/update.html
UPDATE default AS d
SET d.group = ARRAY_APPEND(d.group, {......})
WHERE .....;
UPDATE default AS d
SET g.players = ARRAY_APPEND(g.players, {......}) FOR g IN d.group WHEN g.id = 2 END
WHERE .....;
If you know which document IDs you want to update you can use the key-value subdocument API, which will generally be faster than going via N1QL for a single document update.
This will add a new player to the end of X Group's "players" array:
bucket.mutateIn(docId)
.arrayAppend("group[0].players",
JsonObject.create()
.put("name", "John Smith"))
// ... other player JSON
.execute();
And this will add a new Group Z to the "group" array:
bucket.mutateIn(docId)
.arrayAppend("group",
JsonObject.create()
.put("name", "Z Group"))
// ... other group JSON
.execute();
I am having a problem while querying elastic search. The below is my query
GET _search {
"query": {
"bool": {
"must": [{
"match": {
"name": "SomeName"
}
},
{
"match": {
"type": "SomeType"
}
},
{
"match": {
"productId": "ff134be8-10fc-4461-b620-79s51199c7qb"
}
},
{
"range": {
"request_date": {
"from": "2018-08-22T12:16:37,392",
"to": "2018-08-28T12:17:41,137",
"format": "YYYY-MM-dd'T'HH:mm:ss,SSS"
}
}
}
]
}
}
}
I am using three match queries and a range query in the bool query. My intention is getting docs with these exact matches and with in this date range. Here , if i change name and type value, i wont get the results. But for productId , if i put just ff134be8, i would get results. Anyone knows why is that ? . The exact match works on name and type but not for productId
You need to set the mapping of your productId to keyword to avoid the tokenization. With the standard tokenizer "ff134be8-10fc-4461-b620-79s51199c7qb" will create ["ff134be8", "10fc", "4461", "b620", "79s51199c7qb"] as tokens.
You have different options :
1/ use a term query to check without analyzing the content of the field
...
{
"term": {
"productId": "ff134be8-10fc-4461-b620-79s51199c7qb"
}
},
...
2/ if you are in Elasticsearch 6.X you could change your request to
...
{
"match": {
"productId.keyword": "ff134be8-10fc-4461-b620-79s51199c7qb"
}
},
...
As elasticsearch will create a subfield keyword with the type keyword for all string field
The best option is, of course, the first one. Always use term query if you are trying to match the exact content.
I have a sample json which I want to index into elasticsearch.
Sample Json Indexed:
put test/names/1
{
"1" : {
"name":"abc"
},
"2" : {
"name":"def"
},
"3" : {
"name":"xyz"
}
}
where ,
index name : test,
type name : names,
id :1
Now the default mapping generated by elasticsearch is :
{
"test": {
"mappings": {
"names": {
"properties": {
"1": {
"properties": {
"name": {
"type": "string"
}
}
},
"2": {
"properties": {
"name": {
"type": "string"
}
}
},
"3": {
"properties": {
"name": {
"type": "string"
}
}
},
"metadataFieldDefinition": {
"properties": {
"name": {
"type": "string"
}
}
}
}
}
}
}
}
If the map size increases from 3 ( currently) to suppose thousand or million, then ElasticSearch will create a mapping for each which may cause a performance issue as the mapping collection will be huge .
I tried creating a mapping by setting :
"dynamic":false,
"type":object
but it was overriden by ES. since it didnt match the indexed data.
Please let me know how can I define a mapping so that ES. doesnot creates one like the above .
I think there might be a little confusion here in terms of how we index documents.
put test/names/1
{...
document
...}
This says: the following document belongs to index test and is of type name with id 1. The entire document is treated as type name. Using the PUT API as you currently are, you cannot index multiple documents at once. ES immediately interprets 1, 2, and 3 as a properties of type object, each containing a property name of type string.
Effectively, ES thinks you are trying to index ONE document, instead of three
To get many documents into index test with a type of name, you could do this, using the CURL syntax:
curl -XPUT"http://your-es-server:9200/test/names/1" -d'
{
"name": "abc"
}'
curl -XPUT"http://your-es-server:9200/test/names/2" -d'
{
"name": "ghi"
}'
curl -XPUT"http://your-es-server:9200/test/names/3" -d'
{
"name": "xyz"
}'
This will specify the document ID in the endpoint you are index to. Your mapping will then look like this:
"test": {
"mappings": {
"names": {
"properties": {
"name": {
"type": "string"
}
}
}
}
}
Final Word: Split your indexing up into discrete operations, or check out the Bulk API to see the syntax on how to POST multiple operations in a single request.