I'm trying to create a dummy SearchResponse object by passing the values manually to the constructor. I have a JUnit test class for which I'm using this dummy value to mock the actual method call. Trying with the below method to
public SearchResponse actionGet() throws ElasticsearchException {
ShardSearchFailure[] shardFailures = new ShardSearchFailure[0];
int docId = 0;
String id = "5YmRf-6OTvelt29V5dphmw";
Map<String, SearchHitField> fields = null;
InternalSearchHit internalSearchHit = new InternalSearchHit(docId, id,
null, fields);
InternalSearchHit[] internalSearchHit1 = { internalSearchHit };
InternalSearchResponse EMPTY = new InternalSearchResponse(
new InternalSearchHits(internalSearchHit1, 0, 0), null, null,
null, false);
SearchResponse searchResponse = new SearchResponse(EMPTY, "scrollId",
1, 1, 1000, shardFailures);
return searchResponse;
}
and here is my actual value of json when query directly to elasticsearch.
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"failed": 0
},
"hits": {
"total": 28,
"max_score": null,
"hits": [
{
"_index": "monitoring",
"_type": "quota-management",
"_id": "5YmRf-6OTvelt29V5dphmw",
"_score": null,
"_source": {
"#timestamp": "2014-08-20T15:43:20.762Z",
"category_name": "cat1111",
"alert_message": "the new cpu threshold has been reached 80%",
"alert_type": "Critical",
"view_mode": "unread"
},
"sort": [
1408549226173
]
}
]
}
}
I want to create similar kind of response by creating the actual SearchResponse Object. But I couldn't find any way to send the values in InternalSearchHit[]. Please let me know how can I do this.
This will do what you want:
SearchShardTarget shardTarget = new SearchShardTarget("1", "monitoring", 1);
ShardSearchFailure[] shardFailures = new ShardSearchFailure[0];
float score = 0.2345f;
BytesReference source = new BytesArray("{\"#timestamp\":\"2014-08-20T15:43:20.762Z\",\"category_name\""
+ ":\"cat1111\",\"alert_message\":\"the new cpu threshold has been reached 80%\",\"alert_type\":"
+ "\"Critical\",\"view_mode\":\"unread\"}");
InternalSearchHit hit = new InternalSearchHit(1, "5YmRf-6OTvelt29V5dphmw", new StringText("quota-management"),
null);
hit.shardTarget(shardTarget);
hit.sourceRef(source);
hit.score(score);
InternalSearchHit[] hits = new InternalSearchHit[]{hit};
InternalSearchHits internalSearchHits = new InternalSearchHits(hits, 28, score);
InternalSearchResponse internalSearchResponse = new InternalSearchResponse(internalSearchHits, null, null,
null, false);
SearchResponse searchResponse = new SearchResponse(internalSearchResponse, "scrollId", 1, 1, 1000,
shardFailures);
If you call toString() on searchResponse it returns:
{
"_scroll_id" : "scrollId",
"took" : 1000,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"hits" : {
"total" : 28,
"max_score" : 0.2345,
"hits" : [ {
"_index" : "monitoring",
"_type" : "quota-management",
"_id" : "5YmRf-6OTvelt29V5dphmw",
"_score" : 0.2345,
"_source":{"#timestamp":"2014-08-20T15:43:20.762Z","category_name":"cat1111","alert_message":"the new cpu threshold has been reached 80%","alert_type":"Critical","view_mode":"unread"}
} ]
}
}
It works for me in ElasticsearchS 6.5
BytesReference source = new BytesArray(
"{your json response come here}" );
SearchHit hit = new SearchHit( 1 );
hit.sourceRef( source );
SearchHits hits = new SearchHits( new SearchHit[] { hit }, 5, 10 );
SearchResponseSections searchResponseSections = new SearchResponseSections( hits, null, null, false, null, null, 5 );
SearchResponse searchResponse = new SearchResponse( searchResponseSections, null, 8, 8, 0, 8, new ShardSearchFailure[] {} );
return searchResponse;
Related
this code works perfectly if there is no field called _transferSize,
sample har :
"cookies": [
{
"name": "x-amz-captcha-1",
"value": "1577615972915416",
"path": "/",
"expires": "2020-12-28T08:39:32.000Z",
"httpOnly": false,
"secure": false
},
{
"name": "x-amz-captcha-2",
"value": "PKRgLIISQDY1ubrOgWIOQQ==",
"path": "/",
"expires": "2020-12-28T08:39:32.000Z",
"httpOnly": false,
"secure": false
}
],
"content": {
"size": 0,
"mimeType": "x-unknown"
},
"redirectURL": "/",
"headersSize": -1,
"bodySize": -1,
"_transferSize": 553
},
I was trying to get all properties by HarLib java library.
code
ObjectMapper mapper = new ObjectMapper();
mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
System.out.println("Harpath : " + harPath);
File f = new File(harPath);
HarFileReader r = new HarFileReader();
HarLog log = r.readHarFile(f);
HarEntries entries = log.getEntries();
System.out.println(entries);
// HarHeaders s = entries.getEntries().get(1).getResponse().getHeaders();
// List<HarHeader> ss = s.getHeaders();
// for (HarHeader harHeader : ss) {
// System.out.println(harHeader.getValue());
// }
// System.out.println("elements on entries : " + entries.getEntries().size());
//System.out.println(ss.get(5).getName() +"="+ss.get(5).getValue());
can anyone help me to get values of each KEYS, so that can get map and get all values?
It seems like you tried to customize the ObjectMapper but actually never used it. Try something like this:
ObjectMapper mapper = new ObjectMapper();
mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
// ...
File f = new File(harPath);
JsonParser parser = mapper.getFactory().createParser(f);
List<HarWarning> warnings = new ArrayList<HarWarning>();
HarLog log = r.readHarFile(parser, warnings);
// ...
How to use mongodb java driver to compare dayOfYear of two ISODate objects?
Here are my docs
{"name": "hello", "count": 4, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
{"name": "hello", "count": 5, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
{"name": "goodbye", "count": 6, "TIMESTAMP": ISODate("2017-10-01T02:00:35.098Z")}
{"name": "foo", "count": 6, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
I want to compare the day in "TIMESTAMP" to perform some aggregation
Bson match = Aggregates.match(eq("name": "hello"));
Bson group = Aggregates.group(new Document("name", "$name"), Accumulators.sum("total", 1));
collection.aggregate(Arrays.asList(match, group))
Now I am not sure how to do this aggregation for all the records that belongs to particular day?
so my expected result for "2017-10-02" is
[{"_id": {"name":"hello"}, "total": 9}, {"_id": {"name":"foo"}, "total": 6}]
Given the following documents:
{"name": "hello", "count": 4, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
{"name": "hello", "count": 5, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
{"name": "goodbye", "count": 6, "TIMESTAMP": ISODate("2017-10-01T02:00:35.098Z")}
{"name": "foo", "count": 6, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
The following command ...
db.getCollection('dayOfYear').aggregate([
// project dayOfYear as an attribute
{ $project: { name: 1, count: 1, dayOfYear: { $dayOfYear: "$TIMESTAMP" } } },
// match documents with dayOfYear=275
{ $match: { dayOfYear: 275 } },
// sum the count attribute for the selected day and name
{ $group : { _id : { name: "$name" }, total: { $sum: "$count" } } }
])
... will return:
{
"_id" : {
"name" : "foo"
},
"total" : 6
}
{
"_id" : {
"name" : "hello"
},
"total" : 9
}
I think this meets the requirement expressed in your OP.
Here's the same command expressed using the MongoDB Java driver:
MongoCollection<Document> collection = mongoClient.getDatabase("stackoverflow").getCollection("dayOfYear");
Document project = new Document("name", 1)
.append("count", 1)
.append("dayOfYear", new Document("$dayOfYear", "$TIMESTAMP"));
Document dayOfYearMatch = new Document("dayOfYear", 275);
Document grouping = new Document("_id", "$name").append("total", new Document("$sum", "$count"));
AggregateIterable<Document> documents = collection.aggregate(Arrays.asList(
new Document("$project", project),
new Document("$match", dayOfYearMatch),
new Document("$group", grouping)
));
for (Document document : documents) {
logger.info("{}", document.toJson());
}
Update based on this comment:
One of the problems with project is that it only include fields you specify . The above input is just an example. I have 100 fields in my doc I can't sepecify every single one so if I use project I have to specify all 100 fields in addition to "dayOfYear" field. – user1870400 11 mins ago
You can use the following command to return the same output but without a $project stage:
db.getCollection('dayOfYear').aggregate([
// ignore any documents which do not match dayOfYear=275
{ "$redact": {
"$cond": {
if: { $eq: [ { $dayOfYear: "$TIMESTAMP" }, 275 ] },
"then": "$$KEEP",
"else": "$$PRUNE"
}
}},
// sum the count attribute for the selected day
{ $group : { _id : { name: "$name" }, total: { $sum: "$count" } } }
])
Here's that command in its 'Java form':
MongoCollection<Document> collection = mongoClient.getDatabase("stackoverflow").getCollection("dayOfYear");
Document redact = new Document("$cond", new Document("if", new Document("$eq", Arrays.asList(new Document("$dayOfYear", "$TIMESTAMP"), 275)))
.append("then", "$$KEEP")
.append("else", "$$PRUNE"));
Document grouping = new Document("_id", "$name").append("total", new Document("$sum", "$count"));
AggregateIterable<Document> documents = collection.aggregate(Arrays.asList(
new Document("$redact", redact),
new Document("$group", grouping)
));
for (Document document : documents) {
logger.info("{}", document.toJson());
}
Note: Depending on the size of your collection/your non functional requirements/etc you may want to consider the performance of these solutions and either (a) add a match stage before you start projecting/redacting or (b) extract dayOfYear into its own attribute so that you can avoid this complexity entirely.
I have a survey document in mongodb, each survey have surveyRefId for unique identification. I am not able to understand how to find sub-documents having questionType = hard in documents whose surveyRefid = 377 or 360.
Here is a sample document:
{
"json": {
"surveyRefId": 377,
"surveyName": "survey on sociology",
"questionsVoList": [
{
"questionId": "556708425215763c64b8af3d",
"questionText": "question no 1",
"questionTitle": "",
"questionType": "hard",
"preQuestion": true,
"questionOptions": [
{
"questionRefId": 0,
"optionType": "RADIOBUTTON",
"isanswer": true,
"optionText": "ch1"
}
]
},
{
"questionId": "556708425215763c64b8af3d",
"questionText": "question no 2",
"questionTitle": "",
"questionType": "simple",
"question": true,
"questionOptions": [
{
"questionRefId": 0,
"optionType": "RADIOBUTTON",
"isanswer": true,
"optionText": "ch1"
}
],
},
{
"questionId": "556708425215763c64b8af3d",
"questionText": "question no 3",
"questionTitle": "",
"questionType": "hard",
"questionOptions": [
{
"questionRefId": 0,
"optionType": "RADIOBUTTON",
"isanswer": true,
"optionText": "ch1"
},
{
"questionRefId": 0,
"optionType": "RADIOBUTTON",
"isanswer": false,
"optionText": "ch2"
}
],
}
]
}
}
EDIT-- Solution by using Java driver as per Sylvain Leroux
BasicDBObject matchSurvey = new BasicDBObject();
matchSurvey.put("$match", new BasicDBObject("json.surveyRefId", new BasicDBObject("$in", new Integer[]{377,360})));
BasicDBObject unwind = new BasicDBObject();
unwind.put("$unwind", "$json.questionsVoList");
BasicDBObject matchQuestion = new BasicDBObject();
matchQuestion.put("$match", new BasicDBObject("json.questionsVoList.questionType", "hard"));
HashMap map = new HashMap();
map.put("_id", "$_id");
map.put("questionsVoList", new BasicDBObject("$push", "$json.questionsVoList"));
BasicDBObject group = new BasicDBObject();
group.put("$group",map);
List<BasicDBObject> list = new ArrayList<BasicDBObject>();
list.add(matchSurvey);
list.add(unwind);
list.add(matchQuestion);
list.add(group);
AggregateIterable output = collection.aggregate(list, DBObject.class);
"find sub-documents having questionType = "hard"" can be understood in three different ways:
All documents having a "hard" query
If you only want all documents having an "hard query, you will use find and $elemMatch:
db.test.find({"json.surveyRefId": { $in: [377, 360]},
"json.questionsVoList": {$elemMatch: {"questionType":"hard"}}})
First "hard" query of a document
If you need to find the first "hard" query of a document, you will combine the above query with the $ projection operator:
db.test.find({"json.surveyRefId": { $in: [377, 360]},
"json.questionsVoList": {$elemMatch: {"questionType":"hard"}}}
{"json.surveyRefId":1, "json.questionsVoList.$":1})
All hard queries
If you need to find all "hard" queries of the document, you will have to use the aggregation framework:
db.test.aggregate({$match: { "json.surveyRefId": { $in: [377, 360]} }},
{$unwind: "$json.questionsVoList"},
{$match: { "json.questionsVoList.questionType": "hard"}},
{$group: {_id: "$_id", questionsVoList: {$push: "$json.questionsVoList"}}}
)
The first $match step will filter out unwanted documents based on their surveyRefId
Then $unwind will produce a document for each sub-document
An other $match filters out unwanted documents based on questionType
Finally, a $group will combine all sub-documents back as one for a given _id
Producing:
{
"_id" : ObjectId("556828d002509ae174742d11"),
"questionsVoList" : [
{
"questionId" : "556708425215763c64b8af3d",
"questionText" : "question no 1",
"questionTitle" : "",
"questionType" : "hard",
"preQuestion" : true,
"questionOptions" : [
{
"questionRefId" : 0,
"optionType" : "RADIOBUTTON",
"isanswer" : true,
"optionText" : "ch1"
}
]
},
{
"questionId" : "556708425215763c64b8af3d",
"questionText" : "question no 3",
"questionTitle" : "",
"questionType" : "hard",
"questionOptions" : [
{
"questionRefId" : 0,
"optionType" : "RADIOBUTTON",
"isanswer" : true,
"optionText" : "ch1"
},
{
"questionRefId" : 0,
"optionType" : "RADIOBUTTON",
"isanswer" : false,
"optionText" : "ch2"
}
]
}
]
}
I've been looking for this question one week and I can't understand why it still don't work...
I have this object into my MongoDB database:
{
produc: [
{
cod_prod: "0001",
description: "Ordenador",
price: 400,
current_stock: 3,
min_stock: 1,
cod_zone: "08850"
},
{
cod_prod: "0002",
description: "Secador",
price: 30,
current_stock: 10,
min_stock: 2,
cod_zone: "08870"
},
{
cod_prod: "0003",
description: "Portatil",
price: 500,
current_stock: 8,
min_stock: 4,
cod_zone: "08860"
},
{
cod_prod: "0004",
description: "Disco Duro",
price: 100,
current_stock: 20,
min_stock: 5,
cod_zone: "08850"
},
{
cod_prod: "0005",
description: "Monitor",
price: 150,
current_stock: 0,
min_stock: 2,
cod_zone: "08850"
}
]
}
I would like to query for array elements with specific cod_zone ("08850") for example.
I found the $elemMatch projection that supposedly should return just the array elements which match the query, but I don't know why I'm getting all object.
This is the query I'm using:
db['Collection_Name'].find(
{
produc: {
$elemMatch: {
cod_zone: "08850"
}
}
}
);
And this is the result I expect:
{ produc: [
{
cod_prod: "0001",
denominacion: "Ordenador",
precio: 400,
stock_actual: 3,
stock_minimo: 1,
cod_zona: "08850"
},{
cod_prod: "0004",
denominacion: "Disco Duro",
precio: 100,
stock_actual: 20,
stock_minimo: 5,
cod_zona: "08850"
},
{
cod_prod: "0005",
denominacion: "Monitor",
precio: 150,
stock_actual: 0,
stock_minimo: 2,
cod_zona: "08850"
}]
}
I'm making a Java program using MongoDB Java Connector, so I really need a query for java connector but I think I will be able to get it if I know mongo query.
Thank you so much!
This is possible through the aggregation framework. The pipeline passes all documents in the collection through the following operations:
$unwind operator - Outputs a document for each element in the produc array field by deconstructing it.
$match operator will filter only documents that match cod_zone criteria.
$group operator will group the input documents by a specified identifier expression and applies the accumulator expression $push to each group:
$project operator then reconstructs each document in the stream:
db.collection.aggregate([
{
"$unwind": "$produc"
},
{
"$match": {
"produc.cod_zone": "08850"
}
},
{
"$group":
{
"_id": null,
"produc": {
"$push": {
"cod_prod": "$produc.cod_prod",
"description": "$produc.description",
"price" : "$produc.price",
"current_stock" : "$produc.current_stock",
"min_stock" : "$produc.min_stock",
"cod_zone" : "$produc.cod_zone"
}
}
}
},
{
"$project": {
"_id": 0,
"produc": 1
}
}
])
will produce:
{
"result" : [
{
"produc" : [
{
"cod_prod" : "0001",
"description" : "Ordenador",
"price" : 400,
"current_stock" : 3,
"min_stock" : 1,
"cod_zone" : "08850"
},
{
"cod_prod" : "0004",
"description" : "Disco Duro",
"price" : 100,
"current_stock" : 20,
"min_stock" : 5,
"cod_zone" : "08850"
},
{
"cod_prod" : "0005",
"description" : "Monitor",
"price" : 150,
"current_stock" : 0,
"min_stock" : 2,
"cod_zone" : "08850"
}
]
}
],
"ok" : 1
}
I'm using ElasticSearch 1.4.2 and want to upsert some document in json format to it. When I try to insert a test document I can only find it via id, but not via search of the logName field or others that I tried. Probably I miss a step in the upsert method, it's at the end of the question.
Here the query results:
$ curl -XGET http://localhost:9200/pwotest/_search?q=logName:%22pwotest1%22\&pretty=true
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" : [ ]
}
}
Searching for the id results in a document where logName is pwotest1:
$ curl -XGET http://localhost:9200/pwotest/_search?q=\$oid:%22549954143004ba1bf99a56ba%22\&pretty=true
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 3.1972246,
"hits": [
{
"_index": "pwotest",
"_type": "User",
"_id": "549954143004ba1bf99a56ba",
"_score": 3.1972246,
"_source": {
"_id": {
"$oid": "549954143004ba1bf99a56ba"
},
"logName": "pwotest1",
"modifiedBy": "test",
"modifiedId": "549954143004ba1bf99a56ba",
"modificationDate": 1419334676507,
"internalType": "create",
"dm_Version": "0.8.2",
"creationDate": 1419334676515,
"createdId": "549954143004ba1bf99a56ba"
}
}
]
}
}
The code to upsert the document is in Java and looks like this:
/**
* See ES doc
* #param o is a representation of PWO object
* #throws PWOException
*/
public void update(JsonObject o) throws PWOException {
Preconditions.checkNotNull(index, "index must not be null for update");
getNode();
Client client = node.client();
// this needs to come from somewhere
String type = "User";
String id = GsonHelper.getId(o).get();
String json = new Gson().toJson(o);
IndexRequest indexRequest = new IndexRequest(index, type, id).
source(json);
UpdateRequest upd = new UpdateRequest(index, type, id).
doc(json).
upsert(indexRequest);
Logger.info("Update is %s [%s, %s, %s]", upd, index, type, id);
try {
client.update(upd).get();
} catch (InterruptedException | ExecutionException e) {
throw new PWOException(GsonHelper.createMessageJsonObject("Update in elastic search failed"), e);
}
}
it looks like your _id in your source is messed up:
"_id":{"$oid":"549954143004ba1bf99a56ba"}
if you can get this to be a basic value I'm pretty confident your query will work, e.g.:
"_id":"549954143004ba1bf99a56ba"