Develop Reference

How can i remove redundant brackets from my output JSONFILE?

I am using a maeven dependecie that maintain the order of key values of java, but the prblem is that display a double bracket when i m usin a jsonArray. Can someone tell me where my mistake is.
mssgErreurResult() is a method that return an Arraylist that contains all my error messages. Just imagine that Errors ... in the JsonFile is the return of that method
What i get :
{
"complet": false,
"erreurs": [
[
"Errors ..."
]
]
}
What i want:
{
"complet": false,
"erreurs":[
"Errors ..."
]
}
The method i used:
public void ecrireFichierJSon(String fichierSortie) throws
FichierJsonInvalideException, org.codehaus.jettison.json.JSONException {
org.codehaus.jettison.json.JSONObject completOrNot = new org.codehaus.jettison.json.JSONObject();
completOrNot.put("complet", mssgErreurResult().size() <= 0);
org.codehaus.jettison.json.JSONArray messagesErreur = new org.codehaus.jettison.json.JSONArray();
messagesErreur.put(mssgErreurResult());
completOrNot.put("erreurs", messagesErreur);
try {
Files.write(Paths.get(fichierSortie), completOrNot.toString().getBytes());
} catch (IOException e) {
throw new FichierJsonInvalideException(e.toString());
}
}

It is added with extra [ ... ] by org.codehaus.jettison.json.JSONArray messagesErreur, try to remove it and use the following instead
completOrNot.put("erreurs", mssgErreurResult());

Using computed properties with criteria query

I have a mongodb collection. The documents have two fields called rtd ( number of days, int value) and timestamp (long value). I need to get all the documents satisfy this condition using Criteria query
if a document is x
currentTimestamp - x.timestamp converted to days < x.rtd
try {
return mongoTemplate.find(query(criteria), PredictiveEntity.class).stream().filter(predictiveEntity ->
predictiveEntity.getRtd() >= TimeUnit.DAYS.convert(Instant.now().toEpochMilli() - predictiveEntity.getTimestamp(), TimeUnit.MILLISECONDS)
).collect(Collectors.toList());
} catch (Exception e) {
return null;
}

The following query can get you the expected output:
db.predictiveentry.find({
$expr:{
$lt:[
{
$toInt:{
$divide:[
{
$subtract:[new Date().getTime(), "$timestamp"]
},
86400000
]
}
},
"$rtd"
]
}
})
Since $expr is still not supported in Criteria, we need to follow a different route i.e parse the BSON query directly.
Query query = new BasicQuery("{ $expr:{ $lt:[ { $toInt:{ $divide:[ { $subtract:[new Date().getTime(), '$timestamp'] }, 86400000 ] } }, '$rtd' ] } }");
return mongoTemplate.find(query, PredictiveEntity.class).stream().collect(Collectors.toList());

How do I write mongo aggregation reduce query in Spring?

data in mongo :
enter image description here
db.test2.aggregate([
{
"$project" : {
"contents" : 1,
"comments" : {
"$filter" : {
"input" : "$comments",
"as" : "item",
"cond" : {"$gt" : ['$$item.score', 2]}
},
},
"comments2" : {
"$filter" : {
"input" : "$comments2",
"as" : "item",
"cond" : {"$gt" : ["$$item.score", 5]}
}
}
}
},
{
"$project" : {
"content" : 1,
"commentsTotal" : {
"$reduce" : {
"input" : "$comments",
"initialValue" : 0,
"in" : {"$add" : ["$$value", "$$this.score"]}
}
},
"comments2Total" : {
"$reduce" : {
"input" : "$comments2",
"initialValue" : 0,
"in" : {"$add" : ["$$value", "$$this.score"]}
}
}
}
},
{$skip : 0},
{$limit: 3}
]);
<!-- language: lang-json-->
So you can see, this does the following :
1、filter the comments and comments2 which score is gt 5.
2、count total of the socre in comment array.
and i write the aggregation query in Spring like this:
AggregationExpression reduce = ArithmeticOperators.Add.valueOf("$$value").add("$$this.socre");
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.project().andExclude("_id")
.andInclude("content")
.and("comments").filter("item", ComparisonOperators.Gt.valueOf("item.score").greaterThanValue(3)).as("comments")
.and("comments2").filter("item", ComparisonOperators.Gt.valueOf("item.score").greaterThanValue(3)).as("comments2"),
Aggregation.project("comments", "comments2")
.and(ArrayOperators.Reduce.arrayOf("comments").withInitialValue("0").reduce(reduce)).as("commentsTotal")
);
when i run like up , it will throws exception :
java.lang.IllegalArgumentException: Invalid reference '$$value'!

You can try below aggregation by wrapping $filter inside the $reduce operation.
Something like below
AggregationExpression reduce1 = new AggregationExpression() {
#Override
public DBObject toDbObject(AggregationOperationContext aggregationOperationContext) {
DBObject filter = new BasicDBObject("$filter", new BasicDBObject("input", "$comments").append("as", "item").append("cond",
new BasicDBObject("$gt", Arrays.<Object>asList("$$item.score", 2))));
DBObject reduce = new BasicDBObject("input", filter).append("initialValue", 0).append("in", new BasicDBObject("$add", Arrays.asList("$$value", "$$this.socre")));
return new BasicDBObject("$reduce", reduce);
}
};
Aggregation aggregation = newAggregation(
Aggregation.project().andExclude("_id")
.andInclude("content")
.and(reduce1).as("commentsTotal")
);

This is an old question, but in case some one winds up here like me, here's how I was able to solve it.
You cannot access "$$this" and "$$value" variables directly like this in spring.
AggregationExpression reduce = ArithmeticOperators.Add.valueOf("$$value").add("$$this.socre");
To do this we have to use reduce variable enum, like this:
AggregationExpression reduce = ArithmeticOperators.Add.valueOf(ArrayOperators.Reduce.Variable.VALUE.getTarget()).add(ArrayOperators.Reduce.Variable.THIS.referringTo("score").getTarget());
Hope this helps!

I had to solve next task and hadn't find any solutions. So i hope my answer will help somebody.
User with roles (user have list of rights + list of roles, each role have own list of rights, needed to find full list of rights):
user structure
role structure
First, i lookup roles to roleDto (for example), then i collect rights from roles to 1 list:
ArrayOperators.Reduce reduce = ArrayOperators.Reduce.arrayOf("$roleDto.rights")
.withInitialValue(new ArrayList<>())
.reduce(ArrayOperators.ConcatArrays.arrayOf("$$value").concat("$$this"));
As result in reduce i have this 1 list of rights collected from roles.
After that i make:
SetOperators.SetUnion.arrayAsSet(reduce).union("$rights")
using previous result. Result type is AggregationExpression because AbstractAggregationExpression implements AggregationExpression.
So, finally i get smth like this (sorry for messy code):
private static AggregationExpression getAllRightsForUser() {
// concat rights from list of roles (each role have list of rights) - list of list to list
ArrayOperators.Reduce reduce = ArrayOperators.Reduce.arrayOf("$roleDto.rights")
.withInitialValue(new ArrayList<>())
.reduce(ArrayOperators.ConcatArrays.arrayOf("$$value").concat("$$this"));
// union result with user.rights
return SetOperators.SetUnion.arrayAsSet(reduce).union("$rights");
}
Result of this operation can be finally used somewhere like here ;) :
public static AggregationOperation addFieldOperation(AggregationExpression aggregationExpression, String fieldName) {
return aoc -> new Document("$addFields", new Document(fieldName, aggregationExpression.toDocument(aoc)));
}

I had the same issue, one of the solutions is to create a custom Reduce function, here's Union example:
public class SetUnionReduceExpression implements AggregationExpression {
#Override
public Document toDocument(AggregationOperationContext context) {
return new Document("$setUnion", ImmutableList.of("$$value", "$$this"));
}
}

ElasticSearch completion suggester with Java API

I had tried a few example codes on suggester feature of ElasticSearch on the net but I couldn't solve my problem against the autocomplete solution
my index:
client.prepareIndex("kodcucom", "article", "1")
.setSource(putJsonDocument("ElasticSearch: Java",
"ElasticSeach provides Java API, thus it executes all operations " +
"asynchronously by using client object..",
new Date(),
new String[]{"elasticsearch"},
"Hüseyin Akdoğan")).execute().actionGet();
and I used suggestbuilder to obtain the keyword then scan through the content "field", and here is where the null pointer exception occurs due to no result
CompletionSuggestionBuilder skillNameSuggest = new CompletionSuggestionBuilder("skillNameSuggest");
skillNameSuggest.text("lien");
skillNameSuggest.field("content");
SuggestRequestBuilder suggestRequestBuilder = client.prepareSuggest("kodcucom").addSuggestion(skillNameSuggest);
SuggestResponse suggestResponse = suggestRequestBuilder.execute().actionGet();
Iterator<? extends Suggest.Suggestion.Entry.Option> iterator =
suggestResponse.getSuggest().getSuggestion("skillNameSuggest").iterator().next().getOptions().iterator();
Am I missing some filters or input criteria in order to get result? Any result should ok such as autocomplete or record found.
EDIT 1:
This is where I got the NPE and I could see that none of any result return at suggestResponse from debug mode
Iterator<? extends Suggest.Suggestion.Entry.Option> iterator =
suggestResponse.getSuggest().getSuggestion("skillNameSuggest").iterator().next().getOptions().iterator();
EDIT 2:
I am using 2.1.1 version of ElasticSearch Java API
EDIT 3:
I tried in splitting up the iterator line into several code blocks, the NPE occur at the last line when converting a set of data into iterator, but there is not much helping
Suggest tempSuggest = suggestResponse.getSuggest();
Suggestion tempSuggestion = tempSuggest.getSuggestion("skillNameSuggest");
Iterator tempIterator = tempSuggestion.iterator();
I see that the codes:
SuggestRequestBuilder suggestRequestBuilder = client.prepareSuggest("kodcucom").addSuggestion(skillNameSuggest);
SuggestResponse suggestResponse = suggestRequestBuilder.execute().actionGet();
has already consists a empty array/dataset, am I using the suggest request builder incorrectly?

In order to use completion feature, you need to dedicate one field, which will be called completion and you have to specify a special mapping for it.
For example:
"mappings": {
"article": {
"properties": {
"content": {
"type": "string"
},
"completion_suggest": {
"type": "completion"}
}
}
}
The completion_suggest field is the field we will use for the autocomplete function in the above code sample. After this mapping defination, the data must be indexing as follow:
curl -XPOST localhost:9200/kodcucom/article/1 -d '{
"content": "elasticsearch",
"completion_suggest": {
"input": [ "es", "elastic", "elasticsearch" ],
"output": "ElasticSearch"
}
}'
Then Java API can be used as follows for get suggestions:
CompletionSuggestionBuilder skillNameSuggest = new CompletionSuggestionBuilder("complete");
skillNameSuggest.text("es");
skillNameSuggest.field("completion_suggest");
SearchResponse searchResponse = client.prepareSearch("kodcucom")
.setTypes("article")
.setQuery(QueryBuilders.matchAllQuery())
.addSuggestion(skillNameSuggest)
.execute().actionGet();
CompletionSuggestion compSuggestion = searchResponse.getSuggest().getSuggestion("complete");
List<CompletionSuggestion.Entry> entryList = compSuggestion.getEntries();
if(entryList != null) {
CompletionSuggestion.Entry entry = entryList.get(0);
List<CompletionSuggestion.Entry.Option> options =entry.getOptions();
if(options != null) {
CompletionSuggestion.Entry.Option option = options.get(0);
System.out.println(option.getText().string());
}
}

Following link provides you the details of how to create a suggester index. https://www.elastic.co/blog/you-complete-me
Now, I use asynchronous Suggestionbuilder Java API to generate suggestions based on terms.
SearchRequestBuilder suggestionsExtractor = elasticsearchService.suggestionsExtractor("yourIndexName", "yourIndexType//not necessary", "name_suggest", term);
System.out.println(suggestionsExtractor);
Map<String,Object> suggestionMap = new HashMap<>();
suggestionsExtractor.execute(new ActionListener<SearchResponse>() {
#Override
public void onResponse(SearchResponse searchResponse) {
if(searchResponse.status().equals(RestStatus.OK)) {
searchResponse.getSuggest().getSuggestion("productsearch").getEntries().forEach(e -> {
e.getOptions().forEach(s -> {
ArrayList<Object> contents = new ArrayList<>();
suggestionMap.put(s.getText().string(), s.getScore());
});
});
}
}
#Override
public void onFailure(Exception e) {
Helper.sendErrorResponse(routingContext,new JsonObject().put("details","internal server error"));
e.printStackTrace();
}
});
Following is how suggestionbuilder is created.
public SearchRequestBuilder suggestionsExtractor(String indexName, String typeName, String field, String term) {
CompletionSuggestionBuilder csb = SuggestBuilders.completionSuggestion(field).text(term);
SearchRequestBuilder suggestBuilder = client.prepareSearch()
.suggest(new SuggestBuilder().addSuggestion(indexName, csb));
return suggestBuilder;
}

Pretty print for a groovy ConfigObject?

I have this groovy program that creates a groovy configuration file, using a ConfigObject. Once the ConfigObject is set up, it is written to a file using:
myFile.withWriter {writer -> myConfigObject.writeTo(writer)}
This results in each property of the ConfigObject being written on a single line. So for instance a map will be printed as:
graphs=[["type":"std", "host":"localhost", "name":"cpurawlinux"], ["type":"std", "host":"localhost", "name":"memory"], ["type":"std", "host":"localhost", "name":"udp"] ... ]
which is quite unreadable if someone has to take a look at it.
Is there a way to get a more friendly output? Something like that would be great:
graphs=[
["type":"std", "host":"localhost", "name":"cpurawlinux"],
["type":"std", "host":"localhost", "name":"memory"],
["type":"std", "host":"localhost", "name":"udp"]
...
]
I know I could create my own writeTo, but isn't there already something in Groovy for that?

Unfortunately, you'll need to write your own writeTo as you say.
If you have a config file with structure like:
graphs {
a=["type":"std", "host":"localhost", "name":"cpurawlinux"]
b=["type":"std", "host":"localhost", "name":"memory"]
}
Then writeTo will write it out with structure, but if your config file is just a big old list of things, it will write it out as a big old list

if it helps anyone, i had the same question and wrote this...not pretty (ha), but works:
def prettyPrint(properties, level=1, stringBuilder = new StringBuilder()) {
return properties.inject(stringBuilder) { sb, name, value ->
sb.append("\n").append("\t" * level).append(name)
if (!(value instanceof Map) && !(value instanceof List)) {
return sb.append("=").append(value)
} else {
return prettyPrint(properties.getProperty(name), level+1, sb)
}
}
}

Based on mike's answer above:
def prettyPrint
prettyPrint = {obj, level = 0, sb = new StringBuilder() ->
def indent = { lev -> sb.append(" " * lev) }
if(obj instanceof Map){
sb.append("{\n")
obj.each{ name, value ->
if(name.contains('.')) return // skip keys like "a.b.c", which are redundant
indent(level+1).append(name)
(value instanceof Map) ? sb.append(" ") : sb.append(" = ")
prettyPrint(value, level+1, sb)
sb.append("\n")
}
indent(level).append("}")
}
else if(obj instanceof List){
sb.append("[\n")
obj.each{ value ->
indent(level+1)
prettyPrint(value, level+1, sb).append(",")
sb.append("\n")
}
indent(level).append("]")
}
else if(obj instanceof String){
sb.append('"').append(obj).append('"')
}
else {
sb.append(obj)
}
}
For an input like:
{
grails {
scaffolding {
templates.domainSuffix = "Instance"
}
enable {
native2ascii = true
blah = [ 1, 2, 3 ]
}
mime.disable.accept.header.userAgents = [ "WebKit", "Presto", "Trident" ]
}
}
Produces:
{
grails {
scaffolding {
templates {
domainSuffix = "Instance"
}
}
enable {
native2ascii = true
blah = [
1,
2,
3,
]
}
mime {
disable {
accept {
header {
userAgents = [
"WebKit",
"Presto",
"Trident",
]
}
}
}
}
}
}