How can I aggregate by Hour in the mongodb-async-driver (http://www.allanbank.com/mongodb-async-driver/usage.html)
I have an ISODate-Field in my Collection.
[
{ name = "a", date = ISODate(...)},
{ name = "b", date = ISODate(...)},
...
]
I want to display a graph of how may documents occur per hour.
in the MongoDB-Console. I would do something like this:
db.mycollection.aggregate([{$group : {_id : {day:{ $hour : "$date"}}, count: { $sum: 1 }}}])
but I get stuck at the driver-api:
import static com.allanbank.mongodb.builder.AggregationGroupField.set;
import static com.allanbank.mongodb.builder.AggregationGroupId.id;
Aggregate.Builder builder = new Aggregate.Builder();
builder.group(id().add(???), set("pop").sum("pop"))
You need to make use of the Expressions class. Make use of the group method that takes a Builder and the AggregationGroupField array as input.
public Aggregate.Builder group(AggregationGroupId.Builder id,
AggregationGroupField... aggregations)
Build the hour Expression and pass it as the id.
Builder hour = new Builder();
hour.add(Expressions.set("day",Expressions.hour(Expressions.field("date"))));
Aggregate.Builder builder = Aggregate.builder();
builder.group(
hour,
AggregationGroupField.set("pop").sum("pop")
);
MongoIterator<Document> result = col.aggregate(builder);
while(result.hasNext())
{
System.out.println(result.next());
};
Related
I have API that post and get dates:
this is the data class:
data class PlannerGet(
val date: String,
val endTime: String,
val id: Int,
val location: String,
val note: String,
val startTime: String,
val title: String
)
and i am using this library for the calendar:
https://github.com/VarunBarad/Highlightable-Calendar-View
now in the fragment i was able to highlight certain days like this:
HighlightableCalendarView.dayDecorators = listOf(
DayDecorator(
Calendar.getInstance().apply {
set(Calendar.DAY_OF_MONTH, 4)
},
Color.parseColor("#ffffff"),
Color.parseColor("#ff0000")
),
)
but i want to highlight the days from API
i tried to make it like this:
HighlightableCalendarView.dayDecorators = listOf(
DayDecorator(
Calendar.getInstance().apply {
set(PlannerGet.date)
},
Color.parseColor("#ffffff"),
Color.parseColor("#ff0000")
),
)
but i am having a problem with "set" it show "None of the following functions can be called with the arguments supplied."
i tried to add "toInt()" and still the same problem.
what is the correct way to achieve this?
This is because the params which you are passing do not match the required params.
Calendar.getInstance().apply {
set(Calendar.DAY_OF_MONTH, 4)
}
The set functions accept the int field, int value but you are passing the params as the string
PlannerGet.date
The function set
public void set(int field, int value) {
throw new RuntimeException("Stub!");
}
If you want the date to be passed from the API dates please convert the string dates to java Date object.
SOLUTION:
if (response?.body().toString() == "[]") {
}
else if (response.isSuccessful) {
response.body()?.forEach {
getplanner.add(it)
Log.e("gggg gggg",getplanner.toString())
Log.e("gggg ddddd",getplanner[0].date)
}
val list = arrayListOf<DayDecorator>()
for (dsds in getplanner) {
list.add( DayDecorator(
Calendar.getInstance().apply {
// getplanner[0].date
val input_date = dsds.date
val format1 = SimpleDateFormat("yyyy-MM-dd")
var dt1: Date? = null
dt1 = format1.parse(input_date)
val format2: DateFormat = SimpleDateFormat("dd")
val strMonth: String = format2.format(dt1)
val month = strMonth.toInt()
Log.e("dateinplanner", "" + month)
set(Calendar.DAY_OF_MONTH, month)
},
Color.parseColor("#ffffff"),
Color.parseColor("#1AB7B8")
))
}
HighlightableCalendarView.dayDecorators = list
I have implemented an interface. There I am passing timestamp as parameter to a function. For geting the current time, I have used Instant.now().toString().
The function looks like this:
fun createId() {
val values = Record(name = "ABC", timestamp = Instant.now().toString())
interface_name.store(values)
}
#Test
fun `test1 for createId`() {
val values = Record(name = "ABC", timestamp = "2020-12-28")
every { interface_name.store(values) } just runs
}
This is giving me: Verification failed because the timestamp is different. I am unable to figure out how do I mock the Instant.now() so that I get the static timestamp while testing. Any help is appreciated.
Unit testing is easy when all the dependencies are passed as parameters. Hence instead of creating objects within the function, it is always good to design the function based on abstraction and pass the dependency as arguments.
fun createId(clock: Clock = Clock.systemUTC()): Record {
val values = Record(name = "ABC", timestamp = Instant.now(clock).toString())
interface_name.store(values)
return values
}
#Test
fun `test1 for createId`() {
val clock = Clock.fixed(Instant.parse("2021-01-01T00:00:00Z"), UTC)
val record = createId(clock)
assertEquals(Record(name = "ABC", timestamp = "2021-01-01T00:00:00Z"), record)
}
I have spring-data-mogodb application on java or kotlin, and need create text search request to mongodb by spring template.
In mongo shell it look like that:
db.stores.find(
{ $text: { $search: "java coffee shop" } },
{ score: { $meta: "textScore" } }
).sort( { score: { $meta: "textScore" } } )
I already tried to do something but it is not exactly what i need:
#override fun getSearchedFiles(searchQuery: String, pageNumber: Long, pageSize: Long, direction: Sort.Direction, sortColumn: String): MutableList<SystemFile> {
val matching = TextCriteria.forDefaultLanguage().matching(searchQuery)
val match = MatchOperation(matching)
val sort = SortOperation(Sort(direction, sortColumn))
val skip = SkipOperation((pageNumber * pageSize))
val limit = LimitOperation(pageSize)
val aggregation = Aggregation
.newAggregation(match, skip, limit)
.withOptions(Aggregation.newAggregationOptions().allowDiskUse(true).build())
val mappedResults = template.aggregate(aggregation, "files", SystemFile::class.java).mappedResults
return mappedResults
}
May be someone already working with text searching on mongodb with java, please share your knowledge with us )
Setup Text indexes
First you need to set up text indexes on the fields on which you want to perform your text query.
If you are using Spring data mongo to insert your documents in your database, you can use #TextIndexed annotation and indexes will be built while inserting your document.
#Document
class MyObject{
#TextIndexed(weight=3) String title;
#TextIndexed String description;
}
If your document are already inserted in your database, you need to build your text indexes manually
TextIndexDefinition textIndex = new TextIndexDefinitionBuilder()
.onField("title", 3)
.onField("description")
.build();
After the build and config of your mongoTemplate you can pass your text indexes/
template.indexOps(MyObject.class).ensureIndex(textIndex);
Building your text query
List<MyObject> getSearchedFiles(String textQuery){
TextQuery textQuery = TextQuery.queryText(new TextCriteria().matchingAny(textQuery)).sortByScore();
List<MyObject> result = mongoTemplate.find(textQuery, MyObject.class, "myCollection");
return result
}
My mapping of createdAt:
"createdAt": {
"type": "date"
},
I insert the dates like this:
POST logs/_doc/_bulk?pretty
{"index":{"_id":1}}
{"createdAt":"2018-05-01T07:30:00Z","value":"on"}
When I request the documents
GET logs/_doc/_search
It shows me the date as I inserted it:
"_source": {
"createdAt": "2018-05-01T07:30:00Z",
"value":"on"
}
Now I'd like to compare this date with the current time:
"map_script": {
long timestampLog = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.S").parse(doc.createdAt.value).getTime();
long timestampNow = new Date().getTime();
if (timestampNow < timestampLog) {
// case 1
} else {
// case 2
}
}
Weird:
doc.createdAt.value returns "2018-05-01T07:30:00.000Z", which includes milliseconds that I never added.
This error occurs while parsing:
Cannot cast org.joda.time.MutableDateTime to java.lang.String
When I replace doc.createdAt.value by the string 2018-05-01T07:30:00.000Z, it works.
Any help is appreciated. Thank you very much!
Elasticsearch index fields with date types are org.joda.time.DateTime in painless. Using the SimpleDateFormat is the source of the error. Try this instead:
long timestampLog = doc['createdAt'].value.getMillis();
long timestampNow = new Date().getTime();
The rest works as is.
Tested on Elasticsearch 6.3.0.
Please remove the big S in the formatter, check Date and Time Patterns
long timestampLog = new SimpleDateFormat("yyyy-MM-dd'T'hh:mm:ss").parse(doc.createdAt.value).getTime();
long timestampNow = new Date().getTime();
data in mongo :
enter image description here
db.test2.aggregate([
{
"$project" : {
"contents" : 1,
"comments" : {
"$filter" : {
"input" : "$comments",
"as" : "item",
"cond" : {"$gt" : ['$$item.score', 2]}
},
},
"comments2" : {
"$filter" : {
"input" : "$comments2",
"as" : "item",
"cond" : {"$gt" : ["$$item.score", 5]}
}
}
}
},
{
"$project" : {
"content" : 1,
"commentsTotal" : {
"$reduce" : {
"input" : "$comments",
"initialValue" : 0,
"in" : {"$add" : ["$$value", "$$this.score"]}
}
},
"comments2Total" : {
"$reduce" : {
"input" : "$comments2",
"initialValue" : 0,
"in" : {"$add" : ["$$value", "$$this.score"]}
}
}
}
},
{$skip : 0},
{$limit: 3}
]);
<!-- language: lang-json-->
So you can see, this does the following :
1、filter the comments and comments2 which score is gt 5.
2、count total of the socre in comment array.
and i write the aggregation query in Spring like this:
AggregationExpression reduce = ArithmeticOperators.Add.valueOf("$$value").add("$$this.socre");
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.project().andExclude("_id")
.andInclude("content")
.and("comments").filter("item", ComparisonOperators.Gt.valueOf("item.score").greaterThanValue(3)).as("comments")
.and("comments2").filter("item", ComparisonOperators.Gt.valueOf("item.score").greaterThanValue(3)).as("comments2"),
Aggregation.project("comments", "comments2")
.and(ArrayOperators.Reduce.arrayOf("comments").withInitialValue("0").reduce(reduce)).as("commentsTotal")
);
when i run like up , it will throws exception :
java.lang.IllegalArgumentException: Invalid reference '$$value'!
You can try below aggregation by wrapping $filter inside the $reduce operation.
Something like below
AggregationExpression reduce1 = new AggregationExpression() {
#Override
public DBObject toDbObject(AggregationOperationContext aggregationOperationContext) {
DBObject filter = new BasicDBObject("$filter", new BasicDBObject("input", "$comments").append("as", "item").append("cond",
new BasicDBObject("$gt", Arrays.<Object>asList("$$item.score", 2))));
DBObject reduce = new BasicDBObject("input", filter).append("initialValue", 0).append("in", new BasicDBObject("$add", Arrays.asList("$$value", "$$this.socre")));
return new BasicDBObject("$reduce", reduce);
}
};
Aggregation aggregation = newAggregation(
Aggregation.project().andExclude("_id")
.andInclude("content")
.and(reduce1).as("commentsTotal")
);
This is an old question, but in case some one winds up here like me, here's how I was able to solve it.
You cannot access "$$this" and "$$value" variables directly like this in spring.
AggregationExpression reduce = ArithmeticOperators.Add.valueOf("$$value").add("$$this.socre");
To do this we have to use reduce variable enum, like this:
AggregationExpression reduce = ArithmeticOperators.Add.valueOf(ArrayOperators.Reduce.Variable.VALUE.getTarget()).add(ArrayOperators.Reduce.Variable.THIS.referringTo("score").getTarget());
Hope this helps!
I had to solve next task and hadn't find any solutions. So i hope my answer will help somebody.
User with roles (user have list of rights + list of roles, each role have own list of rights, needed to find full list of rights):
user structure
role structure
First, i lookup roles to roleDto (for example), then i collect rights from roles to 1 list:
ArrayOperators.Reduce reduce = ArrayOperators.Reduce.arrayOf("$roleDto.rights")
.withInitialValue(new ArrayList<>())
.reduce(ArrayOperators.ConcatArrays.arrayOf("$$value").concat("$$this"));
As result in reduce i have this 1 list of rights collected from roles.
After that i make:
SetOperators.SetUnion.arrayAsSet(reduce).union("$rights")
using previous result. Result type is AggregationExpression because AbstractAggregationExpression implements AggregationExpression.
So, finally i get smth like this (sorry for messy code):
private static AggregationExpression getAllRightsForUser() {
// concat rights from list of roles (each role have list of rights) - list of list to list
ArrayOperators.Reduce reduce = ArrayOperators.Reduce.arrayOf("$roleDto.rights")
.withInitialValue(new ArrayList<>())
.reduce(ArrayOperators.ConcatArrays.arrayOf("$$value").concat("$$this"));
// union result with user.rights
return SetOperators.SetUnion.arrayAsSet(reduce).union("$rights");
}
Result of this operation can be finally used somewhere like here ;) :
public static AggregationOperation addFieldOperation(AggregationExpression aggregationExpression, String fieldName) {
return aoc -> new Document("$addFields", new Document(fieldName, aggregationExpression.toDocument(aoc)));
}
I had the same issue, one of the solutions is to create a custom Reduce function, here's Union example:
public class SetUnionReduceExpression implements AggregationExpression {
#Override
public Document toDocument(AggregationOperationContext context) {
return new Document("$setUnion", ImmutableList.of("$$value", "$$this"));
}
}