I've got a table File for which I created a hibernate entity with fields:
archiveDate: Date,
createDate: Date,
copyDdate: Date,
modifiedDate: Date,
fileSourceId: Long,
fileSize: Long
I have to create a DAO for retrieving statistic data like how many files were archived in a given period of time in given intervals (daily, weekly, monthly, annually) for specified fileSourceId. Like if dao receives arguments like this:
fileSourceId = 1, startDay = '2012/01/01', endDate = '2012/02/01', interval = 'weekly'
it should return some objects like this:
{fileSourceId = 1, endDate = (end date of the first week), fileCount = 100, sizeSum = 10000 }
{fileSourceId = 1, endDate = (end date of the second week), fileCount = 120, sizeSum = 30000 }
and so on. I would like to avoid doing calculations like counting and summing on the java side as there's a couple of hundred of thousands files.
Is it possible to create a query or criteria to do this in Hibernate or it's better to do some view in database (Oracle) and query this view. I would like to avoid any oracle specific function in query because we don't want to be bound to one database (especially that we use h2 as our testing database in integration testss). How would you resolve this problem?
Related
Working with queryDSL and mySQL:
I have a field from in the time_check table to which I want to add timetrackAutoStop from the location table as hours and then compare it with the current time.
We managed to get queryDSL to understand the following code:
if (data.getAutostopTimeExceeded() != null) {
QLocationEntity location = QLocationEntity.locationEntity;
query.join(TIME_CHECK.location, location);
Expression<LocalDateTime> currentTimeExpr = Expressions.asDateTime(LocalDateTime.now());
TIME_CHECK.from, Expressions.asNumber(location.timetrackAutoStop));
predicates.add(
Expressions.predicate(Ops.GT,
currentTimeExpr,
Expressions.dateTimeOperation(LocalDateTime.class,
Ops.DateTimeOps.ADD_HOURS,
TIME_CHECK.from,
Expressions.asNumber(location.timetrackAutoStop)
)
)
);
}
However it then generates SQL which causes the mySQL database to complain that the ADD_HOURS function doesn't exist.
Is there are way to get what I'm looking for? I.E. all time_check_records older than now() minus timeTrackAutoStop ?
Update:
We tried the following, but get an error message stating: expecting CLOSE, found locationEntity:
QLocationEntity qLocationEntity = QLocationEntity.locationEntity;
query.join(TIME_CHECK.location, qLocationEntity);
Expression<LocalDateTime> currentTimeExpr = Expressions.asDateTime(LocalDateTime.now());
Expression<LocalDateTime> subExpr = Expressions.template(LocalDateTime.class,
"DATE_ADD({0}, INTERVAL {1} HOUR)",
TIME_CHECK.from,
qLocationEntity.timetrackAutoStop
);
predicates.add(Expressions.predicate(Ops.GT, currentTimeExpr, subExpr));
The MySQL database does not have a built-in function for adding hours to a date. However, you can use the DATE_ADD() function to add an interval of hours to a date. The syntax for this function is as follows:
DATE_ADD(date, INTERVAL expr unit)
Where date is the date to which you want to add the interval, expr is the number of hours to add, and unit is the unit of time (in this case, 'hour').
I am about 3 days into my experience with this tool. Everything was going great, I have a file with 3 record definitions (header, data, trailer) and am using an InputValueSwitch with 3 beanListProcessors.
The data object has about 40 properties all String except for 1, Date of Birth is defined as a date with a #Format annotation. With patientDateOfBirth defined as a Date I can process 34K rows in a couple of seconds, which is very acceptable.
#Parsed(field = "patientDateOfBirth")
#FixedWidth(from = 148, to = 158)
#Format(formats = {"yyyy-MM-dd"}, options = "lenient=false")
private Date patientDateOfBirth;
But, if I change the data type from Date to String the processing time increases to about 5 minutes for the same 34K rows, which does not make sense to me. Does this ring any bells for anyone?
#Parsed(field = "patientDateOfBirth")
#FixedWidth(from = 148, to = 158)
#Validate(allowBlanks = true)
private String patientDateOfBirth;
Say that I've got a collection of users, each with a birthday in ISODate date format. eg. ISODate("1958-03-23T00:00:00.000Z"). How can I use a jongo aggregate query to get users with birthdays on the current day.This is what I have currently but it doesn't return any results even though there are users with birthdays:
Date dateOfBirth = new Date();
Integer month = new DateTime(dateOfBirth).getMonthOfYear();
Integer day = new DateTime(dateOfBirth).getDayOfMonth();
List<User> users= IteratorUtils.toList(userJongo.aggregate("
{$project:_id:1,dateOfBirth:1,name:1}}")
.and("{$match :{dateOfBirth.getDate(): {$eq: '"+day+"'}}}")
.and("{$match :{dateOfBirth.getMonth()+1: {$eq:'"+month+"'}}}")
.and("{$limit:"+limit+"}"
.as(User.class).iterator());
Thank inadvance.
How about saving user birthdays as strings and add index to that filed.
In this case your performance will increase.
And you sure dont want to use aggregation.
Just before query get current date-> format into string that you save into DB -> run db.find({birthDate: "0425(this is april 25)"})->after getting results do whatever you want.
I'm trying to create an aggregate query using mongotemplate where there's a grouping by date (i.e 2016-03-01) instead of datetime (i.e 2016-03-01 16:40:12).
The dateToString operation exists in the mongodb documentation it can be used to extract the date from the datetime using formatting:
https://docs.mongodb.org/manual/reference/operator/aggregation/dateToString/
but I get get it to work with mongotemplate - I get a NullPointerException.
(my db version is 3.2)
List<AggregationOperation> aggregationOperations = new ArrayList<AggregationOperation>();
aggregationOperations.add(
Aggregation.project("blabla", ...).
andExpression("dateToString('%Y-%m-%d',timeCreated).as("date"));
aggregationOperations.add(Aggregation.group("date").sum("blabla").as("blabla"));
AggregationResults<?> aggregationResults = this.mongoTemplate.aggregate(
Aggregation.newAggregation(aggregationOperations),
collectionName,
resultClass);
When I use dayOfMonth(timeCreated) to extract the day, there's no exception, but I couldn't find and example of how to make this work with dateToString. I tried without '' for the date format, and it also didn't work...
This is the exception I get:
java.lang.NullPointerException
at org.bson.BasicBSONEncoder._putObjectField(BasicBSONEncoder.java:226)
at org.bson.BasicBSONEncoder.putObject(BasicBSONEncoder.java:194)
at org.bson.BasicBSONEncoder._putObjectField(BasicBSONEncoder.java:255)
at org.bson.BasicBSONEncoder.putObject(BasicBSONEncoder.java:194)
at org.bson.BasicBSONEncoder._putObjectField(BasicBSONEncoder.java:255)
at org.bson.BasicBSONEncoder.putObject(BasicBSONEncoder.java:194)
at org.bson.BasicBSONEncoder._putObjectField(BasicBSONEncoder.java:255)
at org.bson.BasicBSONEncoder.putIterable(BasicBSONEncoder.java:324)
at org.bson.BasicBSONEncoder._putObjectField(BasicBSONEncoder.java:263)
at org.bson.BasicBSONEncoder.putObject(BasicBSONEncoder.java:194)
at org.bson.BasicBSONEncoder.putObject(BasicBSONEncoder.java:136)
at com.mongodb.DefaultDBEncoder.writeObject(DefaultDBEncoder.java:36)
at com.mongodb.OutMessage.putObject(OutMessage.java:289)
at com.mongodb.OutMessage.writeQuery(OutMessage.java:211)
at com.mongodb.OutMessage.query(OutMessage.java:86)
at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:81)
at com.mongodb.DB.command(DB.java:320)
at com.mongodb.DB.command(DB.java:299)
at com.mongodb.DB.command(DB.java:374)
at com.mongodb.DB.command(DB.java:246)
at org.springframework.data.mongodb.core.MongoTemplate$2.doInDB(MongoTemplate.java:357)
at org.springframework.data.mongodb.core.MongoTemplate$2.doInDB(MongoTemplate.java:355)
at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:442)
at org.springframework.data.mongodb.core.MongoTemplate.executeCommand(MongoTemplate.java:355)
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:1497)
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:1432)
EDIT:
Eventually we decided here on a different solution than what was suggested below, I'm writing it here in case anyone else finds it useful:
In addition to the "timeCreated" field which holds the datetime, we saved another field in the document: "date", that holds just the date (as long).
For example if "timeCreated" = "2015-12-24 16:36:06.657+02:00", then date is "2015-12-24 00:00:00", and we save 1449180000000.
Now we can simply group by "date".
You could try projecting the fields first by using the SpEL andExpression in the projection operation and then group by the new fields in the group operation:
Aggregation agg = newAggregation(
project()
.andExpression("year(timeCreated)").as("year")
.andExpression("month(timeCreated)").as("month")
.andExpression("dayOfMonth(timeCreated)").as("day"),
group(fields().and("year").and("month").and("day"))
.sum("blabla").as("blabla")
);
AggregationResults<BlaBlaModel> result =
mongoTemplate.aggregate(agg, collectionName, BlaBlaModel.class);
List<BlaBlaModel> resultList = result.getMappedResults();
You could try using the DateOperators.DateToString class
aggregationOperations.add(
Aggregation.project("blabla", ...).
and(DateOperators.DateToString.dateOf("timeCreated").toString("%Y-%m-%d"));
There are many similar questions asked. But not exactly similar to the issue i am facing. I have seen almost all the questions and answers around it
So the problem is
I got to insert a date field in my mongo collection
But I can't access the collection directly. I got to use a service. The service takes a string and returns me oid.
So once i construct the BasicDBObject I call toString on it and pass it on to my service.. I even tried inserting it directly in a test collection and mongo is complaining.
BasicDBObject document = new BasicDBObject();
long createdAtSinceEpoch = 0;
long expiresAtSinceEpoch = 0;
createdAtSinceEpoch = System.nanoTime();
Date createdAt = new Date(TimeUnit.NANOSECONDS.toMillis(createdAtSinceEpoch));
document.append("createdAt", createdAt);
expiresAtSinceEpoch = createdAtSinceEpoch + +TimeUnit.SECONDS.toNanos(30);
Date expiresAt = new Date(TimeUnit.NANOSECONDS.toMillis(expiresAtSinceEpoch));
document.append("expiresAt", expiresAt);
service.storeRecord(document.toString());
and the generated JSON String looks like
{
"createdAt": {
"$date": "2015-09-01T20:05:21.641Z"
},
"expiresAt": {
"$date": "2015-09-01T20:05:51.641Z"
}
and Mongo complains that
Unable to parse JSON : Date expecting integer milliseconds, at (3,17)
So If i pass milliseconds alone instead of date object in the document.append() method then it DOES NOT recognize this field as date and considers it as String but inserts into the collection
I need 2 things
1) I want the data to be inserted
2) I am planning to expire that row by adding an index to the expiresAt field. So I want mongo to recognize that its a date field
JSON makes a difference between a numeric field and a text field containing a number. The latter one is only recognized as a String; I assume that this is what you did when you thought you were giving your service the date as an integer. Unfortunately you didn’t show us the relevant code.
When I save the Date info as a non String format, I annotate the field in my DTO as below. This helps the MongoDB know that the field is to be treated as an ISO date which then would be useful for making range search etc.,
#DateTimeFormat(iso = ISO.DATE_TIME) private Date date;
Date date = new Date();
BasicDBObject date= new BasicDBObject("date", date);
Data.insert(date);