Develop Reference

Spring Boot MongoRepository query via Date with Criteria lt & gte gives wrong result

When I query MongoRepository via Date field with Criteria in a Spring Boot application, the result is wrong. Here is my method:
Query query = new Query(new Criteria().andOperator(
Criteria.where("id").is(filter.getId()),
Criteria.where("datas.ts").lt(filter.getEndTime()).gte(filter.getStartTime())
));
List<PhaseData> phaseDatas = mongoOperations.find(query, PhaseData.class);
List<Data> result = new ArrayList<Data>();
for(Data pData : phaseDatas) {
result.addAll(pData.getDatas());
}
return result;
When I query with
{
"id" : "1234",
"startTime" : "2016-08-04 12:00",
"endTime" : "2016-08-04 15:00"
}
it gives me records with hour 16:54 & 21:12 too. How can I solve this issue?

Not sure if this addresses your question directly.
The DB won't return wrong result to the query. So I think it could be one of the following things:
It could be that the when you view the documents in mongodb, it displays date in iso format. So view the documents in the same format as you are creating dates for your query.
It could be timezone issue.
Mongodb dates can be considered as ISODate (MongoDB Date)
When you query, you create date objects in your timezone. So as a first debugging measure, I would see if both my DB and query timezones are the same.
Also, probably it would help if you query by creating date objects in ISODate by using SimpleDateFormat(SDF is not thread safe).
I have found that it could be confusing because the dates that you send are in a different format and the documents that you visually see in mongodb tool are displaying dates in iso format. I think that it could be the issue. The results are good, but probably you are viewing the two things differently and it causes the confusion.

Mongodb java springdata unable to get result for date equals query

I have inserted some test records to the mongo database with following structure.
{
"_id" : ObjectId("5563fe96a826638b48c77c26"),
"date" : ISODate("2015-05-02T07:00:00.326Z"),
"createdDate" : ISODate("2015-05-26T05:03:18.899Z"),
"updatedDate" : ISODate("2015-05-26T05:03:18.899Z"),
"status" : 0
}
Now when I try to query it using Spring data or via MongoDB I am always getting returned result list size to be 0.
Calendar calendar = Calendar.getInstance();
calendar.set(2015, 4, 2, 0, 0, 0);
Query query = new Query();
query.addCriteria(Criteria.where("date").is(calendar.getTime());
List<DateRecord> attendanceList = findAll(query, DateRecord.class);
System.out.println(attendanceList.size());
I am getting a very similar result for BasicDBObject, list of size 0.
DBCursor cursor;
BasicDBObject query1 = new BasicDBObject();
query1.append("date", calendar.getTime());
cursor = collection.find(query1);
System.out.println("Total objects returned "+cursor.size());
Any pointers on same will be highly appreciated. All I just want that data should be returned based upon year,month and day and any timestamp field values should be ignored.

I suggest using a different query - look for date greater than 2015-4-2 00:00:00 and explicitly less than 2015-4-3:00:00:00
Another approach, that I'm less enthusiastic about, would be to to add a field to the document just for the search purpose (e.g. "dateWithoutHour" calculated by java just before saving a document, and assuming data doesn't arrive from other sources). I don't like it, because I prefer my data to be pure logic and not change any time someone comes up with a new search requirement... but sometimes I had to resort to it).
And as always, when facing a difficult query it's tempting to consider $where , but I won't recommend it because it can't use indices.

Avoid Date Overlapping using JPA

I have an entity class with properties 'code','fromDate' and 'toDate' and i need to insert one new record using JPA such a way that for given code date range should not overlap.
For example
If code- ABC of date range 01/Feb/2014-10/Feb/2014 exist in DB.
I am inserting code ABC again with date range
03/Feb/2014-07/Feb/2014 should not accept - from date and to date is Within existing Date range
28/Jan/2014-02/Feb/2014 should not accept - to date is Within existing Date range
05/Feb/2014-21/Feb/2014 should not accept - From date is Within existing Date range
01/Jan/2014-28/Feb/2014 should not accept - The existing date range is within the given date range so Overlapping will happen.
Suppose the data need to be inserted is in a viewObject with similar properties.
Please help me to do the validation for date overlapping using JPA predicates

Before saving the new object you can query the DB to check if an 'overlapping' records exists.
If, a record is returned, then do not save the new object, else save;
String query = "SELECT ent FROM Entity ent WHERE ent.fromDate <= :toDate AND ent.toDate >= :fromDate WHERE ent.id = :entId";
List<Entity> overlappingRecords = JPA.em().createQuery(query).setParameter("entId", id).setParameter("fromDate", fromDate).setParameter("toDate", toDate).getResultList();
if(overlappingRecords.isEmpty())
//Over lap does not exist
else
//Over lap exists
This query assumes rejection of edges overlapping exactly.

More Efficient Way of Doing This SQL Query? A time comparison query?

I have this SQL query which queries the database every 5 seconds to determine who is currently actively using the software. Active users have pinged the server in the last 10 seconds. (The table gets updated correctly on user activity and a I have a thread evicting entries on session timeouts, that all works correctly).
What I'm looking for is a more efficient/quicker way to do this, since it gets called frequently, about every 5 seconds. In addition, there may be up to 500 users in the database. The language is Java, but the question really pertains to any language.
List<String> r = new ArrayList<String>();
Calendar c = Calendar.getInstance();
long threshold = c.get(Calendar.SECOND) + c.get(Calendar.MINUTE)*60 + c.get(Calendar.HOUR_OF_DAY)*60*60 - 10;
String tmpSql = "SELECT user_name, EXTRACT(HOUR FROM last_access_ts) as hour, EXTRACT(MINUTE FROM last_access_ts) as minute, EXTRACT(SECOND FROM last_access_ts) as second FROM user_sessions";
DBResult rs = DB.select(tmpSql);
for (int i=0; i<rs.size(); i++)
{
Map<String, Object> result = rs.get(i);
long hour = (Long)result.get("hour");
long minute = (Long)result.get("minute");
long second = (Long)result.get("second");
if (hour*60*60 + minute*60 + second > threshold)
r.add(result.get("user_name").toString());
}
return r;

If you want this to run faster, then create an index on user_sessions(last_access_ts, user_name), and do the date logic in the query:
select user_name
from user_sessions
where last_access_ts >= now() - 5/(24*60*60);
This does have a downside. You are, presumably, updating the last_access_ts field quite often. An index on the field will also have to be updated. On the positive side, this is a covering index, so the index itself can satisfy the query without resorting to the original data pages.

I would move the logic from Java to DB. This mean you translate if into where, and just select the name of valid result.
SELECT user_name FROM user_sessions WHERE last_access_ts > ?
In your example the c represent current time. It is highly possible that result will be empty.
So your question should be more about date time operation on your database.

Just let the database do the comparison for you by using this query:
SELECT
user_name
FROM user_sessions
where TIMESTAMPDIFF(SECOND, last_access_ts, current_timestamp) > 10
Complete example:
List<String> r = new ArrayList<String>();
Calendar c = Calendar.getInstance();
long threshold = c.get(Calendar.SECOND) + c.get(Calendar.MINUTE)*60 + c.get(Calendar.HOUR_OF_DAY)*60*60 - 10;
// this will return all users that were inactive for longer than 10 seconds
String tmpSql = "SELECT
user_name
FROM user_sessions
where TIMESTAMPDIFF(SECOND, last_access_ts, current_timestamp) > 10";
DBResult rs = DB.select(tmpSql);
for (int i=0; i<rs.size(); i++)
{
Map<String, Object> result = rs.get(i);
r.add(result.get("user_name").toString());
}
return r;
SQLFiddle

The solution is to remove the logic from your code to the sql query to only get the active users from that select, using a where clause.
It is faster to use the sql built-in functions to get fewer records and iterate less in your code.
Add this to your sql query to get the active users only:
Where TIMESTAMPDIFF(SECOND, last_access_ts, current_timestamp) > 10
This will get you all the records whose date is 10 seconds ago or sooner.

Try the MySQL TimeDiff function in your select. This way you can select only the results that are active without having to do any other calculations.
Link: MySQL: how to get the difference between two timestamps in seconds

If I get you right, then you got only 500 entries in your user_sessions table. In this case I wouldn't even care about indexes. Throw them away. The DB engine probably won't use them anyway for such a low record count. The performance gain due to not updating the indexes on every record update could be probably higher than the query overhead.
If you care about DB stress, then lengthen the query/update intervals to 1 minute or more, if your application allows this. Gordon Linoff's answer should give you the best query performance though.
As a side note (because it has bitten me before): If you don't use the same synchronized time for all user callbacks, then your "active users logic" is flawed by design.

db4o query optimisation for a scheduling application

I'm just getting started with db4o in a scheduling application and I'm looking for an efficient way to retrieve rooms which are not booked between certain dates.
So, I have a collection of Room objects each of which has a collection of Booking objects (which can be empty). A Booking has a start date and and end date. I want to say 'get all the rooms that have no Bookings between DateA and DateB'.
I'm sure I could do this using a Native Query but since there's a date range involved (my understanding is date ranges aren't optimzed for NQ) and I need to do this query very frequently (many times per second for potentially more 10,000 rooms - the majority of which have no Bookings) I'm looking for more efficient alternatives.
Is there a way to phrase this using SODA?
Or a better way to arrange my data model to get round this issue?

Yes you can do this by using SODA Query
Date fromDate = null ; // assign reservation start dat
Date toDate = null ; // assign reservation upto
Query query = db.query();
query.constrain(Booking.class);
query.descend ("fromDate").constrain ( fromDate ).greater().equal ().
and (query.descend ("toDate").constrain (toDate).smaller().equal());
ObjectSet<Booking> objectSet = query.execute();
Query for all of the rooms which do not have a booking between fromDate and toDate
Query query = db.query();
query.constrain(Room.class);
query.descend ("bookingStartDate").constrain ( fromDate ).greater().equal ().and
(query.descend ("bookingEndDate").constrain (toDate).smaller().equal()).not();
ObjectSet<Room> objectSet = query.execute();
See Also : Building SODA Queries