In a publishing flow I need to insert a DBObject into a mongo db collection using Java.
I receive the object as a String, and this has been passed to me from a .NET application that used article.ToBsonDocument().ToJson() on a POCO.
On my side of the flow, in Java, I tried just using BasicDBObject doc = (BasicDBObject) JSON.parse(content); but I get a com.mongo.util.JSONParseException on a Date:
"CreationDate" : ISODate("2013-03-18T08:50:53Z")
I can change how the content is generated in C#, and I can change how to write to the DB in java, my only constraint is that it must be passed as a string between the two systems.
Any suggestions?
EDIT Thanks to the tip from #Jim Dagg below, some googling for ISODate and BsonDocument turned out this gem. Changing the c# code to use
article.ToBsonDocument().ToJson(new JsonWriterSettings{OutputMode = JsonOutputMode.Strict});
fixed it.
The ISODate constructor call is what's causing the issue. From an issue on the MongoDB JIRA:
The parser accepts these two date formats: seconds ->
"yyyy-MM-dd'T'HH:mm:ss'Z'" or seconds.milleseconds ->
"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'" Just add seconds to your $date value
and the aggregation command should work. Here's the JSON doc that I
had success with: { "aggregate" : "test", pipeline : [ {$match : {
date : { $date : "2012-05-01T12:30:00Z" } } } ] }
If you remove the ISODate constructor and simply render your date as (for example) "2013-03-18T08:50:53Z", you should be in business.
Related
I have a slight problem with return data from Mongo find() vs a findOneAndReplace().
First, to know I'm working on an API that queries Mongo and returns data in JSON format.
Problem I'm having is that if I do a findOneAndReplace() to update and return the modified document, much like so:
// javaDocument() is just a org.bson.Document
var modDoc = jobsCollection.findOneAndReplace( javaDocument({"_id": jobid}), javaDocument(jobData), foarOptions );
if(isDefined("modDoc")) {
sReturn.DATA = deserializeJSON(modDoc.toJSON());
}
the dates set in my document returns within subkeys named "$date", which I do not want:
It really should read:
"created_at": "2022-02-03T10:15:01.634Z"
Doing a simple jobsCollection.find() however, seems to return me the date appropriately, like so:
var data= [];
jobsCollection.find( javaDocument({}) ).into(data);
sReturn.DATA = data;
What am I missing here ? I could simple copy the "$date" key and fix the structure, but I don't always know where in the structure I'll have dates.... Is there a way I could have this properly returned with FindOneAndReplace() ? I'm thinking in might have something to do with the modDoc.toJSON() but haven't quite found my answer yet...
Thanks for your time. Cheers! Pat
I am having an issue with mongodb. Could anyone explain me why this works (gives back some results):
db.getCollection('test').find({"reportedOn" : { "$gte" : ISODate("2019-03-01T01:20:00"), "$lt" : ISODate("2019-04-01T02:00:00") } });
And this doesn't work (returns zero):
db.getCollection('test').find({"reportedOn" : { "$gte" : { "$date" : 1551398400000 }, "$lt" : { "$date" : 1554073200000 } } });
Thanks a lot.
You may have to check this kind of related topic to understand why you actually can't use an epoch time directly in your query.
Here is the interesting part of querying on Date :
According to Data Types in the mongo Shell both should be equivalent:
The mongo db provides various methods to return the date, either as a string or as a Date object:
Date() method which returns the current date as a string.
new Date() constructor which returns a Date object using the ISODate() wrapper.
ISODate() constructor which returns a Date object using the ISODate() wrapper.
and using ISODate should still return a Date object.
{"$date": "ISO-8601 string"} can be used when strict JSON representation is required.
So you may want to check all ISO-8601 available formats and update your code accordingly to the expected format in order to get this working as expected.
Try the following query
{ "$date" : { "$numberLong" : "<dateAsMilliseconds>" } }
as mentioned in the MongoDB docs instead of comparing with $date only. Link
When I query MongoRepository via Date field with Criteria in a Spring Boot application, the result is wrong. Here is my method:
Query query = new Query(new Criteria().andOperator(
Criteria.where("id").is(filter.getId()),
Criteria.where("datas.ts").lt(filter.getEndTime()).gte(filter.getStartTime())
));
List<PhaseData> phaseDatas = mongoOperations.find(query, PhaseData.class);
List<Data> result = new ArrayList<Data>();
for(Data pData : phaseDatas) {
result.addAll(pData.getDatas());
}
return result;
When I query with
{
"id" : "1234",
"startTime" : "2016-08-04 12:00",
"endTime" : "2016-08-04 15:00"
}
it gives me records with hour 16:54 & 21:12 too. How can I solve this issue?
Not sure if this addresses your question directly.
The DB won't return wrong result to the query. So I think it could be one of the following things:
It could be that the when you view the documents in mongodb, it displays date in iso format. So view the documents in the same format as you are creating dates for your query.
It could be timezone issue.
Mongodb dates can be considered as ISODate (MongoDB Date)
When you query, you create date objects in your timezone. So as a first debugging measure, I would see if both my DB and query timezones are the same.
Also, probably it would help if you query by creating date objects in ISODate by using SimpleDateFormat(SDF is not thread safe).
I have found that it could be confusing because the dates that you send are in a different format and the documents that you visually see in mongodb tool are displaying dates in iso format. I think that it could be the issue. The results are good, but probably you are viewing the two things differently and it causes the confusion.
Before I start, I have already searched around for an answer to this issue and the best answer I could come up with is this question
I have one difference though. I have a table that maintains a history of many documents. Therefore I need to query on an ID as well as the date range. Here is what my query currently looks like in Java
BasicDBObject searchQuery = new BasicDBObject();
searchQuery.put("id", id);
searchQuery.put("dateModified", BasicDBObjectBuilder.start("$gte", fromDate).add("$lte", toDate).get());
DBCursor cursor = table.find(searchQuery);
This returns no results. The MongoQuery that is generated by this block of code looks like this:
db.history.find({ "id" : 12345 , "dateModified" : { "$gte" : { "$date" : "2015-01-19T00:00:00.000Z"} , "$lte" : { "$date" : "2015-01-25T00:00:00.000Z"}}});
When I manually type this into MongoDB command line, this also returns no results. I currently have one record in the database for testing purposes that looks like this:
{
"id" : NumberLong(12345),
"dateModified" : ISODate("2015-01-21T19:42:28.044Z")
}
This object should clearly match the query, yet nothing is returning, any ideas?
EDIT: So it turns out that the string generated by the query object doesn't match the ISODate object in the database. I'd like to clarify that fromDate and toDate are both java.util.Date objects. I'm still not sure how to solve this though.
I figured out the issue. I don't understand the cause, but the issue is with the BasicDBObjectBuilder not using the Date object correctly. I switched to QueryBuilder and built the exact same query and it returned results.
fromDate must be of the type Date not the String representation. An ISODate in the MongoDB storage Engine is not equal to the String representation of the same date and so they do not match.
I am trying to retrieve workitems from TFS Server using OData4j 0.7 in Java.
Here is my code:
public List<TFSWorkItem> getWorkItems(final String projectName)
{
final List<TFSWorkItem> tfsWorkItems = new ArrayList<TFSWorkItem>();
String filter = String.format("Project eq '%s'", projectName);
Enumerable<OEntity> workItems = consumer.getEntities("WorkItems").filter(filter).execute();
for (OEntity workitem : workItems)
{
System.out.println(workitem.getProperty("Title", String.class));
}
}
When I run this code I get a
Exception in thread "main" java.lang.IllegalArgumentException: Illegal datetime format 2013-03-15T14:22:08.077+05:30
at org.odata4j.internal.InternalUtil.parseDateTimeFromXml(InternalUtil.java:96)
On further debugging the code I found that OData4j when trying to map the retrieved date from TFS server finds it incompatible .
Date retrieved from TFS :
2013-03-15T14:22:08.077+05:30
Date expected by OData4j :
2013-03-15T14:22:08.077
Is there a way where I can avoid this?
Updated
For anybody who is facing the same issue.
I have modified my code to :
final String fields = "Id,Project,Title";
Enumerable<OEntity> workItems = consumer.getEntities("WorkItems").filter(filter).select(fields.toString()).execute();
for (OEntity workitem : workItems)
{
System.out.println("Id : " + workitem.getProperty("Id").getValue());
System.out.println("Project : "+workitem.getProperty("Project").getValue());
System.out.println("Title : "+workitem.getProperty("Title").getValue());
}
Since I need only these fields to process I have given a select query to select "Id","Project" and "Title" instead of fetching all fields.
This is a temporary fix unless I find a better solution.
I think I figured this out. Check out this post: OData4J exception
Basically, it boils down to setting the value of an Edm.DateTime column in C# code using DateTime.Now. The OData Producer should be using DateTime.UtcNow.
DateTime.Now includes local time zone information. Microsoft's code sees the local time zone info in the DateTime structure and sends back the Edm.DataTime field formatted as an Edm.DateTimeOffset string. The DateTime.UtcNow property does not include any time zone information, so it is correctly formatted as an Edm.DateTime string when it is sent back to the OData Consumer.
I suspect this might be a bug in Microsoft's WCF Data Services stack or their Entity Framework.