I have a JSON file which has 40k documents, each document contains a date field. I need to query within Java with dates to retrieve data, so I stored the date in numberLong format.
Date dt = new Date();
SimpleDateFormat format = new SimpleDateFormat("MM/dd/yyyy");
dt = format.parse(nextLine[j]);
document.put(ColumnNameAsKey[j], dt.getTime());
where the above code is done in for loop in an API to store data to mongo.
But after entering all those data, and then when I queried numberlong changes automatically for the same date, so that I am unable to retrieve all data for the required date. My query to retrieve is
querygraph.put("Complaint Date (MM/DD/YYYY)", new
BasicDBObject("$gte",startdate.getTime()).append("$lte",EndDate.getTime()));
for eg : if the date 08/01/2012 contains large number of document, the correct numberlong for the date 08/01/2012 is replaced in the date field in mongo. this will continue, but aftr some number of documents, the numberlong keep on changing.. ie if NumberLong is 134353300000 for the date 08/01/2012, then after 6 or more document the numberlong will be different from the former one.. causing unable to retrieve exact data for the date 08/01/2012..
What makes the difference here?
I think see your problem, because you store the millisecond precision of time you are actually getting problems with the long ints representing parts of a day making it impossible to query past, say, midnight.
This is because MongoDB querying does not take this sort of contextual querying into account.
First off a hint, don't store as number longs, store as the $date BSON type using only the lines:
Date dt = new Date();
SimpleDateFormat format = new SimpleDateFormat("MM/dd/yyyy");
You will get more functionality from using the specified BSON date type and the querying ability is the same across the board.
You have the right idea about querying your records:
querygraph.put("Complaint Date (MM/DD/YYYY)", new
BasicDBObject("$gte",startdate.getTime()).append("$lte",EndDate.getTime()));
But I got a feeling you are doing something wrong. When you create the start date and the end date you are actually looking for the 00:00:00 time of the start date and the 23:59:59 time of the end date. This is due to your getTime() function, UNIX timestamp does not return partial times as such it will just return the default which is effectively now().
One way around that could make your life easier is to standardise times on this field so that you specify a time of 00:00:00 for all dates allowing you pick out ranges correctly.
Related
I have certain object that has a date property. When i print the date value in Java i get a normal date like "11/08/2021". The thing is, when i try to check the date of my object (which is stored in mongoDB) in Robo 3T, i get a number like that : 1507037760657.
I searched and i found this kind of converter that allows to convert timeStamp format to date format. But that value : 1507037760657 is converted to some really weird date like 15/2/49726.
Is mongoDB using another timestamp format i don't know? How can i get a good date from this number ? 1507037760657
Thanks
I have java.util.Date field in my document class.
E:g:
#Document(collection = "testdoc")
public class TestDoc {
#Id
String id;
Date startDate;
}
Even if I set the Date with UTC and IST, it always save in my collection as below,
"startDate" : ISODate("2015-08-21T18:30:00.000Z")
How can I save the time zone also in mongo collection? What does Z stand in this case?
The 'Z' signifies that the time is stored in UTC. Mongo internally converts all local representations of time in UTC before storing. However, one suggestion would be to store the time along with the timezone which is received from your application. You can later reconstruct the local time from the UTC time and the timezone in your application logic.
Please go through this link. They have given an example on how to model local time data using JavaScript.
https://docs.mongodb.com/v3.2/tutorial/model-time-data/
Do the conversion before storing and save as UTC always. Then reconvert it in the timezone you want before displaying.
If you desperately want to store the timezone with the offset you may have to deal with it as a separate string in db, but it cannot go with date field for MongoDB.
As currently MongoDB doesn't allow saving of timezone.
Below is the open JIRA issue or the same.
https://jira.mongodb.org/browse/SERVER-6310
Dates in MongoDB are stored in UTC. There isn't timestamp with a timezone data type like in some relational databases. Applications that need to access and modify timestamps, based on local time, should store the timezone offset together with the date and offset dates on an application level.
In the MongoDB shell, this could be done using the following format with JavaScript:
let now = new Date();
db.page_views.save({date: now,
offset: now.getTimezoneOffset()});
Then you need to apply the saved offset to reconstruct the original local time, as in the following example:
let record = db.page_views.findOne();
let localNow = new Date( record.date.getTime() - ( record.offset * 60000 ) );
I guess here you'll find a good guideline how to handle timestamps in different scenarios for different data language independent.
As it's recommendet in the document, ALWAYS use UTC to save the data, even it is local timezone. If necessary, save the time zone in a seperate field ( JSON or xml) and follw the format guideline ISO 8601. ( like you did but there are many possible representations)
As far as I know, correct me if i'm wrong, JSOn does not deal with specific date formats but JS does. Microsoft docs recommends the followind format bases on ECMA YYYY-MM-DDTHH:mm:ss.sssZ this
The Z in the Timestamp shows that it's UTC format with zero offset +00:00
if nothing else is added.
If you want to use the "Z" notation, you have to add or substract the offset within the timestamp instead of writing the zero-offset and add the offset at the end.
I recommend you to follw the w3c guideline because it covers different scenatios for different time saving usecases.
Hope this helps
I'm reading information from a file (.csv) that will be inserted to a database after validation and approval from the end user. The text file is read, validated and its information loaded to a List containing forms which are used to check if the data already exist in database.
The problem arises when parsing the String to Date. The SimpleDateFormat.parse() method returns an unexpected date format even when the pattern for SimpleDateFormat is "yyyy-MM-dd".
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
dateFormat.setLenient(false);
Parsing the loaded values to the travel object:
travel.setDate( dateFormat.parse( form.getTravelDate() ));
In debugger the form shows the date as:
"2012-12-12"
It is read as intended. However when parsed becomes:
Wed Dec 12 00:00:00 CST 2012
I've spent the whole day trying to solve this but I'm out of ideas at this point. Afaik the pattern is alright and I've tried adding a locale to no avail.
Edit: the problem is when I need to insert the values to the database using Hibernate. The not desired format also ends up showing in the Database.
The data is show in a .jsp page using
HttpServletRequest("table",travelList);
The date format I don't need shows here, when in the past this issue never happened. At last the information is sent to the database where the problem persists.
No, it "becomes" a java.util.Date value. If you're then converting it back to a string by calling Date.toString(), you should consult the Date.toString() documentation for what to expect.
The value stored in the Date is just a point in time - the number of milliseconds since the Unix epoch. There's no format in there, no time zone etc. It also doesn't know that it was only a date value rather than a date and time (the naming of Date is one of the many unfortunate aspects of the API).
It's crucial that you mentally separate "the result of the parse operation" from "the string value that is used to represent that result if I call toString"".
I'd also advise you to set the time zone on your SimpleDateFormat to UTC when parsing a date - that way you know that you can't possibly have any ambiguous or skipped times leading to hard-to-predict behaviour. (Of course, you then need to know that you'll have parsed the date to "the start of the UTC date" and handle it appropriately elsewhere.)
You need to use the formatter when printing the Date as well.
dateFormat.format( travel.getDate() );
When your parse the String using a DateFormat, you get a complete Date object with the time units missing in the string initialized to zero. In your example, that's hours, minutes and seconds.
By default, if you do not use a formatter, Date's default string representation (provided by its toString() method) gets printed.
Parsing doesn't mean it format, it simply parse it as text to a java.util.Date object. See parse method in documentation.
You need to use the format method.
dateFormat.format(travel.getDate())
See documentation for more details.
if form.getTravelDate() is returning String then First Parse the Date from String
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
dateFormat.setLenient(false);
Date parsedDate=dateFormat.parse(form.getTravelDate());
// Here parsedDate is in form Wed Dec 12 00:00:00 CST 2012
Now Format the Date using the Same SImpleDateFormat to get the desired output
System.out.println(dateFormat.format(parsedDate));
Your Desired Output
2012-12-12
Update
Assuming data Type of Columns in which we insert this data in Database is of Date Type not varchar then use the below statement
travel.setDate(dateFormat.format(parsedDate));
As I understand parse method returns Date. Below is the parse method syntax.
public Date parse(String source)
throws ParseException
So, you need to parse the string date and store into a Date variable. Then format the Date variable using SimpleDateFormat.
//getTravelDate is "2012-12-12"
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
dateFormat.setLenient(false);
Date dtObj=new Date();
dtObj=dateFormat.parse(form.getTravelDate()); //Store in date variable
System.out.println(dateFormat.format(dtObj)); // Now format the date object
The final output will be 2012-12-12. Hope it will help you.
Is it possible to save date values in the database of this format dd/MM/yy in Grails? I know I can customized the format in the views but I need the values to be returned as json and also return the values of dates in json in that format. Any suggestion will be appreciated.
Try this on your presentation layer, don't save time in that format in database. use following code to format the time according to your need.
Date date = new Date( );
SimpleDateFormat simpleFormat = new SimpleDateFormat ("dd/MM/yy");
System.out.println("Date: " + simpleFormat .format(date));
But if you want to save the data in this format in databse, then remember it returns a string and you will have to save the date in String format in database. Which I wouldn't recommend because of many reason.
You should not store the date as a formatted string because you lose the ability to do many things with that field, such as sort it or compare it. Always use the database's native date format for storage. If you want to change the format there are many places to do it, including the presentation layer (as others have suggested) and the database query layer. Format the date in the query if you want to do minimal processing in Java/Javascript.
Agreed with others you should not save in the database in String format. To format a Date using Groovy you can use the String.format() method.
String.format('%tY-%<tm-%<td', new Date())
See the Groovy dates documentation for further examples.
I want to display the date of a trip. In the application several trips are listed, each has a long value with the currentTimeMillis of its generation. I want to turn this long number into a date but the output is always something around 1970... Is System.currentTimeMillis() the right way to store the time of the trip generation?
Date date = new Date(trip.getStartTime()); // 195342322
SimpleDateFormat test = new SimpleDateFormat("yyyy-MM-dd");
startTime.setText(String.valueOf(test.format(date)));
-> Output of startTime: 1970-01-01
Try to use Calendar for simple formatting
We don't know how trip.getStartTime() is implemented, however it returns a wrong value. 195342322 is Wed, 10 Mar 1976 21:45:22 GMT (you can check it online)
Since the only non-deprecated Date constructor accepts a long argument, I guess you used that. It's better to store the date(time) in a database-friendly way, such as ISO 8601. Note that if you store it on the client in a SQLite database, SQLite doesn't have a time type, and you have to use a text field for that.
Once you switch to the text representation for serialization, you will be able to get back a Date by using SimpleDateFormat.parse()
use something like this:
Calendar calendar=Calendar.getInstance();
calendar.setTimeInMillis(trip.getStartTime());
Date date =calendar.getTime ();
as shown here .