We are trying to compare a date stored in mySQL with a plain text date field read from the original of a Gmail message, using java date compare.
First time through, the Gmail message plain text Date is read and stored in a mySQL database field “date_sent” type TIMESTAMP.
The next time that message is checked, it gets the message plain text Date and, using the Java date compare function, compares that with the stored date_sent value.
This compare usually works. However -- if the date and time of the message Date being compared is during the 1am hour on a day in which the time changed (daylight savings to standard), the compare fails.
Has anyone experienced this? how were you able to fix it?
It seems like you've got an uphill battle. If a message arrives, dated 1:30am, on the date when Daylight Savings ends, I don't think there's any way you can tell whether it's the FIRST occurrence of 1:30am (before the clocks go back) or the SECOND.
Presumably, you'll never get a message dated 1:30am on the date when Daylight Savings starts, because this time won't actually exist.
So if you do this by converting the text field to a date and storing it, you'll always have an issue of how to compare such dates. Your comparison might be wrong sometimes if you pick the wrong 1:30am, and I don't think there's too much you can do about it, if you want your timezones to be correct, for both Daylight Savings Time and Standard Time.
One thing you might consider doing is storing the timestamps as text, not as dates, so that the conversion never happens. If you use a format like yyyy-MM-dd HH:mm:ss then you should be able to do a text comparison instead of a date comparison, and get the correct results.
I've never experienced this because I've never done it. What I think you'll have to do is get/know the timezone of the text date, convert it into a Date object (using that timezone), then compare the dates that way.
You say you're using the Date compare method, but that says it requires a Date as input. So are you already converting? Please show us that code. Are you passing in a timezone?
Also, this question has very useful information that you can use.
Related
I have a MySQL database which is storing a datetime value, let's say 2020-10-11 12:00:00. (yyyy-mm-dd hh:mm:ss format)
The type of this date (in mysql) is DATETIME
When I retrieve this data in my controller, it has the java 7 type "Date". But it adds a timezone CEST due to my locale I suspect. Here I already find confusing that when displaying this date which is not supposed to have a timezone attached it actually has... and the debugger says it is "2020-10-11 12:00:00 CEST".
My problem is that date was not stored with the CEST timezone. It was stored with the America/New_York one, for example. EDIT: What I mean with this line, is that the date was stored from new york using the timezone of new york. So, it was really 12:00:00 AM there, but here in Madrid it was 18:00:00 PM. I need that 18:00:00.
So in New York, someone did an insert at that time. Which means that the time in Europe was different. I need to calculate which time was in Europe when in America was 12AM. But my computer keeps setting that date to CEST when I retrieve it so all my parsing attempts are failing... This was my idea:
Date testingDate // This date is initialized fetching the "2020-10-11 12:00:00" from mySql
Calendar calendar = new GregorianCalendar()
calendar.setTime(testingDate)
calendar.setTimeZone(TimeZone.getTimeZone("America/New_York")
SimpleDateFormat localDateFormatter = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
TimeZone localTimeZone = TimeZone.getTimeZone("Europe/Madrid")
localDateFormatter.setTimeZone(localTimeZone)
String localStringDate = localDateFormatter.format(calendar.getTime())
Date newDate = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse(localStringDate)
Here my idea is that: I create a brand new calendar, put on it the time that I had on America and I also say hey this calendar should have the America Timezone. So when I get the time of it using a formatter from Europe it should add the corresponding hours to it. It makes a lot of sense in my head but it is just not working in the code D: And I really don't want to calculate the time difference by myself and adding or substracting the hours because that would look extremely hardcoded in my opinion.
Can any one give me some ideas of what I'm interpreting wrong or how should I tackle this problem in a better way?
Important: I'm using java 7 and grails 2.3.6.
My problem is that date was not stored with the CEST timezone. It was stored with the America/New_York one, for example.
From what I know of MySQL, this is impossible.
Calendar calendar = new GregorianCalendar()
No, don't. The calendar API is a disaster. Use java.time, the only time API in java that actually works and isn't completely broken / very badly designed. If you can't (java 7 is extremely out of date and insecure, you must upgrade!), there's the jsr310 backport. Add that dependency and use it.
Let me try to explain how to understand time first, because otherwise, any answer to this question cannot be properly understood:
About time!
There are 3 completely different concepts and they are all often simplified to mean 'time', but you shouldn't simplify them - these 3 different ideas are not really related and if you ever confuse one for another, problems always occur. You cannot convert between these 3 concepts unless you do so deliberately!
"solarflares time": These describe the moment in time as a universal global concept that something occurred or will occur. "That solar flare was observed at X" is a 'solarflares' time. Best way to store this is millis since epoch.
"appointment time": These describe a specific moment in time as it was or will be in some localized place, but stated in a globally understandable way. "We have a zoom meeting next tuesday at 5" is one of these. It's not actually constant, because locales can decide to adopt new timezones or e.g. move the 'switch date' for daylight savings. For example, if you have an appointment at your dentist on 'november 5th, at 17:00, 2021', and you want to know how many hours are left until your appointment starts, then the value should not change just because you flew to another timezone and are looking at this number from there. However, it should change if the country where you made the appointment in decided to abolish daylight savings time. That's the difference between this one and the 'solarflares' one. This one can still change due to political decisions.
"wake-up-alarm time": These describe a more mutable concept: Some way humans refer to time which doesn't refer to any specific instant or is even trying to. Think "I like to wake up at 8", and thus the amount of time until your alarm will go off next is continually in flux if you are travelling across timezones.
Now, on to your question:
I have a MySQL database which is storing a datetime value, let's say 2020-10-11 12:00:00. (yyyy-mm-dd hh:mm:ss format)
Not so fast. What exact type does that column have? What is in your CREATE TABLE statement? The key thing to figure out here is what is actually stored on disk? Is it solarflare, appointment, or wakeup-alarm? There's DATE, DATETIME and TIMESTAMP, and over the years, mysql has significantly changed how these things are stored.
I believe that, assuming you are using the modern takes on storage (So, newish mysql and no settings to explicitly emulate old behaviour), e.g. a DATETIME stores sign, year, day, hour, minute, and second under the hood, which means it is wakeup alarm style: There is no timezone info in this, therefore, the actual moment in time is not set at all and depends on who is asking.
Contrast to TIMEZONE which is stored as UTC epoch seconds, so it's solarflares time, and it doesn't include any timezone at all. You'd have to store that separately. As far as I know, the most useful of the 3 time representations (appointment time) is not a thing in mysql. That's very annoying; mysql tends to be, so perhaps par for the course.
In java, all 3 concepts exist:
solarflares time is java.time.Instant. java.util.Date, java.sql.Timestamp, System.currentTimeMillis() are also solarflares time. That 'Date' is solarflares timestamp is insane, but then there is a reason that API was replaced.
appointment time is java.time.ZonedDateTime
wakeup-alarm time is java.time.LocalDateTime.
When I retrieve this data in my controller, it has the java 7 type "Date".
Right. So, solarflares time.
Here's the crucial thing:
If the type of time stored in MySQL does not match the type of time on the java side, pain happens.
It sure sounds like you have wakeup-alarm time on disk, and it ends up on java side as solarflares time. That means somebody involved a timezone conversion. Could have happened internally in mysql, could have happened in flight between mysql and the jdbc driver (mysql puts it 'on the wire' converted), or the jdbc driver did it to match java.sql.Timestamp.
The best solution is not to convert at all, and the only real way to do that is to either change your mysql table def to match java's, so, make that CREATE TABLE (foo TIMESTAMP), as TIMESTAMP is also solarflares time, or, to use, at the JDBC level, not:
someResultSet.getTimestamp(col);
as that returns solarflares time, but:
someResultSet.getObject(col, LocalDateTime.class);
The problem is: Your JDBC driver may not support this. If it doesn't, it's a crappy JDBC driver, but that happens sometimes.
This is still the superior plan - plan A. So do not proceed to the crappy plan B alternative unless there is no other way.
Plan B:
Acknowledge that conversion happens and that this is extremely annoying and errorprone. So, make sure you manage it, carefully and explicitly: Make sure the proper SET call is set up so that mysql's sense of which timezone we are at matched. Consider adding storing the timezone as a column in your table if you really need appointment time. etcetera.
Thanks to #rzwitserloot I was able to find out a solution.
First I'll get the data from the database. I'll get rid of any timezone added by the driver / mysql by converting it to a LocalDateTime. Then, I'll create a new ZonedDateTime using the Timezone that was used when storing the data in the database.
Once I have a ZonedDateTime, it is time to convert it using my current timezone. I'll get a new ZonedDateTime object with the proper time.
Then I just add a few more lines to convert it back to my main "Date" class:
I've used the ThreeTen backport as suggested.
Date dateMySQL //Initialized with the date from mysql
Calendar calendar = new GregorianCalendar()
calendar.setTime(dateMySQL)
org.threeten.bp.LocalDateTime localDateTime = org.threeten.bp.LocalDateTime.of(calendar.get(Calendar.YEAR), calendar.get(Calendar.MONTH)+1,
calendar.get(Calendar.DAY_OF_MONTH), calendar.get(Calendar.HOUR_OF_DAY), calendar.get(Calendar.MINUTE),
calendar.get(Calendar.SECOND))
String timezone //Initialized with the timezone from mysql (Ex: "America/New_York")
ZonedDateTime zonedDateTime = ZonedDateTime.of(localDateTime, ZoneId.of(timezone))
ZonedDateTime utcDate = zonedDateTime.withZoneSameInstant(ZoneId.of("Europe/Madrid"))
calendar.setTimeInMillis(utcDate.toInstant().toEpochMilli())
Date desiredDate = calendar.time
dateMySQL: "2020-10-11 10:00:00" // CEST due to my driver
timezone: "America/New_York"
desiredDate: "2020-10-11 19:00:00" // CEST Yay!
I want to create an instance of Calendar with string date coming from server. Now I don't know what format server is sending .
It can be changed for different countries. I know I can ask them to add another key for dateFormat and create Calendar from it. But still I want to know Is there any way to create Calendar Instance without knowing current string date format.
I have gone through this and this. But none fulfill my requirement
This is impossible.
If the server sends the value "1/2/2017", you have no way of knowing if this refers to January 2nd or February 1st.
If the server sends the value "מָחָר", in theory you could realize that this might be a Hebrew translation of the word "tomorrow" (at least, according to Google Translate), but even then, it is not clear whether this is to be taken relative to today or some other date.
If the server sends the value "I want to create an instance of Calendar with string date coming from server", you have no means of creating a date from that, at least using any algorithm that would make sense to people.
And so on.
The only reason a server should return a date in an arbitrary format is if the date would only ever be read by the user who provided the value in the first place and presented as plain text verbatim, without parsing. Otherwise, the server should supply the date in a standardized format, with the UI consuming that date being responsible for formatting it in a user-friendly (and, ideally, locale-aware) fashion.
You're welcome to try to brute-force the problem, iterating over a series of date formats and seeing if any result in a seemingly-valid date. This fails the 1/2/2017 scenario (as there are at least two formats that would return a seemingly-valid date), but perhaps you know enough about the server to narrow down the possible formats to reduce the odds of collisions like this.
The Joda Date & Time API has a date parser which can parse date strings in many formats. Note that some datetime strings can be ambiguous: 10-09-2003 could mean October 9 or September 10.
I have a String like this "15:30:10". Is there anyway to get TimeZone object from this String?
I received a Time String "HH:MM:SS" from another application in other countries (not same with my country). And I have to show the TimeZone. That string is all I have.
Is there anyway to get TimeZone object from this String?
No.
The String contains no timezone information, and you cannot extract information that isn't there.
I received a Time String "HH:MM:SS" from another application in other countries (not same with my country). And I have to show the TimeZone. That String is all I have.
Same answer. You can't do it.
Thinking outside the box a little bit ...
If the time string was supposed to represent the time >>now<< in some unknown timezone, then you could calculate the offset from UTC for that timezone. (It just requires some simple arithmetic which is too trivial to mention.)
But that doesn't give you a real TimeZone. For example, you won't be able to tell the difference between the timezones for France and Namibia.
What is the base timezone of your application? Is there any date associated with this time? If you have date and know the base timezone of your application then it is possible. Otherwise forget it
Is there any function or library which gets a date in milliseconds, given a String?
This question shows how to convert a formatted String to a Date object, but is there any way to do this with an unformatted String?
Basically, the task is impossible. Here's an example:
01/04/2012
In the US, that means January 4th 2012. In Australia, that mean 1st April 2012.
Without knowing where you are and what date formats conventionally mean, it is impossible to accurately map an arbitrary date-like string to a date time value that matches what the user actually meant.
And even if you do know about the relevant local conventions, users have a remarkable propensity to be oblivious to ambiguity. Dealing with that may require deep domain knowledge (or mind reading skills!) to disambiguate the possible meanings.
When you think about it, this is why modern user interfaces typically use a date-picker widget of some kind when the user needs to enter a date / time
first convert the string to Date. From there you can get time in milis using Date.getTime() method
So me and my partner have been working on this project for a while now. We work with dates A LOT in this project, and we recently noticed an issue, and we are rather deep in at this point.
We store our times in SQLlite (Android project) as a formatted string, since a lot of the time they are directly bound to listviews and such.
The problem we noticed, which i found kind of odd, is that that SimpleDateTimeFormat object, when used to format to 24h time (its a medical based project, so 24h time is the convention here) 12:00am-12:59am are formatted to 24:00-24:59, instead of 00:00-00:59...
This isn't too much of an issue until we query the database and order the results by the dates, any data that is between 12:00am and 12:59am will show up at the end of the list, but it should show up at the beginning...
Anyone else encountered this problem? or know a way around it? The best thing possible would be a way to store the data as 00:00 not 24:00.
Cheers
I strongly suspect you're using the wrong pattern. We've got to guess as you haven't posted any code (hint, hint), but I suspect you're using a pattern such as
kk:mm:ss
instead of
HH:mm:ss
Sample code:
import java.util.*;
import java.text.*;
public class Test {
public static void main(String[] args) throws Exception {
SimpleDateFormat broken = new SimpleDateFormat("kk:mm:ss");
broken.setTimeZone(TimeZone.getTimeZone("Etc/UTC"));
SimpleDateFormat working = new SimpleDateFormat("HH:mm:ss");
working.setTimeZone(TimeZone.getTimeZone("Etc/UTC"));
Date epoch = new Date(0);
System.out.println(broken.format(epoch));
System.out.println(working.format(epoch));
}
}
Additionally, as others have pointed out, you shouldn't be storing your values in string format to start with... avoid string conversions wherever you can, as each conversion is a potential pain point.
Please read this and this about how SQLite stores dates (or doesn't store dates). SQLite doesn't have a "Date" type, so it is stored as a string. You should store your date as an integer (milliseconds), and then you can use date and time functions to pull them out (from the first link).
From the documentation
1.2 Date and Time Datatype
SQLite does not have a storage class set aside for storing dates
and/or times. Instead, the built-in Date And Time Functions of SQLite
are capable of storing dates and times as TEXT, REAL, or INTEGER
values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS"). REAL as Julian
day numbers, the number of days since noon in Greenwich on November
24, 4714 B.C. according to the proleptic Gregorian calendar. INTEGER
as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Applications can chose to store dates and times in any of these
formats and freely convert between formats using the built-in date and
time functions.
I prefer INTEGER / Unix time storage, then use the built in date and time functions to format when pulling from DB.
EDIT: Also, this will take care of sorting. I'm guessing your current "sorting" of the dates in SQLite is string based, which is bad mmmmkay.
What is the format string you are passing to your SimpleDateFormat? According to the docs, using 'H' for the hours should get you 0-23, using 'k' should get you 1-24.