I have a MySQL database which is storing a datetime value, let's say 2020-10-11 12:00:00. (yyyy-mm-dd hh:mm:ss format)
The type of this date (in mysql) is DATETIME
When I retrieve this data in my controller, it has the java 7 type "Date". But it adds a timezone CEST due to my locale I suspect. Here I already find confusing that when displaying this date which is not supposed to have a timezone attached it actually has... and the debugger says it is "2020-10-11 12:00:00 CEST".
My problem is that date was not stored with the CEST timezone. It was stored with the America/New_York one, for example. EDIT: What I mean with this line, is that the date was stored from new york using the timezone of new york. So, it was really 12:00:00 AM there, but here in Madrid it was 18:00:00 PM. I need that 18:00:00.
So in New York, someone did an insert at that time. Which means that the time in Europe was different. I need to calculate which time was in Europe when in America was 12AM. But my computer keeps setting that date to CEST when I retrieve it so all my parsing attempts are failing... This was my idea:
Date testingDate // This date is initialized fetching the "2020-10-11 12:00:00" from mySql
Calendar calendar = new GregorianCalendar()
calendar.setTime(testingDate)
calendar.setTimeZone(TimeZone.getTimeZone("America/New_York")
SimpleDateFormat localDateFormatter = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
TimeZone localTimeZone = TimeZone.getTimeZone("Europe/Madrid")
localDateFormatter.setTimeZone(localTimeZone)
String localStringDate = localDateFormatter.format(calendar.getTime())
Date newDate = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse(localStringDate)
Here my idea is that: I create a brand new calendar, put on it the time that I had on America and I also say hey this calendar should have the America Timezone. So when I get the time of it using a formatter from Europe it should add the corresponding hours to it. It makes a lot of sense in my head but it is just not working in the code D: And I really don't want to calculate the time difference by myself and adding or substracting the hours because that would look extremely hardcoded in my opinion.
Can any one give me some ideas of what I'm interpreting wrong or how should I tackle this problem in a better way?
Important: I'm using java 7 and grails 2.3.6.
My problem is that date was not stored with the CEST timezone. It was stored with the America/New_York one, for example.
From what I know of MySQL, this is impossible.
Calendar calendar = new GregorianCalendar()
No, don't. The calendar API is a disaster. Use java.time, the only time API in java that actually works and isn't completely broken / very badly designed. If you can't (java 7 is extremely out of date and insecure, you must upgrade!), there's the jsr310 backport. Add that dependency and use it.
Let me try to explain how to understand time first, because otherwise, any answer to this question cannot be properly understood:
About time!
There are 3 completely different concepts and they are all often simplified to mean 'time', but you shouldn't simplify them - these 3 different ideas are not really related and if you ever confuse one for another, problems always occur. You cannot convert between these 3 concepts unless you do so deliberately!
"solarflares time": These describe the moment in time as a universal global concept that something occurred or will occur. "That solar flare was observed at X" is a 'solarflares' time. Best way to store this is millis since epoch.
"appointment time": These describe a specific moment in time as it was or will be in some localized place, but stated in a globally understandable way. "We have a zoom meeting next tuesday at 5" is one of these. It's not actually constant, because locales can decide to adopt new timezones or e.g. move the 'switch date' for daylight savings. For example, if you have an appointment at your dentist on 'november 5th, at 17:00, 2021', and you want to know how many hours are left until your appointment starts, then the value should not change just because you flew to another timezone and are looking at this number from there. However, it should change if the country where you made the appointment in decided to abolish daylight savings time. That's the difference between this one and the 'solarflares' one. This one can still change due to political decisions.
"wake-up-alarm time": These describe a more mutable concept: Some way humans refer to time which doesn't refer to any specific instant or is even trying to. Think "I like to wake up at 8", and thus the amount of time until your alarm will go off next is continually in flux if you are travelling across timezones.
Now, on to your question:
I have a MySQL database which is storing a datetime value, let's say 2020-10-11 12:00:00. (yyyy-mm-dd hh:mm:ss format)
Not so fast. What exact type does that column have? What is in your CREATE TABLE statement? The key thing to figure out here is what is actually stored on disk? Is it solarflare, appointment, or wakeup-alarm? There's DATE, DATETIME and TIMESTAMP, and over the years, mysql has significantly changed how these things are stored.
I believe that, assuming you are using the modern takes on storage (So, newish mysql and no settings to explicitly emulate old behaviour), e.g. a DATETIME stores sign, year, day, hour, minute, and second under the hood, which means it is wakeup alarm style: There is no timezone info in this, therefore, the actual moment in time is not set at all and depends on who is asking.
Contrast to TIMEZONE which is stored as UTC epoch seconds, so it's solarflares time, and it doesn't include any timezone at all. You'd have to store that separately. As far as I know, the most useful of the 3 time representations (appointment time) is not a thing in mysql. That's very annoying; mysql tends to be, so perhaps par for the course.
In java, all 3 concepts exist:
solarflares time is java.time.Instant. java.util.Date, java.sql.Timestamp, System.currentTimeMillis() are also solarflares time. That 'Date' is solarflares timestamp is insane, but then there is a reason that API was replaced.
appointment time is java.time.ZonedDateTime
wakeup-alarm time is java.time.LocalDateTime.
When I retrieve this data in my controller, it has the java 7 type "Date".
Right. So, solarflares time.
Here's the crucial thing:
If the type of time stored in MySQL does not match the type of time on the java side, pain happens.
It sure sounds like you have wakeup-alarm time on disk, and it ends up on java side as solarflares time. That means somebody involved a timezone conversion. Could have happened internally in mysql, could have happened in flight between mysql and the jdbc driver (mysql puts it 'on the wire' converted), or the jdbc driver did it to match java.sql.Timestamp.
The best solution is not to convert at all, and the only real way to do that is to either change your mysql table def to match java's, so, make that CREATE TABLE (foo TIMESTAMP), as TIMESTAMP is also solarflares time, or, to use, at the JDBC level, not:
someResultSet.getTimestamp(col);
as that returns solarflares time, but:
someResultSet.getObject(col, LocalDateTime.class);
The problem is: Your JDBC driver may not support this. If it doesn't, it's a crappy JDBC driver, but that happens sometimes.
This is still the superior plan - plan A. So do not proceed to the crappy plan B alternative unless there is no other way.
Plan B:
Acknowledge that conversion happens and that this is extremely annoying and errorprone. So, make sure you manage it, carefully and explicitly: Make sure the proper SET call is set up so that mysql's sense of which timezone we are at matched. Consider adding storing the timezone as a column in your table if you really need appointment time. etcetera.
Thanks to #rzwitserloot I was able to find out a solution.
First I'll get the data from the database. I'll get rid of any timezone added by the driver / mysql by converting it to a LocalDateTime. Then, I'll create a new ZonedDateTime using the Timezone that was used when storing the data in the database.
Once I have a ZonedDateTime, it is time to convert it using my current timezone. I'll get a new ZonedDateTime object with the proper time.
Then I just add a few more lines to convert it back to my main "Date" class:
I've used the ThreeTen backport as suggested.
Date dateMySQL //Initialized with the date from mysql
Calendar calendar = new GregorianCalendar()
calendar.setTime(dateMySQL)
org.threeten.bp.LocalDateTime localDateTime = org.threeten.bp.LocalDateTime.of(calendar.get(Calendar.YEAR), calendar.get(Calendar.MONTH)+1,
calendar.get(Calendar.DAY_OF_MONTH), calendar.get(Calendar.HOUR_OF_DAY), calendar.get(Calendar.MINUTE),
calendar.get(Calendar.SECOND))
String timezone //Initialized with the timezone from mysql (Ex: "America/New_York")
ZonedDateTime zonedDateTime = ZonedDateTime.of(localDateTime, ZoneId.of(timezone))
ZonedDateTime utcDate = zonedDateTime.withZoneSameInstant(ZoneId.of("Europe/Madrid"))
calendar.setTimeInMillis(utcDate.toInstant().toEpochMilli())
Date desiredDate = calendar.time
dateMySQL: "2020-10-11 10:00:00" // CEST due to my driver
timezone: "America/New_York"
desiredDate: "2020-10-11 19:00:00" // CEST Yay!
Let's assume that I have client's time saved in my database as 2020-09-22T10:50:37.276240900
I need to present this date in web-service for client app depending on client timezone, for example I need to add 2 hours to saved date if client lives in UTC+2 timezone.
So what am I doing for ?
Getting date from entity and adding timezone to time taken from database (startDate: LocalDateTime)
entity.startDate.atZone(ZoneId.of("Europe/Vienna"))
what gives me the value of ZonedDateTime 2020-09-22T10:50:37.276240900+02:00[Europe/Vienna]
This value is what I'm expecting for, basically "initial time plus 2 hours". After that I would to format this time to have output with this 2 hours of being added, some kind of this
12:50 22.09.2020
but when I do format like this
entity.startDate
.atZone(ZoneId.of("Europe/Vienna"))
.format(DateTimeFormatter.ofPattern(NotificationListener.EUROPEAN_DATE_FORMAT, Locale.ENGLISH))
where const val EUROPEAN_DATE_FORMAT = "HH:mm dd.MM.yyyy"
I get this output 10:50 22.09.2020 which looks like my format is not applied properly, so I cannot see my 2 hours.
So my questions are:
am I correct to adding timezone of client app in described way ?
how to apply timezone in more precise way and format this date to see timezone zone applied ?
LocalDateTime.atZone does not "move" the point in time. In fact it tries to present the point in time where the local time in the given timezone is exactly what the LocalDateTime shows.
In other words: if your LocalDateTime represented 10:00 at some date, then the ZonedDateTime output of atZone will also represent 10:00 local time at the specified time zone (except in cases where that local time doesn't exist due to DST changes).
So if your stored time is actually in UTC, you need to add one more step:
ZonedDateTime utcTime = entity.startDate.atZone(ZoneOffset.UTC);
ZonedDateTime localTime = utcTime.withZoneSameInstant(ZoneId.of("Europe/Vienna"));
Alternatively you can avoid calculating the localTime each time and instead configure the DateTimeFormatter to use a given time zone (which means it'll do the necessary calculations internally) using DateTimeFormatter.withZone. If you do this then you can pass the utcTime to it directly.
I have a customized GWT Date -Time widget which is a combination of two text boxes, one to hold the date and one to hold time. When I enter the date 04/09/1956 12:00 AM (in the Date widget), internally in the ValueChangeHandler for my widget I run it through the GWT's DateTimeFormat class' format() method which takes in the date and time zone information and gives me a formatted Date String that is User friendly when displayed and then based on the date in that widget, I set the time in the Time part of it.
The issue is when I put in dates not too old (my observation was dates not older than those in the year 1981), there seems to be no problem at all. When I put in dates older than that say 1956 in my case, there is some weird Daylight savings logic that messes my format of my string by adjusting it back by 1 hour and gives me 04/08/1956 11:00 PM instead of 04/09/1956 12:00 AM. Even though the date object still represents the date I intended to, the formatted string is messed up with a different date representation.
This is issue is reproducible only when I run the app in production mode. When I run it locally on my machine in the hosted mode, I do not see this problem at all. That is the worst part.
I understand that GWT reads from a javascript file called noCache.js when we run in production mode as opposed to the Web-INF/lib folder in hosted mode.
Also, I run the java.util.Date object through GWT's formatter in several other places where I have a date in hand but never have this issue..... It comes when I run it in a ValueChangeHandler.
Did any one encounter this weird behavior before?
OK. Here is the sample code:
Date date = new Date();
date.setYear(1956 - 1900);
date.setMonth(3);
date.setDate(9);
date.setHours(0);
date.setMinutes(0);
date.setSeconds(0);
// date here is 04/09/1956 12:00 AM
DateTimeFormat dateTimeFormat = new DateTimeFormat(somePattern); //pattern is a string //which represents which pattern you want to use
String formattedDateString = dateTimeFormat.format(date, timeZone); // timeZone is an //instance of com.google.gwt.i18n.client.TimeZone
// formattedDateString is 04/8/1956 11:00 PM. The time got pushed back by one hour.
Windows suport Daylight saving from the year 1987. This is why any date before that year is not correctly shown. check it at Wikipedia
I want to convert Strings like "20000603163334 GST" or "20000603163334 -0300" to UTC time. The problem is that time zones in my strings can be 'general time zones', I mean they can be strings as CET, GST etc. etc. And I don't know how to convert these ones.
Because of these string time zones I can not use Joda Time's DateTimeFormat.forPattern("yyyyMMddhhmmss z").withZone(DateTimeZone.UTC);, because according to the documentation: "Time zone names ('z') cannot be parsed".
So, one question I have is if you know a method to go around this limitation in Joda Time? I would prefer to use Joda Time, if possible, instead of the standard Java API.
Another in which I thought I can solve this problem with time zone's names is to use the Java's SimpleDateFormat.
So I make something like:
SimpleDateFormat f = new SimpleDateFormat("yyyyMMddhhmmss z");
//f.setTimeZone(TimeZone.getTimeZone("UTC"));
f.setCalendar(new GregorianCalendar(TimeZone.getTimeZone("UTC")));
Date time = f.parse("20000603163334 GST");
The SimpleDateFormat parses the String (I don't care here about the problem that there are multiple time zones with the same name - what this class parses it's good for me).
The problem is that I don't know how to convert it from here to UTC. How can I do this?
The fact that I set the f's time zone to UTC (in both the two ways from above) doesn't help. I hope someone can help me fix this, I read a lot of questions and answers on this theme here, on stackoverflow, but I haven't found a solution yet.
I found two solutions to your problem. The first was to set the default time zone to UTC:
TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
I'm not sure what other side effect this might have.
The second solution I found was to use a different SimpleDateFormat for output.
SimpleDateFormat f = new SimpleDateFormat("yyyyMMddhhmmss z");
f.setTimeZone(TimeZone.getTimeZone("UTC"));
Date time = f.parse("20000603163334 GST");
System.out.println(time);
System.out.println("(yyyyMMddhhmmss z): " + f.format(time));
SimpleDateFormat utc = new SimpleDateFormat("yyyyMMddhhmmss z");
utc.setTimeZone(TimeZone.getTimeZone("UTC"));
System.out.println("(yyyyMMddhhmmss z): " + utc.format(time));
Using two SimpleDateFormat objects allowed the output to be put in UTC Time. Here is the output from running this code:
Sat Jun 03 08:33:34 EDT 2000
(yyyyMMddhhmmss z): 20000603043334 GST
(yyyyMMddhhmmss z): 20000603123334 UTC
Here may be the reason why Joda does not support 3 letter zone ids. This is from the TimeZone ( http://download.oracle.com/javase/6/docs/api/java/util/TimeZone.html ) JavaDoc. As far as Joda goes, I didn't see a workaround, but I'm not very familiar with that library.
Three-letter time zone IDs For
compatibility with JDK 1.1.x, some
other three-letter time zone IDs (such
as "PST", "CTT", "AST") are also
supported. However, their use is
deprecated because the same
abbreviation is often used for
multiple time zones (for example,
"CST" could be U.S. "Central Standard
Time" and "China Standard Time"), and
the Java platform can then only
recognize one of them.
I need to store the timezone an email was sent from. Which is the best way to extract it from the email's 'Date:' header (an RFC822 date)? And what is the recommended format to store it in the database (I'm using hibernate)?
I recommend you use Mime4J.
The library is designed for parsing all kinds of email crap.
For parsing dates you would use its DateTimeParser.
int zone = new DateTimeParser(new StringReader("Fri, 27 Jul 2012 09:13:15 -0400")).zone();
After that I usually convert the datetimes to Joda's DateTime. Don't use SimpleDateFormatter as will not cover all the cases for RFC822.
Below will get you the Joda TimeZone (from the int zone above) which is superior to Java's TZ.
// Stupid hack in case the zone is not in [-+]zzzz format
final int hours;
final int minutes;
if (zone > 24 || zone < -24 ) {
hours = zone / 100;
minutes = minutes = Math.abs(zone % 100);
}
else {
hours = zone;
minutes = 0;
}
DateTimeZone.forOffsetHoursMinutes(hours, minutes);
Now the only issue is that the Time Zone you will get always be a numeric time zone which may still not be the correct time zone of the user sending the email (assuming the mail app sent the users TZ and not just UTC).
For example -0400 is not EDT (ie America/New_York) because it does not take Daylight savings into account.
Probably easiest to parse with JodaTime as it supports ISO8601 see Date and Time Parsing and Formatting in Java with Joda Time.
DateTimeFormatter parser2 = ISODateTimeFormat.dateTimeNoMillis();
System.out.println(parser2.parseDateTime(your_date_string));
Times must always be stored in UTC (GMT) with a timezone - i.e. after parsing convert from the timezone to GMT and remove daylight savings offset and save the original timezone.
You must store the date with the timezone after converting to UTC.
If you remove or don't handle the timezone it will cause problems when dealing with data that has come from a different timezone.
Extract the data from the header using some sort of substring or regular expression. Parse the date with a SimpleDateFormatter to create a Date object.
The timezone in the email will not show in which timezone it was send. Some programs use ever UTC or GMT. Of course the time zone is part of the date time value and must also be parse.
Why do you want know it.
- Do you want normalize the timestamp? Then use a DateFormat for parsing it.
- Do you want detect the timezome of the user that send the email? This will not correctly work.
It looks like you already mentioned this in one of your comments, but I think it's your best answer. The JavaMail library contains RFC822 Date header parsing code in javax.mail.internet.MailDateFormat. Unfortunately it doesn't expose the TimeZone parsing directly, so you will need to copy the necessary code directly from javax.mail.internet.MailDateParser, but it's worth taking advantage of the careful work already done.
As for storing it, the parser will give you the date as an offset, so you should be able to store it just fine as an int (letting Hibernate translate that to your database for you).