We have a Java webapp that talks to an Oracle database.
We are keeping track of a particular time (timestamp format in Oracle) and we were wondering what would be the best time format in java to use.
The database we are talking to could be in the US or Europe so we need a java type that will allow us to use Locales.
What in peoples opinion would be the best Java type to use?
You should use java.sql.Timestamp together with a PreparedStatement then you don't need to worry about a "format" at all.
What is the datatype of the Oracle field that you are storing timestamp?
Make sure it preserves timezone data.
Related
I am reaching out to the community here because either I found an issue with Hibernate or I just don't understand how to use java.sql.Date and java.time.LocalDate.
I'm running into a problem where my DB is in UTC time zone while my client is in EST. I have a field in the DB called ETA of type DATE that for example is set to 2019-09-10 on a record. When I read the date in EST it becomes 2019-09-09. The DATE field according to documentation has no timezone information.
When Hibernate reads the value, it uses the DateTypeDescriptor.java class. However, the problem with that class is that it will first try to read the value as a java.sql.Date (in the rs.getDate( name ) part) and then call java.sql.Date.toLocalDate() in the javaTypeDescriptor.wrap() part, which is implemented poorly because what it does is it strips off the timezone offset information and just plainly returns a date. This makes 2019-09-09T20:00:00.000-0400 (which is 2019-09-10 in the database) to become 2019-09-09 without the timezone offset part.
What I consider the problem is the part where Hibernate calls rs.getDate() because that must return java.sql.Date. Now, the MySQL driver contains a method public LocalDate getLocalDate(int columnIndex) to obtain a LocalDate, so I don't understand why Hibernate isn't using that method.
I found that someone has already brought up this same problem with them but they don't seem to consider it an issue.
Therefore, I'm reaching out here to understand - is there a bug with Hibernate or am I just not understanding correctly how to convert between DATE (DB) and LocalDate (Java) types.
PS: I use latest Hibernate 5.4.5 and latest JPA 2.2.
Try Hibernate 5 and the configuration option hibernate.jdbc.time_zone. That's an official way to force it to use JDBC APIs which use an Calendar instance in the UTC timezone.
That said, I've found that there are a lot of problems in this area. JDBC drivers and people using databases often get this subtly wrong. That's why we eventually gave up and just put the milliseconds (in UTC) into the database. So the column type is always NUMBER(16) for all three temporal types.
That allows us to write special converters which return predictable results, independent of JDBC drivers, database/VM/OS time zones.
Programmers can then concentrate on trying to understand why timezones don't work they way they think they should :-)
What is the proper way to save date or time based data in the database?
What are the proper "field mappings" for java to postgresql(or to some other database)?
That data should be stored in utc format without timezones.
-> timestamp and date based stuff fails in here, those will add current timezone (http://docs.oracle.com/javase/7/docs/api/java/util/Date.html)
-> what are the other options?
should I use "plain epoch/integer" column and other column for timezone? But then I cannot use all the functions etc. that the database is providing for me.
I could use hibernate with some jodatime magic, but in my current stack I don't have hibernate in use.
Possible solutions:
1). Change the computer/java timezone -> java will in the UCT (eg. export TZ="GMT" or -Duser.timezone=UCT)
2). Use epoch/Integer/Long values in date/time fields / types -> works but now I cannot use build in database functions.
3). Use Jodatime with custom hibernate datatypes?
4). Use Java8 new time and date apis?
In most cases, it is best to use the Postgres data type timestamptz (short for timestamp with time zone) when dealing with multiple time zones or when you want to save all timestamps as UTC.
Don't let the name mislead you, the time zone is not actually saved. But (as opposed to timestamp [without time zone]) the time zone from textual input is taken into account as modifier to compute the actual UTC timestamp value, which is saved.
On output, the text representation of the value is formatted according to your current time zone setting: timestamp is shifted and the according time zone modifier attached to it.
Note that timestamps without appended time zone are interpreted according to the current time zone setting of your session. If you want to enter a literal UTC value disregarding the current time zone, it has to be:
'2014-08-21 16:39:09+0'::timestamptz
not:
'2014-08-21 16:39:09'::timestamptz -- would assume current time zone
Detailed explanation in this related answer:
Ignoring timezones altogether in Rails and PostgreSQL
As you say, it would be best to store dates as UTC on the database. In oracle you can use a DATE or TIMESTAMP datatype. You can then use the java layer to present your dates in local time to the user and with a java.sql.timestamp column. Joda is essentially built in to the latest version of java so definitely use that for any conversions etc. The alternative would be to store timestamp with timezone in oracle and perhaps use oracle date functions in your sql and stored procedures to convert the date as required. We do the former, but it may depend on your team (db people vs java people) and your audience - are there likely to be lots of different timezones in the user base or is timing on the DST changeover going to break your app.
If you can describe particular situations you are concerned about I'm sure someone will help out. Storing your data in UTC will at least ensure that your data is solid but may require many conversions in the presentation layer.
There is not a single correct way but I my opinion is that you should store time as a long unix timestamp and single dates as epoch days. Java 8 has nice functions to deal with them. Avoid locking yourself with jodatime and hibernate just to manage a date.
What do you mean functions that the database is providing you?
You can always do SQL selects with integers and long since they are called from your program.
If you need a lot of manual use of the database (not programmatic) then you may want to use human readable dates.
A lot have been said (and written on SO) on parts of the subject, but not in a comprehensive, complete way, so we can have one "ultimate, covering-it-all" solution for everyone to use.
I have an Oracle DB where I store date+time+timezone of global events, so original TZ must be preserved, and delivered to the client side upon request. Ideally, it could work nicely by using standard ISO 8601 "T" format which can be nicely stored in Oracle using "TIMESTAMP WITH TIME ZONE" column type ("TSTZ").
Something like '2013-01-02T03:04:05.060708+09:00'
All I need to do is to retrieve the above value from DB and send it to client without any manipulations.
The problem is that Java lacks support of ISO 8601 (or any other date+time+nano+tz data type) and the situation is even worse, because Oracle JDBC driver (ojdbc6.jar) has even less support of TSTZ (as opposed to Oracle DB itself where it's well supported).
Specifically, here's what I shouldn't or cannot do:
Any mapping from TSTZ to java Date, Time, Timestamp (e.g. via JDBC getTimestamp() calls) won't work because I lose TZ.
Oracle JDBC driver doesn't provide any method to map TSTZ to java Calendar object (this could be a solution, but it isn't there)
JDBC getString() could work, but Oracle JDBC driver returns string in format '2013-01-02 03:04:05.060708 +9:00', which is not compliant with ISO 8601 (no "T", no trailing 0 in TZ, etc.). Moreover, this format is hard-coded (!) inside Oracle JDBC driver implementation, which also ignores JVM locale settings and Oracle session formatting settings (i.e. it ignores NLS_TIMESTAMP_TZ_FORMAT session variable).
JDBC getObject(), or getTIMESTAMPTZ(), both return Oracle's TIMESTAMPTZ object, which is practically useless, because it doesn't have any conversion to Calendar (only Date, Time and Timestamp), so again, we lose TZ information.
So, here are the options I'm left with:
Use JDBC getString(), and string-manipulate it to fix and make ISO 8601 compliant. This is easy to do, but there's a danger to die if Oracle changes internal hard-coded getString() formatting. Also, by looking at the getString() source code, seems like using getString() would also result in some performance penalty.
Use Oracle DB "toString" conversion: "SELECT TO_CHAR(tstz...) EVENT_TIME ...". This works fine, but has 2 major disadvatages:
Each SELECT now has to include TO_CHAR call which is a headache to remember and write
Each SELECT now has to add EVENT_TIME column "alias" (needed e.g. to serialize the result to Json automatically)
.
Use Oracle's TIMESTAMPTZ java class and extract relevant value manually from its internal (documented) byte array structure (i.e. implement my own toString() method which Oracle forgot to implement there). This is risky if Oracle changes internal structure (unlikely) and demands relatively complicated function to implement and maintain.
I hope there's 4th, great option, but from looking all over the web and SO - I can't see any.
Ideas? Opinions?
UPDATE
A lot of ideas have been given below, but it looks like there is no proper way to do it. Personally, I think using method #1 is the shortest and the most readable way (and maintains decent performance, without losing sub-milliseconds or SQL time-based query capabilities).
This is what I eventually decided to use:
String iso = rs.getString(col).replaceFirst(" ", "T");
Thanks for good answers everyone,
B.
JDBC getObject(), or getTIMESTAMPTZ(), both return Oracle's TIMESTAMPTZ object, which is practically useless, because it doesn't have any conversion to Calendar (only Date, Time and Timestamp), so again, we lose TZ information.
That would be my recommendation as the only reliable way to get the information you seek.
If you are on Java SE 8 and have ojdbc8 then you can use getObject(int, OffsetDateTime.class). Be aware that when you use getObject(int, ZonedDateTime.class) you may be affected by bug 25792016.
Use Oracle's TIMESTAMPTZ java class and extract relevant value manually from its internal (documented) byte array structure (i.e. implement my own toString() method which Oracle forgot to implement there). This is risky if Oracle changes internal structure (unlikely) and demands relatively complicated function to implement and maintain.
This is what we ultimately went with until bug free JSR-310 support is available in the Oracle JDBC driver. We determined this was the only reliable way to get the information we want.
A slight improvement to #2:
CREATE OR REPLACE PACKAGE FORMAT AS
FUNCTION TZ(T TIMESTAMP WITH TIME ZONE) RETURN VARCHAR2;
END;
/
CREATE OR REPLACE PACKAGE BODY FORMAT AS
FUNCTION TZ(T TIMESTAMP WITH TIME ZONE) RETURN VARCHAR2
AS
BEGIN
RETURN TO_CHAR(T,'YYYYMMDD"T"HH24:MI:SS.FFTZHTZM');
END;
END;
/
The in SQL this becomes:
SELECT FORMAT.TZ(tstz) EVENT_TIME ...
It's more readable.
If you ever need to change it, it's 1 place.
The downside is it is an extra function call.
you need two values: time utc in millis since 1970 and timezone offset fom utc.
So store them as a pair and forward them as a pair.
class DateWithTimeZone {
long timestampUtcMillis;
// offset in seconds
int tzOffsetUtcSec;
}
A Date is a pair of numbers. It is not a String. So a machine interface should not contain a date represented by a iso string, although that is handy to debug.
If even java cannot parse that iso date, how do you think that your clients can do?
If you design an interface to your clients, think how they can parse that. And in advance write a code that shows that.
This is untested, but seems like it ought to be a workable approach. I'm not sure about parsing the TZ name out, but just treating the two parts of the TZTZ object as separate inputs to Calendar seems like the was to go.
I'm not sure whether longValue() will return the value in local or GMT/UCT. If it's not GMT, you should be able to load a calendar as UTC and ask it for a Calendar converted to local TZ.
public Calendar toCalendar(oracle.sql.TIMESTAMPTZ myOracleTime) throws SQLException {
byte[] bytes = myOracleTime.getBytes();
String tzId = "GMT" + ArrayUtils.subarray(bytes, ArrayUtils.lastIndexOf(bytes, (byte) ' '), bytes.length);
TimeZone tz = TimeZone.getTimeZone(tzId);
Calendar cal = Calendar.getInstance(tz);
cal.setTimeInMillis(myOracleTime.longValue());
return cal;
}
Do you really care about sub-millisecond precision? If not converting from a UTC millisecond + timezone-offset to your required string is a one-liner using joda-time:
int offsetMillis = rs.getInt(1);
Date date = rs.getTimestamp(2);
String iso8601String =
ISODateTimeFormat
.dateTime()
.withZone(DateTimeZone.forOffsetMillis(offsetMillis))
.print(date.getTime());
Prints, for example (current time in +9:00):
2013-07-18T13:05:36.551+09:00
Regarding the database: Two columns, one for the offset, one for the date. The date column could be an actual date type (thus making many, timezone-independent anyway, db date functions available). For time-zone dependent queries (such as the mentioned global hourly histogram) perhaps a view could expose columns: local_hour_of_day, local_minute_of_hour, etc.
This is likely how one would have to do it if no TSTZ datatype was available--which, considering Oralce's poor support, is the nearly the case for practical purposes. Who wants to use an Oracle specific features anyway! :-)
Since it looks like there's no magical way of doing this right, the simplest and the shortest method would be #1. Specifically, this is all the code needed:
// convert Oracle's hard-coded: '2013-01-02 03:04:05.060708 +9:00'
// to properly formatted ISO 8601: '2013-01-02T03:04:05.060708 +9:00'
String iso = rs.getString(col).replaceFirst(" ", "T");
it seems that just adding 'T' is enough, although a perfectionist would probably put more cosmetics (regex can optimized, of course), e.g.: rs.getString(col).replaceFirst(" ", "T").replaceAll(" ", "").replaceFirst("\+([0-9])\:", "+0$1:");
B.
The solution with oracle is SELECT SYSTIMESTAMP FROM DUAL
I am trying to read several dates from my database but under certain circumstances I get a ' java.sql.SQLException Bad format for DATE'. Here is my code :
Date entryDateD = res.getDate("entryDate");
In debug mode I see that the content of entryDateD '1996-9-15' as is in my database..
Although I would have to mention that I read other dates too from my database which I notice are of the format 'xxxx-0y-zz'. What I want to say is that in case of a month being less than 10 there is a zero added in front of it which in this case is not added. I suspect that this might have something to do with it.
(this zero does not appear in the database itself though not only in this date but in any date)
thanx in advance :)
Just a guess, but what MySQL column type do you have for entryDate??
By default a date type in MySQL will generate an yyyy-mm--dd format; the missing zero leads me to believe that the column type may be varchar or other non-date type at DB level.
This could be the cause of your problems at Java level...
If you represent it as java.util.Date, you have the advantage of allowing the JDBC driver to worry about handling any format issues with the database.
As for how it's rendered in your display, that's up to you and your use of the java.text.DateFormat class.
You're doing the right thing by representing a date as java.util.Date in your app, but you need to understand that formatting is a separate issue from the type.
Java has a child class of java.util.Date, called java.sql.Date
The easiest way to create a Java SQL Date is by calling the constructor with a Utility Date.
java.util.Date uDate = res.getDate("entryDate");
java.sql.Date sDate = new java.sql.Date(uDate.getTime());
If you want to get res.getDate("entryDate") as YYYY-MM-DD then you need to cast as date in your query like this:
SELECT
...
CAST(entryDate as DATE)as entryDate,
...
FROM your_table;
If you feel this is helpful then do upVote to make it useful for others.
I have a simple java object with several date properties and I always seem to change my mind on how to define them. Should the properties be defined as date objects or strings? The object is going to be used in struts 1.3 application with iBatis as the persistence layer and mysql as the database. The database columns are defined as datetime and they can possibly be null and I usually don’t care about the time portion.
public Date getForcastDate();
or
public String getForcastDate();
Most of the existing code base uses strings, but that just doesn’t seem quite right to me.
Keep your dates as Dates. That way you can change formatting depending on locales, check for invalid dates, sort by them etc.
By keeping them as strings you're potentially throwing away data (e.g. milliseconds if your formatter doesn't use them) and definitely behaviour.
Using strong-typing (e.g. keeping them as Dates) will aid in terms of development. Your method signatures become clearer, refactoring using IDE tooling becomes easier etc. Otherwise you end up with APIs that talk in nothing but strings, it's trivial to mix up parameters, and it becomes impossible to work out what's going on.
Tip: Check out Joda-Time as a better alternative to the standard java.util.Date.
I would use Date object because it cleaner to store a Date and convert it to a String when needed. Otherwise you have to hard code a formatted date into a String field.
I would never use Strings in this cas as what would today be 8/3/11 or 3/8/11 or 2011-03-08. This is really a specific case of trying to use the most restrictive type/class possible for a variable. This is so that you can understand its behaviour more fully, both by having a restricted or specialised set of methods and by helping documentation of other classes using it. Using a Date here would allow you to use a Calendar object to add days or months. Conversion to or from a string only needs to be done for input and output.
In practice if they were only dates I would crete my own Date class so could ignore times or use JodaTime which provides easier manipulation than the java Date
In my code I always use the most high level object. In this case I would suggest - Calendar. Here is separate discussion about Date and Calendar. I always think this way - converting Calendar/Date to String is simple - use SimpleDateFormatter. But very often you will need to do something with the date (add several days or hours, subtract a year, handle timezones etc) and then each time you would have to convert it from String to Calendar/Date.
Date if you had to, but java.util.Calendar would probably be more appropriate nowadays. With a String you'd have to worry about format like #jzd mentioned. With Calendar, you can easily switch between formats. Also Calendar lets you get at the date with Calendar.getTime()