Application: Java + ExtJS
There are a lot of different entity with properties of java.util.Date type: startDate and iesendDate (endDate could be NULL). Both dates could be selected with or without time part (e.g. time part is always persisted, event if it is not selected). For example, like this:
2010-07-01 00:00:00
Possible problems start when user selects endDate without time. For example, period starts on 2010-07-01 and ends on 1010-07-04. Right now in database it is stored like:
startDate="2010-07-01 00:00:00"
endDate="2010-07-04 00:00:00".
So it seems that period ends on the FIRST second of 2010-07-04. But as user assume, that endDate is implicitly included, e.g. period ends on LAST second of 2010-07-04. There are a lot of date comparisons for different periods in the system.
How in this case to store end date properly?
I thought about possible solutions, but all of them seems a bit wrong:
To store time part for end date like this: "2010-07-04 23:59:99". But then seems that end date day is not 24h - but (24h - 0.(9) millisecond), that could be potential problem. Also time part looks quite ugly.
To modify ExtJs component that it will add 1 day to date selected by user on persistance stage and substract 1 day again when this date will be shown to user (except cases when the user explicitly set time part). I don't like here that dates with time part and without it are treated differently.
To save only start date as Date object, and then save length of period in seconds, for example. This approach seems quite good - but a have to rework the whole application and possible it will be no very easy to use different comparisons on end dates.
Just use current one - save non-enclusive end date without time and be very careful during dates comparisons
Could someone explain the most widely used practices to solve such problem?
I've recently changed from your first approach to your second approach.
Approach 1: "2010-07-04 23:59:99" should be "2010-07-04 23:59:59" but anyway it is indeed ugly and technically there is a lost second. Another issue I did have is that I wanted to start one record with the same data/time as another record was stopped. So it was possible to find the successor of a record. And with approach 1 this cannot be done as the time will differentiate 1 second.
Approach 2: The end date will be "2010-07-05" and the query condition will be < endDate instead of <= endDate for approach 1. Here the end date is meaning 'till' or 'until' or 'up to' and not like in approach 1 'up to and including'. For this reason in new projects I do use tillDate or actually I use something like serviceStart / serviceTill.
The disadvantage is indeed to format it -1 day when showing to the user. But for me this makes sense.
Add end date as +1 day of user selected value.
Separate model and presentation
Do separate the stored end time in your database completely from the presentation of the end time to the user.
Store end date and time in database: Since you always store both date and time, you should clearly store the true date and time. If the period end at midnight between July 4 and 5, store this time, so 2010-07-05 00:00:00.
Present to the user: How your user would like to see that end time, I cannot say, and you should ask the users themselves. Maybe they will tell you:
As it is. 2010-07-05 00:00:00.
As 24 hours on the last day of the period, 2010-07-04 24:00:00.
As the last day as date only 2010-07-04 (I consider this unlikely, especially of the start time is not at midnight).
Or something else.
So yes, it’s very likely that you will have to treat end at midnight specially for presentation. It is not nearly as bad as if you had needed to treat it specially in your business logic.
Related
I want to use Instant type to put it to MySQL database (timestamp field). Unfortunately, when using POJO and Record#from(), jOOQ doesn't let me do that for some reason. I have the following to my gradle configuration:
forcedTypes {
forcedType {
name = "Instant"
types = "timestamp"
}
}
The code is being generated correctly, but doesn't work and gives me errors in runtime:
Data truncation: Incorrect datetime value: '2021-01-16 05:01:25.457+00:00' for column `test`.`messages`.`time_sent` at row 1
I tried to add my own converter and binder, but they don't seem to work if name is set in gradle config, as a warning appears during build. But without name I can't get jOOQ to generate Instant for timestamp field.
Is there a way to use Instant with SQL timestamp when using jOOQ's POJO?
MySQL's timestamp data type is a bit, well, idiotic.
It stores time as seconds since the epoch (jan 1st 1970, UTC, midnight), in a 32-bit integer. This has a few issues:
There is no room for milliseconds. Your Instant has .457 milliseconds and this needs to be stripped out, which is why JOOQ is refusing to do this; that's destroying data. I assume you don't care about those millis, but JOOQ doesn't know that. You can try to strip the millis out of the instant, if you must, JOOQ would presumably allow saving an Instant, if that Instant has the property that it the epochmillis that it wraps around is divisible by 1000 (has no millis part). Except, that was annoying, so at some point, eventhough it's stored as 32-bit seconds-since-epoch, the data type now also contains, separately, a fractional second, from 0 to 999999. Either you have an old version of MySQL or the underlying table structure that doesn't allow this, or JOOQ isn't aware that MySQL does at least support millis.
At 2038-01-19 03:14:07 UTC, your app explodes. That's it. That's the last timestamp representable by a MySQL TIMESTAMP object. We're less than 18 years removed from this. Therefore this datatype is effectively unusable, and you should use something else. (This may not sound believable to you. Peruse MySQL8's user manual §11.2.2 if you need some convincing, straight from the source). Java's core instant storage system doesn't suffer from the dreaded Y2K38, as java uses millis-since-epoch in 64-bit; we got a few billion years to go before that runs out of numbers.
Note that the printed message is a bit misleading. Instants are stored as milliseconds since epoch and do not have a timezone, that +00.00 in the printout suggests that it does. It doesn't - and thus the fact that mysql's TIMESTAMP type also doesn't isn't the problem.
Solutions
The best solution, by far, is to use a database engine that isn't broken like this. For example, try postgres.
A distant second solution is to peruse JOOQ issue #9665 where #lucaseder (core contributor of JOOQ; he's the genius doing the magic over there :P) says it doesn't work yet, but there's some code there you may be able to use.
Note that if you actually care about zones, this becomes a lot more complicated. There are 3 different ways to represent time; these are distinct and cannot be converted to each other without additional info and caveats; most tools incorrectly silently convert them anyway, and pain ensues when this happens. Thus, you may want to think about which of the 3 kinds of time you have here:
solarflares time: A moment time something happened or will happen, such as a solarflare. If some political entity decides to change timezone, it has no effect. The 'time until event occurs' / 'time since event occurred' goes up by 1 second every second no matter what happens with timezones. In java terms, java.time.Instant.
appointment time: I call my barber in Amsterdam and I have an appointment on Jan 5th, 2023, 14:00. You'd think this is like solarflares time, but, no. If the dutch parliament slams the gavel and adopts a law that the Netherlands will no longer observe daylight savings and will remain in summertime, then the moment that gavel comes down, the # of seconds until my appointment goes up by an hour (is it it down by an hour?). This is not exotic at all - see EU Directive 2000/84/EC - it is, in fact, likely. Solarflares time should not shift like this, and appointment time does. Best represented as year+month+day+hour+minute+second+millis+a full zone (which is Europe/Amsterdam, not silly useless stuff like +0800 or PST). In java terms, ZonedDateTime.
Alarmclock time: Just years, months, day, hour, minute, second. That's it - it doesn't represent anything particular but just the concept. If I set my alarm to wake me up at 08 in the morning and I take a trip across the pacific, the time until my alarm goes off should change drastically as I move timezones. In java terms, LocalDateTime and friends.
I'm assuming you have a solarflares time (for example, to track 'user X change their password at this time'). This answer assumes you meant that. If you did not, my advice would change a bit. Mostly that whole 'do not use mysql' thing would become even stronger: What you really want is the datatype TIMESTAMP WITH TIME ZONE such as postgres has.
We are trying to compare a date stored in mySQL with a plain text date field read from the original of a Gmail message, using java date compare.
First time through, the Gmail message plain text Date is read and stored in a mySQL database field “date_sent” type TIMESTAMP.
The next time that message is checked, it gets the message plain text Date and, using the Java date compare function, compares that with the stored date_sent value.
This compare usually works. However -- if the date and time of the message Date being compared is during the 1am hour on a day in which the time changed (daylight savings to standard), the compare fails.
Has anyone experienced this? how were you able to fix it?
It seems like you've got an uphill battle. If a message arrives, dated 1:30am, on the date when Daylight Savings ends, I don't think there's any way you can tell whether it's the FIRST occurrence of 1:30am (before the clocks go back) or the SECOND.
Presumably, you'll never get a message dated 1:30am on the date when Daylight Savings starts, because this time won't actually exist.
So if you do this by converting the text field to a date and storing it, you'll always have an issue of how to compare such dates. Your comparison might be wrong sometimes if you pick the wrong 1:30am, and I don't think there's too much you can do about it, if you want your timezones to be correct, for both Daylight Savings Time and Standard Time.
One thing you might consider doing is storing the timestamps as text, not as dates, so that the conversion never happens. If you use a format like yyyy-MM-dd HH:mm:ss then you should be able to do a text comparison instead of a date comparison, and get the correct results.
I've never experienced this because I've never done it. What I think you'll have to do is get/know the timezone of the text date, convert it into a Date object (using that timezone), then compare the dates that way.
You say you're using the Date compare method, but that says it requires a Date as input. So are you already converting? Please show us that code. Are you passing in a timezone?
Also, this question has very useful information that you can use.
I have 2 different computers, each with different TimeZone.
In one computer im printing System.currentTimeMillis(), and then prints the following command in both computers:
System.out.println(new Date(123456)); --> 123456 stands for the number came in the currentTimeMillis in computer #1.
The second print (though typed hardcoded) result in different prints, in both computers.
why is that?
How about some pedantic detail.
java.util.Date is timezone-independent. Says so right in the javadoc.
You want something with respect to a particular timezone? That's java.util.Calendar.
The tricky part? When you print this stuff (with java.text.DateFormat or a subclass), that involves a Calendar (which involves a timezone). See DateFormat.setTimeZone().
It sure looks (haven't checked the implementation) like java.util.Date.toString() goes through a DateFormat. So even our (mostly) timezone-independent class gets messed up w/ timezones.
Want to get that timezone stuff out of our pure zoneless Date objects? There's Date.toGMTString(). Or you can create your own SimpleDateFormatter and use setTimeZone() to control which zone is used yourself.
why is that?
Because something like "Oct 4th 2009, 14:20" is meaningless without knowing the timezone it refers to - which you can most likely see right now, because that's my time as I write this, and it probably differs by several hours from your time even though it's the same moment in time.
Computer timestamps are usually measured in UTC (basically the timezone of Greenwich, England), and the time zone has to be taken into account when formatting them into something human readable.
Because that milliseconds number is the number of milliseconds past 1/1/1970 UTC. If you then translate to a different timezone, the rendered time will be different.
e.g. 123456 may correspond to midday at Greenwich (UTC). But that will be a different time in New York.
To confirm this, use SimpleDateFormat with a time zone output, and/or change the timezone on the second computer to match the first.
javadoc explains this well,
System.currentTimeMillis()
Note that while the unit of time of the return value is a millisecond, the granularity of the value depends on the underlying operating system and may be larger. For example, many operating systems measure time in units of tens of milliseconds.
See https://docs.oracle.com/javase/7/docs/api/java/util/Date.html#toString().
Yes, it's using timezones. It should also print them out (the three characters before the year).
Using JodaTime library (although I am a bit flexible). I realized some of the inputs coming in are breaking Joda time because the days of the month are above 31 or below 1 (because of client-side code).
I am using the LocalDate object for calendar manipulation. Is there a library or method to easily sanitize the dates so the input doesn't start throwing exceptions?
Some Scala code I am using now: EDIT: Fixed code
val now = new LocalDate();
val workingDate = now.withYear(y).withMonthOfYear(m).withDayOfMonth(d).withDayOfWeek(DateTimeConstants.SUNDAY)
ymdStart = toTimestampAtStart( workingDate )
For clarification, the goal here is to convert the date to a proper date, so if a user submitted July 38, it would convert to August 7. There's an incoming URL structure causing a lot of this and it looks like /timeline/2012/07/30.
For reasons of pure exercise (I agree normalization seems to be bad practice) I'm now just purely curious if there are libraries that deal with such a problem.
Thanks!
Final Update:
Like the answer points out, normalization was a poor idea. I did a lot of re-factoring on the client side to fix the incoming variables. This is the code I ended up using:
ymdStart = new Timestamp( toTimestampAtStart( new LocalDate(y,m,d).withDayOfWeek(1) ).getTime - 86400000 )
ymdEnd = new Timestamp( ymdStart.getTime + 691200000 )
First of all, a LocalDate is immutable, so each chained with...() is creating a new date.
Second, it is a well-known antipattern to update pieces of a date one at a time. The end result will depend on the current value of the date, the order in which you update the pieces, and whether or not the implementation "normalizes" dates.
In other words NEVER update a date/time piecemeal.
Assume for a minute that the implementation "normalizes" (i.e. corrects for overflow) invalid dates. Given your code, if today's date was 31-Jan-2011 and you did
now.setMonth(FEBRUARY);
now.setDayOfMonth(12);
the result will be 12-March-2011. The first statement sets the date to 31-February, which gets normalized to 03-March, then the day gets set to 12. Ah, you say, you can just set the day-of-month first. But that doesn't work for different starting points (construction of which is left as an exercise).
And from your question I surmise that JodaTime throws exceptions rather than normalize, which is anothe reason for not doing it this way.
I'm an intermediate java coder, writing a librarian Java app, and for some reason I just can't figure out whether I should use dueDate.after(today) OR dueDate.before(today) method as to deciding if the book is overdue. I've gotten quite some contradictory values by typing both the methods. Hence, I'm also assuming there is some other bug in my code as well, so it would be nice if you can conform which is the correct method so that I can move on the fixing the other bug.
You need dueDate.before(today): the due date is before today; the due date is in the past, so the book is past due.
Maybe it's easier if you swap the objects around? You'd get today.after(dueDate) and if you read that out loud, it suddenly becomes quite clear: "if today is after the due date, then ..."
Remember that the before and after methods perform a < (or >) comparison, rather than a <= (or >= comparison). That is what is meant by the word "strictly" in the API documentation.
Also, Java Date objects are really an instant in time, rather than what people usually think of as a "date". That is, it won't just compare the day, but the time of day.
If you want to compare only the day, without checking the time during that day, create all of your due dates to be at a specific time, like midnight. For example, suppose that a book is due October 26. The due date could be midnight, October 27.
boolean overdue = !now.before(dueDate);
The somewhat awkward negation accounts for the case when it is now exactly 12:00 AM Oct. 27.
dueDate.before(today) would translate to the actual dueDate of the book occurring before today, meaning that it is actually overdue. This is probably what you want (assuming you are checking for true)
the other side of things
dueDate.after(today) would translate to the actual dueDate of the book occurring after today, meaning that it is not yet over due.
The only difference between !dueDate.after(today) and dueDate.before(today) is the result when both dates are exactly the same - presumably a book is not overdue when returned on the due date, so dueDate.before(today) should be correct.
As for your unspecified other problems: are you aware that java.util.Date represents a millisecond-precision point in time, not a calendar date? That means that in order to use its comparison methods for calendar dates, you have to make very sure you set the time components to zero when creating your Date instances. Another cause of problems could be time zone differences.
It depends on what you want to achieve. Is dueDate the date you want to set when lending a book? The due date is derived from the lending date of the book, so I would try an approach like book.isDue(today) assuming that the book object contains the lending and due dates as attributes.
With questions like this it helps if you make it explicit for yourself which objects are involved and what their relations are before you design the actions between the objects.