Exasol is transforming old dates incorrectly:
SELECT ADD_SECONDS('1970-01-01 00:00:00',-30610224000.000)
-- 0999-12-27 00:00:00
SELECT ADD_SECONDS('1970-01-01 00:00:00',-30609792000.000)
-- 1000-01-01 00:00:00
While in java:
System.out.println(Instant.ofEpochMilli(0).plus(-30610224000L, ChronoUnit.SECONDS));
System.out.println(Instant.ofEpochMilli(0).plus(-30609792000L, ChronoUnit.SECONDS));
1000-01-01T00:00:00Z
1000-01-06T00:00:00Z
Do you know why that diference?
Not knowing Exasol I bet it’s the difference between the Julian and the proleptic Gregorian calendar.
The Julian calendar (named after Julius Caesar) has leap years every 4 years always. At a point in history they discovered that this gave a bit too many leap years. So under pope Gregor the Gregorian calendar was introduced, leaving out leap years for years that are divisible by 100 but not by 400 (so 1900 was not a leap year, 2000 was and 2100 will not be). The change is known as the Gregorian change or changeover.
The proleptic Gregorian calendar is a newer invention. It extrapolates the Gregorian calendar back into times before the Gregorian change, thus using dates that are in disagreement with the calendar actually used back in those times. The advantages being that calculations are simpler, and we are free from deciding when to make the Gregorian change, which is nice since each jurisdiction had their own date for that.
Instant and the other classes from java.time use the proleptic Gregorian calendar, so give inaccurate dates for years 999 and 1000. If Exasol uses the Julian calendar (which I do not know), this could be the explanation for the differences you observed.
Links
Julian calendar
Proleptic Gregorian calendar
Related question: Java: How to get the timestamp of '1000-01-01 00:00:00' in UTC?
Related
How would one calculate a number of days in 1582. Yes, that is the year of introduction of the Georgian Calendar (in some countries). I assume October 1582 should not have 31 days as some of the dates never existed.
Yet when I tried Joda Time (Java/Groovy) it says 30 days:
LocalDate start = new LocalDate("1582-10-01");
LocalDate end = new LocalDate("1582-10-31");
println Days.daysBetween(start, end).getDays();
Same for SQL
-- PostgreSQL
SELECT DATE_PART('day', '1582-10-31'::date - '1582-10-01'::timestamp);
-- MSSQL
SELECT DATEDIFF(dd, '1582-10-31', '1582-10-01');
So is there some agreement/specification to actually treat 1582-10-14 as if it would actually exist? Or is there some easy way to calculate correct diff for year 1582 and earlier?
I have not used Java in many years, but I am familiar with dealing with several calendars in other languages. From the "Key Concepts" subtab of the "Documentation" tab of the Joda Time website we find the "Chronology" page which states
The default chronology in Joda-Time is ISO. This calendar system is
the same as that used by business in the majority of the world today.
The ISO system is unsuitable for historical work before 1583 as it
applies the leap year rules from today back in time (it is a proleptic
calendar). As a result, users requiring a more historically accurate
calendar system are forced to think about their actual requirements,
which we believe is a Good Thing.
Proleptic means that from a known day and date that virtually everyone agrees about, such as the Meter Convention having been signed in Paris on 20 May 1875, the rules of the calendar are applied backward to find any date desired, even if it is before the calendar was created.
As for computing the interval in one calendar, such as the Julian calendar, to a date in a different calendar, such as the Gregorian calendar, a common approach is to convert them both to a count-of-days from a chosen epoch, such as the modified julian date, which counts from midnight universal time at the beginning of November 17, 1858. Then one simply subtracts one day count from the other to find the number of days between them. A quick glance at the Joda Time documentation did not show any facility for computing a day count.
I am currently not set up to program in Java. Ole V.V. comment about using the Gregorian-Julian chronology of Joda-Time seems useful, but I have not tried it:
LocalDate first = new LocalDate(1582, 10, 1, GJChronology.getInstance());
LocalDate last = new LocalDate(1582, 10, 31, GJChronology.getInstance());
int countOfDaysDiff = Days.daysBetween(first, last).getDays();
System.out.println(countOfDaysDiff);
Output according to Ole V.V.:
20
I think I will go ahead and close with that both answers are probably correct. October 1582 did and didn't have 31 days. I mean that 14th October didn't exist (as in no one was born on that day in Gregorian Calendar) and for the purpose of accounting all debts were pushed by ten days. So I guess the only way is to manually count days and don't use any libraries for that.
When establishing Gregorian Calendar it was said that:
we direct and ordain:
that ten days shall be removed from the month of October of the year 1582
But also:
But in order that nobody suffers prejudice by this our subtraction of ten days, in connection with any annual or monthly payments, the judges in any controversies that may arise over this, shall by reason of the said subtraction add ten days to the due date for any such payment.
Source: https://en.wikisource.org/wiki/Translation:Inter_gravissimas
I try to provide a tool to convert datetime from Java to C#. But there is a serious problem.
In Java, I read '0001-01-01' from the SQL Server database via java.sql.Date, and get the millisecond -62135798400000.
I also consider the timezone offset.
private static long getMilliSecondWithoutTimeZone(long origin) {
return origin + (ZonedDateTime.now().getOffset().getLong(OFFSET_SECONDS) * 1000);
}
And the final millisecond is -62135769600000.
In C#, I use this millisecond to new Datetime
var ticks = new DateTime(1970, 1, 1).Ticks + (-62135769600000 * 10000);
var date = new DateTime(ticks);
When the code runs, it will throw the exception:
System.ArgumentOutOfRangeException: 'Ticks must be between DateTime.MinValue.Ticks and DateTime.MaxValue.Ticks. (Parameter 'ticks')'
However, the conversion is correct after '1600-01-01' according to my test.
Before '1600-01-01', there always is a few days of error.
It makes me very confused.
I find the remarks in https://learn.microsoft.com/en-us/dotnet/api/system.globalization.juliancalendar?view=net-5.0#remarks
The Gregorian calendar was developed as a replacement for the Julian calendar (which is represented by the JulianCalendar class) and was first introduced in a small number of cultures on October 15, 1582. When working with historic dates that precede a culture's adoption of the Gregorian calendar, you should use the original calendar if it is available in the .NET Framework. For example, Denmark changed from the Julian calendar to the Gregorian calendar on February 19 (in the Julian calendar) or March 1 (in the Gregorian calendar) of 1700. In this case, for dates before the adoption of the Gregorian calendar, you should use the Julian calendar. However, note that no culture offers intrinsic support for the JulianCalendar class. You must use the JulianCalendar class as a standalone calendar. For more information, see Working with calendars.
The actual reason is:
C# uses the Gregorian calendar all the time.
Java uses the Gregorian calendar after October 15, 1582, and uses the Julian calendar before.
The solution:
import java.sql.Date;
import java.time.chrono.IsoChronology;
import java.time.*;
public class Test {
public static Long getMilliSeconds(Date date) {
if (null == date) {
return null;
}
IsoChronology ISO = IsoChronology.INSTANCE;
LocalDate ld = date.toLocalDate();
return ISO.localDateTime(LocalDateTime.of(ld.getYear(), ld.getMonth(), ld.getDayOfMonth(), 0, 0, 0)).toInstant(ZoneOffset.UTC).toEpochMilli();
}
}
It seems like the millisecond value that you mention, -62_135_798_400_000, comes out of an old-fashioned java.sql.Date object created in a timezone that is assumed to be at UTC offset +08:00 back then, perhaps just Etc/GMT-8. With this assumption, the value is historically correct since it was the Julian calendar that was used back then, and Date does use that.
I don’t know the .NET classes that C# uses, but I consider it a likely that a few days error are caused by them using the proleptic Gregorian calendar, that is, pretending that the Gregorian calendar was used in all past even though it didn’t come into existence before 1582. The modern Java date and time API does this and therefore gives you millisecond values that usually differ by a few days.
long milliseconds = LocalDate.of(1, 1, 1)
.atStartOfDay(ZoneOffset.ofHours(8))
.toInstant()
.toEpochMilli();
System.out.format(Locale.ENGLISH, "%,d%n", milliseconds);
Output:
-62,135,625,600,000
It is 48 hours — or 2 days — later than the time you mentioned. See if it solves your issue.
Link
Oracle tutorial: Date Time explaining how to use java.time.
You forgot to account for time zone offset.
If we set the time zone to UTC, you'll see this:
TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
System.out.println(new Date(-62135798400000L));
Output
Fri Dec 31 16:00:00 UTC 1
It is actually year 1 BC, not year 1 AD.
The time 16:00 indicates a time zone offset of 8 hours, so if we change to GMT+8 we get:
TimeZone.setDefault(TimeZone.getTimeZone("GMT+8"));
System.out.println(new Date(-62135798400000L));
Output
Sat Jan 01 00:00:00 GMT+08:00 1
That is correctly year 1 AD.
Which means that you need to adjust the millisecond value by 8 hours, aka 28800000 milliseconds.
For the date 0001-01-01 00:00 UTC, the correct value for milliseconds is -62135769600000. Anything less than that will be rejected by the C# DateTime class.
If I try to set java.sql.Date as
new java.sql.Date(1582-1900,09,14)
It returns me
1582-10-24
So there is a difference of 10 days. How to solve this problem?
Are you sure that date exists?
Wikipedia says the Gregorian Calender (which is what you are probably using) started on October 15, 1582.
When the new calendar was put in use, the error accumulated in the 13 centuries since the Council of Nicaea was corrected by a deletion of 10 days. The Julian calendar day Thursday, 4 October 1582 was followed by the first day of the Gregorian calendar, Friday, 15 October 1582 (the cycle of weekdays was not affected).
If you need to deal with days before that, you probably have to write some more involved code.
This is due to the calendar being switched from Julian to Gregorian in that year. (The latter has the 100 and 400 leap year corrections that the Julian calendar lacks. This accounts for the 10 day difference that had accumulated.)
Note that some countries - in particular England - did not adopt that calendar until 1752. And Russia, for example, didn't adopt it until well into the 20th century!
As a rule of thumb, if you're working with dates before 1752 then you ought to consult an historian.
Consider this code:
Date date = new SimpleDateFormat("MMddyyyy").parse("01011500");
LocalDate localDateRight = LocalDate.parse(formatter.format(date), dateFormatter);
LocalDate localDateWrong = LocalDateTime.ofInstant(date.toInstant(), ZoneId.systemDefault()).toLocalDate();
System.out.println(date); // Wed Jan 01 00:00:00 EST 1500
System.out.println(localDateRight); // 1500-01-01
System.out.println(localDateWrong); // 1500-01-10
I know that 1582 is the cutoff between the Julian and Gregorian calendars. What I don't know is why this happens, or how to adjust for it.
Here's what I've figured out so far:
The date Object has a BaseCalender set to JulianCalendar
date.toInstant() just returns Instant.ofEpochMilli(getTime())
date.getTime() returns -14830974000000
-14830974000000 is Wed, 10 Jan 1500 05:00:00 GMT Gregorian
So it seems like either the millis returned by getTime() is wrong (unlikely) or just different than I expect and I need to account for the difference.
LocalDate handles the proleptic gregorian calendar only. From its javadoc:
The ISO-8601 calendar system is the modern civil calendar system used
today in most of the world. It is equivalent to the proleptic
Gregorian calendar system, in which today's rules for leap years are
applied for all time. For most applications written today, the
ISO-8601 rules are entirely suitable. However, any application that
makes use of historical dates, and requires them to be accurate will
find the ISO-8601 approach unsuitable.
In contrast, the old java.util.GregorianCalendar class (which is indirectly also used in toString()-output of java.util.Date) uses a configurable gregorian cut-off defaulting to 1582-10-15 as separation date between julian and gregorian calendar rules.
So LocalDate is not useable for any kind of historical dates.
But bear in mind that even java.util.GregorianCalendar often fails even when configured with correct region-dependent cut-off date. For example UK started the year on March 25th before 1752. And there are many more historical deviations in many countries. Outside of Europe even the julian calendar is not useable before introduction of gregorian calendar (or best useable only from a colonialist perspective).
UPDATE due to questions in comment:
To explain the value -14830974000000 let's consider following code and its output:
SimpleDateFormat format = new SimpleDateFormat("MMddyyyy", Locale.US);
format.setTimeZone(TimeZone.getTimeZone("America/New_York"));
Date d = format.parse("01011500");
long t1500 = d.getTime();
long tCutOver = format.parse("10151582").getTime();
System.out.println(t1500); // -14830974000000
System.out.println(tCutOver); // default gregorian cut off day in "epoch millis"
System.out.println((tCutOver - t1500) / 1000); // output: 2611699200 = 30228 * 86400
It should be noted that the value -12219292800000L mentioned in your earlier comment is different by 5 hours from tCutOver due to timezone offset difference between America/New_York and UTC. So in timezone EST (America/New_York) we have exactly 30228 days difference. For the timespan in question we apply the rules of julian calendar that is every fourth year is a leap year.
Between 1500 and 1582 we have 82 * 365 days + 21 leap days. Then we have also to add 273 days between 1582-01-01 and 1582-10-01, finally 4 days until cut-over (remember 4th of Oct is followed by 15th of Oct). At total: 82 * 365 + 21 + 273 + 4 = 30228 (what was to be proved).
Please explain to me why you have expected a value different from -14830974000000 ms. It looks correct for me since it handles the timezone offset of your system, the julian calendar rules before 1582 and the jump from 4th of Oct 1582 to cut-over date 1582-10-15. So for me your question "how do I tell the date object to return the ms to the correct Gregorian date?" is already answered - no correction needed. Keep in mind that this complex stuff is a pretty long time in production use and can be expected to work correctly after so many years.
If you really want to use JSR-310 for that stuff I repeat that there is no support for gregorian cut-over date. The best thing is that you might do your own work-around.
For example you might consider the external library Threeten-Extra which contains a proleptic julian calendar since release 0.9. But it will still be your effort to handle the cut-over between old julian calendar and new gregorian calendar. (And don't expect such libraries to be capable of handling REAL historic dates due to many other reasons like new year start etc.)
Update in year 2017: Another more powerful option would be using HistoricCalendar of my library Time4J which handles much more than just julian/gregorian-cutover.
When I parse a date with the year 0000 it appears to be stored as the year 0001.
See below for code:
String dateStr = "00000102";
System.out.println(dateStr);
DateFormat dateFormat = new SimpleDateFormat("yyyyMMdd");
Date date = dateFormat.parse("00000102");
String convertedStr = dateFormat.format(date);
System.out.println(convertedStr);
The output is as per below:
00000102
00010102
Is there a way to represent the year 0000 in Java using the standard Java API?
I don't believe it's possible, since java.util.Date is based on UTC, which is based on the Gregorian calendar, and the Gregorian calendar has no year zero.
...the traditional proleptic Gregorian calendar (like the Julian calendar) does not have a year 0 and instead uses the ordinal numbers 1, 2, … both for years AD and BC. Thus the traditional time line is 2 BC, 1 BC, AD 1, and AD 2.
(Source: The Wikipedia article on the Gregorian calendar)
I don't think the calendar is zero-based. Before 1 AD there was 1 BC. No 0.
Also: what kind of application are you building that needs to handle dates from that era? And if you need to cover that area, consider this: "Dates obtained using GregorianCalendar are historically accurate only from March 1, 4 AD onward, when modern Julian calendar rules were adopted. Before this date, leap year rules were applied irregularly, and before 45 BC the Julian calendar did not even exist."
Year 0 does not exist in the Gregorian calendar. From Year 0 at Wikipedia:
"Year zero" does not exist in the widely used Gregorian calendar or in its predecessor, the Julian calendar. Under those systems, the year 1 BC is followed by AD 1.
...
The absence of a year 0 leads to some confusion concerning the boundaries of longer decimal intervals, such as decades and centuries. For example, the third millennium of the Gregorian calendar began on Monday, 1 January, 2001, rather than the widely celebrated Saturday, 1 January, 2000. Likewise, the 20th century began on 1 January 1901.
...
java.time
I recommend that you use java.time, the modern Java date and time API, for your date work.
As others have said, the Julian/Gregorian calendar that the old Date and SimpleDateFormat classes used does not have a year zero. Before year one in the current era (CE also known as AD) came year 1 before the current era (BCE also known as BC).
A side effect of using java.time is that you do get a year zero! That’s right. java.time uses the proleptic Gregorian calendar, a modern inventions that not only extends the rules of the Gregorian back into times before the Gregorian calendar was invented, but also includes a year 0 before year 1, and a year -1 (minus one) before that. You may say that year 0 corresponds to 1 BCE and -1 to 2 BCE, etc.
So parsing your string is no problem. There’s even a built-in formatter for it.
String dateStr = "00000102";
LocalDate date = LocalDate.parse(dateStr, DateTimeFormatter.BASIC_ISO_DATE);
System.out.println("Parsed date is " + date);
String convertedStr = date.format(DateTimeFormatter.BASIC_ISO_DATE);
System.out.println(convertedStr);
Output:
Parsed date is 0000-01-02
00000102
We see that in both output lines year 0000 is printed back as expected.
What went wrong in your code?
When we all agree that there was no year 0, we should have expected your parsing to fail with an exception because of the invalid year. Why didn’t it? It’s one of the many problems with the old SimpleDateFormat class: with default settings it just extrapolates and takes year 0000 to mean the year before year 0001, so year 1 BCE. And falsely pretends that all is well. This explains why year 0001 was printed back: it meant year 1 BCE, but since you didn’t print the era too, this was really hard to tell.
Links
Oracle tutorial: Date Time explaining how to use java.time.
Proleptic Gregorian calendar on Wikipedia.