How to convert decimal timestamp to date in Java with trailing decimals - java

I have been trying to figure out how to convert a timestamp to a date but with the trailing decimals at the end, so for example:
Timestamp - C50204EC EC42EE92 is equivalent to Sep 27, 2004 03:18:04.922896299 UTC.
The timestamp format includes the first 32-bit unsigned seconds as a field spanning 136 years and the 32-bit fraction field resolving 232 picoseconds. In the timestamp formats, the prime epoch, or base date of era 0, is 0 h 1 January 1900 UTC, when all bits are zero.
This is what I have written for my code so far:
BigDecimal bi = new BigDecimal("1096255084000");
double decimal_timestamp = bi.doubleValue();
DateFormat formatter = new SimpleDateFormat("dd/MM/yyyy hh:mm:ss.SSS");
formatter.setTimeZone(TimeZone.getTimeZone("UTC"));
Calendar calendar = Calendar.getInstance();
calendar.setTimeInMillis(decimal_timestamp);
String date = formatter.format(calendar.getTime());
System.out.println(decimal_timestamp + " = " + date);
My thought is that it is probably not possible with calendar, so I'll have to do it from scratch, but I have no idea how to go about doing that.

java.time
Using the example from the explanation:
Timestamp - C50204EC EC42EE92 is equivalent to Sep 27, 2004
03:18:04.922896299 UTC.
Instant epoch = OffsetDateTime.of(1900, 1, 1, 0, 0, 0, 0, ZoneOffset.UTC).toInstant();
BigInteger timeStamp = new BigInteger("C50204ECEC42EE92", 16);
// To get the whole part and the fraction right, divide by 2^32
double secondsSince1900 = timeStamp.doubleValue() / 0x1_0000_0000L;
// Convert seconds to nanos by multiplying by 1 000 000 000
Instant converted = epoch.plusNanos(Math.round(secondsSince1900 * 1_000_000_000L));
System.out.println(converted);
Output is:
2004-09-27T03:18:04.922896384Z
It’s off by 85 nanoseconds. Likely better floating-point arithmetic can do even better. Edit: A little loss of precision is unavoidable since the original time stamp has a resolution of 2^-32 seconds, which is more than 4 times as fine as the nanosecond (10^-9 second) resolution of Instant.
The Calendar class that you were trying to use was always poorly designed and is now long outdated. Instead I do as Amongalen suggested in a comment, I am using java.time, the modern Java date and time API. Edit: For comparison Calendar has millisecond resolution, so would at best give you a substabtial loss of precision.
Edit: More precise math
I couldn’t let the 85 nanoseconds be. Here’s a version that preserves precision as far as possible and gives the expected result:
BigDecimal timeStamp = new BigDecimal(new BigInteger("C50204ECEC42EE92", 16));
// To get the whole part and the fraction right, divide by 2^32
BigDecimal bit32 = new BigDecimal(0x1_0000_0000L);
BigDecimal secondsSince1900 = timeStamp.divide(bit32);
// Convert seconds to nanos by multiplying by 1 000 000 000; round to long
long nanosSince1900 = secondsSince1900.multiply(new BigDecimal(TimeUnit.SECONDS.toNanos(1)))
.setScale(0, RoundingMode.HALF_UP)
.longValueExact();
Instant converted = epoch.plusNanos(nanosSince1900);
2004-09-27T03:18:04.922896300Z
1 nano too much? This is because I used half-up rounding in the call to setScale. If instead I truncate (using RoundingMode.FLOOR), I get the exact result from the explanation. So my version doesn’t lose more precision than theirs.
Link
Oracle tutorial: Date Time explaining how to use java.time.

Related

converting epoch to ZonedDateTime in Java

How to convert epoch like 1413225446.92000 to ZonedDateTime in java?
The code given expects long value hence this will throw NumberFormatException for the value given above.
ZonedDateTime.ofInstant(Instant.ofEpochMilli(Long.parseLong(dateInMillis)), ZoneId.of(TIME_ZONE_PST));
java.time can directly parse your string
Edit: If your millisecond value is always non-negative, the following DateTimeFormatter can parse it.
private static final String TIME_ZONE_PST = "America/Los_Angeles";
private static final DateTimeFormatter epochFormatter = new DateTimeFormatterBuilder()
.appendValue(ChronoField.INSTANT_SECONDS, 1, 19, SignStyle.NEVER)
.optionalStart()
.appendFraction(ChronoField.NANO_OF_SECOND, 0, 9, true)
.optionalEnd()
.toFormatter()
.withZone(ZoneId.of(TIME_ZONE_PST));
Now parsing into a ZonedDateTime is just one method call:
ZonedDateTime zdt = ZonedDateTime.parse(dateInMillis, epochFormatter);
System.out.println(zdt);
Output is:
2014-10-13T11:37:26.920-07:00[America/Los_Angeles]
It will not work correctly with a negative value: the fraction would still be parsed as positive, which I am assuming would be incorrect. To be sure to be notified in case of a negative value I have specified in the formatter that the number cannot be signed.
A more general solution: use BigDecimal
If you need a more general solution, for example including negative numbers, I think it’s best to let BigDecinmal parse the number and do the math.
BigDecimal bd = new BigDecimal(dateInMillis);
BigDecimal[] wholeAndFractional = bd.divideAndRemainder(BigDecimal.ONE);
long seconds = wholeAndFractional[0].longValueExact();
int nanos = wholeAndFractional[1].movePointRight(9).intValue();
ZonedDateTime zdt = Instant.ofEpochSecond(seconds, nanos)
.atZone(ZoneId.of(TIME_ZONE_PST));
Output is the same as before. Only now we can also handle negative numbers according to expectations:
String dateInMillis = "-1.5";
1969-12-31T15:59:58.500-08:00[America/Los_Angeles]
Even scientific notation is accepted:
String dateInMillis = "1.41322544692E9";
2014-10-13T11:37:26.920-07:00[America/Los_Angeles]
If finer precision than nanoseconds is possible in the string, consider how you want to truncate or round, and instruct BigDecimal accordingly, there are a number of options.
Original answer
Basil Bourque’s answer is a good one. Taking out the nanoseconds from the fractional part into an integer for nanoseconds may entail a pitfall or two. I suggest:
String dateInMillis = "1413225446.92000";
String[] secondsAndFraction = dateInMillis.split("\\.");
int nanos = 0;
if (secondsAndFraction.length > 1) { // there’s a fractional part
// extend fractional part to 9 digits to obtain nanoseconds
String nanosecondsString
= (secondsAndFraction[1] + "000000000").substring(0, 9);
nanos = Integer.parseInt(nanosecondsString);
// if the double number was negative, the nanos must be too
if (dateInMillis.startsWith("-")) {
nanos = -nanos;
}
}
ZonedDateTime zdt = Instant
.ofEpochSecond(Long.parseLong(secondsAndFraction[0]), nanos)
.atZone(ZoneId.of("Asia/Manila"));
System.out.println(zdt);
This prints
2014-10-14T02:37:26.920+08:00[Asia/Manila]
We don’t need 64 bits for the nanoseconds, so I am just using an int.
Assumption: I have assumed that your string contains a floating-point number and that it may be signed, for example -1.50 would mean one and a half seconds before the epoch. If one day your epoch time comes in scientific notation (1.41322544692E9), the above will not work.
Please substitute your desired time zone in the region/city format if it didn’t happen to be Asia/Manila, for example America/Vancouver, America/Los_Angeles or Pacific/Pitcairn. Avoid three letter abbreviations like PST, they are ambiguous and often not true time zones.
Split the number into a pair of 64-bit long integers:
Number of whole seconds since the epoch reference date of first moment of 1970 in UTC
A number of nanoseconds for the fractional second
Pass those numbers to the factory method Instant.ofEpochSecond​(long epochSecond, long nanoAdjustment)
With an Instant in hand, proceed to assign a time zone to get a ZonedDateTime.
ZoneId z = ZoneId.of( "America/Los_Angeles" ) ;
ZonedDateTime zdt = instant.atZone( z ) ;
Expanding on Basil's and Ole's answers here, for the special case of a negative timestamp i.e. before epoch. Is that even possible? Here's what Jon Skeet writes in "All about java.util.Date":
The Date class uses “milliseconds since the Unix epoch” – that’s the
value returned by getTime(), and set by either the Date(long)
constructor or the setTime() method. As the moon walk occurred before
the Unix epoch, the value is negative: it’s actually -14159020000.
The only real difference between Ole's answer (besides a few extra asserts) is that here, we do not reverse the sign on nanos if the date string starts with a negative sign. The reason for this is, when passing the nanos to the Instant constructor, that is an adjustment, so if we send the nanos as a negative, it will actually adjust the seconds back, and thus the entire ZonedDateTime value is off by the nanos.
This is from the JavaDoc, note the interesting behavior:
This method allows an arbitrary number of nanoseconds to be passed in.
The factory will alter the values of the second and nanosecond in
order to ensure that the stored nanosecond is in the range 0 to
999,999,999. For example, the following will result in the exactly the
same instant:
Instant.ofEpochSecond(3, 1);
Instant.ofEpochSecond(4,-999_999_999);
Instant.ofEpochSecond(2, 1000_000_001);
So the 2nd argument, nanos, we are not setting the value, it is an adjustment. So just the same as for a positive timestamp (after epoch), we want to send in the actual nanos.
Taking Ole's code as a base and adding the above mentioned changes:
String strDateZoned = "Jul 20 1969 21:56:20.599 CDT"; // yes, should use America/Chicago here as Ole points out
DateTimeFormatter dtfFormatter = DateTimeFormatter.ofPattern("MMM dd yyyy HH:mm:ss.SSS zzz");
ZonedDateTime originalZoned = ZonedDateTime.parse(strDateZoned, dtfFormatter);
long epochSeconds = originalZoned.toInstant().getEpochSecond();
int nanoSeconds = originalZoned.toInstant().getNano();
String dateInMillis = epochSeconds + "." + nanoSeconds;
String[] secondsAndFraction = dateInMillis.split("\\.");
int nanos = 0;
if (secondsAndFraction.length > 1) { // there’s a fractional part
// extend fractional part to 9 digits to obtain nanoseconds
String nanosecondsString
= (secondsAndFraction[1] + "000000000").substring(0, 9);
nanos = Integer.parseInt(nanosecondsString);
}
ZonedDateTime zdt = Instant
.ofEpochSecond(Long.parseLong(secondsAndFraction[0]), nanos)
.atZone(ZoneId.of("America/Chicago"));
String formattedZdt = dtfFormatter.format(zdt);
System.out.println("zoneDateTime expected = " + strDateZoned);
System.out.println("zoneDateTime from millis = " + formattedZdt);
assertEquals("date in millis is wrong", "-14159020.599000000", dateInMillis);
assertEquals("date doesn't match expected",strDateZoned, dtfFormatter.format(zdt));
Output from code:
zoneDateTime expected = Jul 20 1969 21:56:20.599 CDT
zoneDateTime from millis = Jul 20 1969 21:56:20.599 CDT
If we reverse the sign on nanos for the case where the seconds part is negative, we can see the difference in the formatted ZonedDateTime:
org.junit.ComparisonFailure: date doesn't match expected
Expected :Jul 20 1969 21:56:20.599 CDT
Actual :Jul 20 1969 21:56:19.401 CDT
P.S. A few more thoughts from the 'All About Dates' post on what Jon Skeet calls "leniency", and elsewhere I have seen called 'normalization' which is perhaps due to POSIX influences:
It’s lenient for no obvious reason: “In all cases, arguments given to
methods for these purposes need not fall within the indicated ranges;
for example, a date may be specified as January 32 and is interpreted
as meaning February 1.” How often is that useful?

Calculate time difference between two times represented as longs

I am trying to calculate the difference between two times, which are represented as longs in the Format HHmm 24 hour time. E.g 4:30pm is represented by the long 0430.
I am happy for the difference to be in minutes.
Is there a simple calculation that can be done to achieve this? I am aware of Java's Date class, however I want to avoid having to store dummy date information just for a calculation on time.
Thanks!
Putting aside the fact that this is a really, really bad way to store times, the easiest way to do this is to convert the HHMM time to minutes since the start of the day:
long strangeTimeFormatToMinutes(long time) {
long minutes = time % 100;
long hours = time / 100;
return minutes + 60 * hours;
}
Then just use plain old subtraction to get the difference.
You may also want to add validation that minutes and hours are in the ranges you expect, i.e. 0-59 and 0-23.
You mentioned that you didn't want to use the Date class because it required you to use a dummy date. The LocalTime class does not require that.
LocalTime start = LocalTime.of(6,15,30,200); // h, m, s, nanosecs
LocalTime end = LocalTime.of(6,30,30,320);
Duration d = Duration.between(start, end);
System.out.println(d.getSeconds()/60);
Pad zeros
First convert your integer to a 4-character string, padding with leading zeros.
For example, 430 becomes 0430 and parsed as 04:30. Or, 15 becomes 0015 and parsed as quarter past midnight, 00:15.
String input = String.format( "%04d", yourTimeAsInteger );
LocalDate
The LocalTime class represents a time-of-day value with no date and no time zone.
DateTimeFormatter f = DateTimeFormatter.ofPattern( "HHmm" );
LocalTime ld = LocalTime.parse( input , f ) ;

Found weird behavior with Joda API

I started looking at joda API for handling timezone related issues for my web application. I found very weird behavior while doing so. I hope there must be some workaround. here is my snap-code.
DateTime utcDateTime = new DateTime(System.currentTimeMillis(), DateTimeZone.UTC);
DateTime currentDateTime = DateTime.now();
int seconds = Seconds.secondsBetween(utcDateTime.toLocalDateTime(), currentDateTime.toLocalDateTime()).getSeconds();
System.out.println("hours::" + (((float)seconds)/(60*60)));
My local timezone is : kolkata, India (GMT + 5:30)
then I should get 5.5 as myn console output, but I am getting hours::5.499722
Don't know Why? Am I doing anything wrong?
(5.5 hours - 5.499722 hours) in ms
=
1000.8 ms
1 second, to the rounding/floating point precision. You call
DateTime utcDateTime = new DateTime(System.currentTimeMillis(), DateTimeZone.UTC);
DateTime currentDateTime = DateTime.now();
Seconds.secondsBetween().getSeconds() gives you an integer number of seconds. Your calls are successive meaning they can be off by even 1 millisecond, which can be rounded to 1 second when converted to int.
Given that you want hours and minutes (an Hours object can only hold an integer number of hours) you probably want:
Period period = new Period(utcDateTime.toLocalDateTime(),
currentDateTime.toLocalDateTime(),
PeriodType.time());
Or alternatively, and more cleanly IMO:
int offsetMillis = DateTimeZone.getDefault().getOffset(Instant.now());
Duration offset = Duration.millis(offsetMillis);
You should very rarely need to know the offset in this fashion though, in my experience.

TOD clock time to java.util.Date or milliseconds

I have a database table, which is filled with data from a mainframe via ETL.
One column of that table is called "TOD" as in Time-Of-Day.
This columns stores values such as :
"CAE7631DC43DC686"
"CAE7631C4AC6DC0B"
"CAE6216DF2BC0D04"
"CAE621D8F9916E8E"
all these values are around Feb 10th 2013 and Feb 11th 2013.
now, on mainframe, this is a time-date representation (TOD clock).
it represents the time past from 01.01.1900 in macroseconds (1/1 000 000 of a second).
What I need is a java library / method / algorithm implementation that could convert these strings to java.util.Date's.
Found these sites on the web :
http://paul.saers.com/Tod_howto.html
http://www.longpelaexpertise.com.au/toolsTOD.php
This page explains how to calculate it, but it's a little too much for me.
I'm sure I'd do some errors somewhere.
So, my question is; do you know about a library (Joda Time ?) that I could use ?
I need to convert these value to a java.util.Date and a Date object to a string representation, (like "CAE621D8F9916E8E").
Thanks in advance.
In my use case I have a getter method that directly reads the 8 bytes TOD as byte array and translates it into a long, but here to adhere to the poster:
BigInteger bi = new BigInteger ("CAE7631DC43DC686", 16); // no strip off of 686
long tod = bi2.longValue();
I used the following to avoid the BigDecimal calculation overhead:
tod = tod >>> 12; // remove rightmost 3 bytes and replace with zeros
tod = tod - 2208988800000000l; // substract 1970
tod = tod/1000; // make millis out of micros
// timeformatter and dateformatter without Joda
SimpleDateFormat timeFormatter = new SimpleDateFormat("HH:mm:ss.SS z Z", Locale.getDefault());
SimpleDateFormat dateFormatter = new SimpleDateFormat("dd.MM.yyyy", Locale.getDefault());
// Display
System.out.println(timeFormatter.format(new Date(tod)));
System.out.println(dateFormatter.format(new Date(tod)));
The output will be:
22:59:46.420 CET +0100
10.02.2013
Step by step, using Joda:
Data used in the calculation can be found on the website you referred to The other reference you gave states that TOD is expressed in UTC
// we start with your string minus the three last digits
// which are some internal z/Series cruft
BigInteger bi = new BigInteger ("CAE7631DC43DC", 16); // 686 stripped off
// then, from tables the website we get the TOD value for start of epoch
// here also, minus the three last digits
BigInteger startOfEpoch70 = new BigInteger ("7D91048BCA000", 16); // 000 stripped off
// using that we calculate the offset in microseconds in epoch
BigInteger microsinepoch = bi.subtract(startOfEpoch70);
// and reduce to millis
BigInteger millisinepoch = microsinepoch.divide(new BigInteger("1000"));
// which we convert to a long to feed to Joda
long millisinepochLong = millisinepoch.longValue();
// Et voila, the result in UTC
DateTime result = new DateTime(millisinepochLong).withZone(DateTimeZone.UTC);
// Now, if you want a result in some other timezone, that's equally easy
// with Joda:
DateTime result2 = result.toDateTime(DateTimeZone.forID("EET"));
System.out.println("The result is " + result + " or represented in timezone EET "
+ result2);
Which gives this output:
The result is 2013-02-10T21:59:46.420Z or represented in timezone
EET 2013-02-10T23:59:46.420+02:00
The "cruft" I refer to is explained as follows:
We skip the last 12 bits (normally,some of these bits are used by MVS to tell what processor was used to read the TOD clock and what LPAR was active).
Of course, instead of brutally snipping these bytes off the string, one could also do
bi = bi.divide(new BigInteger("1000", 16));
as dividing by hex 1000 will also get rid of the last 12 bits.
EDIT: as Mehmet pointed out in the comments, TOD is in UTC and this means that the resulting DateTime should be told so. For convenience I also showed how to transpose that DateTime to another time zone (using EET as an example)
Parse your hex date using BigInteger:
new BigInteger("CAE7631DC43DC686", 16);
Then do the necessary conversions to the Unix epoch using the various methods offered by BigInteger (multiply, ...).

Bug in Java Calendar / Date for 2nd October 2010?

I'm not sure what I'm doing wrong, but I've got a piece of code which calculates the number of days between two dates, and which looks something like the following:
final Calendar first = new GregorianCalendar(2010, Calendar.OCTOBER, 1);
final Calendar last = new GregorianCalendar(2010, Calendar.NOVEMBER, 1);
final long difference = last.getTimeInMillis() - first.getTimeInMillis();
final long days = difference / (1000 * 60 * 60 * 24);
System.out.println("difference: " + difference);
System.out.println("days: " + days);
To summarise, the code block above calculates the number of days between 1st October 2010 and 1 November 2010. I'm expecting to see it return 31 days (seeing as there's 31 days in October)
difference: xxxx
days: 31
but instead it's showing 30 days in October!
difference: 2674800000
days: 30
I've managed to narrow it down to between the the dates 2 October 2010 and 3 October 2010, which seems to only have 82800000 milliseconds, instead of a full 86400000 milliseconds (exactly one hour missing).
Does anyone have any ideas what I'm doing wrong? Or is the 2nd October a special date which has one minute less than a regular day?
(86400000 - 82800000)/1000 = 3600, which is one hour. You're seeing daylight savings time, combined with the rounding of integer math
You could get around it by doing the calculations with floating point numbers and rounding at the end, or you could check out a library like Joda time which offers much better date math than what's built in.
You may be better off comparing the year and day or year instead of the milliseconds that pass in a day.
int lastYear= last.get(Calendar.YEAR);
int firstYear= first.get(Calendar.YEAR);
int lastDayofYear = last.get(Calendar.DAY_OF_YEAR);
int firstDayofYear = first.get(Calendar.DAY_OF_YEAR);
int nDaysElapse = lastDayofYear - firstDayofYear;
int nYearsElapse = lastYear- firstYear;
int days = (nYearsElapse*365)+nDaysElapse;
You should read this post to get a better understanding of how Calendar is interrelated with date/time stamps.
Having read that site, my initial questions were:
What do you mean by days? Do you mean '24-hour blocks' or do you mean calendar days? In the same vein, do you care if you are off slightly due to daylight savings etc?
If you mean Calendar days, your best bet is probably to:
final Calendar first = new GregorianCalendar(2010, 9, 1);
final Calendar last = new GregorianCalendar(2010, 10, 1);
Calendar difference = Calendar.getInstance();
difference.setTimeInMillis(last.getTimeInMillis() - first.getTimeInMillis());
int numDays = difference.get(Calendar.DAY_OF_YEAR) - difference.getMinimum(Calendar.DAY_OF_YEAR);
Of course, the above code will only work if the number of days < 365. You will need to create a rolling calculation e.g.
int yearDiff = last.get(Calendar.YEAR) - first.get(Calendar.YEAR);
Calendar tmp = new GregorianCalendar();
tmp.setTimeInMillis(first.getTimeInMillis());
for(int i = 0; i < yearDiff; i++) {
numDays += tmp.getActualMaximum(Calendar.DAY_OF_YEAR);
i++;
tmp.add(Calendar.YEAR, 1);
}
This should allow you to get the number of days in a correct and consistent manner, without worrying about Daylight Savings, Leap Years etc.
That said, JodaTime probably has this functionality built in.
The answer by Brad Mace is correct.
Use a Library
This question is a good example of why you should use a good date-time library wither than roll your own. For Java that means using either Joda-Time or the new java.time package in Java 8.
Joda-Time
Example code in Joda-Time.
DateTimeZone timeZone = DateTimeZone.forID( "Australia/Melbourne" );
DateTime theFirst = new DateTime( 2014, DateTimeConstants.OCTOBER, 1, 0, 0, 0, timeZone ).withTimeAtStartOfDay();
DateTime nextMonth = theFirst.plusMonths( 1 ).withTimeAtStartOfDay();
int days = Days.daysBetween( theFirst, nextMonth ).getDays();
Or if you don't care about time-of-day, use the LocalDate class.
java.time
Java 8 and later comes with a new java.time framework to supplant the old java.util.Date/.Calendar classes. Inspired by Joda-Time, defined by JSR 310, and extended by the ThreeTen-Extra project.
Example code using Java 8. Note that the enum ChronoUnit returns a 64-bit long rather than int.
LocalDate firstOfOctober = LocalDate.of( 2010 , java.time.Month.OCTOBER , 1 );
LocalDate nextMonth = firstOfOctober.plusMonths( 1 );
long daysInMonth = ChronoUnit.DAYS.between( firstOfOctober , nextMonth );
The code you put in your post is calculating the time between September 1 and October 1, not October 1 and November 1. The output is correct for the code you posted.

Categories

Resources