Develop Reference

Parse timestamp string in Java with variable number of nanoseconds

I have a CSV that contains timestamps in the following formats:
yyyy-MM-dd HH:mm:ssX
yyyy-MM-dd HH:mm:ss.SX
yyyy-MM-dd HH:mm:ss.SSX
yyyy-MM-dd HH:mm:ss.SSSX
yyyy-MM-dd HH:mm:ss.SSSSX
yyyy-MM-dd HH:mm:ss.SSSSSX
yyyy-MM-dd HH:mm:ss.SSSSSSX
How can I parse a string that could contain any one of the above formats?
The following code can parse the timestamp when 3-6 nanoseconds are present, but fails when the nano seconds aren't present or are less than 3:
String time = "2018-11-02 11:39:03.0438-04";
DateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSSSSSX");
Date date = sdf.parse(time);
System.out.println("Date and Time: " + date.getTime());
I currently have a method that iterates from 0-6 and generates a string with a number of "S" equal to the value of the iterated variable. The method attempts to parse the string within a try/catch until the string is successfully parsed. For example, the string 2018-11-02 11:39:03.0438-04 will attempt to be parsed five times before being successful.
The CSV is an export of a PostgreSQL table that has columns with type TIMESTAMP WITH TIME ZONE and appears to cut off trailing "0" nanosecond places.
I'm using Java 8 and am open to any external libraries (Joda?).

You'd better use Java Time API1, from the package java.time.
Date, SimpleDateFormatter and Calendar classes are flawed and obsolete.
The DateTimeFormatter class provides numerous options, so you can configure all you need. Note that by using the method appendFraction, the nanos are right-padded.
String[] dateStrs = {
"2018-11-02 11:39:03.4-04",
"2018-11-02 11:45:22.71-04",
"2018-11-03 14:59:17.503-04"
};
DateTimeFormatter f = new DateTimeFormatterBuilder()
.appendPattern("yyyy-MM-dd HH:mm:ss.")
.appendFraction(ChronoField.NANO_OF_SECOND, 1, 9, false)
.appendPattern("X")
.toFormatter();
// Single item:
LocalDateTime date = LocalDateTime.parse("2018-11-02 11:39:03.7356562-04", f);
// Multiple items:
List<LocalDateTime> dates = Arrays.asList(dateStrs).stream()
.map(t -> LocalDateTime.parse(t, f))
.collect(Collectors.toList());
1 Java 8 new Date and Time API is heavily influenced by Joda Time. In fact the main author is Stephen Colebourne, the author of Joda Time.

The first 19 characters are identical.
Also, you have different lengths in the different cases. You can use a switch to test the length of the String and handle the separate cases for the different possible values.

I'm not sure but something like this seems to work for me:
String time = "2018-11-02 11:39:03.0438-04";
DateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSSSSSX");
Date date = sdf.parse(time);
System.out.println("Date and Time: " + date.getTime());
In general, you want to you the longest format possible, with 6x S in this case.

SimpleDateFormat parse 2018-11-08_21h34m46sZ

Hi want to convert UTC times to local time and i am doing this
public class TimeZoneN {
public static void main( String[] args ) throws ParseException {
String timeStr1 = "2018-11-08_21h34m46sZ";
String formatStr1 = "yyyy-MM-dd_HH'h'mm'm'ss's'Z";
SimpleDateFormat formatter = new SimpleDateFormat( formatStr1 );
formatter.setTimeZone( TimeZone.getTimeZone( "UTC" ) );
Date date1 = formatter.parse( timeStr1 );
System.out.println( date1 );
}
}
i am getting a parse exception because od the Z, but when i do
String formatStr1 = "yyyy-MM-dd_HH'h'mm'm'ss'sZ'";
It passes, is that valid though ? Woudn't that consider sZ as a single constant and not Z as the timezone token ?

You are correct that Z denotes Zulu time, another name for UTC. You may also think of it as offset zero from UTC. So you will want to parse Z as an offset to ensure that your time is interpreted correctly.
java.time
However, don’t use SimpleDateFormat. It’s notoriously troublesome and long outdated. Don’t use Date either, it too is long outdated and it too has design problems.
DateTimeFormatter formatter
= DateTimeFormatter.ofPattern("uuuu-MM-dd_H'h'm'm's's'X");
String timeStr1 = "2018-11-08_21h34m46sZ";
Instant instant1 = formatter.parse(timeStr1, Instant::from);
System.out.println(instant1);
Output:
2018-11-08T21:34:46Z
For the sake of completeness:
want to convert … to local time
Taking America/Toronto as a random example:
ZonedDateTime dateTime = instant1.atZone(ZoneId.of("America/Toronto"));
System.out.println(dateTime);
Output:
2018-11-08T16:34:46-05:00[America/Toronto]
In a comment you asked about Z:
what difference would that make if i dont include it at all ?
Two things:
You want to make sure the string includes the required offset and object if not, or you risk that errors pass unnoticed. It’s called input validation.
As I mentioned you want to parse Z as an offset or you cannot extract an unambiguous point in time from the parsed values.
Link: Oracle tutorial: Date Time explaining how to use java.time.

What is the Standard way to Parse different Dates passed as Query-Params to the REST API?

I am working on a REST API which supports Date as a query param. Since it is Query param it will be String. Now the Date can be sent in the following formats in the QueryParams:
yyyy-mm-dd[(T| )HH:MM:SS[.fff]][(+|-)NNNN]
It means following are valid dates:
2017-05-05 00:00:00.000+0000
2017-05-05 00:00:00.000
2017-05-05T00:00:00
2017-05-05+0000
2017-05-05
Now to parse all these different date-times i am using Java8 datetime api. The code is as shown below:
DateTimeFormatter formatter = new DateTimeFormatterBuilder().parseCaseInsensitive()
.append(DateTimeFormatter.ofPattern("yyyy-MM-dd[[ ][['T'][ ]HH:mm:ss[.SSS]][Z]"))
.toFormatter();
LocalDateTime localDateTime = null;
LocalDate localDate = null;
ZoneId zoneId = ZoneId.of(ZoneOffset.UTC.getId());
Date date = null;
try {
localDateTime = LocalDateTime.parse(datetime, formatter);
date = Date.from(localDateTime.atZone(zoneId).toInstant());
} catch (Exception exception) {
System.out.println("Inside Excpetion");
localDate = LocalDate.parse(datetime, formatter);
date = Date.from(localDate.atStartOfDay(zoneId).toInstant());
}
As can be seens from the code I am using DateTimeFormatter and appending a pattern. Now I am first trying to parse date as LocalDateTime in the try-block and if it throws an exception for cases like 2017-05-05 as no time is passed, I am using a LocalDate in the catch block.
The above approach is giving me the solution I am looking for but my questions are that is this the standard way to deal with date sent as String and is my approach is in line with those standards?
Also, If possible what is the other way I can parse the different kinds of date (shown as the Valid dates above) except some other straightforward solutions like using an Array list and putting all the possible formats and then using for-loop trying to parse the date?

DateTimeFormatter formatter = new DateTimeFormatterBuilder()
.append(DateTimeFormatter.ISO_LOCAL_DATE)
// time is optional
.optionalStart()
.parseCaseInsensitive()
.appendPattern("[ ]['T']")
.append(DateTimeFormatter.ISO_LOCAL_TIME)
.optionalEnd()
// offset is optional
.appendPattern("[xx]")
.parseDefaulting(ChronoField.HOUR_OF_DAY, 0)
.parseDefaulting(ChronoField.OFFSET_SECONDS, 0)
.toFormatter();
for (String queryParam : new String[] {
"2017-05-05 00:00:00.000+0000",
"2017-05-05 00:00:00.000",
"2017-05-05T00:00:00",
"2017-05-05+0000",
"2017-05-05",
"2017-05-05T11:20:30.643+0000",
"2017-05-05 16:25:09.897+0000",
"2017-05-05 22:13:55.996",
"2017-05-05t02:24:01"
}) {
Instant inst = OffsetDateTime.parse(queryParam, formatter).toInstant();
System.out.println(inst);
}
The output from this snippet is:
2017-05-05T00:00:00Z
2017-05-05T00:00:00Z
2017-05-05T00:00:00Z
2017-05-05T00:00:00Z
2017-05-05T00:00:00Z
2017-05-05T11:20:30.643Z
2017-05-05T16:25:09.897Z
2017-05-05T22:13:55.996Z
2017-05-05T02:24:01Z
The tricks I am using include:
Optional parts may be included in either optionalStart/optionalEnd or in [] in a pattern. I use both, each where I find it easier to read, and you may prefer differently.
There are already predefined formatters for date and time of day, so I reuse those. In particular I take advantage of the fact that DateTimeFormatter.ISO_LOCAL_TIME already handles optional seconds and fraction of second.
For parsing into an OffsetDateTime to work we need to supply default values for the parts that may be missing in the query parameter. parseDefaulting does this.
In your code you are converting to a Date. The java.util.Date class is long outdated and has a number of design problems, so avoid it if you can. Instant will do fine. If you do need a Date for a legacy API that you cannot change or don’t want to change just now, convert in the same way as you do in the question.
EDIT: Now defaulting HOUR_OF_DAY, not MILLI_OF_DAY. The latter caused a conflict when only the millis were missing, but it seems the formatter is happy with just default hour of day when the time is missing.

I usually use the DateUtils.parseDate which belongs to commons-lang.
This method looks like this:
public static Date parseDate(String str,
String... parsePatterns)
throws ParseException
Here is the description:
Parses a string representing a date by trying a variety of different parsers.
The parse will try each parse pattern in turn. A parse is only deemed successful if it parses the whole of the input string. If no parse patterns match, a ParseException is thrown.
The parser will be lenient toward the parsed date.

#Configuration
public class DateTimeConfig extends WebMvcConfigurationSupport {
/**
* https://docs.spring.io/spring-framework/docs/current/reference/html/core.html#format-configuring-formatting-globaldatetimeformat
* #return
*/
#Bean
#Override
public FormattingConversionService mvcConversionService() {
DefaultFormattingConversionService conversionService = new DefaultFormattingConversionService(false);
conversionService.addFormatterForFieldAnnotation(new NumberFormatAnnotationFormatterFactory());
// Register JSR-310 date conversion with a specific global format
DateTimeFormatterRegistrar dateTimeRegistrar = new DateTimeFormatterRegistrar();
dateTimeRegistrar.setDateTimeFormatter(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
dateTimeRegistrar.setDateTimeFormatter(DateTimeFormatter.ofPattern("yyyy-MM-dd"));
dateTimeRegistrar.setDateTimeFormatter(DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss'Z'"));
dateTimeRegistrar.registerFormatters(conversionService);
// Register date conversion with a specific global format
DateFormatterRegistrar dateRegistrar = new DateFormatterRegistrar();
dateRegistrar.setFormatter(new DateFormatter("yyyy-MM-dd"));
dateRegistrar.setFormatter(new DateFormatter("yyyy-MM-dd HH:mm:ss"));
dateRegistrar.setFormatter(new DateFormatter("yyyy-MM-dd'T'HH:mm:ss'Z'"));
dateRegistrar.registerFormatters(conversionService);
return conversionService;
}
}

DateTimeFormatter Accepting Multiple Dates and Converting to One (java.time library)

I am trying to write a DateTimeFormatter that will allow me to take in multiple different String formats, and then convert the String formats to a specific type. Due to the scope of the project and the code that already exists, I cannot use a different type of formatter.
E.g., I want to accept MM/dd/yyyy as well as yyyy-MM-dd'T'HH:mm:ss but then when I print I only want to print to MM/dd/yyyy format and have it in the format when I call LocalDate.format(formatter);
Could someone suggest ideas on how to do this with the java.time.format.*;
Here is how I could do it in org.joda:
// MM/dd/yyyy format
DateTimeFormatter monthDayYear = DateTimeFormat.forPattern("MM/dd/yyyy");
// array of parsers, with all possible input patterns
DateTimeParser[] parsers = {
// parser for MM/dd/yyyy format
monthDayYear.getParser(),
// parser for yyyy-MM-dd'T'HH:mm:ss format
DateTimeFormat.forPattern("yyyy-MM-dd'T'HH:mm:ss").getParser()
};
DateTimeFormatter parser = new DateTimeFormatterBuilder()
// use the monthDayYear formatter for output (monthDayYear.getPrinter())
// and parsers array for input (parsers)
.append(monthDayYear.getPrinter(), parsers)
// create formatter (using UTC to avoid DST problems)
.toFormatter()
.withZone(DateTimeZone.UTC);
I have not found a good/working example of this online.

I've tested this with JDK 1.8.0_131 for Mac OS X and JDK 1.8.0111 for Windows (both worked).
I've created a DateTimeFormatter with optional sections (delimited by []), to parse both cases (MM/dd/yyyy and yyyy-MM-dd'T'HH:mm:ss).
The same formatter worked for your case (LocalDate), but there are some considerations below.
// parse both formats (use optional section, delimited by [])
DateTimeFormatter parser = DateTimeFormatter.ofPattern("[MM/dd/yyyy][yyyy-MM-dd'T'HH:mm:ss]");
// parse MM/dd/yyyy
LocalDate d1 = LocalDate.parse("10/16/2016", parser);
// parse yyyy-MM-dd'T'HH:mm:ss
LocalDate d2 = LocalDate.parse("2016-10-16T10:20:30", parser);
// parser.format(d1) is the same as d1.format(parser)
System.out.println(parser.format(d1));
System.out.println(parser.format(d2));
The output is:
10/16/2016
10/16/2016
PS: this works only with LocalDate. If I try to format an object with time fields (like LocalDateTime), both formats are used:
System.out.println(parser.format(LocalDateTime.now()));
This prints:
06/18/20172017-06-18T07:40:55
Note that it formatted with both patterns. My guess is that the formatter checks if the object has the fields in each optional section. As the LocalDate has no time fields (hour/minute/second), the second pattern fails and it prints only the first one (MM/dd/yyyy). But the LocalDateTime has all the time fields, and both patterns are valid, so both are used to format.
My conclusion is: this isn't a general solution (like the Joda-Time's version), it's more like a "lucky" case where the patterns involved created the desired situation. But I wouldn't rely on that for all cases.
Anyway, if you are only using LocalDate, you can try to use this code. But if you're working with another types, then you'll probably have to use another formatter for the output, like this:
// parser/formatter for month/day/year
DateTimeFormatter mdy = DateTimeFormatter.ofPattern("MM/dd/yyyy");
// parser for both patterns
DateTimeFormatter parser = new DateTimeFormatterBuilder()
// optional MM/dd/yyyy
.appendOptional(mdy)
// optional yyyy-MM-dd'T'HH:mm:ss (use built-in formatter)
.appendOptional(DateTimeFormatter.ISO_LOCAL_DATE_TIME)
// create formatter
.toFormatter();
// parse MM/dd/yyyy
LocalDate d1 = LocalDate.parse("10/16/2016", parser);
// parse yyyy-MM-dd'T'HH:mm:ss
LocalDate d2 = LocalDate.parse("2016-10-16T10:20:30", parser);
// use mdy to format
System.out.println(mdy.format(d1));
System.out.println(mdy.format(d2));
// format object with time fields: using mdy formatter to avoid multiple pattern problem
System.out.println(mdy.format(LocalDateTime.now()));
The output is:
10/16/2016
10/16/2016
06/18/2017

The parsing part can be written, and has been added in the ThreeTen-Extra library. The relevant code is here and is included below for clarity. The key trick is using parseUnresolved() to find out which format is correct:
public static <T> T parseFirstMatching(CharSequence text, TemporalQuery<T> query, DateTimeFormatter... formatters) {
Objects.requireNonNull(text, "text");
Objects.requireNonNull(query, "query");
Objects.requireNonNull(formatters, "formatters");
if (formatters.length == 0) {
throw new DateTimeParseException("No formatters specified", text, 0);
}
if (formatters.length == 1) {
return formatters[0].parse(text, query);
}
for (DateTimeFormatter formatter : formatters) {
try {
ParsePosition pp = new ParsePosition(0);
formatter.parseUnresolved(text, pp);
int len = text.length();
if (pp.getErrorIndex() == -1 && pp.getIndex() == len) {
return formatter.parse(text, query);
}
} catch (RuntimeException ex) {
// should not happen, but ignore if it does
}
}
throw new DateTimeParseException("Text '" + text + "' could not be parsed", text, 0);
}
Unfortunately, there is no way to write a single DateTimeFormatter that supports flexible parsing and prints using a specific output format as per Joda-Time.

What you're asking is not possible.
DateTimeFormatter is a final class, so you cannot subclass it to implement your own behavior.
The constructor is package-private, so you can't call it yourself. The only way to create a DateTimeFormatter is by using a DateTimeFormatterBuilder. Note that the static helper methods for creating a DateTimeFormatter are internally using DateTimeFormatterBuilder, e.g.
public static DateTimeFormatter ofPattern(String pattern) {
return new DateTimeFormatterBuilder().appendPattern(pattern).toFormatter();
}
DateTimeFormatterBuilder is also a final class, and cannot be subclassed, and it doesn't provide any methods for supplying multiple alternate formats to use, like you want.
In short, DateTimeFormatter is closed and cannot be extended. If your code can only use a DateTimeFormatter, then you are out of luck.

The Answer by Andreas is correct and should be accepted.
Check length of string
As an alternative, you can simply test the length of your string and apply one of two formatters.
DateTimeFormatter fDateOnly = DateTimeFormatter.ofPattern( "MM/dd/uuuu" ) ;
DateTimeFormatter fDateTime = DateTimeFormatter.ISO_LOCAL_DATE_TIME ;
LocalDate ld = null ;
if( input.length() == 10 ) {
try {
ld = LocalDate.parse( input , fDateOnly ) ;
} catch (DateTimeParseException e ) {
…
}
} else if ( input.length() == 19 ) {
try {
LocalDateTime ldt = LocalDateTime.parse( input , fDateTime ) ;
ld = ldt.toLocalDate() ;
} catch (DateTimeParseException e ) {
…
}
} else {
// Received unexpected input.
…
}
String output = ld.format( fDateOnly ) ;
Be aware that you can let java.time automatically localize when generating a string representing the value of your date-time rather than hard-code a specific format. See DateTimeFormatter.ofLocalizedDate.

java.lang.IllegalArgumentException: Invalid format

This code:
DateTimeParser[] parsers = { DateTimeFormat.forPattern("dd/MM/yyyy HH:mm:ss zzz").getParser(),
DateTimeFormat.forPattern("dd/MM/yyyy HH:mm:ss").getParser(), DateTimeFormat.forPattern("dd/MM/yyyy HH:mm").getParser(),
DateTimeFormat.forPattern("HH:mm").getParser() };
DateTimeFormatter formatter = new DateTimeFormatterBuilder().append(null, parsers).toFormatter();
Session session;
DateTime dTime = null;
Calendar calendar;
try{
if (completedTime != null && !completedTime.equalsIgnoreCase("")){
LocalDateTime jt = LocalDateTime.parse(completedTime, formatter);
LocalDateTime dt;
LocalDateTime retDate;
produces the error:
java.lang.IllegalArgumentException: Invalid format: "09/05/2015 04:00:00 GDT" is malformed at " GDT"
at the LocalDateTime jt = LocalDateTime.parse(completedTime, formatter); line
I can't for the life of me work out why it is failing. I am pretty sure it is something simple, but I haven't spotted it.

You may want to refer to this thread (or one of the many others like it). My best advice would be to try cutting to only one "z" in your parser.

You need to manually specify a mapping from timezone abbreviation to timezone. For example:
return new DateTimeFormatterBuilder()
.appendPattern("dd/MM/yyyy HH:mm:ss ")
.appendTimeZoneShortName(UK_TIMEZONE_SYMBOLS)
.toFormatter();
Here UK_TIMEZONE_SYMBOLS is a Map<String,DateTimeZone> which contains our view of timezone names (so BST is British summer time, not Bangladesh standard time)
Here's how we build ours:
public static Map<String, String> buildTimeZoneSymbolMap(Locale locale) {
Map<String, String> timeZoneSymbols = Maps.newLinkedHashMap();
for (String[] zoneInfo : DateFormatSymbols.getInstance(locale).getZoneStrings()) {
String timeZone = zoneInfo[0];
if (!timeZoneSymbols.containsKey(zoneInfo[2])) {
timeZoneSymbols.put(zoneInfo[2], timeZone);
}
if (zoneInfo[4] != null && !timeZoneSymbols.containsKey(zoneInfo[4])) {
timeZoneSymbols.put(zoneInfo[4], timeZone);
}
}
timeZoneSymbols.put("UTC", "GMT");
return timeZoneSymbols;
}
public static Map<String, DateTimeZone> buildDateTimeZoneSymbolMap(Locale locale) {
return Maps.transformValues(buildTimeZoneSymbolMap(locale), input -> DateTimeZone.forTimeZone(TimeZone.getTimeZone(input)));
}
public static final Map<String, DateTimeZone> UK_TIMEZONE_SYMBOLS = ImmutableMap.copyOf(buildDateTimeZoneSymbolMap(Locale.UK));

First thing to note:
What is "GDT"? The website http://www.timeanddate.com/time/zones/ does not yield an answer. So if it really exists and is not a typo then what is your locale? Remember that time zone names and abbreviations are highly localized.
Second: The count of pattern symbols "z" is okay - for classes like SimpleDateFormat etc. - see its documentation. Either four letters for the full name or less than four letters for the abbreviation:
General time zone: Time zones are interpreted as text if they have
names. Text: For formatting, if the number of pattern letters is 4 or
more, the full form is used; otherwise a short or abbreviated form is
used if available. For parsing, both forms are accepted, independent
of the number of pattern letters.
But you use Joda-Time. Its documentation clearly states:
Zone names: Time zone names ('z') cannot be parsed.
I have verified this non-support using the newest Joda-Time version 2.7 by following code:
DateTimeFormatter formatter = DateTimeFormat.forPattern("dd/MM/yyyy HH:mm:ss z").withLocale(Locale.GERMANY);
DateTime dt = formatter.parseDateTime("09/05/2015 04:00:00 MESZ");
System.out.println("Joda-Time: " + dt);
// Exception in thread "main" java.lang.IllegalArgumentException: Invalid format: "09/05/2015 04:00:00 MESZ" is malformed at "MESZ"
Of course, "MESZ" is correct and must be interpreted as Europe/Berlin in context of given german locale.
However, since version update (2.2) the same code set to Locale.US works for some timezones names like "EDT", "PST" etc., see also this commit. So we can finally say, the parsing support of Joda-Time for timezone names and abbreviations is best to say very limited. Once again, what is your Locale? If it is not US then I can understand why you get the exception. And you will also get an exception for the input "GDT" even if we consider it as valid due to the limited capabilities of Joda-Time-parser.