I have a directory where I receive files matching pattern ABC_STOCK_List_YYYYMMDD_YYYYMMDD.csv.
I am writing a scheduled service in java where I need to check the file is for today;s date and then do my process on that file.
ABC_STOCK_List_20200220_20200220.csv
ABC_STOCK_List_20200219_20200219.csv
ABC_STOCK_List_20200218_20200218.csv
ABC_STOCK_List_20200217_20200217.csv
I have this so far:
private Optional<File> findLatestFile(final String dir) {
return Stream.of(new File(dir).listFiles())
.filter(
file -> !file.isDirectory()
&& file.getName().startsWith(prefix)
&& file.getName().endsWith(".csv")
&& isTodaysFile(file)
)
.findFirst();
}
private boolean isTodaysFile(File file) {
return false;
}
I need help with isTodaysFile() that should check the latter YYYYMMDD is today's date. It should not only rely on date from file name, but also filesystem created or modified time that should also be today.
Here's my approach to this problem. At first, I've created two helper functions, one that retrieves the end date from the filename, and the other one that retrieves the filesystem modified time. I don't think it's necessary to check for the created time, since always created time is less than or equal to modified date time.
The function that retrieves the file name's end date:
private LocalDate getEndDate(File file) {
String fileName = file.getName();
String fileNameWithoutExtension = fileName.substring(0, fileName.lastIndexOf("."));
String[] fileNameChunks = fileNameWithoutExtension.split("_");
String endDateAsString = fileNameChunks[fileNameChunks.length - 1];
return LocalDate.parse(endDateAsString, DateTimeFormatter.ofPattern("yyyyMMdd"));
}
Next, the function that retrieves the filesystem modified date for a file. For this, I'm using the Files#getLastModifiedTime to retrieve the modified date:
private LocalDate getLastModifiedDate(File file, ZoneId zoneId) {
try {
return ZonedDateTime
.ofInstant(Files.getLastModifiedTime(file.toPath()).toInstant(), zoneId)
.toLocalDate();
} catch (IOException e) {
throw new RuntimeException("Could not read file attributes: " + file.getAbsolutePath());
}
}
In the end, it's just using calling these functions and performing the validations:
boolean isTodaysFile(File file) {
Clock systemUTCClock = Clock.systemUTC();
LocalDate localDateNow = LocalDate.now(systemUTCClock);
LocalDate fileEndDate = getEndDate(file);
// first check - validate that the file name's end date is today
if (!fileEndDate.isEqual(localDateNow)) {
return false;
}
LocalDate lastModifiedDate = getLastModifiedDate(file, systemUTCClock.getZone());
// second check - validate that the modified that is today
// no need to check the creation date, since creation date is always less or equal to the last modified date
return lastModifiedDate.equals(localDateNow);
}
I'm using Clock.systemUTC() and instantiating all dates based on this, to make sure we are always using UTC.
LocalDate.now(systemUTCClock)
systemUTCClock.getZone()
If the input files in the directory are:
ABC_STOCK_List_20200220_20200220.csv
ABC_STOCK_List_20200219_20200219.csv
ABC_STOCK_List_20200218_20200218.csv
ABC_STOCK_List_20200217_20200217.csv
ABC_STOCK_List_20200305_20200305.csv
At the date of writing this, the current date is 03-05-2020. The output file, when calling findLatestFile is:
ABC_STOCK_List_20200217_20200305.csv
Notes:
I didn't perform any validations on the file name format. If something is wrong with the format, you may get some errors when retrieving the end date.
Related
I know I'm probably doing something wrong, but I am trying to format a Date that is currently stored inside of a string but it won't let me parse it to a String because it doesn't recognize it as a Date (because it's in a String variable) and won't let me format it because it cannot format it in its current state. For reference, I am making a time clock application.
I apologize if I'm doing something stupid but I am fairly new to this and have never used SimpleDateFormat before. I put some snippets of code below:
ArrayList<String> punchHistoryTimes = new ArrayList<String>();
SimpleDateFormat sdf =new SimpleDateFormat("dd-MM-yyyy HH:mm:ss");
public void updatePunchHistory(Sheet sheet){
for(int rowNum:rowNumbers){
punchHistoryTimes.add(sheet.getRow(rowNum).getCell(1).getStringCellValue());
punchHistory.add(new JLabel(sheet.getRow(rowNum).getCell(0).getRichStringCellValue().getString()+ " " + sheet.getRow(rowNum).getCell(1).getRichStringCellValue().getString()+ " " + sheet.getRow(rowNum).getCell(2).getRichStringCellValue().getString()));
}
}
//other code is above this but not relevant to the issue
currentEmployee.setEndTime(sdf.format(currentDate));
if(punchHistory.get(punchHistory.size()-1).getText().contains("Clocked In")){
calcTimeWorked(punchHistory.get(punchHistoryTimes.size()-1).getText(),currentEmployee.getEndTime());
}else{
//This line below is where the error is happening
//value of currentEmployee.getStartTime() at error: 1654653731536
//value of currentEmployee.getEndTime() at error: 07-06-2022 21:02:12
//Both currentEmployee.getStartTime() and currentEmployee.getEndTime() are stored as Strings
calcTimeWorked(currentEmployee.getStartTime(),currentEmployee.getEndTime());
}
currentEmployee.setHoursWorked(differenceInTime);
I tried using the debugger and it shows the error is that it cannot parse 1654653731536. I understand the issue but cannot get a solution. I believe the issue is because when it stores the value in the excel file it is storing the date as a string but then when it pulls the date back out of the excel later (the application would have been closed between these events) it views it as a string and does not recognize that there is a Date inside of the String. Is there any way to cast the String 1654653731536 to a Date?
Here's my code:
public static String getStringFormat(Date inputDate, String timeZone){
String strFormat = null;
try{
final TimeZone computedTimeZone = TimeZone.createTimeZone(TimeZoneInfo.buildTimeZoneData(timeZone));
DateTimeFormat dateTimeFormat = DateTimeFormat.getFormat(DateTimeFormat.PredefinedFormat.ISO_8601);
strFormat = dateTimeFormat.format(inputDate, computedTimeZone);
Date d = new Date(strFormat);
strFormat = dateTimeFormat.format(d, TimeZone.createTimeZone(0));
String[] s = strFormat.split("\\+");
strFormat = s[0];
}catch(Exception e){
Console.log(e.getMessage());
}
return strFormat;
}
For input, new Date() and Etc/GMT+3 this function returns null. What could be wrong?
Error
Error: NullPointerException: undefined
at NBF_g$.QBF_g$ [as createError_0_g$] (NullPointerException.java:40)
at NBF_g$.ub_g$ [as initializeBackingError_0_g$] (Throwable.java:113)
at NBF_g$.bb_g$ (Throwable.java:61)
at NBF_g$.Ib_g$ (Exception.java:25)
at NBF_g$.avq_g$ (RuntimeException.java:25)
at NBF_g$.gfs_g$ (JsException.java:34)
at new NBF_g$ (NullPointerException.java:27)
at new wou_g$ (JSONString.java:43)
The method TimeZoneInfo.buildTimeZoneData(String tzJSON) doesn't accept the name of the zone, but needs a JSON string full of the details of how that zone works. It turns out that the browser doesn't come to you with all of the details of how all time zones work, so your app has to already be prepared to handle them.
GWT ships with all of the timezones (though they are currently a little out of date, and should be updated in this next release), but you have to tell the compiler which ones you want, or it will compile them out. The full list of all possible timezones and their offsets, etc is not small, so I would encourage you to limit the list.
These are stored in the constants interface TimeZoneConstants. Here is how you might use it:
TimeZoneConstants constants = GWT.create(TimeZoneConstants.class);
// This is the shorthand for TimeZone.createTimeZone(TimeZoneInfo.buildTimeZoneData(...))
TimeZone computedTimeZone = TimeZone.createTimeZone(constants.americaAdak());
//...
If you want to use the timezone string instead, say, passed from the server, you could build a map of the possible timezones that are supported. Be aware though that the full map is very large (200KB just for the timezones in the "America/..." group).
computedTimeZone = TimeZone.createTimeZone(constants.americaAdak());
zones.put(computedTimeZone.getID(), computedTimeZone);
computedTimeZone = TimeZone.createTimeZone(constants.americaAnchorage());
zones.put(computedTimeZone.getID(), computedTimeZone);
computedTimeZone = TimeZone.createTimeZone(constants.americaAnguilla());
zones.put(computedTimeZone.getID(), computedTimeZone);
//...
Then you can read out a specific item from the map as needed:
String tzName = Window.prompt("Enter a timezone name", "America/Chicago");
DateTimeFormat dateTimeFormat = DateTimeFormat.getFormat(DateTimeFormat.PredefinedFormat.ISO_8601);
String strFormat = dateTimeFormat.format(inputDate, zones.get(tzName));
//...
In your comment, you clarified the question, that you only need to deal with offsets, not the full TimeZone string format, i.e. Etc/GMT+3, meaning "Offset of +3 hours from GMT". This is easier to handle - simply parse out the +3 into a number, and use the TimeZone.createTimeZone(int timeZoneOffsetInMinutes) method. This will not understand daylight savings time, but that wouldn't be possible without the full name of the timezone or list of offsets, etc (which gets to why that JSON is so large).
//TODO, implement parse(), don't forget about negative numbers
int offsetInHours = parse(timeZone);
TimeZone computedTimeZone = TimeZone.createTimeZone(60 * offsetInHours);
//...
I have the following code to check the last modification on a file saved on network drive.
private long determineLastEdit(ILoaderData file) {
String localDir = "c:\\Software\\log\\";
String localPDF = "empty28.pdf";
String originDir = "smb:\\ProdName\\ShareName\\Temp\\username\\path\\to\\folder\\";
//My company remote storage
File localFile = new File(originDir + localPDF)
//this does not work
//File localFile = new File(localDir + localPDF)
//this works as expected
Date currentTime = new Date();
long timeCurrent = currentTime.getTime();
long timeFile = localFile.lastModified();
//this returns 0 on remote, correct time on local
boolean fileEx = localFile.isFile(); //returns false on remote, true on local
boolean fileTp = localFile.isAbsolute(); //returns false on remote, true on local
long difference = Math.abs(timeCurrent - timeFile);
return difference;
}
The parameter given to constructor of a file is following:
smb:\\\ProdName\\\ShareName\\\Temp\\\username\\\path\\\to\\\folder\\\empty28.pdf
However, the lastModified() method returns 0 for some reason, what am I doing wrong? The file has no lock of any kind, it is regular (altough empty PDF).
EDIT1: I tested the method on local file, the path was:
c:\\\Software\\\log\\\empty28.pdf
And the value returned was correct, my suspicion was that method was not allowed to execute on a given file since it's on network drive. However, this check happens on one thread that is already authorized. No clue where the error is.
EDIT2: I updated code to provide better examples. Right now, it seems the problem is with reading file from network drive
EDIT3 I Tried using different method. Imported:
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.BasicFileAttributes;
and added code:
Path path = Paths.get(localDir + localPDF);
BasicFileAttributes attr = Files.readAttributes(path, BasicFileAttributes.class);
with same result, again, local drive works and remote is not working.
According to the Javadocs the method lastModified:
Returns a long value representing the time the file was last modified, measured in milliseconds since the epoch (00:00:00 GMT, January 1, 1970), or 0L if the file does not exist or if an I/O error occurs.
So, check out your URL you passes to the File's constructor.
It's as simple as this (note: I included a date format):
String localPDF = "empty28.pdf";
String originDir = "\\\\smb\\ProdName\\ShareName\\Temp\\username\\path\\to\\file\\";
File file = new File(originDir + localPDF);
SimpleDateFormat sdf = new SimpleDateFormat("dd/MM/yyyy HH:mm:ss");
System.out.println(sdf.format(file.lastModified()));
I'm doing an integration testing with DBUnit (2.49) + Hibernate (4.1.3) following this tutorial.
Production database : Oracle 10
Test database : Hsqldb 2.3.3
Context
My data contains the current format of date : yyyy/MM/dd. However,according to DBUnit faq, DBUnit only supports this format yyyy-mm-dd hh:mm:ss.fffffffff, so I had to create a new format for TimeStamp.
How I tried to fix it
I created a CustomTimeStampDataType based on this tutorial. I changed this part:
String formats[] = {"yyyy-MM-dd HH:mm", "yyyy-MM-dd HH:mm a", "yyyy-MM-dd HH:mm:ss.fffffffff"};
into this one:
String formats[] = {"yyyy/MM/dd"};
I created a CustomeDataTypeFactory following the same tutorial. I only make it extend Oracle10DataTypeFactory rather than DefaultDatatTypeFactory.
In HibernateDBUnitTestCase, I override setDatabaseConfig() with the following:
#Override
protected void setUpDatabaseConfig(DatabaseConfig config){
config.setProperty(DatabaseConfig.PROPERTY_DATATYPE_FACTORY, new CustomDataTypeFactory());
}
But I got new errors
I ran a unit test and got this error.
org.dbunit.dataset.datatype.TypeCastException: Unable to typecast value <1997/02/14> of type <java.lang.String> to TIMESTAMP
at org.dbunit.dataset.datatype.TimestampDataType.typeCast(TimestampDataType.java:120)
at org.dbunit.dataset.datatype.TimestampDataType.setSqlValue(TimestampDataType.java:176)
at org.dbunit.database.statement.SimplePreparedStatement.addValue(SimplePreparedStatement.java:73)
at org.dbunit.operation.RefreshOperation$RowOperation.execute(RefreshOperation.java:189)
at org.dbunit.operation.RefreshOperation.execute(RefreshOperation.java:113)
at org.dbunit.AbstractDatabaseTester.executeOperation(AbstractDatabaseTester.java:190)
at org.dbunit.AbstractDatabaseTester.onSetup(AbstractDatabaseTester.java:103)
at org.dbunit.DatabaseTestCase.setUp(DatabaseTestCase.java:156)
at test.HibernateDbUnitTestCase.setUp(HibernateDbUnitTestCase.java:85)
at test.PlayerTest.setUp(PlayerTest.java:117)
Caused by: java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]
at java.sql.Timestamp.valueOf(Unknown Source)
at org.dbunit.dataset.datatype.TimestampDataType.typeCast(TimestampDataType.java:116)
... 20 more
That was weird, it seemed like my CustomTimeStamp was not called, so I changed the date in the dataset using the default format : 1997-02-14 00:00:00.0, and ran the unit test again. Then I got:
org.dbunit.dataset.datatype.TypeCastException: Unable to typecast value <1997-02-14 00:00:00.0> of type <java.lang.String> to TIMESTAMP
at test.CustomTimestampDataType.typeCast(CustomTimestampDataType.java:69)
at test.CustomTimestampDataType.setSqlValue(CustomTimestampDataType.java:84)
at org.dbunit.database.statement.SimplePreparedStatement.addValue(SimplePreparedStatement.java:73)
at org.dbunit.operation.RefreshOperation$RowOperation.execute(RefreshOperation.java:189)
at org.dbunit.operation.RefreshOperation.execute(RefreshOperation.java:113)
at org.dbunit.AbstractDatabaseTester.executeOperation(AbstractDatabaseTester.java:190)
at org.dbunit.AbstractDatabaseTester.onSetup(AbstractDatabaseTester.java:103)
at org.dbunit.DatabaseTestCase.setUp(DatabaseTestCase.java:156)
at test.HibernateDbUnitTestCase.setUp(HibernateDbUnitTestCase.java:85)
at test.PlayerTest.setUp(PlayerTest.java:117)
That means CustomTimeStamp was actually called. Seems like, the problem stemed from DatabaseTestCase.setUp which somehow called the wrong TimeStampDataType.
How could I fix this issue?
My first option was to replace every yyyy/MM/dd into yyyy-mm-dd in the dataset using regular expressions. This worked fine, until I had to test a method that selected a date based on a request (so the format is yyyy-mm-dd) and compared it to the current date. ( so the format is yyyy / mm / dd). Hsqldb can't compare two dates with different format.
My second option was to decompile dbunit.jar, rewrite TimeStampDataType based on the tutorial. I'm unfamiliar with bytecode writing so before entering uncharted waters, I wanted to know if you had another solution.
Thank you in advance
Fixed it!
So I ended up using my second option.
This is the detailed path for those who need it.
Download dbUnit.2.2.source.jar
Unzip the jar
Go to Eclipse, File > New > Java Project
Uncheck "Use default location"
In Location : specify the path to the new folder created from the jar
Click on Finish
Modify the TimestampDataType.java (if needed)
Instead of ts = java.sql.Timestamp.valueOf(stringValue); use the code below
String formats[] =
{"dd/MM/yyyy HH:mm:ss.SS"}; //and more depending on your need
Timestamp ts = null;
for (int i = 0; i < formats.length; i++)
{
SimpleDateFormat sdf = new SimpleDateFormat(formats[i]);
try {
java.util.Date date = sdf.parse(stringValue);
ts = new Timestamp(date.getTime());
return ts;
}
catch( ParseException e) {
}
Modify the DateDataType.java (if needed)
Instead of return java.sql.Date.valueOf(stringValue); , use the code below
String formats[] =
{"dd/MM/yyyy"}; //and more depending on your need
for (int i = 0; i < formats.length; i++)
{
SimpleDateFormat sdf = new SimpleDateFormat(formats[i]);
try {
java.util.Date date = sdf.parse(stringValue);
java.sql.Date datesql = new java.sql.Date(date.getTime());
return datesql;
}
catch( ParseException e) {
}
}
Right-click on your project, then Export
Select JAR file, then Next
Fill the export destination then Finish.
You just have to add this new jar to the library to make it work.
I'm automating processing of flat files received from a mainframe and am confused on creating and modified times. The files are created on the mainframe then emailed to the required individuals. The individuals then save the file from email (Outlook) and do whatever it is they do with them.
For that automation, the file will be processed automatically. I need to let the user know when the last file was loaded and give them the option to load a newer file.
My issue is with the times: I'm getting a creation date/time that is later than the lastmodified time. Using this code:
try {
DateFormat utcFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'");
DateFormat cstFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
utcFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
cstFormat.setTimeZone(TimeZone.getTimeZone("America/Chicago"));
Path filePath = Paths.get("C:/data/KWJFLTD.XLS");
BasicFileAttributes basicAttr = Files.readAttributes(filePath, BasicFileAttributes.class);
FileTime creationTime = basicAttr.creationTime();
FileTime modifiedTime = basicAttr.lastModifiedTime();
String cTime = creationTime.toString();
String mTime = modifiedTime.toString();
Date dc = utcFormat.parse(cTime);
Date dm = utcFormat.parse(mTime);
cTime = cstFormat.format(dc);
mTime = cstFormat.format(dm);
System.out.println("Creation Time: " + cTime);
System.out.println("Modified Time: " + mTime);
} catch (IOException ex) {
Logger.getLogger(FAULoad.class.getName()).log(Level.SEVERE, null, ex);
} catch (ParseException ex){
Logger.getLogger(FAULoad.class.getName()).log(Level.SEVERE, null, ex);
}
I get this result:
Creation Time: 2015-06-24 15:25:12
Modified Time: 2015-06-24 02:28:05
I end up with a creation time after the last modified time. Is this because the creation time is changed when it's saved from email? In this situation, given that there's (theoretically) no modification going on since it was generated, would the last modified time be the same as the creation date?
According to Microsoft Documentation (the link relates to Win XP but most like also applies up to Win 10) the modification time is the time the file's content was last written while the creation time is the time a specific file has been created.
Thus if you copy a file the modification date would not change since no content has been written/changed but a new file (the copy) has been created and thus the creation time would be after the modification time.