I have the following code to check the last modification on a file saved on network drive.
private long determineLastEdit(ILoaderData file) {
String localDir = "c:\\Software\\log\\";
String localPDF = "empty28.pdf";
String originDir = "smb:\\ProdName\\ShareName\\Temp\\username\\path\\to\\folder\\";
//My company remote storage
File localFile = new File(originDir + localPDF)
//this does not work
//File localFile = new File(localDir + localPDF)
//this works as expected
Date currentTime = new Date();
long timeCurrent = currentTime.getTime();
long timeFile = localFile.lastModified();
//this returns 0 on remote, correct time on local
boolean fileEx = localFile.isFile(); //returns false on remote, true on local
boolean fileTp = localFile.isAbsolute(); //returns false on remote, true on local
long difference = Math.abs(timeCurrent - timeFile);
return difference;
}
The parameter given to constructor of a file is following:
smb:\\\ProdName\\\ShareName\\\Temp\\\username\\\path\\\to\\\folder\\\empty28.pdf
However, the lastModified() method returns 0 for some reason, what am I doing wrong? The file has no lock of any kind, it is regular (altough empty PDF).
EDIT1: I tested the method on local file, the path was:
c:\\\Software\\\log\\\empty28.pdf
And the value returned was correct, my suspicion was that method was not allowed to execute on a given file since it's on network drive. However, this check happens on one thread that is already authorized. No clue where the error is.
EDIT2: I updated code to provide better examples. Right now, it seems the problem is with reading file from network drive
EDIT3 I Tried using different method. Imported:
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.BasicFileAttributes;
and added code:
Path path = Paths.get(localDir + localPDF);
BasicFileAttributes attr = Files.readAttributes(path, BasicFileAttributes.class);
with same result, again, local drive works and remote is not working.
According to the Javadocs the method lastModified:
Returns a long value representing the time the file was last modified, measured in milliseconds since the epoch (00:00:00 GMT, January 1, 1970), or 0L if the file does not exist or if an I/O error occurs.
So, check out your URL you passes to the File's constructor.
It's as simple as this (note: I included a date format):
String localPDF = "empty28.pdf";
String originDir = "\\\\smb\\ProdName\\ShareName\\Temp\\username\\path\\to\\file\\";
File file = new File(originDir + localPDF);
SimpleDateFormat sdf = new SimpleDateFormat("dd/MM/yyyy HH:mm:ss");
System.out.println(sdf.format(file.lastModified()));
Related
I have a directory where I receive files matching pattern ABC_STOCK_List_YYYYMMDD_YYYYMMDD.csv.
I am writing a scheduled service in java where I need to check the file is for today;s date and then do my process on that file.
ABC_STOCK_List_20200220_20200220.csv
ABC_STOCK_List_20200219_20200219.csv
ABC_STOCK_List_20200218_20200218.csv
ABC_STOCK_List_20200217_20200217.csv
I have this so far:
private Optional<File> findLatestFile(final String dir) {
return Stream.of(new File(dir).listFiles())
.filter(
file -> !file.isDirectory()
&& file.getName().startsWith(prefix)
&& file.getName().endsWith(".csv")
&& isTodaysFile(file)
)
.findFirst();
}
private boolean isTodaysFile(File file) {
return false;
}
I need help with isTodaysFile() that should check the latter YYYYMMDD is today's date. It should not only rely on date from file name, but also filesystem created or modified time that should also be today.
Here's my approach to this problem. At first, I've created two helper functions, one that retrieves the end date from the filename, and the other one that retrieves the filesystem modified time. I don't think it's necessary to check for the created time, since always created time is less than or equal to modified date time.
The function that retrieves the file name's end date:
private LocalDate getEndDate(File file) {
String fileName = file.getName();
String fileNameWithoutExtension = fileName.substring(0, fileName.lastIndexOf("."));
String[] fileNameChunks = fileNameWithoutExtension.split("_");
String endDateAsString = fileNameChunks[fileNameChunks.length - 1];
return LocalDate.parse(endDateAsString, DateTimeFormatter.ofPattern("yyyyMMdd"));
}
Next, the function that retrieves the filesystem modified date for a file. For this, I'm using the Files#getLastModifiedTime to retrieve the modified date:
private LocalDate getLastModifiedDate(File file, ZoneId zoneId) {
try {
return ZonedDateTime
.ofInstant(Files.getLastModifiedTime(file.toPath()).toInstant(), zoneId)
.toLocalDate();
} catch (IOException e) {
throw new RuntimeException("Could not read file attributes: " + file.getAbsolutePath());
}
}
In the end, it's just using calling these functions and performing the validations:
boolean isTodaysFile(File file) {
Clock systemUTCClock = Clock.systemUTC();
LocalDate localDateNow = LocalDate.now(systemUTCClock);
LocalDate fileEndDate = getEndDate(file);
// first check - validate that the file name's end date is today
if (!fileEndDate.isEqual(localDateNow)) {
return false;
}
LocalDate lastModifiedDate = getLastModifiedDate(file, systemUTCClock.getZone());
// second check - validate that the modified that is today
// no need to check the creation date, since creation date is always less or equal to the last modified date
return lastModifiedDate.equals(localDateNow);
}
I'm using Clock.systemUTC() and instantiating all dates based on this, to make sure we are always using UTC.
LocalDate.now(systemUTCClock)
systemUTCClock.getZone()
If the input files in the directory are:
ABC_STOCK_List_20200220_20200220.csv
ABC_STOCK_List_20200219_20200219.csv
ABC_STOCK_List_20200218_20200218.csv
ABC_STOCK_List_20200217_20200217.csv
ABC_STOCK_List_20200305_20200305.csv
At the date of writing this, the current date is 03-05-2020. The output file, when calling findLatestFile is:
ABC_STOCK_List_20200217_20200305.csv
Notes:
I didn't perform any validations on the file name format. If something is wrong with the format, you may get some errors when retrieving the end date.
I want to write data fetched from a database table every time to new text files with different names.
I use Calendar.getInstance().getTime() to make the filename unique and create the corresponding new txt file in the required directory:
String path = "/home/username/Documents/Netbeans Projects/cashbackEngine/Reports";
Date currentTime = Calendar.getInstance().getTime();
SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMdd_HHmm");
SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyyy-MM-dd");
String dateString = simpleDateFormat.format(currentTime);
String reportFileName2 = "ICC_Cashback_Report_SS_$1$.txt".replace("$1$", sdf.format(currentTime));
String reportFilePath2 = path + "/" + reportFileName2;
File file = new File(reportFilePath2);
writer = new FileWriter(file, true);
Then I iterate over the dataset and write the data to the file by using writer.write();
Is there any other simple way to do this... or any inbuilt string builder method in Java to automatically generates the new file names with constant prefix and unique suffix numbers.
I don't want to use a timestamp to distinguish it from others
and every time new data will get added to the newly generated text file whenever the Java code is executed...
You could use a simple counter as a file prefix. You'd have to persist the current state of the counter between different runs of the program, though, e.g. also in the database. Here's a question dealing with the same issue and answers containing your current approach and others.
I have to validate and read .txt files from a date wise folder. Need suggestions on best possible ways to achieve this
I will have a property file which will have following information
value active channel
5092 Y 11
5092 Y 12
5092 Y 13
5093 N 10
5093 N 11
from this property file first i need get active value(i.e. 5092) and their channel information i.e 11,12,13
based on the above information need to iterate files from a date wise folder.
Input(folder)
10JUN2017
HW_5092_ABC_11.txt
HW_5092_ABC_12.txt
HW_5092_ABC_13.txt
11JUN2017
HW_5092_ABC_11.txt
HW_5092_ABC_12.txt
Based on the property file information (i.e. 5092 is active file and it has channel 11,12,13) look for the current date folder i.e. 11JUN2017 then look for files of 5092. If 11JUN2017 has files related to 5092 (i.e. all 3 files 11,12 and 13) then need to read files from 11JUN2017 folder and process the files. Else need to go back to previous date then look for the files.
In the above example, 11JUN2017 does not have all the files so, i need to go back to previous date i.e. 10JUN2017 and look for the files if found then process it. If 10JUN2017 also does not have all files then go back to previous date (max no of days to traverse back is 43 days).
Update
There is some change in my requirement. There will not be date wise folder instead file name itself contains date in YYYYMMdd format and all files will be in single folder, for example, filename will be like as follows BIG_ABCHINE_MATERIAL_2092_11_20170614-150136-243.txt.
So below is what I am trying
public void fileLoadingProcess() {
//Reading text file which contains all the information
in = new BufferedReader(new FileReader("C:\\OrgDetails.txt"));
in.readLine();
String str;
while ((str = in.readLine()) != null) {
values = str.split("\t");
//Adding config property values to Map
if (values[5].equalsIgnoreCase("Y")) {
System.out.println("Active sales org " + values[0]);
if (distChannelMap.containsKey(values[0])) {
list = distChannelMap.get(values[0]);
list.add(Integer.parseInt(values[6]));
} else {
list = new ArrayList<Integer>();
list.add(Integer.parseInt(values[6]));
distChannelMap.put(values[0], list);
}
} else {
System.out.println("Inactive Sales org " + values[0]);
}
}
Set set = distChannelMap.entrySet();
Iterator iterator = set.iterator();
while (iterator.hasNext()) {
Map.Entry mentry = (Map.Entry) iterator.next();
System.out.println("Key --> " + mentry.getKey() + " Value(s) -->" + mentry.getValue());
//Calling this method to check file needs to be processed based on active and inactive states
isFileNeedsToBeProcessed(mentry.getKey().toString(), (ArrayList<Integer>) mentry.getValue());
}
//Check whether this file is exists or not
//
}
public void isFileNeedsToBeProcessed(String salesOrg, ArrayList<Integer> distChannel) {
System.out.println("Previous Day " + new TxtToXMLCommon().previousDate(1));
//String previousDate = new TxtToXMLCommon().getCurrentDate();
for (int i = 0; i < distChannel.size(); i++) {
int x =1;
previousDate = new TxtToXMLCommon().previousDate(x);
System.out.println("Distribution channel " + distChannel.get(i));
File folder = new File(Constants.INPUT_FOLDER);
File[] files = folder.listFiles();
if (files.length == 0) {
_logger.info("***No files present to process***");
return;
} else {
for (int k = 0; k < files.length; k++) {
if (files[k].getName().contains("MATERIAL")) {
if (salesOrg.equalsIgnoreCase(files[k].getName().substring(21, 25))
&& (distChannel.get(i) == Integer.parseInt(files[k].getName().substring(26, 28)))&&
previousDate.equalsIgnoreCase(files[k].getName().substring(29, 37))) {
System.out.println("File present to process " + files[k].getName());
break;
// if () {
// //processingFilesMap.put("MATERIAL", files[k].getName());
// System.out.println("File present to process " + files[k].getName());
// }else{
//
// }
}else{
//previousDate = new TxtToXMLCommon().previousDate(x+1);
}
//previousDate = new TxtToXMLCommon().previousDate(2);
}//end of if(MATERIAL)
}// end of files length
}
}
}
But I am little bit stuck here while finding for the file in a folder(single source folder). Say, as you see, I have put the property file information in the distChannelMap and then for each value I am iterating source folder and trying to find the file which contains org value values[0] and key values[6] which I have stored in arraylist. because for one value 5092 there are 3 channel.
Now if I do not find file matching org value (5092), channel (11) and date (20170615), I need to iterate the folder again with date-1 means value 5092, channel 11 and date 20170614 like this till 43 days. once i found the file I am thinking to put it into map so that I have all files which are ready to process. But I am little bit stuck in going to previous date when i do not the file for particular channel. once this channel is find, I need to do the same process for other channels like 12 and 13 for the org 5092.
I think it’s simplest to take one channel at a time.
Construct the file name from active value and channel number, for example
String.format("HW_%d_ABC_%d.txt", activeValue, channel)
Use LocalDate for the folder names.
DateTimeFormatter dateFormatter = DateTimeFormatter.ofPattern("ddMMMuuuu", Locale.ENGLISH);
LocalDate date = LocalDate.now(ZoneId.systemDefault());
String folderName = date.format(dateFormatter).toUpperCase(Locale.ROOT);
Think twice about the time zone to use. ZoneId.systemDefault() will give you the JVM’s current setting, it may not be what you need. I am typing this on June 10th and understand that in your time zone it’s already June 11th. Would be a pity to miss today’s files because of incorrect time zone.
In a loop, look for the file in the folder. If found, exit the loop. If not found, do
date = date.minusDays(1);
folderName = date.format(dateFormatter).toUpperCase(Locale.ROOT);
This will give you the folder name of the previous day’s folder. Then look there.
This will work across the beginning of the month and the beginning of the year. For the limit of looking 43 days back, either use a counter or set
LocalDate limit = date.minusDays(43);
and then give up when date.isBefore(limit).
Edit If you want to start from the folders and files, look into Files.newDirectoryStream(Path, String). You may use it first to find the folders of this month and the previous two months (that should cover 43 days and more), maybe filter and sort them. Next for each folder you can use the same method to determine which files are present. You will have a challenge keeping track of which files are present in more than one folder and which one is the newest, it can be solved. I am not immediately convinced about the advantage of this approach, but I trust you to make a good decision for yourself.
I'm doing an integration testing with DBUnit (2.49) + Hibernate (4.1.3) following this tutorial.
Production database : Oracle 10
Test database : Hsqldb 2.3.3
Context
My data contains the current format of date : yyyy/MM/dd. However,according to DBUnit faq, DBUnit only supports this format yyyy-mm-dd hh:mm:ss.fffffffff, so I had to create a new format for TimeStamp.
How I tried to fix it
I created a CustomTimeStampDataType based on this tutorial. I changed this part:
String formats[] = {"yyyy-MM-dd HH:mm", "yyyy-MM-dd HH:mm a", "yyyy-MM-dd HH:mm:ss.fffffffff"};
into this one:
String formats[] = {"yyyy/MM/dd"};
I created a CustomeDataTypeFactory following the same tutorial. I only make it extend Oracle10DataTypeFactory rather than DefaultDatatTypeFactory.
In HibernateDBUnitTestCase, I override setDatabaseConfig() with the following:
#Override
protected void setUpDatabaseConfig(DatabaseConfig config){
config.setProperty(DatabaseConfig.PROPERTY_DATATYPE_FACTORY, new CustomDataTypeFactory());
}
But I got new errors
I ran a unit test and got this error.
org.dbunit.dataset.datatype.TypeCastException: Unable to typecast value <1997/02/14> of type <java.lang.String> to TIMESTAMP
at org.dbunit.dataset.datatype.TimestampDataType.typeCast(TimestampDataType.java:120)
at org.dbunit.dataset.datatype.TimestampDataType.setSqlValue(TimestampDataType.java:176)
at org.dbunit.database.statement.SimplePreparedStatement.addValue(SimplePreparedStatement.java:73)
at org.dbunit.operation.RefreshOperation$RowOperation.execute(RefreshOperation.java:189)
at org.dbunit.operation.RefreshOperation.execute(RefreshOperation.java:113)
at org.dbunit.AbstractDatabaseTester.executeOperation(AbstractDatabaseTester.java:190)
at org.dbunit.AbstractDatabaseTester.onSetup(AbstractDatabaseTester.java:103)
at org.dbunit.DatabaseTestCase.setUp(DatabaseTestCase.java:156)
at test.HibernateDbUnitTestCase.setUp(HibernateDbUnitTestCase.java:85)
at test.PlayerTest.setUp(PlayerTest.java:117)
Caused by: java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]
at java.sql.Timestamp.valueOf(Unknown Source)
at org.dbunit.dataset.datatype.TimestampDataType.typeCast(TimestampDataType.java:116)
... 20 more
That was weird, it seemed like my CustomTimeStamp was not called, so I changed the date in the dataset using the default format : 1997-02-14 00:00:00.0, and ran the unit test again. Then I got:
org.dbunit.dataset.datatype.TypeCastException: Unable to typecast value <1997-02-14 00:00:00.0> of type <java.lang.String> to TIMESTAMP
at test.CustomTimestampDataType.typeCast(CustomTimestampDataType.java:69)
at test.CustomTimestampDataType.setSqlValue(CustomTimestampDataType.java:84)
at org.dbunit.database.statement.SimplePreparedStatement.addValue(SimplePreparedStatement.java:73)
at org.dbunit.operation.RefreshOperation$RowOperation.execute(RefreshOperation.java:189)
at org.dbunit.operation.RefreshOperation.execute(RefreshOperation.java:113)
at org.dbunit.AbstractDatabaseTester.executeOperation(AbstractDatabaseTester.java:190)
at org.dbunit.AbstractDatabaseTester.onSetup(AbstractDatabaseTester.java:103)
at org.dbunit.DatabaseTestCase.setUp(DatabaseTestCase.java:156)
at test.HibernateDbUnitTestCase.setUp(HibernateDbUnitTestCase.java:85)
at test.PlayerTest.setUp(PlayerTest.java:117)
That means CustomTimeStamp was actually called. Seems like, the problem stemed from DatabaseTestCase.setUp which somehow called the wrong TimeStampDataType.
How could I fix this issue?
My first option was to replace every yyyy/MM/dd into yyyy-mm-dd in the dataset using regular expressions. This worked fine, until I had to test a method that selected a date based on a request (so the format is yyyy-mm-dd) and compared it to the current date. ( so the format is yyyy / mm / dd). Hsqldb can't compare two dates with different format.
My second option was to decompile dbunit.jar, rewrite TimeStampDataType based on the tutorial. I'm unfamiliar with bytecode writing so before entering uncharted waters, I wanted to know if you had another solution.
Thank you in advance
Fixed it!
So I ended up using my second option.
This is the detailed path for those who need it.
Download dbUnit.2.2.source.jar
Unzip the jar
Go to Eclipse, File > New > Java Project
Uncheck "Use default location"
In Location : specify the path to the new folder created from the jar
Click on Finish
Modify the TimestampDataType.java (if needed)
Instead of ts = java.sql.Timestamp.valueOf(stringValue); use the code below
String formats[] =
{"dd/MM/yyyy HH:mm:ss.SS"}; //and more depending on your need
Timestamp ts = null;
for (int i = 0; i < formats.length; i++)
{
SimpleDateFormat sdf = new SimpleDateFormat(formats[i]);
try {
java.util.Date date = sdf.parse(stringValue);
ts = new Timestamp(date.getTime());
return ts;
}
catch( ParseException e) {
}
Modify the DateDataType.java (if needed)
Instead of return java.sql.Date.valueOf(stringValue); , use the code below
String formats[] =
{"dd/MM/yyyy"}; //and more depending on your need
for (int i = 0; i < formats.length; i++)
{
SimpleDateFormat sdf = new SimpleDateFormat(formats[i]);
try {
java.util.Date date = sdf.parse(stringValue);
java.sql.Date datesql = new java.sql.Date(date.getTime());
return datesql;
}
catch( ParseException e) {
}
}
Right-click on your project, then Export
Select JAR file, then Next
Fill the export destination then Finish.
You just have to add this new jar to the library to make it work.
I'm automating processing of flat files received from a mainframe and am confused on creating and modified times. The files are created on the mainframe then emailed to the required individuals. The individuals then save the file from email (Outlook) and do whatever it is they do with them.
For that automation, the file will be processed automatically. I need to let the user know when the last file was loaded and give them the option to load a newer file.
My issue is with the times: I'm getting a creation date/time that is later than the lastmodified time. Using this code:
try {
DateFormat utcFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'");
DateFormat cstFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
utcFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
cstFormat.setTimeZone(TimeZone.getTimeZone("America/Chicago"));
Path filePath = Paths.get("C:/data/KWJFLTD.XLS");
BasicFileAttributes basicAttr = Files.readAttributes(filePath, BasicFileAttributes.class);
FileTime creationTime = basicAttr.creationTime();
FileTime modifiedTime = basicAttr.lastModifiedTime();
String cTime = creationTime.toString();
String mTime = modifiedTime.toString();
Date dc = utcFormat.parse(cTime);
Date dm = utcFormat.parse(mTime);
cTime = cstFormat.format(dc);
mTime = cstFormat.format(dm);
System.out.println("Creation Time: " + cTime);
System.out.println("Modified Time: " + mTime);
} catch (IOException ex) {
Logger.getLogger(FAULoad.class.getName()).log(Level.SEVERE, null, ex);
} catch (ParseException ex){
Logger.getLogger(FAULoad.class.getName()).log(Level.SEVERE, null, ex);
}
I get this result:
Creation Time: 2015-06-24 15:25:12
Modified Time: 2015-06-24 02:28:05
I end up with a creation time after the last modified time. Is this because the creation time is changed when it's saved from email? In this situation, given that there's (theoretically) no modification going on since it was generated, would the last modified time be the same as the creation date?
According to Microsoft Documentation (the link relates to Win XP but most like also applies up to Win 10) the modification time is the time the file's content was last written while the creation time is the time a specific file has been created.
Thus if you copy a file the modification date would not change since no content has been written/changed but a new file (the copy) has been created and thus the creation time would be after the modification time.