Given the below FileRepository class, how can I optimise the file search as I am dealing with 500 or more client directories. Perhaps I can re-write the below using Streams?
I need to look at all customer directories at first level, and then foreach customer directories look a level below and only take into consideration yesterday's folder which is something like COB02Oct2010. I have written a DateHelper to return me this this previous working day date to then only consider the sub-directories that is relevant...then I look at the matching file pattern that resides in that directory to get the file to send.
Can I simply this using Paths and DirectoryStream?
public class FileRepository {
public List<File> getFilesToSend(String sourcePath, String pattern, String format) {
List<File> files = new ArrayList<>();
File[] customerDir = getCustomerDirs(sourcePath);
for (int i = 0; i < clientDirs.length; i++) {
files.addAll(processClientDirectory(clientDirs[i], pattern, format));
}
return files;
}
private List<File> processClientDirectory(File clientDir, String pattern, String format) {
List<File> result = new ArrayList<>();
pattern = pattern.toLowerCase(Locale.ENGLISH);
format = Constants.EXTENSION_SEPARATOR + format.toLowerCase(Locale.ENGLISH); //add a "."
File cobDir = new File(clientDir, "COB" + DateHelper.getPreviousWorkingDay());
getFilesToProcess(result, cobDir, pattern, format);
return result;
}
private void getFilesToProcess(List<File> result, File cobDir, String pattern, String format) {
if (!cobDir.exists()) {
return;
}
File[] files = cobDir.listFiles(pathName -> {
if (pathName.isDirectory()) {
return true;
}
if (!pathName.isFile()) {
return false;
}
String name = pathName.getName().toLowerCase(Locale.ENGLISH);
if (!name.startsWith(pattern)) {
return false;
}
if (!name.endsWith(format)) {
return false;
}
return true;
});
for (int i = 0; i < files.length; i++) {
File current = files[i];
if (current.isDirectory()) {
getFilesToProcess(result, current, pattern, format);
continue;
}
result.add(current);
}
}
public File[] getCustomerDirs(String sourcePath) {
File[] directories = new File(sourcePath).listFiles(File::isDirectory);
return directories;
}
}
I have not sure how I can write a filter maybe for example like this:
try (DirectoryStream<Path> stream = Files.newDirectoryStream(directoryPath, filter)) {
for (Path path : stream) {
if (Files.isRegularFile(path)) {
consumer.accept(path);
}
}
}
Here is a sample solution by my library: AbacusUtil
public List<File> getFilesToSend(String sourcePath, String pattern, String format) {
final Predicate<File> isTargetFile = f -> f.isFile()
&& N.startsWithIgnoreCase(f.getName(), pattern)
&& N.endsWithIgnoreCase(f.getName(), "." + format);
return Stream.list(new File(sourcePath))
.filter(File::isDirectory)
.map(f -> new File(f, "COB" + getPreviousWorkingDay()))
.flatMap(cobDir -> Stream.list(cobDir, true).filter(isTargetFile))
.toList();
}
Related
I am trying to read package and class name from file. In the below code fileEntry.getName() is giving me output C:User\mywork\Myproject\target\generated-source\java\demo\Project.java. I want to get only demo.Project as an output. Appreciate suggestion thank you
public void listFilesForFolder(final File folder) {
for (final File fileEntry : folder.listFiles()) {
if (fileEntry.isDirectory()) {
listFilesForFolder(fileEntry);
} else {
System.out.println(fileEntry.getName());
//C:User\mywork\Myproject\target\generated-source\java\demo\Project.java
}
}
}
I suppose you only need to find java class files and you want inner packages declared as package1.package2.demo.Project. You can do some String manipulation to get that output.
public static String FILE_SEPARATOR = System.getProperty("file.separator");
public static void listFilesForFolder(final File folder) {
for (final File fileEntry : folder.listFiles()) {
if (fileEntry.isDirectory()) {
listFilesForFolder(fileEntry);
} else {
if (!fileEntry.getName().endsWith(".java")) {
continue;
}
String absolutePath = fileEntry.getAbsolutePath();
String javaPackageSeparator = FILE_SEPARATOR + "java" + FILE_SEPARATOR;
int indexOf = absolutePath.indexOf(javaPackageSeparator);
if (indexOf > -1) {
String javaFilePath = absolutePath.substring(indexOf, absolutePath.length());
String strippedJavaSeparator = javaFilePath.replace(javaPackageSeparator, "");
String[] constructProperPackageName = strippedJavaSeparator.split(Pattern.quote(FILE_SEPARATOR));
String packageName = "";
String className = "";
for (int i = 0; i < constructProperPackageName.length; i++) {
if (i == constructProperPackageName.length - 1) {
className = constructProperPackageName[i].replace(".java", "");
continue;
}
packageName += constructProperPackageName[i] + ".";
}
System.out.println(packageName + className);
}
}
}
}
Produces output as following:
com.company.exception.handler.AbstractExceptionHandler
com.company.exception.handler.IExceptionHandler
com.company.exception.handler.APIRequestExceptionHandler
...
A simple scan across the root / source directory using Files.find will return all java files, and then you can adjust the path to generate package name.
Path srcDir = Path.of("src");
BiPredicate<Path, BasicFileAttributes> dotjava = (p,a) -> a.isRegularFile() && p.getFileName().toString().endsWith(".java");
try(var java = Files.find(srcDir, Integer.MAX_VALUE, dotjava)) {
java.map(p -> srcDir.relativize(p).toString().replaceAll("\\.java$", "").replace(File.separator, "."))
.forEach(System.out::println);
}
Date dir = new java.util.Date(System.currentTimeMillis());
String baseDir = "/home/gaurav/usr/logs/ESBegin/";
String newDir = createDateBasedDirectory(baseDir, dir);
Logger logger = Logger.getLogger("MyLog1");
FileHandler fh;
try {
// This block configure the logger with handler and formatter
fh = new FileHandler(newDir+"/data.log");
logger.addHandler(fh);
SimpleFormatter formatter = new SimpleFormatter();
fh.setFormatter(formatter);
// the following statement is used to log any messages
logger.info(stringifiedJson);
} catch (SecurityException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
this will create a folder of todays date but i want to create new folder for everyday and store the log file in new folder....means everyday's folder must have that day's log file
i have following method to create a folder
public static String createDateBasedDirectory(String baseDirectory, Date argDate) {
String newDir = null;
if (baseDirectory != null && argDate != null) {
try {
String format = "yyyy-MM-dd";
DateFormat dateFormatter = new SimpleDateFormat(format);
String date = dateFormatter.format(argDate);
File f = new File(baseDirectory);
File files[] = f.listFiles();
String dir = null;
int baseDirLength = baseDirectory.length();
int checkingLength = baseDirLength + format.length() + 3;
int maxSeqNo = 0;
for (int i = 0; i < files.length; i++) {
if (files[i].isDirectory()) {
dir = files[i].toString();
if (dir.length() == checkingLength) {
String existingDirDate = dir.substring(baseDirLength, baseDirLength + 10);
if (date.equalsIgnoreCase(existingDirDate)) {
int dirSeqNo =
Integer.parseInt(dir.substring(baseDirLength + 11, baseDirLength + 10 + 3));
if (dirSeqNo > maxSeqNo) {
maxSeqNo = dirSeqNo;
}
}
}
}
}
String currSeq = "" + (maxSeqNo + 1);
if (currSeq.length() == 1) {
currSeq = "0" + currSeq;
}
newDir = baseDirectory + date;
new File(newDir).mkdir();
} catch (Exception e) {
e.printStackTrace();
}
}
return newDir;
}
what should i change if in want to create a new folder everyday
You can use the functionality of your logging framework to do so. For example use Log4J's RollingFileAppender.
You can use the fileName parameter to create new directories. From the documentation:
fileName: The name of the file to write to. If the file, or any of its parent directories, do not exist, they will be created.
Check if the directory for the current day exists, then create it if it doesn't exist or simply return it if it does exist.
/*
* to make sure everyone knows what's going on
* name this method like getOrCreateDailyLogDirectory
*/
public static String createDateBasedDirectory(String baseDirectory, Date argDate) {
String newDir = null;
if (baseDirectory != null && argDate != null) {
try {
String format = "yyyy-MM-dd";
DateFormat dateFormatter = new SimpleDateFormat(format);
String date = dateFormatter.format(argDate);
// check if the directory exists:
String todaysLogDir = baseDirectory + "\\" + date; // create the path as String
// then create a Path (java.nio, alternatives possible)
Path todaysDirectoryPath = Paths.get(todaysLogDir);
// and check if this Path exists
if (Files.exists(todaysDirectoryPath) {
// if present, just return it in order to write (into) a log file there
return todaysDirectoryPath.toUri().toString();
} else {
// create it the way you want and return the path as String
...
}
...
} catch (Exception e) {
e.printStackTrace();
}
}
return newDir;
}
Doing it this (or a similar) way always returns the directory for the current day and creates it before once a day on first log attempt.
I have return code to retrieve a list of all filepaths within a directory but I'm only getting the contents of the last folder. I have two folders, each has 3 files.
Here is my code:
import java.io.*;
import java.util.*;
class Filedirexts
{
public static void main(String[] args) throws IOException
{
String Dirpath = "E:/Share/tstlpatches";
String fieldirpath ="";
File file = new File(Dirpath);
List<String> strfilelst = new ArrayList<String>();
strfilelst = Filedirexts.getsubdir(file);
System.out.println(strfilelst.size());
for(int i=0;i<strfilelst.size();i++)
{
fieldirpath = strfilelst.get(i);
System.out.println("fieldirpath : "+fieldirpath);
}
}
public static List<String> getsubdir(File file) throws IOException
{
File[] filelist = file.listFiles();
List<String> strfileList = new ArrayList<String>();
System.out.println("filelist" + filelist.length);
for (int i=0; i< filelist.length ; i++)
{
if(filelist[i].exists())
{
if(filelist[i].isFile())
{
file = filelist[i];
System.out.println( " fileeach file : "+fileeach.getAbsolutePath());
strfileList.add(file.getAbsolutePath());
}
else if (filelist[i].isDirectory())
{
file = filelist[i];
System.out.println( " fileeach Directory : "+fileeach.getCanonicalPath());
strfileList = Filedirexts.getsubdir(file);
strfileList.add(file.getCanonicalPath().toString());
}
}
}
return strfileList;
}
}
This is my folder structure:
MainPath E:\Share\tstlpatches which is used in code itself
E:\Share\tstlpatches\BE
E:\Share\tstlpatches\BE\graphical
E:\Share\tstlpatches\BE\graphical\data1.txt
E:\Share\tstlpatches\BE\graphical\data2.txt
E:\Share\tstlpatches\BE\graphical\data3.txt
E:\Share\tstlpatches\BE\test
E:\Share\tstlpatches\BE\test\1.txt
E:\Share\tstlpatches\BE\test\2.txt
E:\Share\tstlpatches\BE\test\readme.txt
I'm only getting
E:\Share\tstlpatches\BE
E:\Share\tstlpatches\BE\test
E:\Share\tstlpatches\BE\test\1.txt
E:\Share\tstlpatches\BE\test\2.txt
E:\Share\tstlpatches\BE\test\readme.txt
If I use the normal method, it works fine, but when I use with the list I'm only getting the constents of the last folder.
What do I need to do to make the code work properly?
Your recursive method is incorrectly creating a new ArrayList<String> in each call and returning this new ArrayList in each invocation. This is why you are only seeing the contents of the last call.
Here's how to fix this:
1) Change the getsubdir to be void and pass in the List as a parameter.
public static void getsubdir(File file, List<String> strfileList) throws IOException
{
File[] filelist = file.listFiles();
System.out.println("filelist " + filelist.length);
for (int i=0; i< filelist.length ; i++)
{
if(filelist[i].exists())
{
if(filelist[i].isFile())
{
file = filelist[i];
System.out.println( " fileeach file : "+file.getAbsolutePath());
strfileList.add(file.getAbsolutePath());
}
else if (filelist[i].isDirectory())
{
file = filelist[i];
System.out.println( " fileeach Directory : "+file.getCanonicalPath());
// Note: Since you want the directory first in the list,
// add it before the recursive call
strfileList.add(file.getCanonicalPath().toString());
Filedirexts.getsubdir(file, strfileList);
}
}
}
}
2) Change the way you call it from main:
Instead of:
strfilelst = Filedirexts.getsubdir(file);
Just use:
Filedirexts.getsubdir(file, strfilelst);
building an Android app i came across the need to do some file copying. I would like a way to get new filenames in the event of a filename allready being used by adding a "(increment)" string in the filename. for example
text.txt ---> text(1).txt
The algorith should account for the following
1) if text.txt exists the new file name should NEVER be text.txt(1)
2) if text(1).txt exists then new filename should be text(2).txt not text(1)(1).txt
3) if text(1)foo.txt exists the new filename should be text(1)foo(1).txt
I've allready done the first but I'm having difficulties with the second. Regular expressions is not my strong suit!(It's not mandatory to use Regex. every approach is welcome) Some help ?
ANSWER:
combining my original code and one of the answers here I ended up with this which works very well for me in all cases regardless of file having an extension or not:
public static File getFinalNewDestinationFile(File destinationFolder, File fileToCopy){
String destFolderPath = destinationFolder.getAbsolutePath()+File.separator;
File newFile = new File(destFolderPath + fileToCopy.getName());
String filename=fileToCopy.getName();
String nameWithoutExtentionOrIncrement;
String extension = getFileExtension(filename);
if(extension!=null){
extension="."+extension;
int extInd = filename.lastIndexOf(extension);
nameWithoutExtentionOrIncrement = new StringBuilder(filename).replace(extInd, extInd+extension.length(),"").toString();
}
else{
extension="";
nameWithoutExtentionOrIncrement = filename;
}
int c=0;
int indexOfClose = nameWithoutExtentionOrIncrement.lastIndexOf(")");
int indexOfOpen = nameWithoutExtentionOrIncrement.lastIndexOf("(");
if(indexOfClose!=-1 && indexOfClose!=-1 && indexOfClose==nameWithoutExtentionOrIncrement.length()-1 && indexOfClose > indexOfOpen && indexOfOpen!=0){
String possibleNumber = nameWithoutExtentionOrIncrement.substring(indexOfOpen+1, indexOfClose);
try{
c = Integer.parseInt(possibleNumber);
nameWithoutExtentionOrIncrement=nameWithoutExtentionOrIncrement.substring(0, indexOfOpen);
}catch(Exception e){c=0;}
}
while(newFile.exists()){
c++;
String path = destFolderPath + nameWithoutExtentionOrIncrement +"(" + Integer.toString(c) + ")" + extension;
newFile = new File(path);
}
return newFile;
}
public static String getFileExtension(String filename) {
if (filename == null) { return null; }
int lastUnixPos = filename.lastIndexOf('/');
int lastWindowsPos = filename.lastIndexOf('\\');
int indexOfLastSeparator = Math.max(lastUnixPos, lastWindowsPos);
int extensionPos = filename.lastIndexOf('.');
int lastSeparator = indexOfLastSeparator;
int indexOfExtension = lastSeparator > extensionPos ? -1 : extensionPos;
int index = indexOfExtension;
if (index == -1) {
return null;
} else {
return filename.substring(index + 1).toLowerCase();
}
}
Using one regex pattern:
final static Pattern PATTERN = Pattern.compile("(.*?)(?:\\((\\d+)\\))?(\\.[^.]*)?");
String getNewName(String filename) {
if (fileExists(filename)) {
Matcher m = PATTERN.matcher(filename);
if (m.matches()) {
String prefix = m.group(1);
String last = m.group(2);
String suffix = m.group(3);
if (suffix == null) suffix = "";
int count = last != null ? Integer.parseInt(last) : 0;
do {
count++;
filename = prefix + "(" + count + ")" + suffix;
} while (fileExists(filename));
}
}
return filename;
}
The regex pattern explanation:
(.*?) a non greedy "match everything" starting at the beginning
(?:\\((\\d+)\\))? a number in parenthesis (optional)
(?:____________) - is a non capturing group
___\\(______\\)_ - matches ( and )
______(\\d+)____ - matches and captures the one or more digits
(\\.[^.]+)? a dot followed by anything but a dot (optional)
Here's one way of doing it:
String fileName;
File file = new File(fileName);
if(file.exists()) {
int dot = fileName.lastIndexOf('.'), open = fileName.lastIndexOf('('), incr;
boolean validNum = false;
if(fileName.charAt(dot-1) == ')' && open != -1){
String n = fileName.substring(open+1, dot-1);
try {
incr = Integer.parseInt(n);
validNum = true;
} catch(NumberFormatException e) {
validNum = false;
}
}
if(validNum) {
String pre = fileName.substring(0, open+1), post = fileName.substring(0, dot-1);
while(new File(pre + ++incr + post).exists());
fileName = pre + incr + post;
} else {
fileName = fileName.substring(0, dot) + "(1)" + fileName.substring(dot);
}
}
I assume a couple of things:
1) A method called fileExists(String fileName) is available. It returns true if a file with the specified name is already present in the file system.
2) There is a constant called FILE_NAME which in your example case is equal to "text".
if (!fileExists(FILE_NAME)) {
//create file with FILE_NAME.txt as name
}
int availableIndex = 1;
while (true) {
if (!fileExists(currentName)) {
//create file with FILE_NAME(availableIndex).txt
break;
}
availableIndex++;
}
I am not very sure about Android but since its a Java program, you may be able to create File object of the directory in which you want to write.
Once you have this you can see the list of file names already present inside it and other related information. Then you can decide the file name as per your above logic.
File dir = new File("<dir-path>");
if(dir.isDirectory()){
String[] files = dir.list();
for(String fileName : files){
<logic for finding filename>
}
}
If all filenames have an extenstion you could do something like this (just an example you will have to change it to work in your case):
public static void main(String[] args)
{
String test = "test(1)foo.txt";
String test1 = "test(1)foo(1).txt";
Pattern pattern = Pattern.compile("((?<=\\()\\d+(?=\\)\\.))");
Matcher matcher = pattern.matcher(test);
String fileOutput = "";
String temp = null;
int newInt = -1;
while(matcher.find())
{
temp = matcher.group(0);
}
if(temp != null)
{
newInt = Integer.parseInt(temp);
newInt++;
fileOutput = test.replaceAll("\\(\\d+\\)(?=\\.(?!.*\\.))", "(" + newInt + ")");
}
else
{
fileOutput = test;
}
System.out.println(fileOutput);
matcher = pattern.matcher(test1);
fileOutput = "";
temp = null;
while(matcher.find())
{
temp = matcher.group(0);
}
if(temp != null)
{
newInt = Integer.parseInt(temp);
newInt++;
fileOutput = test1.replaceAll("\\(\\d+\\)(?=\\.(?!.*\\.))", "(" + newInt + ")");
}
else
{
fileOutput = test1;
}
System.out.println(fileOutput);
}
Output:
test(1)foo.txt
test(1)foo(2).txt
This uses regex to look for a number in the () right before the last ..
Update
replaceAll() changed to handle case where there is a . after the first (1) in test(1).foo(1).txt.
I am trying to find the value of f1 which I am able to print, but somehow it is not returning the same. Can anybody tell what's wrong with it.
public String walk(String folderpath) {
String f1 = "";
File root = new File(folderpath);
File[] list = root.listFiles();
if (list == null) return f1;
for (File f : list) {
if (f.isDirectory()) {
walk(f.getAbsolutePath());
f1 = f.getAbsolutePath();
if (f1.contains("foldername")) {
System.out.println(+f1);
}
}
}
return f1;
}
Though it finding the folder name, the loop still goes on in your current code and it keep assigning the latest absolute path to the variable f1 and you'll get the last value in the loop.
To avoid that you can return right away in if condition.
public String walk(String folderpath) {
String f1 = "";
File root = new File(folderpath);
File[] list = root.listFiles();
if (list == null)
return f1;
for (File f : list) {
if (f.isDirectory()) {
walk(f.getAbsolutePath());
f1 = f.getAbsolutePath();
if (f1.contains("foldername")) {
return f1; // return after you found it
}
}
}
return f1;
}
public String walk( String folderpath ) {
String f1 = "";
File root = new File( folderpath );
File[] list = root.listFiles();
if (list == null) return f1;
for ( File f : list ) {
if ( f.isDirectory() ) {
walk( f.getAbsolutePath() );
f1=f.getAbsolutePath();
if(f1.contains("foldername"))
{
System.out.println(+f1);
return f1;
}
}
}
return "noFolderName";
}
Here you directly return the value when you got it.
And if you need to continue your foreach, store the f1 value in another variable instead of returning it in the if and return this new value at the end.