Develop Reference

Create directory in SFTP using Java [duplicate]

I am making an application for file uploading in Java using jSch. I want to put my file in different directories based on their creation date etc.
I have a main directory "/var/local/recordingsbackup/" in which I am creating other directories and putting data in them.
To achieve this:
I have to create Dir'y like
"/var/local/recordingsbackup/20140207/root/SUCCESS/WN/" and put
data in it.
I've tried this so far:
private void fileTransfer(ChannelSftp channelTarget, temp_recording_log recObj, String filePath) {
int fileNameStartIndex = filePath.lastIndexOf("/") + 1;
String date = new SimpleDateFormat("yyyyMMdd").format(recObj.getCalldate());
String fileName = filePath.substring(fileNameStartIndex);
String staticPath = "/var/local/recordingsbackup/";
String completeBackupPath = staticPath + date + "/" + recObj.getUsername() + "/" + recObj.getStatus() + "/" + recObj.getDisposition() + "/";
try {
InputStream get = SourceChannel.get(filePath);
try {
channelTarget.put(get, completeBackupPath + fileName);
} catch (SftpException e) {
System.out.println("Creating Directory...");
channelTarget.mkdir(completeBackupPath); // error on this line
channelTarget.put(get, completeBackupPath + fileName);
}
} catch (SftpException e) {
log.error("Error Occured ======== File or Directory dosen't exists === " + filePath);
e.printStackTrace();
}
}
If I'm creating single dir like /var/local/recordingsbackup/ then no error occurs and files successfully uploaded.
Please help me in this...how can I create these Nested Directories???

Finally, I've done it.
This is how I got succeed :
try {
channelTarget.put(get, completeBackupPath + fileName);
} catch (SftpException e) {
System.out.println("Creating Directory...");
String[] complPath = completeBackupPath.split("/");
channelTarget.cd("/");
for (String dir : complPath) {
if (folder.length() > 0) {
try {
System.out.println("Current Dir : " + channelTarget.pwd());
channelTarget.cd(folder);
} catch (SftpException e2) {
channelTarget.mkdir(folder);
channelTarget.cd(folder);
}
}
}
channelTarget.cd("/");
System.out.println("Current Dir : " + channelTarget.pwd());
channelTarget.put(get, completeBackupPath + fileName);
}

I don't think what you want to do is possible in the SFTP protocol. You will have to create each sub-directory in turn.

public static void mkdirs(ChannelSftp ch, String path) {
try {
String[] folders = path.split("/");
if (folders[0].isEmpty()) folders[0] = "/";
String fullPath = folders[0];
for (int i = 1; i < folders.length; i++) {
Vector ls = ch.ls(fullPath);
boolean isExist = false;
for (Object o : ls) {
if (o instanceof LsEntry) {
LsEntry e = (LsEntry) o;
if (e.getAttrs().isDir() && e.getFilename().equals(folders[i])) {
isExist = true;
}
}
}
if (!isExist && !folders[i].isEmpty()) {
ch.mkdir(fullPath + folders[i]);
}
fullPath = fullPath + folders[i] + "/";
}
} catch (Exception e) {
e.printStackTrace();
}
}
I used this implementation to create nested folders.
I tried not to use Exception. Keep in mind that filesystem is linux based.
The OP already found a solution but I wanted to append to it.
Simply I do mkdir if the folder that I wanted to create doesn't exist in "ls" result.

Correction of the previous script:
public static void mkdirs(ChannelSftp ch, String path) {
try {
String[] folders = path.split("/");
if (folders[0].isEmpty()) folders[0] = "/";
String fullPath = folders[0];
for (int i = 1; i < folders.length; i++) {
Vector ls = ch.ls(fullPath);
boolean isExist = false;
for (Object o : ls) {
if (o instanceof LsEntry) {
LsEntry e = (LsEntry) o;
if (e.getAttrs().isDir() && e.getFilename().equals(folders[i])) {
isExist = true;
}
}
}
if (!isExist && !folders[i].isEmpty()) {
// Add separator path
ch.mkdir(fullPath + "/" + folders[i]);
}
// Add separator path
fullPath = fullPath + "/" + folders[i] + "/";
}
} catch (Exception e) {
e.printStackTrace();
}
}
Another solution is execute shell command:
String remotePath = "fake/folders/recursive/on/sftp/server";
ChannelExec channel = (ChannelExec) session.openChannel("exec");
// NOTE: the provided paths are expected to require no escaping
channel.setCommand("mkdir -p " + remotePath);
channel.connect();
while (!channel.isClosed()) {
// dir creation is usually fast, so only wait for a short time
Thread.sleep(SHORT_WAIT_MSEC);
}
channel.disconnect();
if (channel.getExitStatus() != 0) {
throw new IOException("Creating directory failed: " + remotePath);
}

find pattern matches files recursively

Given the below FileRepository class, how can I optimise the file search as I am dealing with 500 or more client directories. Perhaps I can re-write the below using Streams?
I need to look at all customer directories at first level, and then foreach customer directories look a level below and only take into consideration yesterday's folder which is something like COB02Oct2010. I have written a DateHelper to return me this this previous working day date to then only consider the sub-directories that is relevant...then I look at the matching file pattern that resides in that directory to get the file to send.
Can I simply this using Paths and DirectoryStream?
public class FileRepository {
public List<File> getFilesToSend(String sourcePath, String pattern, String format) {
List<File> files = new ArrayList<>();
File[] customerDir = getCustomerDirs(sourcePath);
for (int i = 0; i < clientDirs.length; i++) {
files.addAll(processClientDirectory(clientDirs[i], pattern, format));
}
return files;
}
private List<File> processClientDirectory(File clientDir, String pattern, String format) {
List<File> result = new ArrayList<>();
pattern = pattern.toLowerCase(Locale.ENGLISH);
format = Constants.EXTENSION_SEPARATOR + format.toLowerCase(Locale.ENGLISH); //add a "."
File cobDir = new File(clientDir, "COB" + DateHelper.getPreviousWorkingDay());
getFilesToProcess(result, cobDir, pattern, format);
return result;
}
private void getFilesToProcess(List<File> result, File cobDir, String pattern, String format) {
if (!cobDir.exists()) {
return;
}
File[] files = cobDir.listFiles(pathName -> {
if (pathName.isDirectory()) {
return true;
}
if (!pathName.isFile()) {
return false;
}
String name = pathName.getName().toLowerCase(Locale.ENGLISH);
if (!name.startsWith(pattern)) {
return false;
}
if (!name.endsWith(format)) {
return false;
}
return true;
});
for (int i = 0; i < files.length; i++) {
File current = files[i];
if (current.isDirectory()) {
getFilesToProcess(result, current, pattern, format);
continue;
}
result.add(current);
}
}
public File[] getCustomerDirs(String sourcePath) {
File[] directories = new File(sourcePath).listFiles(File::isDirectory);
return directories;
}
}
I have not sure how I can write a filter maybe for example like this:
try (DirectoryStream<Path> stream = Files.newDirectoryStream(directoryPath, filter)) {
for (Path path : stream) {
if (Files.isRegularFile(path)) {
consumer.accept(path);
}
}
}

Here is a sample solution by my library: AbacusUtil
public List<File> getFilesToSend(String sourcePath, String pattern, String format) {
final Predicate<File> isTargetFile = f -> f.isFile()
&& N.startsWithIgnoreCase(f.getName(), pattern)
&& N.endsWithIgnoreCase(f.getName(), "." + format);
return Stream.list(new File(sourcePath))
.filter(File::isDirectory)
.map(f -> new File(f, "COB" + getPreviousWorkingDay()))
.flatMap(cobDir -> Stream.list(cobDir, true).filter(isTargetFile))
.toList();
}

How to increment the filename number if the file exists

How can I increment the filename if the file already exists?
Here's the code that I am using -
int num = 0;
String save = at.getText().toString() + ".jpg";
File file = new File(myDir, save);
if (file.exists()) {
save = at.getText().toString() + num + ".jpg";
file = new File(myDir, save);
num++;
}
This code works, but only two files are saved, like file.jpg and file2.jpg.

This problem is to always initialize num = 0, so if file exists, it saves file0.jpg and does not check whether file0.jpg exists.
So, to code work. You should check until it is available:
int num = 0;
String save = at.getText().toString() + ".jpg";
File file = new File(myDir, save);
while(file.exists()) {
save = at.getText().toString() + (num++) + ".jpg";
file = new File(myDir, save);
}

Try this:
File file = new File(myDir, at.getText().toString() + ".jpg");
for (int num = 0; file.exists(); num++) {
file = new File(myDir, at.getText().toString() + num + ".jpg");
}
// Now save/use your file here

In addition to the first answer, I made some more changes:
private File getUniqueFileName(String folderName, String searchedFilename) {
int num = 1;
String extension = getExtension(searchedFilename);
String filename = searchedFilename.substring(0, searchedFilename.lastIndexOf("."));
File file = new File(folderName, searchedFilename);
while (file.exists()) {
searchedFilename = filename + "(" + (num++) + ")" + extension;
file = new File(folderName, searchedFilename);
}
return file;
}

int i = 0;
String save = at.getText().toString();
String filename = save +".jpg";
File f = new File(filename);
while (f.exists()) {
i++;
filename =save+ Integer.toString(i)+".jpg";
f = new File(filename);
}
f.createNewFile();

You can avoid the code repetition of some of the answers here by using a do while loop
Here's an example using the newer NIO Path API introduced in Java 7
Path candidate = null;
int counter = 0;
do {
candidate = Paths.get(String.format("%s-%d",
path.toString(), ++counter));
} while (Files.exists(candidate));
Files.createFile(candidate);

Kotlin version:
private fun checkAndRenameIfExists(name: String): File {
var filename = name
val extension = "pdf"
val root = Environment.getExternalStorageDirectory().absolutePath
var file = File(root, "$filename.$extension")
var n = 0
while (file.exists()) {
n += 1
filename = "$name($n)"
file = File(root, appDirectoryName + File.separator + "$filename.$extension")
}
return file
}

Another simple logic solution to get the unique file name under a directory using Apache Commons IO using WildcardFileFilter to match the file name and get the number of exists with the given name and increment the counter.
public static String getUniqueFileName(String directory, String fileName) {
String fName = fileName.substring(0, fileName.lastIndexOf("."));
Collection<File> listFiles = FileUtils.listFiles(new File(directory), new WildcardFileFilter(fName + "*", IOCase.INSENSITIVE), DirectoryFileFilter.DIRECTORY);
if(listFiles.isEmpty()) {
return fName;
}
return fName.concat(" (" + listFiles.size() + ")");
}

This is the solution I use to handle this case. It works for folders as well as for files.
var destination = File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS), "MyFolder")
if (!destination.exists()) {
destination.mkdirs()
} else {
val numberOfFileAlreadyExist =
destination.listFiles().filter { it.name.startsWith("MyFolder") }.size
destination = File(
Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS),
"MyFolder(${numberOfFileAlreadyExist + 1})"
)
destination.mkdirs()
}

Having needed to solve this problem in my own code, I took Tejas Trivedi's answer, made it work like Windows when you happen to download the same file several times.
// This function will iteratively to find a unique file name to use when given a file: example (###).txt
// More or less how Windows will save a new file when one already exists: 'example.txt' becomes 'example (1).txt'.
// if example.txt already exists
private File getUniqueFileName(File file) {
File originalFile = file;
try {
while (file.exists()) {
String newFileName = file.getName();
String baseName = newFileName.substring(0, newFileName.lastIndexOf("."));
String extension = getExtension(newFileName);
Pattern pattern = Pattern.compile("( \\(\\d+\\))\\."); // Find ' (###).' in the file name, if it exists
Matcher matcher = pattern.matcher(newFileName);
String strDigits = "";
if (matcher.find()) {
baseName = baseName.substring(0, matcher.start(0)); // Remove the (###)
strDigits = matcher.group(0); // Grab the ### we'll want to increment
strDigits = strDigits.substring(strDigits.indexOf("(") + 1, strDigits.lastIndexOf(")")); // Strip off the ' (' and ').' from the match
// Increment the found digit and convert it back to a string
int count = Integer.parseInt(strDigits);
strDigits = Integer.toString(count + 1);
} else {
strDigits = "1"; // If there is no (###) match then start with 1
}
file = new File(file.getParent() + "/" + baseName + " (" + strDigits + ")" + extension); // Put the pieces back together
}
return file;
} catch (Error e) {
return originalFile; // Just overwrite the original file at this point...
}
}
private String getExtension(String name) {
return name.substring(name.lastIndexOf("."));
}
Calling getUniqueFileName(new File('/dir/example.txt') when 'example.txt' already exists while generate a new File targeting '/dir/example (1).txt' if that too exists it'll just keep incrementing number between the parentheses until a unique file is found, if an error happens, it'll just give the original file name.
I hope this helps some one needing to generate a unique file in Java on Android or another platform.

This function returns the exact new file with an increment number for all kind of extensions.
private File getFileName(File file) {
if (file.exists()) {
String newFileName = file.getName();
String simpleName = file.getName().substring(0, newFileName.indexOf("."));
String strDigit = "";
try {
simpleName = (Integer.parseInt(simpleName) + 1 + "");
File newFile = new File(file.getParent() + "/" + simpleName + getExtension(file.getName()));
return getFileName(newFile);
}
catch (Exception e){
}
for (int i=simpleName.length()-1; i>=0; i--) {
if (!Character.isDigit(simpleName.charAt(i))) {
strDigit = simpleName.substring(i + 1);
simpleName = simpleName.substring(0, i+1);
break;
}
}
if (strDigit.length() > 0) {
simpleName = simpleName + (Integer.parseInt(strDigit) + 1);
}
else {
simpleName += "1";
}
File newFile = new File(file.getParent() + "/" + simpleName + getExtension(file.getName()));
return getFileName(newFile);
}
return file;
}
private String getExtension(String name) {
return name.substring(name.lastIndexOf("."));
}

Reading all files in a path and checking on them java

I am reading all files in the paths which mentioned in the code
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;
public class folderReader {
public static void main(String[] args) {
//Extension > .wav files
File folderWav = new File(
"/home/bassem/Desktop/LatestData3/Nemlar_RMC/nemlar_RMC");
File[] listOfWavs = folderWav.listFiles();
//Extension > .insent
File folderInsent = new File(
"/home/bassem/Desktop/LatestData3/outputInsent");
File[] listOfInsents = folderInsent.listFiles();
//Extension > .ctl
File folderCtl = new File("/home/bassem/Desktop/LatestData3/outputCtl");
File[] listOfCtls = folderCtl.listFiles();
for (File file : listOfWavs) {
for (File file2 : listOfInsents) {
for (File file3 : listOfCtls) {
if (file.isFile() && file2.isFile() && file3.isFile()) {
if ((file.getName().substring(0,
file.getName().length() - 4).equals(file3
.getName().substring(0,
file3.getName().length() - 4)))) {
if ((file2.getName().substring(0,
file2.getName().length() - 7).equals(file3
.getName().substring(0,
file3.getName().length() - 4)))) {
System.out.println(file3.getAbsolutePath());
}
}
}
}
}
}
}
}
After Reading all the files I am checking on their names with using substring to remove (.ctl - .insent - .wav) and if they are equaled I should do some stuff I am trying here to print path to check if the code is working and it prints nothing
I tried to use this and it worked well it printed them all.
for (File file : listOfWavs) {
for (File file2 : listOfInsents) {
if (file.isFile() && file2.isFile()) {
if ((file.getName().substring(0,
file.getName().length() - 4).equals(file2.getName()
.substring(0, file2.getName().length() - 7)))) {
System.out.println(file2.getAbsolutePath());
}
}
}
}
the problem only happens when i am checking on the 3 folders together when I run the code it does nothing just an empty console !

String folder = "/home/bassem/Desktop/LatestData3/";
List<String> insents = readFolderBasenames(folder + "outputInsent", "insent");
List<String> wavs = readFolderBasenames(folder + "Nemlar_RMC/nemlar_RMC", "wav");
List<String> ctls = readFolderBasenames(folder + "outputCtl", "ctl");
insents.retainAll(wavs);
insents.retainAll(ctls);
for(String file : insents) {
System.out.println(folder + file + ".insent");
}
private List<String> readFolderBasenames(String path, String extension) {
File[] arr = new File(path).listFiles());
for(int i=0; i<arr.length; i++) {
arr[i] = arr[i].getName().substring(0, arr[i].getName().length() - extension.length() - 1);
}
return Arrays.asList(arr);
}

Java - renaming output file if name already exists with an increment, taking into account existing increments

building an Android app i came across the need to do some file copying. I would like a way to get new filenames in the event of a filename allready being used by adding a "(increment)" string in the filename. for example
text.txt ---> text(1).txt
The algorith should account for the following
1) if text.txt exists the new file name should NEVER be text.txt(1)
2) if text(1).txt exists then new filename should be text(2).txt not text(1)(1).txt
3) if text(1)foo.txt exists the new filename should be text(1)foo(1).txt
I've allready done the first but I'm having difficulties with the second. Regular expressions is not my strong suit!(It's not mandatory to use Regex. every approach is welcome) Some help ?
ANSWER:
combining my original code and one of the answers here I ended up with this which works very well for me in all cases regardless of file having an extension or not:
public static File getFinalNewDestinationFile(File destinationFolder, File fileToCopy){
String destFolderPath = destinationFolder.getAbsolutePath()+File.separator;
File newFile = new File(destFolderPath + fileToCopy.getName());
String filename=fileToCopy.getName();
String nameWithoutExtentionOrIncrement;
String extension = getFileExtension(filename);
if(extension!=null){
extension="."+extension;
int extInd = filename.lastIndexOf(extension);
nameWithoutExtentionOrIncrement = new StringBuilder(filename).replace(extInd, extInd+extension.length(),"").toString();
}
else{
extension="";
nameWithoutExtentionOrIncrement = filename;
}
int c=0;
int indexOfClose = nameWithoutExtentionOrIncrement.lastIndexOf(")");
int indexOfOpen = nameWithoutExtentionOrIncrement.lastIndexOf("(");
if(indexOfClose!=-1 && indexOfClose!=-1 && indexOfClose==nameWithoutExtentionOrIncrement.length()-1 && indexOfClose > indexOfOpen && indexOfOpen!=0){
String possibleNumber = nameWithoutExtentionOrIncrement.substring(indexOfOpen+1, indexOfClose);
try{
c = Integer.parseInt(possibleNumber);
nameWithoutExtentionOrIncrement=nameWithoutExtentionOrIncrement.substring(0, indexOfOpen);
}catch(Exception e){c=0;}
}
while(newFile.exists()){
c++;
String path = destFolderPath + nameWithoutExtentionOrIncrement +"(" + Integer.toString(c) + ")" + extension;
newFile = new File(path);
}
return newFile;
}
public static String getFileExtension(String filename) {
if (filename == null) { return null; }
int lastUnixPos = filename.lastIndexOf('/');
int lastWindowsPos = filename.lastIndexOf('\\');
int indexOfLastSeparator = Math.max(lastUnixPos, lastWindowsPos);
int extensionPos = filename.lastIndexOf('.');
int lastSeparator = indexOfLastSeparator;
int indexOfExtension = lastSeparator > extensionPos ? -1 : extensionPos;
int index = indexOfExtension;
if (index == -1) {
return null;
} else {
return filename.substring(index + 1).toLowerCase();
}
}

Using one regex pattern:
final static Pattern PATTERN = Pattern.compile("(.*?)(?:\\((\\d+)\\))?(\\.[^.]*)?");
String getNewName(String filename) {
if (fileExists(filename)) {
Matcher m = PATTERN.matcher(filename);
if (m.matches()) {
String prefix = m.group(1);
String last = m.group(2);
String suffix = m.group(3);
if (suffix == null) suffix = "";
int count = last != null ? Integer.parseInt(last) : 0;
do {
count++;
filename = prefix + "(" + count + ")" + suffix;
} while (fileExists(filename));
}
}
return filename;
}
The regex pattern explanation:
(.*?) a non greedy "match everything" starting at the beginning
(?:\\((\\d+)\\))? a number in parenthesis (optional)
(?:____________) - is a non capturing group
___\\(______\\)_ - matches ( and )
______(\\d+)____ - matches and captures the one or more digits
(\\.[^.]+)? a dot followed by anything but a dot (optional)

Here's one way of doing it:
String fileName;
File file = new File(fileName);
if(file.exists()) {
int dot = fileName.lastIndexOf('.'), open = fileName.lastIndexOf('('), incr;
boolean validNum = false;
if(fileName.charAt(dot-1) == ')' && open != -1){
String n = fileName.substring(open+1, dot-1);
try {
incr = Integer.parseInt(n);
validNum = true;
} catch(NumberFormatException e) {
validNum = false;
}
}
if(validNum) {
String pre = fileName.substring(0, open+1), post = fileName.substring(0, dot-1);
while(new File(pre + ++incr + post).exists());
fileName = pre + incr + post;
} else {
fileName = fileName.substring(0, dot) + "(1)" + fileName.substring(dot);
}
}

I assume a couple of things:
1) A method called fileExists(String fileName) is available. It returns true if a file with the specified name is already present in the file system.
2) There is a constant called FILE_NAME which in your example case is equal to "text".
if (!fileExists(FILE_NAME)) {
//create file with FILE_NAME.txt as name
}
int availableIndex = 1;
while (true) {
if (!fileExists(currentName)) {
//create file with FILE_NAME(availableIndex).txt
break;
}
availableIndex++;
}

I am not very sure about Android but since its a Java program, you may be able to create File object of the directory in which you want to write.
Once you have this you can see the list of file names already present inside it and other related information. Then you can decide the file name as per your above logic.
File dir = new File("<dir-path>");
if(dir.isDirectory()){
String[] files = dir.list();
for(String fileName : files){
<logic for finding filename>
}
}

If all filenames have an extenstion you could do something like this (just an example you will have to change it to work in your case):
public static void main(String[] args)
{
String test = "test(1)foo.txt";
String test1 = "test(1)foo(1).txt";
Pattern pattern = Pattern.compile("((?<=\\()\\d+(?=\\)\\.))");
Matcher matcher = pattern.matcher(test);
String fileOutput = "";
String temp = null;
int newInt = -1;
while(matcher.find())
{
temp = matcher.group(0);
}
if(temp != null)
{
newInt = Integer.parseInt(temp);
newInt++;
fileOutput = test.replaceAll("\\(\\d+\\)(?=\\.(?!.*\\.))", "(" + newInt + ")");
}
else
{
fileOutput = test;
}
System.out.println(fileOutput);
matcher = pattern.matcher(test1);
fileOutput = "";
temp = null;
while(matcher.find())
{
temp = matcher.group(0);
}
if(temp != null)
{
newInt = Integer.parseInt(temp);
newInt++;
fileOutput = test1.replaceAll("\\(\\d+\\)(?=\\.(?!.*\\.))", "(" + newInt + ")");
}
else
{
fileOutput = test1;
}
System.out.println(fileOutput);
}
Output:
test(1)foo.txt
test(1)foo(2).txt
This uses regex to look for a number in the () right before the last ..
Update
replaceAll() changed to handle case where there is a . after the first (1) in test(1).foo(1).txt.