My FtpClient class is producing an error when I try to upload file on ftp server. I get this message:
220 (vsFTPd 3.0.3)
USER newftpuser
331 Please specify the password.
PASS ftp
230 Login successful.
TYPE I
200 Switching to Binary mode.
PORT 192,168,1,7,235,73
200 PORT command successful. Consider using PASV.
STOR /upload/logo.png
425 Failed to establish connection.
FtpClient.java
import org.apache.commons.net.PrintCommandListener;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
import org.apache.commons.net.ftp.FTPReply;
import java.io.*;
import java.util.Arrays;
import java.util.Collection;
import java.util.stream.Collectors;
public class FtpClient {
private String server;
private int port;
private String user;
private String password;
private FTPClient ftp;
public void open() throws IOException {
ftp = new FTPClient();
ftp.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out)));
ftp.enterLocalPassiveMode();
ftp.connect(server, port);
int reply = ftp.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
ftp.disconnect();
throw new IOException("Exception in connecting to FTP Server");
}
ftp.login(user, password);
ftp.setFileType(FTP.BINARY_FILE_TYPE);
}
public void close() throws IOException {
ftp.disconnect();
}
public void putFileToPath(InputStream inputStream, String path) throws IOException {
ftp.storeFile(path, inputStream);
}
}
Tests
#Test
public void dropFileOnFtpServer() throws IOException, URISyntaxException {
ftpClient = new FtpClient(...);
ftpClient.open();
InputStream inputStream = this.getClass().getResourceAsStream("/images/logo.png");
ftpClient.putFileToPath(inputStream, "/upload/logo.png");
assertTrue(ftpClient.listFiles("/upload").contains("logo.png"));
ftpClient.close();
}
Solved
Reason of problem is passive mode. FTPClient of appache.commons.net need to enable passive mode manually before downloading or uploading files, so programs didn't work correct.
Solution
public void putFileToPath(InputStream inputStream, String path) throws IOException {
ftp.enterLocalPassiveMode();
/* I enable this mode before connecting ftp, so client doesnt work*/
ftp.storeFile(path, inputStream);
}
Please check Android FTP error - 425 Can't open data connection and Error: 425 Can't open data connection and edit your configurations based on these guides.
I'm using this class for ftp handling, you can implement your own exception handling and configuration management.
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPSClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ClassPathResource;
import org.springframework.stereotype.Component;
import java.io.IOException;
import java.io.InputStream;
#Component
#Slf4j
public class FTPFileWriterImpl implements FTPFileWriter {
private final FTPProperties ftpProperties;
protected FTPClient ftpClient;
#Autowired
public FTPFileWriterImpl(#Autowired FTPProperties ftpProperties) {
this.ftpProperties = ftpProperties;
}
//#PostConstruct
public void init() {
if (this.ftpProperties.isAutoStart()) {
log.debug("Autostarting connection to FTP server.");
this.open();
}
}
public boolean open() {
close();
log.debug("Connecting and logging in to FTP server.");
if (ftpProperties.getProtocolType().equals("SSL"))
ftpClient = new FTPSClient();
else
ftpClient = new FTPClient();
ftpClient.enterLocalPassiveMode();
boolean loggedIn = false;
try {
ftpClient.connect(ftpProperties.getServer(), ftpProperties.getPort());
loggedIn = ftpClient.login(ftpProperties.getUsername(), ftpProperties.getPassword());
if (loggedIn) {
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
if (ftpProperties.getKeepAliveTimout() > 0)
ftpClient.setControlKeepAliveTimeout(ftpProperties.getKeepAliveTimout());
} else {
log.error("Failed login to FTP server");
ftpClient.logout();
ftpClient.disconnect();
}
} catch (Exception e) {
log.error(e.getMessage());
}
return loggedIn;
}
public void close() {
if (ftpClient != null) {
try {
if (ftpClient.isConnected()) {
ftpClient.logout();
ftpClient.disconnect();
}
} catch (Exception e) {
log.error(e.getMessage(), e);
}
}
}
public InputStream loadFile(String fileName) {
try {
log.debug("Trying to retrieve a file from remote path " + fileName);
this.open();
return ftpClient.retrieveFileStream(fileName);
} catch (IOException e) {
log.error(e.getMessage(), e);
throw new StorageFileNotFoundException("File not found!");
} finally {
this.close();
}
}
public boolean delete(String fileName) throws IOException {
this.open();
boolean result = ftpClient.deleteFile(fileName);
this.close();
return result;
}
public String saveFile(InputStream inputStream, String dir, String fileName, boolean append) throws Exception {
log.debug("Trying to store a file to destination path " + fileName);
boolean result;
this.open();
try {
String saveDir = this.ftpProperties.getRemoteDir() + "/" + dir;
if (!directoryExists(saveDir)) {
boolean makeDirResult = this.ftpClient.makeDirectory(saveDir);
if(!makeDirResult) {
log.error("Storage directory {} does not exist on ftp server, failed to create it!", saveDir);
throw new StorageException("Storage directory {} does not exist on ftp server, failed to create it!");
}
}
String remote = saveDir + "/" + fileName;
if (append)
result = ftpClient.appendFile(remote, inputStream);
else
result = ftpClient.storeFile(remote, inputStream);
if (!result) {
log.error("Cannot save file on server, Server response is: {}", ftpClient.getReplyCode());
throw new StorageException(String.format("Cannot save file on server: %s", ftpClient.getReplyCode()));
}
return remote;
}
finally {
this.close();
}
}
public String saveFile(String sourcePath, String fileName, boolean append) throws Exception {
InputStream inputStream;
inputStream = new ClassPathResource(sourcePath).getInputStream();
return this.saveFile(inputStream, "", fileName, append);
}
private boolean directoryExists(String dirPath) throws IOException {
ftpClient.changeWorkingDirectory(dirPath);
int returnCode = ftpClient.getReplyCode();
return returnCode != 550;
}
public boolean isConnected() {
boolean connected = false;
if (ftpClient != null) {
try {
connected = ftpClient.sendNoOp();
} catch (Exception e) {
log.error(e.getMessage(), e);
}
}
log.debug("Checking for connection to FTP server. Is connected: " + connected);
return connected;
}
}
FTPFileWriter interface:
import java.io.IOException;
import java.io.InputStream;
public interface FTPFileWriter {
/**
* Connects to a server and tries to log in the user.
*
* #return boolean True if successful, False otherwise.
*/
boolean open();
/**
* Logouts the current user and disconnects from the server.
*/
void close();
/**
* Retrieve a file from the ftp server.
*
* #param remotePath Remote path for the file to retrieve.
* #return boolean True if successful, False otherwise.
*/
InputStream loadFile(String remotePath);
boolean delete(String remotePath) throws IOException;
/**
* Store a file on the ftp server.
*
* #param inputStream Stream the new file is read from.
* #param dir
* #param destPath Remote path the file should be placed at.
* #param append Append to an existing file or write as a new file.
* #return boolean True if successful, False otherwise.
*/
String saveFile(InputStream inputStream, String dir, String destPath, boolean append) throws Exception;
/**
* Store a file on the ftp server.
*
* #param sourcePath Local path the file is read from.
* #param destPath Remote path the file should be placed at.
* #param append Append to an existing file or write as a new file.
* #return boolean True if successful, False otherwise.
*/
String saveFile(String sourcePath, String destPath, boolean append) throws Exception;
/**
* Does a NOOP to see if the connection is valid.
*
* #return boolean True if connected, False otherwise.
*/
boolean isConnected();
}
FTPProperties is a class that provides your ftp settings:
public class FTPProperties {
private String server;
private String username;
private String password;
#Min(0)
#Max(65535)
private int port;
private int keepAliveTimout;
private boolean autoStart;
private String protocolType;
private String remoteDir;
#PostConstruct
public void init() {
if (port == 0) {
port = 21;
}
}
}
You can implement your own exception classes instead of StorageException and StorageFileNotFoundException or throw traditional exceptions
In case you got 553 Could not create file error, it relates to lack of privilege for writing to disk for ftp user and you can resolve it by running following command on Linux-based ftp host:
chown -R ftpusername /var/ftpfiles
Replace /var/ftpfiles with path to ftp storage folder.
Related
I have a task to edit a file inside of a zip on SecureCRT.
I am able to run Linux commands remotely using JSCH library (com.jcraft.jsch)
Here is part of my code:
Session session = setUpSession(testParameters, softAsserter);
Channel channel = session.openChannel("exec");
((ChannelExec)channel).setCommand(command);
channel.setInputStream(null);
((ChannelExec)channel).setErrStream(System.err);
InputStream inputStream = channel.getInputStream();
channel.connect();
I wish to know what is the best way, or the right commands in order to edit a file (for example Test.txt) inside of a zip file on a SecureCRT server.
The contends inside the zip file can be modified in significant number of ways.
I have mentioned some ways which actually might work for you. In order to do that
We should securely transfer the source file/compiled file from local machine to server. The below link would help to transfer the file securely .
https://www.vandyke.com/int/drag_n_drop.html
As a first step , We should develop a snippet which is capable of modifying the contends of the zip file, Then we should copy the file to the server . Then we execute the command to run the file so that the contends inside the zip gets modified.
The below approach has been mentioned only to modify the zip contends.
Approach 1: Using a Simple Java snippet to achieve
We can write a simple java snippet which can open the zip file and edit , Keep the file in the machine and then execute the class file by just running "java filename" which would actually modify contends in the zip file.
Link which would help :
Modifying a text file in a ZIP archive in Java
import java.io.*;
import java.nio.file.*;
class RemoteEditFileContends {
/**
* Edits the text file in zip.
*
* #param zipFilePathInstance
* the zip file path instance
* #throws IOException
* Signals that an I/O exception has occurred.
*/
public static void editTextFileInZip(String zipFilePathInstance) throws IOException {
Path pathInstance = Paths.get(zipFilePathInstance);
try (FileSystem fileSystemIns = FileSystems.newFileSystem(pathInstance, null)) {
Path pathSourceInstance = fileSystemIns.getPath("/abc.txt");
Path tempCopyIns = generateTempFile(fileSystemIns);
Files.move(pathSourceInstance, tempCopyIns);
streamCopy(tempCopyIns, pathSourceInstance);
Files.delete(tempCopyIns);
}
}
/**
* Generate temp file.
*
* #param fileSystemIns
* the file system ins
* #return the path
* #throws IOException
* Signals that an I/O exception has occurred.
*/
public static Path generateTempFile(FileSystem fileSystemIns) throws IOException {
Path tempCopyIns = fileSystemIns.getPath("/___abc___.txt");
if (Files.exists(tempCopyIns)) {
throw new IOException("temp file exists, generate another name");
}
return tempCopyIns;
}
/**
* Stream copy.
*
* #param sourecInstance
* the src
* #param destinationInstance
* the dst
* #throws IOException
* Signals that an I/O exception has occurred.
*/
public static void streamCopy(Path sourecInstance, Path destinationInstance) throws IOException {
try (
BufferedReader bufferInstance = new BufferedReader(new InputStreamReader(Files.newInputStream(sourecInstance)));
BufferedWriter writerInstance = new BufferedWriter(
new OutputStreamWriter(Files.newOutputStream(destinationInstance)))) {
String currentLine = null;
while ((currentLine = bufferInstance.readLine()) != null) {
currentLine = currentLine.replace("key1=value1", "key1=value2");
writerInstance.write(currentLine);
writerInstance.newLine();
}
}
}
public static void main(String[] args) throws IOException {
editTextFileInZip("test.zip");
}
}
Approach 2: Using python to modify the zip files
How to update one file inside zip file using python
Approach 3 : Writing a shell script to modify the contends of zip file directly, So that we can copy the shell script to the server and then execute directly the shell script.
https://superuser.com/questions/647674/is-there-a-way-to-edit-files-inside-of-a-zip-file-without-explicitly-extracting
The below snippet would help you to connect and execute using the library.
import java.io.IOException;
import java.io.InputStream;
import java.util.logging.Level;
import java.util.logging.Logger;
import com.jcraft.jsch.Channel;
import com.jcraft.jsch.ChannelExec;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
public class ConnetionManager {
private static final Logger _logger = Logger.getLogger(ConnetionManager.class.getName());
private JSch jschSSHChannel;
private String strUserName;
private String strConnectionIP;
private int intConnectionPort;
private String strPassword;
private Session sesConnection;
private int intTimeOut;
private void doCommonConstructorActions(String userNameInstance, String tokenpassword, String connetionServerIo,
String hostFileName) {
jschSSHChannel = new JSch();
try {
jschSSHChannel.setKnownHosts(hostFileName);
} catch (JSchException exceptionInstance) {
_logError(exceptionInstance.getMessage());
}
strUserName = userNameInstance;
strPassword = tokenpassword;
strConnectionIP = connetionServerIo;
}
public ConnetionManager(String userName, String password, String connectionIP, String knownHostsFileName) {
doCommonConstructorActions(userName, password, connectionIP, knownHostsFileName);
intConnectionPort = 22;
intTimeOut = 60000;
}
public ConnetionManager(String userName, String password, String connectionIP, String knownHostsFileName,
int connectionPort) {
doCommonConstructorActions(userName, password, connectionIP, knownHostsFileName);
intConnectionPort = connectionPort;
intTimeOut = 60000;
}
public ConnetionManager(String userName, String password, String connectionIP, String knownHostsFileName,
int connectionPort, int timeOutMilliseconds) {
doCommonConstructorActions(userName, password, connectionIP, knownHostsFileName);
intConnectionPort = connectionPort;
intTimeOut = timeOutMilliseconds;
}
public String connect() {
String errorMessage = null;
try {
sesConnection = jschSSHChannel.getSession(strUserName, strConnectionIP, intConnectionPort);
sesConnection.setPassword(strPassword);
sesConnection.connect(intTimeOut);
} catch (JSchException exceptionInstance) {
errorMessage = exceptionInstance.getMessage();
}
return errorMessage;
}
private String _logError(String errorMessage) {
if (errorMessage != null) {
_logger.log(Level.SEVERE, "{0}:{1} - {2}", new Object[] { strConnectionIP, intConnectionPort, errorMessage });
}
return errorMessage;
}
private String _logWarnings(String warnMessage) {
if (warnMessage != null) {
_logger.log(Level.WARNING, "{0}:{1} - {2}", new Object[] { strConnectionIP, intConnectionPort, warnMessage });
}
return warnMessage;
}
public String sendCommand(String executionCommand) {
StringBuilder outputBuffer = new StringBuilder();
try {
Channel channelInstance = sesConnection.openChannel("exec");
((ChannelExec) channelInstance).setCommand(executionCommand);
InputStream commandOutputStream = channelInstance.getInputStream();
channelInstance.connect();
int readByte = commandOutputStream.read();
while (readByte != 0xffffffff) {
outputBuffer.append((char) readByte);
readByte = commandOutputStream.read();
}
channelInstance.disconnect();
} catch (IOException ioExceptionInstance) {
_logWarnings(ioExceptionInstance.getMessage());
return null;
} catch (JSchException schExceptionInstance) {
_logWarnings(schExceptionInstance.getMessage());
return null;
}
return outputBuffer.toString();
}
public void close() {
sesConnection.disconnect();
}
}
I am working on a Java application in which I am trying to create a Multipart file out of downloaded InputStream. Unfortunately, it is not working and the Multipart file is empty. I checked the size of savedFile on disk before copying it to Multipart, and it has correct size, attributes, content.
What am I doing wrong in the conversion, there is no stacktrace, as I am catching it.
Code :
// InputStream contains file data.
byte[] bytes = IOUtils.toByteArray(inputStream);
File file = new File(msg + "temp");
if (file.exists() && file.isDirectory()) {
OutputStream outputStream = new FileOutputStream(new File(msg + "temp" + "/" +
groupAttachments.getFileName()));
outputStream.write(bytes);
outputStream.close();
}
java.io.File savedFile = new java.io.File(msg + "temp" + "/" +
groupAttachments.getFileName());
DiskFileItem fileItem = new DiskFileItem("file", "text/plain", false,
savedFile.getName(), (int) savedFile.length(), savedFile.getParentFile());
fileItem.getOutputStream();
MultipartFile multipartFile = new CommonsMultipartFile(fileItem);
System.out.println("Saved file size is "+savedFile.length());
if (multipartFile.isEmpty()) {
System.out.println("Dropbox uploaded multipart file is empty");
} else {
System.out.println("Multipart file is not empty.");
}
this.dropboxTask.insertFile(multipartFile, "",
savedPersonalNoteObject.getNoteid(), (long) 0, true);
Path path = Paths.get(msg + "temp" + "/" + groupAttachments.getFileName());
Console output :
Multipart file is not empty
Bytes are not null
File path is /My Group
Input stream is not null
Saved file size is 4765
Dropbox uploaded multipart file is empty
Multipart file is empty
Bytes are not null
What am I doing wrong in the conversion? Any help would be nice. Thanks a lot.
The DiskFileItem uses a DeferredFileOutputStream which uses an in-memory byte-array that is only filled when bytes are actually transferred.
Since files are used directly and no bytes are actually copied,
the byte-array is never filled. See for yourself in the source code:
Source code CommonsMultipartFile
Source code DiskFileItem
Source code DeferredFileOutputStream
So, instead of just calling fileItem.getOutputStream();, transfer the bytes to fill the in-memory byte-array:
try (OutputStream out = fileItem.getOutputStream();
InputStream in = Files.newInputStream(file.toPath())) {
IOUtils.copy(in, dfos);
}
and then the tranferTo call will work.
This appears to be a bit cumbersome for just moving a file: CommonsMultipartFile only calls fileItem.write((File)dest) in the transferTo method.
Below are two test cases, one using the DiskFileItem and one using the LocalFileItem. The code for LocalFileItem is shown further below.
I used dependencies org.springframework:spring-web:4.2.2.RELEASE, commons-fileupload:commons-fileupload:1.3.1 and junit:junit:4.12
Test class CommonMp:
import static org.junit.Assert.*;
import java.io.*;
import java.nio.charset.*;
import java.nio.file.*;
import org.apache.commons.fileupload.disk.DiskFileItem;
import org.apache.commons.io.IOUtils;
import org.junit.Test;
import org.springframework.web.multipart.commons.CommonsMultipartFile;
public class CommonMp {
private final Charset CS = StandardCharsets.UTF_8;
#Test
public void testLocalMp() {
Path testInputFile = null, testOutputFile = null;
try {
testInputFile = prepareInputFile();
LocalFileItem lfi = new LocalFileItem(testInputFile);
CommonsMultipartFile cmf = new CommonsMultipartFile(lfi);
System.out.println("Empty: " + cmf.isEmpty());
testOutputFile = testInputFile.getParent().resolve("testMpOutput.txt");
cmf.transferTo(testOutputFile.toFile());
System.out.println("Size: " + cmf.getSize());
printOutput(testOutputFile);
} catch (Exception e) {
e.printStackTrace();
fail();
} finally {
deleteSilent(testInputFile, testOutputFile);
}
}
#Test
public void testMp() {
Path testInputFile = null, testOutputFile = null;
try {
testInputFile = prepareInputFile();
DiskFileItem di = new DiskFileItem("file", "text/plain", false, testInputFile.getFileName().toString(),
(int) Files.size(testInputFile), testInputFile.getParent().toFile());
try (OutputStream out = di.getOutputStream();
InputStream in = Files.newInputStream(testInputFile)) {
IOUtils.copy(in, out);
}
CommonsMultipartFile cmf = new CommonsMultipartFile(di);
System.out.println("Size: " + cmf.getSize());
testOutputFile = testInputFile.getParent().resolve("testMpOutput.txt");
cmf.transferTo(testOutputFile.toFile());
printOutput(testOutputFile);
} catch (Exception e) {
e.printStackTrace();
fail();
} finally {
deleteSilent(testInputFile, testOutputFile);
}
}
private Path prepareInputFile() throws IOException {
Path tmpDir = Paths.get(System.getProperty("java.io.tmpdir"));
Path testInputFile = tmpDir.resolve("testMpinput.txt");
try (OutputStream out = Files.newOutputStream(testInputFile)){
out.write("Just a test.".getBytes(CS));
}
return testInputFile;
}
private void printOutput(Path p) throws IOException {
byte[] outBytes = Files.readAllBytes(p);
System.out.println("Output: " + new String(outBytes, CS));
}
private void deleteSilent(Path... paths) {
for (Path p : paths) {
try { if (p != null) p.toFile().delete(); } catch (Exception ignored) {}
}
}
}
The custom LocalFileItem class, YMMV!
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.UnsupportedEncodingException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.FileItemHeaders;
public class LocalFileItem implements FileItem {
private static final long serialVersionUID = 2467880290855097332L;
private final Path localFile;
public LocalFileItem(Path localFile) {
this.localFile = localFile;
}
#Override
public void write(File file) throws Exception {
Files.move(localFile, file.toPath(), StandardCopyOption.REPLACE_EXISTING);
}
#Override
public long getSize() {
// Spring's CommonsMultipartFile caches the file size and uses it to determine availability.
long size = -1L;
try {
size = Files.size(localFile);
} catch (IOException ignored) {}
return size;
}
#Override
public void delete() {
localFile.toFile().delete();
}
/* *** properties and unsupported methods *** */
private FileItemHeaders headers;
private String contentType;
private String fieldName;
private boolean formField;
#Override
public FileItemHeaders getHeaders() {
return headers;
}
#Override
public void setHeaders(FileItemHeaders headers) {
this.headers = headers;
}
#Override
public InputStream getInputStream() throws IOException {
throw new IOException("Only method write(File) is supported.");
}
public void setContentType(String contentType) {
this.contentType = contentType;
}
#Override
public String getContentType() {
return contentType;
}
#Override
public String getName() {
return localFile.getFileName().toString();
}
#Override
public boolean isInMemory() {
return false;
}
#Override
public byte[] get() {
throw new RuntimeException("Only method write(File) is supported.");
}
#Override
public String getString(String encoding)
throws UnsupportedEncodingException {
throw new RuntimeException("Only method write(File) is supported.");
}
#Override
public String getString() {
throw new RuntimeException("Only method write(File) is supported.");
}
#Override
public String getFieldName() {
return fieldName;
}
#Override
public void setFieldName(String name) {
this.fieldName = name;
}
#Override
public boolean isFormField() {
return formField;
}
#Override
public void setFormField(boolean state) {
this.formField = state;
}
#Override
public OutputStream getOutputStream() throws IOException {
throw new IOException("Only method write(File) is supported.");
}
}
.Net guy working on a java app
I am uploading using the following example as my starting point ( I have this working). This shows using FileContent needing a java.io.File which does not contain the actual file only a pointer to the actual file.
We are uploading from a web site and attempting to insert into the drive, I would prefer to do this using a memory Stream like the .Net example. I cannot see that in looking at the FileContent class. So my questing is: Is there a way to insert a file in Google drive that is in memory and not first on the hard drive?
private static File insertFile(Drive service, String title, String description,
String parentId, String mimeType, String filename) {
// File's metadata.
File body = new File();
body.setTitle(title);
body.setDescription(description);
body.setMimeType(mimeType);
// Set the parent folder.
if (parentId != null && parentId.length() > 0) {
body.setParents(
Arrays.asList(new ParentReference().setId(parentId)));
}
// File's content.
java.io.File fileContent = new java.io.File(filename);
FileContent mediaContent = new FileContent(mimeType, fileContent);
try {
File file = service.files().insert(body, mediaContent).execute();
// Uncomment the following line to print the File ID.
// System.out.println("File ID: " + file.getId());
return file;
} catch (IOException e) {
System.out.println("An error occured: " + e);
return null;
}
}
Override the AbstractInputStream build your own FileContent
package com;
import java.io.IOException;
import java.io.InputStream;
import com.google.api.client.http.AbstractInputStreamContent;
import com.google.api.client.util.Preconditions;
public class FileContent extends AbstractInputStreamContent {
private InputStream inputStream = null;
private long inputLength = 0;
public FileContent(String type, InputStream pInputStream) throws IOException {
super(type);
this.inputStream = Preconditions.checkNotNull(pInputStream);
this.inputLength = this.inputStream.available();
}
public long getLength() throws IOException {
return this.inputLength;
}
public boolean retrySupported() {
return false;
}
#Override
public InputStream getInputStream() throws IOException {
return this.inputStream;
}
}
I am trying to download All files in a directory to my local machine using apache commons like this:
import java.io.FileOutputStream;
import org.apache.commons.net.ftp.FTPClient;
import java.io.IOException;
import java.net.SocketException;
import org.apache.commons.net.ftp.FTPFile;
public class FTPExample {
public static void main(String[] args) throws SocketException, IOException {
FTPClient client = new FTPClient();
client.connect("MyHostName");
client.enterLocalPassiveMode();
client.login("username", "password");
FTPFile[] files = client.listFiles("/App/");
for (FTPFile file : files) {
System.out.println(file.getName());
FileOutputStream fos = new FileOutputStream("Ftp Files/" + file.getName());
client.retrieveFile(file.getName(),fos);
}
}
}
Am able to list the Files in the Directory but I am Getting FilenotFound Exception when trying to Download the files. Please help.
My Error is:
Exception in thread "main" java.io.FileNotFoundException: Ftp Files\01 (The system cannot find the path specified)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
at java.io.FileOutputStream.<init>(FileOutputStream.java:104)
at ftpexample.FTPExample.main(FTPExample.java:30)
Java Result: 1
EDIT: I need the files to be stored in the Folder Ftp File/ in their original file names.
Thank you to those who tried to help. I found the answer to my problem here. this is How I did it:
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.OutputStream;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
import org.apache.commons.net.ftp.FTPReply;
public class FTPExample {
public static void main(String[] args) {
try {
//new ftp client
FTPClient ftp = new FTPClient();
//try to connect
ftp.connect("MyHhostName");
//login to server
if (!ftp.login("username", "password")) {
ftp.logout();
}
int reply = ftp.getReplyCode();
//FTPReply stores a set of constants for FTP reply codes.
if (!FTPReply.isPositiveCompletion(reply)) {
ftp.disconnect();
}
//enter passive mode
ftp.enterLocalPassiveMode();
//get system name
System.out.println("Remote system is " + ftp.getSystemType());
//change current directory
ftp.changeWorkingDirectory("/App/PMIGENV/BACK/Finacle/FC/app/CDCI_LOGS/log/UBSADMIN");
System.out.println("Current directory is " + ftp.printWorkingDirectory());
//get list of filenames
FTPFile[] ftpFiles = ftp.listFiles();
if (ftpFiles != null && ftpFiles.length > 0) {
//loop thru files
for (FTPFile file : ftpFiles) {
if (!file.isFile()) {
continue;
}
System.out.println("File is " + file.getName());
//get output stream
OutputStream output;
output = new FileOutputStream("FtpFiles" + "/" + file.getName());
//get the file from the remote system
ftp.retrieveFile(file.getName(), output);
//close output stream
output.close();
//delete the file
// ftp.deleteFile(file.getName());
}
}
ftp.logout();
ftp.disconnect();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
Here is the Working Code . i Have been trying it for a long time but now it's working fine.
Previously it was downloading files with 0Kb size.
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
public class FTPDownload {
public void ftpDownload() throws IOException
{
String host="your_host_name";
String uname="your_user_name";
String pass="your_password";
String remoteDIR="/public_html/tmp";
String localDIR="D://FTP";
//Object for FTPClient class
FTPClient ftp=new FTPClient();
ftp.connect(host);
boolean login=ftp.login(uname, pass);
ftp.enterLocalPassiveMode();
ftp.changeWorkingDirectory(remoteDIR);
FTPFile[] files=ftp.listFiles();
try{
if(login){
System.out.println("Your Are Logged In "+ftp.getStatus());
System.out.println("Working Directory is "+ftp.printWorkingDirectory());
System.out.println("Local Directory is "+localDIR);
System.out.println("Total Files Are "+files.length);
if(files != null && files.length >0 )
{
for(FTPFile fl:files)
{
if(!fl.isFile())
{
continue;
}
System.out.println(fl.getName());
OutputStream out;
out=new FileOutputStream(localDIR+"/"+fl.getName());
ftp.retrieveFile(fl.getName(), out);
out.close();
}
}
}
else
{
System.out.println("Sorry");
}
ftp.logout();
ftp.disconnect();
}
catch(Exception e){
System.out.println(e);
}
}
public static void main(String[] args) throws IOException {
FTPDownload ft=new FTPDownload();
ft.ftpDownload();
}
}
Is it possible to do file/directory sync in Java using JSch ? I need to sync directory from a remote linux machine to my local windows machine. Is this possible ?
-Tivakar
The easiest way to download files from SCP server is using Commons VFS along with JSch:
import java.io.*;
import org.apache.commons.io.FileUtils;
import org.apache.commons.vfs2.*;
public class CopyRemoteFile {
public static void copyRemoteFiles(String host, String user, String remotePath, String localPath) throws IOException {
FileSystemOptions fsOptions = new FileSystemOptions();
SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(fsOptions, "no");
SftpFileSystemConfigBuilder.getInstance().setIdentities(fsOptions,
new File[] { new File(FileUtils.getUserDirectoryPath() + "/.ssh/id_dsa") });
DefaultFileSystemManager fsManager = (DefaultFileSystemManager) VFS.getManager();
String uri = "sftp://" + user + "#" + host + "/" + remotePath;
FileObject fo = fsManager.resolveFile(uri, fsOptions);
FileObject[] files = fo.getChildren();
for (FileObject file : files) {
// We will be dealing with the files here only
if (file.getType() == FileType.FILE) {
FileUtils.copyInputStreamToFile(file.getContent().getInputStream(),
new File(localPath + "/" + file.getName().getBaseName()));
}
file.close();
}
fo.close();
fsManager.close();
}
}
It's just an example I got in my Wiki, so nothing fancy. But do keep in mind that if you'll close fsManager, you will not be able to open it again in the same VM. I got this issue while testing this solution...
Although the example above does not import any JSch classes, you need to put it in the classpath anyway.
The above example is using private key to authenticate with the remote host. You can easily change that by providing password and modifying the uri to include that.
If you need to sync files, you can compare dates of the files on the local file system (or DB, or any other source of the information) and the remote files:
import java.io.*;
import org.apache.commons.io.*;
import org.apache.commons.vfs2.*;
import org.apache.commons.vfs2.impl.*;
import org.apache.commons.vfs2.provider.sftp.*;
public class CopyRemoteFile {
public static void copyRemoteFiles(final String host, final String user, final String remotePath, final String localPath)
throws IOException {
FileSystemOptions fsOptions = new FileSystemOptions();
SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(fsOptions, "no");
SftpFileSystemConfigBuilder.getInstance().setIdentities(fsOptions,
new File[] { new File(FileUtils.getUserDirectoryPath() + "/.ssh/id_dsa") });
DefaultFileSystemManager fsManager = (DefaultFileSystemManager) VFS.getManager();
String uri = "sftp://" + user + "#" + host + "/" + remotePath;
FileObject fo = fsManager.resolveFile(uri, fsOptions);
FileObject[] files = fo.getChildren();
for (FileObject file : files) {
// We will be dealing with the files here only
File newFile = new File(localPath + "/" + file.getName().getBaseName());
if (file.getType() == FileType.FILE && newFile.lastModified() != file.getContent().getLastModifiedTime()) {
FileUtils.copyInputStreamToFile(file.getContent().getInputStream(), newFile);
newFile.setLastModified(file.getContent().getLastModifiedTime());
}
file.close();
}
fo.close();
fsManager.close();
}
}
Look at: http://the-project.net16.net/Projekte/projekte/Projekte/Programmieren/sftp-synchronisierung.html
There is a whole Programm uploadet.
Here is the sync Part:
import java.io.File;
import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.Vector;
import com.jcraft.jsch.ChannelSftp.LsEntry;
import com.jcraft.jsch.SftpException;
/*
* This is the heart of the whole Program. I hope, the descriptions are precise enought.
*/
public class Sync{
public String ServerPath;
public File LocalFolder;
public sFTPclient client;
public ArrayList<String> serverContentList;
public ArrayList<String> pathList;
public Sync(File local, String to, sFTPclient client){
this.LocalFolder = local;
this.ServerPath = to;
this.client = client;
}
/*
* Executed once. Sets the Server Directory if it exists.
* If the local folder doesn't exist on the Server, it creates it.
*/
public void setServerDirectory() throws SftpException{
try{
client.sftpChannel.cd(ServerPath);
}catch(Exception e){
GUI.addToConsole(ServerPath + " don't exist on your server!");
}
String serverFolder = ServerPath.substring(ServerPath.lastIndexOf('/')+1, ServerPath.length());
if(!LocalFolder.getName().equals(serverFolder)){
try{
client.sftpChannel.mkdir(LocalFolder.getName());
client.sftpChannel.cd(LocalFolder.getName());
} catch (Exception e){
client.sftpChannel.cd(LocalFolder.getName());
}
this.ServerPath = ServerPath + "/" + LocalFolder.getName();
GUI.setNewServerFolder(ServerPath);
}
serverContentList = new ArrayList<String>();
pathList = new ArrayList<String>();
}
//The contentlist contains all Filenames, that should be synchronized
public void setToContentList(String ServerFolder) throws SftpException{
#SuppressWarnings("unchecked")
Vector<LsEntry> fileList = client.sftpChannel.ls(ServerFolder);
int size = fileList.size();
for(int i = 0; i < size; i++){
if(!fileList.get(i).getFilename().startsWith(".")){
serverContentList.add(fileList.get(i).getFilename());
pathList.add(ServerFolder);
}
}
}
/*
* Deletes the synchronized elements from the Lists
*/
public void deleteFromLists(String name){
int position = serverContentList.lastIndexOf(name);
if(position >= 0){
serverContentList.remove(position);
pathList.remove(position);
}
}
/*
* Main function for synchronizing. Works recursive for local folders.
*/
#SuppressWarnings("unchecked")
public void synchronize(File localFolder, String ServerDir) throws SftpException, FileNotFoundException{
if(client.sftpChannel.pwd() != ServerDir){
client.sftpChannel.cd(ServerDir);
}
setToContentList(ServerDir);
File[] localList = localFolder.listFiles();
Vector<LsEntry> ServerList = client.sftpChannel.ls(ServerDir);
ServerList.remove(0); ServerList.remove(0);
/*
* Upload missing Files/Folders
*/
int size = localList.length;
for(int i = 0; i < size; i++){
if(localList[i].isDirectory()){
if(checkFolder(localList[i], ServerDir)){
synchronize(localList[i], ServerDir + "/" + localList[i].getName());
deleteFromLists("SubFolder");
}else {
newFileMaster(true, localList[i], ServerDir);
}
} else {
checkFile(localList[i], ServerDir);
}
deleteFromLists(localList[i].getName());
}
}
/*
* Deletes all files on the server, which are not in the local Folder. Deletes also all missing folders
*/
public void deleteRest() throws SftpException, FileNotFoundException{
int size = serverContentList.size();
for(int i = 0; i < size; i++){
client.sftpChannel.cd(pathList.get(i));
newFileMaster(false, null, serverContentList.get(i));
}
}
/*
* Copy or delete Files/Folders
*/
public void newFileMaster(boolean copyOrNot, File sourcePath, String destPath) throws FileNotFoundException, SftpException{
FileMaster copy = new FileMaster(copyOrNot, sourcePath, destPath, client.sftpChannel);
copy.runMaster();
}
/*
*Useful to find errors - Prints out the content-List every time you call the method.
*If you have Problems, call it before and after every changes of the serverContentList!
*/
/*public void printServerContent(){
System.out.println("SERVER-Content: " + "\n");
for(int i = 0; i < serverContentList.size(); i++){
System.out.println(serverContentList.get(i) + " in " + pathList.get(i));
}
}*/
/*
* Looks ond the server, if the file is there. If not, or the local file has changed, it copies the file on the server.
*/
public void checkFile(File file, String path) throws SftpException, FileNotFoundException{
client.sftpChannel.cd(path);
if(!serverContentList.contains(file.getName())){
newFileMaster(true, file, ServerPath);
} else {
Long localTimeStamp = file.lastModified();
Long timeStamp = client.sftpChannel.stat(file.getName()).getATime()*1000L;
if(localTimeStamp > timeStamp){
newFileMaster(false, null, path + "/" + file.getName());
newFileMaster(true, file, path);
}
}
deleteFromLists(file.getName());
}
/*
* The same as the checkFile function. But it returns a boolean. (Easier to handle in the synchronized funtion)
* Don't check, if the folder has changed (I think this can't be the case)
*/
public boolean checkFolder(File folder, String path) throws SftpException{
client.sftpChannel.cd(path);
if(serverContentList.contains(folder.getName())){
return true;
}else { return false; }
}
}