I am trying to create a simple java application that downloads a file from URL, stores the file and then uploads the file to FTP to store it.
I have a (local) working code that downloads the file to my local machine on C:\ and uses the local file to upload to FTP.
I would like to move this application to OpenShift and run it in Tomcat6 from there. This means that I have to change the C:\ drive reference to a directory in OpenShift. I referenced the "tmp" directory.
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.net.URL;
import java.text.SimpleDateFormat;
import java.util.Date;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.log4j.Logger;
public class AppRun {
public static void main(String[] args) {
Logger log = Logger.getLogger(AppRun.class);
Date date = new Date();
SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMdd");
String formattedDate = sdf.format(date);
downloadFile(formattedDate);
uploadFile(formattedDate);
}
// FILE DOWNLOAD
public static void downloadFile(String formattedDate){
Logger log = Logger.getLogger(AppRun.class);
String sourceURL = "http://websiteURL/pdf/"+formattedDate+".pdf";
String destinationLocation = "/tmp/"+formattedDate+".pdf";
try {
org.apache.commons.io.FileUtils.copyURLToFile(
new URL(sourceURL),
new File(destinationLocation)
);
log.warn("/tmp/"+formattedDate+".pdf downloaded successfully!" );
}
catch (Exception e) {
log.error(e);
log.error("Source URL : " + "http://websiteURL/pdf/"+formattedDate+".pdf" + " Destination URL : " + "/tmp/"+formattedDate+".pdf");
System.out.println("No file found!");
}
}
// FILE UPLOAD
public static void uploadFile(String formattedDate){
Logger log = Logger.getLogger(AppRun.class);
FTPClient client = new FTPClient();
FileInputStream fis = null;
try {
client.connect("ftp.domain.com");
client.login("user", "pass");
// Create an InputStream of the file to be uploaded
String originalFile = "/tmp/"+formattedDate+".pdf";
fis = new FileInputStream(originalFile);
//
// Store file to server
//
String destinationFileName = formattedDate +".pdf";
client.storeFile(destinationFileName, fis);
log.warn("File " + formattedDate +".pdf uploaded successfully!");
client.logout();
} catch (IOException e) {
log.error(e);
e.printStackTrace();
} finally {
try {
if (fis != null) {
fis.close();
}
client.disconnect();
} catch (IOException e) {
log.error(e);
e.printStackTrace();
}
}
}
}
When I run the code, I can download the file but the upload part fails:
/var/lib/openshift/XXXXXXXXXXXXXXXXXXXX/app-root/runtime/repo//.openshift/cron/minutely/java:
2015-09-15 22:27:08 WARN AppRun:41 - /tmp/20150915.pdf downloaded
successfully! 2015-09-15 22:27:09 ERROR AppRun:77 -
java.net.BindException: Permission denied java.net.BindException:
Permission denied
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.(ServerSocket.java:237)
at javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231)
at org.apache.commons.net.ftp.FTPClient.openDataConnection(FTPClient.java:797)
at org.apache.commons.net.ftp.FTPClient._storeFile(FTPClient.java:633)
at org.apache.commons.net.ftp.FTPClient.__storeFile(FTPClient.java:624)
at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:1976)
at AppRun.uploadFile(AppRun.java:72)
at AppRun.main(AppRun.java:25)
Any suggestions how to fix this would be greatly appreciated! Thanks.
Configure the client for FTP PASSIVE mode.
Try adding ftp.enterLocalPassiveMode() after login.
Related
Basically I need to download list of matching files for the search from a FTP server. I have the code to download a specific file from a FTP server. But I need to download all the matching files with my wildcard search. How is that possible in Java?
Here is code for file downloading of a specific filename, from a FTP server -
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
public class FTPDownloadFileDemowithoutmodandfilefilter {
public static void main(String[] args) {
String server = "test.rebex.net";
int port = 21;
String user = "demo";
String pass = "password";
FTPClient ftpClient = new FTPClient();
try {
ftpClient.connect(server, port);
ftpClient.login(user, pass);
ftpClient.enterLocalPassiveMode();
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
File localFile = new File("C:\\project\\readme1.txt");
FTPFile remoteFile = ftpClient.mdtmFile("/readme.txt");
if (remoteFile != null)
{
OutputStream outputStream =
new BufferedOutputStream(new FileOutputStream(localFile));
if (ftpClient.retrieveFile(remoteFile.getName(), outputStream))
{
System.out.println("File downloaded successfully.");
}
outputStream.close();
localFile.setLastModified(remoteFile.getTimestamp().getTimeInMillis());
}
} catch (IOException ex) {
System.out.println("Error: " + ex.getMessage());
ex.printStackTrace();
} finally {
try {
if (ftpClient.isConnected()) {
ftpClient.logout();
ftpClient.disconnect();
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
Use FTPClient.mlistDir (recommended, if the server supports it) or
FTPClient.listFiles to retrieve list of files. And then filter them according to your needs.
The following example downloads all files matching a regular expression .*\.jpg:
FTPFile[] remoteFiles = ftpClient.listFiles(remotePath);
Pattern pattern = Pattern.compile(".*\\.jpg");
Stream<FTPFile> matchingFiles =
Arrays.stream(remoteFiles).filter(
(FTPFile remoteFile) -> pattern.matcher(remoteFile.getName()).matches());
for (Iterator<FTPFile> iter = matchingFiles.iterator(); iter.hasNext(); ) {
FTPFile remoteFile = iter.next();
System.out.println("Found file " + remoteFile.getName() + ", downloading ...");
File localFile = new File(localPath + "\\" + remoteFile.getName());
OutputStream outputStream =
new BufferedOutputStream(new FileOutputStream(localFile));
if (ftpClient.retrieveFile(remotePath + "/" + remoteFile.getName(), outputStream))
{
System.out.println(
"File " + remoteFile.getName() + " downloaded successfully.");
}
outputStream.close();
}
I have a java API for FTP client. I downloaded the jar files that support this FTP API. For some reason, a Usage: java ftp.ftptry Properties_file error was detected.
Here is my code:
package ftp;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.OutputStream;
import java.util.Properties;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
import org.apache.commons.net.ftp.FTPReply;
public class ftptry {
static Properties props;
public static void main(String[] args) {
ftptry getMyFiles = new ftptry();
if (args.length < 1)
{
System.err.println("Usage: java " + getMyFiles.getClass().getName()+
" Properties_file");
System.exit(1);
}
String propertiesFile = args[0].trim();
getMyFiles.startFTP(propertiesFile);
}
public boolean startFTP(String propertiesFile){
props = new Properties();
try {
props.load(new FileInputStream("properties/" + propertiesFile));
String serverAddress = props.getProperty("serverAddress").trim();
String userId = props.getProperty("userId").trim();
String password = props.getProperty("password").trim();
String remoteDirectory = props.getProperty("remoteDirectory").trim();
String localDirectory = props.getProperty("localDirectory").trim();
//new ftp client
FTPClient ftp = new FTPClient();
//try to connect
ftp.connect(serverAddress);
//login to server
if(!ftp.login(userId, password))
{
ftp.logout();
return false;
}
int reply = ftp.getReplyCode();
//FTPReply stores a set of constants for FTP reply codes.
if (!FTPReply.isPositiveCompletion(reply))
{
ftp.disconnect();
return false;
}
//enter passive mode
ftp.enterLocalPassiveMode();
//get system name
System.out.println("Remote system is " + ftp.getSystemType());
//change current directory
ftp.changeWorkingDirectory(remoteDirectory);
System.out.println("Current directory is " + ftp.printWorkingDirectory());
//get list of filenames
FTPFile[] ftpFiles = ftp.listFiles();
if (ftpFiles != null && ftpFiles.length > 0) {
//loop thru files
for (FTPFile file : ftpFiles) {
if (!file.isFile()) {
continue;
}
System.out.println("File is " + file.getName());
//get output stream
OutputStream output;
output = new FileOutputStream(localDirectory + "/" + file.getName());
//get the file from the remote system
ftp.retrieveFile(file.getName(), output);
//close output stream
output.close();
//delete the file
ftp.deleteFile(file.getName());
}
}
ftp.logout();
ftp.disconnect();
}
catch (Exception ex)
{
ex.printStackTrace();
return false;
}
return true;
}
}
and my properties file contains:
#Properties File FTP to server
serverAddress=192.168.xxx.xxx
userId=xxx
password=xxx
remoteDirectory=/LogFiles
localDirectory=/LogFiles
What does this error mean? and how will I run this program without this error?
You need to provide the name of the property file on the command line when invoking your application.
(You need to understand how the check on args.length works)
Hello Every one i am working one project where i need to upload file on my ftp server with my java standalone application
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.SocketException;
import org.apache.commons.io.IOUtils;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
public class Ftpdemo {
public static void main(String args[]) {
// get an ftpClient object
FTPClient ftpClient = new FTPClient();
ftpClient.setConnectTimeout(300);
FileInputStream inputStream = null;
try {
// pass directory path on server to connect
ftpClient.connect("ftp.mydomain.in");
// pass username and password, returned true if authentication is
// successful
boolean login = ftpClient.login("myusername", "mypassword");
if (login) {
System.out.println("Connection established...");
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
ftpClient.enterLocalPassiveMode();
inputStream = new FileInputStream("/home/simmant/Desktop/mypic.png");
boolean uploaded = ftpClient.storeFile("user_screens/test3.png",inputStream);
if (uploaded) {
System.out.println("File uploaded successfully !");
} else {
System.out.println("Error in uploading file !");
}
// logout the user, returned true if logout successfully
boolean logout = ftpClient.logout();
if (logout) {
System.out.println("Connection close...");
}
} else {
System.out.println("Connection fail...");
}
} catch (SocketException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
ftpClient.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
its working fine for the data which is 1 or 2 kb but when i try to upload the file which is of 50 kb and 100 kb then it not working fine. The image uploaded on the server is blank .
As far as I can see, there is nothing wrong with the code. As you're saying 1 or 2 kb of data uploading works fine. So, In my opinion, there is problem with your internet. Might be it's too slow to upload a file size of 50 kb or more.
There is noting wrong with your code only issue with the file or internet speed.Code is working fine here and Also there is recommended that NOT TO USE FTP DETAILS DIRECTLY please avoid this stuff from application you have better option to use web service for your server related stuff.
Good Luck
It seems you had not made connection Instances properly.
Please refer my working code:
public class FTPUploader {
FTPClient ftp = null;
public FTPUploader() throws Exception {
ftp = new FTPClient();
ftp.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(new FileOutputStream("c:\\ftp.log"))));
int reply;
ftp.connect(Constant.FTP_HOST);
reply = ftp.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
ftp.disconnect();
throw new Exception(Constant.FTP_SERVER_EXCEPTION);
}
ftp.login(CommonConstant.FTP_USERNAME, Constant.FTP_PASSWORD);
ftp.setFileType(FTP.TELNET_TEXT_FORMAT);
ftp.enterLocalPassiveMode();
}
public boolean uploadFile(File localFileFullName, String fileName) throws Exception {
InputStream input = new FileInputStream(localFileFullName);
boolean reply = this.ftp.storeFile("'" + fileName +"'", input);
disconnect();
return reply;
}
public void disconnect() {
if (this.ftp.isConnected()) {
try {
this.ftp.logout();
this.ftp.disconnect();
} catch (IOException e) {
System.out.println(this.getClass()+ e);
}
}
}
public static void main(String[] args) throws Exception {
System.out.println("Start");
FTPUploader ftpUploader = new FTPUploader();
System.out.println(ftpUploader.uploadFile(new File("C:\\test.txt"), "Destinate_DIR"));
System.out.println("Done");
}
}
All the best ...!!!
I'm getting this error (550 the filename, directory name, or volume label syntax is incorrect. )
I think the url is correct (obviously not though). Any thoughts?
Here is the url:
STOR /images/report/6F81CB22-3D04-4BA3-AC3F-3D34663449E0**9.png
Here is the invocation method:
private void uploadImageToFtp(String location, String imageName) throws Exception{
File imageFile = new File(location);
System.out.println("Start");
FTPUploader ftpUploader = new FTPUploader("ftp.xxx.com", "user", "password");
ftpUploader.uploadFile(imageFile, imageName, "/images/report/");
imageFile.delete();
ftpUploader.disconnect();
System.out.println("Done");
}
Here is the
ftp class:
package server;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintWriter;
import org.apache.commons.net.PrintCommandListener;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPReply;
public class FTPUploader {
FTPClient ftp = null;
public FTPUploader(String host, String user, String pwd) throws Exception{
ftp = new FTPClient();
ftp.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out)));
int reply;
ftp.connect(host);
reply = ftp.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
ftp.disconnect();
throw new Exception("Exception in connecting to FTP Server");
}
ftp.login(user, pwd);
ftp.setFileType(FTP.BINARY_FILE_TYPE);
ftp.enterLocalPassiveMode();
}
public void uploadFile(File file, String fileName, String hostDir)
throws Exception {
try {
InputStream input = new FileInputStream(file);
this.ftp.storeFile(hostDir + fileName, input);
} catch (Exception e) {
// TODO: handle exception
e.printStackTrace();
}
}
public void disconnect(){
if (this.ftp.isConnected()) {
try {
this.ftp.logout();
this.ftp.disconnect();
} catch (IOException f) {
// do nothing as file is already saved to server
f.printStackTrace();
}
}
}
}
If the FTP server is running Windows, the '*' characters are the problem. Windows file names may not have asterisks.
Try changing working directory first before uploading
ftp.changeWorkingDirectory(DESTPATH);
It worked for me.
I have used this code
import org.apache.commons.net.ftp.FTPClient;
import java.io.FileInputStream;
import java.io.IOException;
public class FTPClientExample {
public static void main(String[] args) {
FTPClient client = new FTPClient();
FileInputStream fis = null;
try {
client.connect("hostname");
client.login("user", "pwd");
String filename = "D:\\Task\\try.txt";
fis = new FileInputStream(filename);
client.storeFile(filename, fis);
client.logout();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (fis != null) {
fis.close();
}
client.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
when i run this,i get the message task complete.But i couldnt find out which folder i should look for the file.some one pls help me?
you are trying to upload on the path D:\\Task\\try.txt .I guess that is your source file path.
You should write something like
client.storeFile(ftpPath + filename, fis);
where ftpPath should be the FTP server location where you want to upload the file.
Edit:: File path structure
ftp://"+username+":"+password+"#"+ip+"/"+dir+"/"+fileName
OK change
String filename = "D:\\Task\\try.txt";
to String filename = "/home/user_name/Desktop";where user_name is your user name for linux .. give it a try the file should be on your Desktop and remember linux is case sensitive.
In case of linux the path String filename = "D:\\Task\\try.txt"; changes to String filename = "/media/your_drive_name/try.txt";
here is linux directory structure explained.