I am using Apache Commons net FTPClient to login and read files from an FTP Server. I manage to login and I can see it logs in successfully, because it shows the working directory path in the header string. However it shows no files when I use listFiles(). (I have also tried using listDirectories() and listNames() but with no success)
Below is a snippet:
try {
client.connect(ftpHost);
} /*catch (SocketException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}*/ catch (IOException ex) {
// TODO Auto-generated catch block
ex.printStackTrace();
}
String header ="";
InputStream stream=null;
BufferedReader reader=null;
try{
client.login(ftpUser, ftpPass);
client.changeWorkingDirectory(targetWorkingDir);
header = client.printWorkingDirectory();
//client.setFileType(FTP.BINARY_FILE_TYPE);
for(int i=0; i<client.listFiles().length;i++){
header+=client.listFiles()[i].getName() +"\n";
}
}
catch (IOException ex){
ex.printStackTrace();
header="ERROR 1: " + ex.getMessage();
for(int i=0;i<ex.getStackTrace().length;i++){
header += "\n" + ex.getStackTrace()[i];
}
}
catch(NullPointerException e){
header = "ERROR 2: "+ e.getMessage()+"\n";
for(int i=0;i<e.getStackTrace().length;i++){
header+= e.getStackTrace()[i] + "\n";
}
}
finally{
if(reader!=null){
try{reader.close();}catch(IOException e){e.printStackTrace();}
try{stream.close();}catch(IOException e){e.printStackTrace();}
}
}
I have also tried using something like this to read a file:
try {
stream = ftpClient.retrieveFileStream("klasa.csv");
reader = new BufferedReader(new InputStreamReader(stream));
header = reader.readLine();
} finally {
if (reader != null) try { reader.close(); } catch (IOException logOrIgnore){}
}
In both cases I'm pretty sure I'm at the right directory, and I make sure that my file is there via FileZilla, but the client can't seem to read any file.
Try to avoid calling listFiles() in the loop. Every call will perform the entire FTP LIST command sequence, so eventually, you will add unnecessary traffic to every call.
You can try to simplify your program first like this:
private static void ftpTest() {
FTPClient f = new FTPClient();
try {
f.connect("{UOUR FTP SERVER}");
f.login("{USER}", "{PASSWORD}");
FTPFile[] files = f.listFiles(".");
for (FTPFile fi: files) {
System.out.printf("f%s\n", fi.getName());
}
} catch (IOException e) {
e.printStackTrace();
}
}
If this program will not give you the list of files in the root directory of your server, you can try to compare your FileZilla FTP options (PASSIVE/ACTIVE mode in particular): https://commons.apache.org/proper/commons-net/apidocs/org/apache/commons/net/ftp/FTPClient.html#enterLocalPassiveMode()
If it won't help you can try to sniff the network traffic using WireShark or tcpdump and compare the commands set to the FTP server.
Related
I want to know how can I use java to confirm a file is a picture file.
I have tried the following code:
public static void main(String[] args) {
// get image format in a file
File file = new File("C:/Users/dell、/Desktop/4.xlsx");
// create an image input stream from the specified fileDD
ImageInputStream iis = null;
try {
iis = ImageIO.createImageInputStream(file);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// get all currently registered readers that recognize the image format
Iterator<ImageReader> iter = ImageIO.getImageReaders(iis);
if (!iter.hasNext()) {
System.out.println("Not a picture file");
throw new RuntimeException("No readers found! Unable to read the uploaded file");
}
// get the first reader
ImageReader reader = iter.next();
try {
System.out.println("Format: " + reader.getFormatName());
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// close stream
if (iis != null){
try {
iis.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally {
try {
iis.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
But it doesn't work perfectly! It shows an exception once the file is not a picture file, so I want to find a better way.
There are too many image extensions. Maybe the best way to validate if a file is an image, is using Regular Expressions. Something like this...
([^\s]+(\.(?i)(jpg|png|gif|bmp|MORE|IMAGE|EXTENSIONS))$)
Here is a complete example of the implementation.
Use ImageIO#read.
public static boolean isPictureFile(File file){
try{
return ImageIO.read(file) != null;
}catch(Exception ex){
return false;
}
}
Basically, the method ImageIO.read(File) will return a BufferedImage object when it successfully read the image file, a null otherwise. All we have to do is to let ImageIO read the file and check if it returns a null or not, and if there it throws an exception for whatever reason, we can safely assume the file is not a picture file.
When i use command prompt on PC and connect device and run abd to execute below command
abd logcat -d > abc.txt
It returns a big file "abc.txt" with all the entries of logcat of all apps. But when I run the same command in an app on android to store it on SD card by making a file it just stores the recent logcat entries and only logs for this app and not even previous logs. Can somebody tell me why is this happening I have also given permission to READ_LOGS
Here is the android code
String fullName = "appLog.txt";
File file = new File (Environment.getExternalStorageDirectory(), fullName);
//clears a file
if(file.exists()){
file.delete();
}
//write log to file
try {
String command = String.format("logcat -d");
Process process = Runtime.getRuntime().exec(command);
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
StringBuilder result = new StringBuilder();
String currentLine = null;
while ((currentLine = reader.readLine()) != null) {
result.append(currentLine);
result.append("\n");
}
FileWriter out = new FileWriter(file);
out.write(result.toString());
out.close();
} catch (IOException e) {
Toast.makeText(getApplicationContext(), e.toString(), Toast.LENGTH_SHORT).show();
}
//clear the log
try {
Runtime.getRuntime().exec("logcat -c");
} catch (IOException e) {
Toast.makeText(getApplicationContext(), e.toString(), Toast.LENGTH_SHORT).show();
}
public void appendLog(String text)
{
File logFile = new File("/sdcard/log.txt");
if (!logFile.exists())
{
try
{
logFile.createNewFile();
}
catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
try
{
BufferedWriter buf = new BufferedWriter(new FileWriter(logFile, true));
buf.append(text);
buf.newLine();
buf.close();
}
catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
And use the method to log from anywhere you need.
appendLog("your message" + anyvalueyouwanttolog );
In Jelly Bean this has changed. App can read only it's own logs and don't need anymore READ_LOGS permission.
As alternative you can use log4j for android. You can use file appender to store all the logs in some file and also LogCatAppender to print logs to logcat, so you can watch logs while developement within eclipse for e.g.
For catching all uncaught exceptions you can create class that implements UncaughtExceptionHandler and then use it with Thread.setDefaultUncaughtExceptionHandler(). This is then useful for printing crash reports (stack traces) to your log and/or to separate file.
I am using NIO File Channel to manage files, and locking them. So far it works, however when I lock a File using NIO File Lock it locks the file so the file Content cannot be changed. As for example if I try to edit a text file on notepad it will show me the following error message:
And that is the expected result, however if I try to delete the file from windows explorer(I haven't tested on other OS likely will be possible as well) it will allow me, and this is undesired, I would like to know if it is possible to Open a File Handle
Code Used:
private static final byte[] MessageBytes;
static {
byte tmp[];
try {
tmp = "Hello World".getBytes("UTF-8");
} catch (UnsupportedEncodingException ex) {
//if fail get the bytes in whatever the java VM charset sets as default
tmp = "Hello World".getBytes();
}
MessageBytes = tmp;
}
private static final String Filename = "Lock_test.txt";
private static void createFileandLock() {
Path FilePath = Paths.get(Filename);
FileChannel OpenFCh;
try {
OpenFCh = FileChannel.open(FilePath, StandardOpenOption.CREATE,
StandardOpenOption.READ, StandardOpenOption.WRITE
// ,StandardOpenOption.APPEND
);
System.out.println("File Channel is Open.");
} catch (IOException err) {
OpenFCh = null;
}
if (OpenFCh != null) {
FileLock Lock = null;
try {
Lock = OpenFCh.lock();
} catch (IOException err) {
System.out.println("Unable To Lock the File.");
}
try {
OpenFCh.write(ByteBuffer.wrap(MessageBytes));
OpenFCh.force(false);
System.out.println("Message Recorded");
} catch (IOException ex) {
System.out.println("Unable To write data into file");
}
try {
// at this point file still locked and open.
//lets wait for input and meanwhile ask to delete the file.
System.out.print("Please Try to delete file at: ");
System.out.println(FilePath.toString());
System.out.println("Press Enter to Continue");
System.in.read();
} catch (IOException ex) {
}
if (Lock != null) {
try {
Lock.close();
} catch (IOException ex) {
}
}
try {
OpenFCh.close();
} catch (IOException ex) {
}
}
}
After further research I notice that using RandomAccessFile Will lock the file avoiding deletion as it creates a File Descriptor that basically open a Handle on the underline Operative system.
So using the RAF does provide the desired result:
Code Used:
private static void createRAFileandLock() {
RandomAccessFile RAf;
try {
RAf = new RandomAccessFile(Filename, "rw");
} catch (FileNotFoundException ex) {
//since is open as RW shold not trigger.
RAf = null;
}
if (RAf != null) {
FileChannel OpenFCh = RAf.getChannel();
FileLock Lock = null;
try {
Lock = OpenFCh.lock();
} catch (IOException err) {
System.out.println("Unable To Lock the File.");
}
try {
OpenFCh.write(ByteBuffer.wrap(MessageBytes));
OpenFCh.force(false);
System.out.println("Message Recorded");
} catch (IOException ex) {
System.out.println("Unable To write data into file");
}
// at this point file still locked and open.
//lets wait for input and meanwhile ask to delete the file.
try {
System.out.print("Please Try to delete file at: ");
System.out.println(Filename);
System.out.println("Press Enter to Continue");
System.in.read();
} catch (IOException ex) {
}
if (Lock != null) {
try {
Lock.close();
} catch (IOException ex) {
}
}
try {
OpenFCh.close();
RAf.close();
} catch (IOException ex) {
}
}
}
However I would like to know if it is possible to archive this using only NIO. As Random Access File is on IO package.
FileLock isn't specified to prevent deletion. It's only specified to interact with other file locks, so you're already deep into platform-dependent behaviour. If RandomAccessFile somehow does what you want you may be stuck with it, but you can't rely on it.
NB of course FileChannel.open() uses a FileDescriptor, handle, etc.
I implemented java code to upload files to server with org.apache.commons.net.ftp.FTPClient
For multiple files the ftp upload speed is very slow.
How can I improve the speed.
-Change library?
What is the powerful FTP client class library for uploading multiple files?
-Use multiple threads?
How can I implement ftp upload function with multiple thread?
Could someone show me an example?
I am a new for multiple threading programming.
After I read all answer, I try to change my code and test it.
Following is a sample FTPClient code:
// create instance of FTPClient
FTPClient ftp = new FTPClient();
ftp.setControlEncoding("UTF-8");
ftp.setDefaultTimeout(30000);
// connect to server
try
{
ftp.connect("10.1.1.1", 990);
}
catch(Exception e)
{
System.out.println("Cannot connect to server");
return;
}
// login to server
if (!ftp.login("username", "password"))
{
ftp.logout();
System.out.println("Cannot login to server");
return;
}
try
{
ftp.setFileTransferMode(FTP.BINARY_FILE_TYPE);
ftp.enterLocalPassiveMode();
// ftp.setBufferSize(0); <-- someone suggest me to set buffer size to 0, but it throw error sometime.
}
catch(Exception e)
{
}
// create directory on server
// dirs is list of required directories on server
for (String dir : dirs)
{
try
{
ftp.makeDirectory(dir);
}
catch(IOException e)
{
}
}
// files is a map of local file and string of remote file
// such as
// file on client is "C://test/a.txt"
// location on server is "/test/a.txt"
for (Map.Entry<File, String> entry : files.entrySet())
{
File localFile = entry.getKey();
String remoteFile = entry.getValue();
FileInputStream input = null;
try
{
input= new FileInputStream(localFile);
ftp.storeFile(remoteFile, input);
}
catch (Exception e)
{
try
{
ftp.deleteFile(remoteFile);
}
catch (IOException e1)
{
}
}
finally
{
if (input != null)
{
try
{
input.close();
}
catch (IOException e)
{
}
}
}
}
// disconnect
if (ftp != null && ftp.isConnected())
{
try
{
ftp.disconnect();
}
catch (IOException f)
{
// do nothing
}
}
When I uploaded 1050 files (each file is about 1-20 KB), it took about 49406 - 51000 millisec (This is upload time only).
I would like to improve the speed.
Some people suggest me to use ftp4j, but when I test the library with 1050 files, the upload speed of ftp4j is slowly than FTPClient about 10000 millisec.
it took about 60000 millisec.
Following is sample ftp4j code:
// create instance of FTPClient
FTPClient ftp = new FTPClient();
ftp.setCharset("UTF-8");
// connect to server
try
{
ftp.connect("10.1.1.1", 990);
}
catch(Exception e)
{
System.out.println("Cannot connect to server")
return;
}
// login to server
try
{
ftp.login("username", "password");
}
catch (Exception e)
{
try
{
ftp.logout();
}
catch (Exception e1)
{
}
System.out.println("Cannot login to server")
return;
}
try
{
ftp.setType(FTPClient.TYPE_BINARY);
ftp.setPassive(true);
}
catch(Exception e)
{
}
// create directory on server
// dirs is list of required directories on server
for (String dir : dirs)
{
try
{
ftp.createDirectory(dir);
}
catch (Exception e)
{
}
}
// files is a map of local file and string of remote file
// such as
// file on client is "C://test/a.txt"
// location on server is "/test/a.txt"
for (Map.Entry<File, String> entry : files.entrySet())
{
final File localFile = entry.getKey();
final String remoteFile = entry.getValue();
BufferedInputStream input = null;
boolean success = false;
try
{
input = new BufferedInputStream(new FileInputStream(localFile));
// ftp.upload(localFile); <-- if I use ftp.upload(File), it will took more time.
ftp.upload(remoteFile, input, 0, 2048, new MyTransferListener());
success = true;
}
catch (Exception e)
{
}
finally
{
if (input != null)
{
try
{
input.close();
}
catch (IOException e)
{
}
}
if (!success)
{
try
{
ftp.deleteFile(remoteFile);
}
catch (Exception e)
{
}
}
}
}
// disconnect
if (ftp != null && ftp.isConnected())
{
try
{
ftp.disconnect();
}
catch (IOException f)
{
// do nothing
}
}
I try to use multiple threads.
Following is multiple threading code:
final CountDownLatch latch = new CountDownLatch(files.size());
ExecutorService pool = Executors.newFixedThreadPool(10);
for (Map.Entry<File, String> entry : files.entrySet())
{
final File localFile = entry.getKey();
final String remoteFile = entry.getValue();
pool.execute(new Runnable() {
public void run()
{
FileInputStream input = null;
try
{
input= new FileInputStream(localFile);
ftp.storeFile(remoteFile, input);
}
catch (Exception e)
{
try
{
ftp.deleteFile(remoteFile);
}
catch (IOException e1)
{
}
}
finally
{
if (input != null)
{
try
{
input.close();
}
catch (IOException e)
{
}
}
latch.countDown();
}
}
});
}
try
{
// waiting for all threads finish
// see: http://stackoverflow.com/questions/1250643/how-to-wait-for-all-threads-to-finish-using-executorservice
latch.await();
}
catch(Exception e)
{
}
Is it correct? It work correctly but it cannot improve speed.
it took about 49000 - 51000 millisec same as the code without thread.
I test the speed with intranet. It will take more time for internet.
How should I do for improve upload speed?
I don't know why, but Apache Commons FTP is pretty slow in uploading, i had the same problem and i couldn't solve it.
Now i use FTP4j, it's pretty similar to apache commons ftp but uploads are really fast.
This is an example:
FTPClient client = new FTPClient();
client.connect("www.yoursite.com");
client.login("login", "password");
client.setPassive(true);
client.setType(FTPClient.TYPE_BINARY);
client.changeDirectory("a");
File f = new File("path/to/your/file");
client.upload(f);
client.disconnect(true);
With this library i uplodaded a 340KB file in less than one second, while with Apache Commons FTP it took about 1 minute.
If you want to transfer different files with threads, try to put every client.upload(f) into a different thread, but i'm not sure it will boost up the transfer.
Quoting #fge previous answer:
Basically, chances are, you can't.
Don't forget that FTP has two types of channels: the command channel and the data channels. One upload is initiated by sending the instructions over the command channel to open a data channel for the upload proper.
Now:
most FTP servers are configured so that one command channel can only open one data channel at anyone time;
there are bandwidth limits: your upstream bandwidth, and the server's downstream bandwidth.
Were it possible to upload several files in parallel, ie, opening more than one data channel, you'd have the problem that the overhead of TCP itself would in fact slow down the upload process in general.
Basically: keep one data channel open at any time. Trying and opening more than one is just not worth it. It may work in ~1% of cases in general. This is just not worth the hassle.
This Q&A some possible explanations of what is going on: why is ftp upload slow in java 7
And furthermore, it offers a couple of workarounds:
Upgrade to the 3.3 snapshot which can (currently) be found here
Call FTPClient.setBufferSize(0).
Apparently there is also a regression in Java 7 for Windows where the firewall application filter for the FTP client is blocking the client from using PASV mode FTP. It is not clear what the best solution to that is, but you could try the following:
Change your Windows firewall to disable the firewall application filter (as described in the Microsoft KB page.
Change your FTP application to use "active" mode ... though this requires that the FTP server can initiate connections to the machine running your clients.
Note: that there seem to be more than one explanation for the problem ... or maybe more than one possible problems.
Following is my java function to write a csv file in ftp location.The file gets created to ftp location but the file size is 0 bytes and file is empty.Kindly help as I am stuck
public int WriteFileToFtp(String FileName, String FileData) {
//get these details for the version??
//??
FTPClient ftp= new FTPClient();
try {
InputStream is = new ByteArrayInputStream(FileData.getBytes("ISO-8859-1"));
ftp.connect(ftpIP);
boolean isConnection = ftp.login(userName,password);
if(!isConnection){
logger.error("Connection failed");
return -1;
}
ftp.enterLocalActiveMode();
ftp.setFileType(FTP.BINARY_FILE_TYPE);//setting fileType
//?? go to directory using the circle code
if(ftpDirectoryToBeUpdate!=null && ftpDirectoryToBeUpdate.trim().length()>0)
{
logger.error("Changing directory for write="+ftpDirectoryToBeUpdate+" lcd="+ftp.printWorkingDirectory());
ftp.changeWorkingDirectory(ftpDirectoryToBeUpdate);
logger.error("Changed directory for write="+ftpDirectoryToBeUpdate+" lcd="+ftp.printWorkingDirectory());
}else
{
logger.error("Changed directory for write failed lcd="+ftp.printWorkingDirectory());
logger.error("DirectoryToReadFrom="+ftpDirectoryToBeUpdate);
}
ftp.storeFile(FileName,is) ;
logger.error(ftp.getReplyString());
is.close();
if(ftp.isConnected())
ftp.disconnect();
} catch (SocketException e) {
//logger.error(LoggerKeyWord.ERROR_ALERT + " FTP WRITE ERROR");
logger.error(e,e);
e.printStackTrace();
return -1;
} catch (IOException e) {
// logger.error(LoggerKeyWord.ERROR_ALERT + " FTP WRITE ERROR");
logger.error(e,e);
e.printStackTrace();
return -1;
}
try {
if(ftp.isConnected())
ftp.disconnect();
} catch (IOException e) {
logger.error (e,e);
e.printStackTrace();
return -1;
}
return 1;
}
have you tried closing input stream before saving the file?