I am trying to upload a file in android to an FTP server using the apache.commons.ftp library.
Note: I am using storeFileStream so that I can track the progress of the upload.
ftpClient.enterLocalPassiveMode();
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
InputStream inputStream = new FileInputStream(localFile);
OutputStream outputStream = ftpClient.storeFileStream(remoteFile);
byte[] buffer = new byte[8192];
int bytesRead;
logMessage("Starting to upload file");
while ((bytesRead = inputStream.read(buffer)) != -1) {
total += bytesRead;
outputStream.write(buffer, 0, bytesRead); //output stream is null here
latestPercentDone = (int) ((total / (float) totalMegaBytes) * 100);
if (percentDone != latestPercentDone) {
percentDone = latestPercentDone;
publishProgress(""+percentDone);
}
}
inputStream.close();
outputStream.close();
ftpClient.completePendingCommand();
I am using the speedtest.tele2.net server for testing. Using an anonymous login.
The log keeps saying the outputstream is null.
EDIT: Using what Martin Prikryl and getting the error code, i found the problem to be the location of the remote file.
Upload to a location rather than a directory.
For example if the file is caled item1.txt the remote file should be /item1.txt or /someFolder/AnotherFolder/item1.txt rather than /someFolder/AnotherFolder/
The FTPClient.storeFileStream method can return null:
An OutputStream through which the remote file can be written. If the data connection cannot be opened (e.g., the file does not exist), null is returned (in which case you may check the reply code to determine the exact reason for failure).
Though the "file does not exist" note is nonsense for upload, it's obviously a copy-and-paste error from the .retrieveFileStream (download).
Use the .getReplyCode and .getReplyString, to see what went wrong.
Related
I have list of files that needs to be read from FTP server.
I have a method readFile(String path, FTPClient client) which reads and prints the file.
public byte[] readFile(String path,FTPClient client){
InputStream inStream = null;
ByteArrayOutputStream os = null;
byte[] finalBytes = new byte[0];
int reply;
int len;
byte[] buffer = new byte[1024];
try{
os = new ByteArrayOutputStream();
inStream = client.retrieveFileStream(path);
reply = client.getReplyCode();
log.warn("In getFTPfilebytes() :: Reply code -"+reply);
while ((len = inStream.read(buffer)) != -1) {
// write bytes from the buffer into output stream
os.write(buffer, 0, len);
}
finalBytes = os.toByteArray();
if(inStream == null){
throw new Exception("File not found");
}
inStream.close();
}catch(Exception e){
}finally{
try{ inStream.close();} catch(Exception e){}
}
return finalBytes;
}
I am calling above method in loop of list which contains strings of file path.
Issue - In loop only first file is getting read properly. Afterwards, it does not read file and throws an exception. inStream gives NULL for second iteration/second file. Also while iterating first file reply code after retrieveFileStream is "125(Data connection already open; transfer starting.)"
In second iteration it gives "200 (The requested action has been successfully completed.)"
I am not able to understand what is wrong here.
Have not closing inputstream connection properly?
You have to call FTPClient.completePendingCommand and close the input stream, as the documentation for FTPClient.retrieveFileStream says:
Returns an InputStream from which a named file from the server
can be read. If the current file type is ASCII, the returned
InputStream will convert line separators in the file to
the local representation. You must close the InputStream when you
finish reading from it. The InputStream itself will take care of
closing the parent data connection socket upon being closed.
To finalize the file transfer you must call completePendingCommand and
check its return value to verify success.
If this is not done, subsequent commands may behave unexpectedly.
inStream = client.retrieveFileStream(path);
try {
while ((len = inStream.read(buffer)) != -1) {
// write bytes from the buffer into output stream
os.write(buffer, 0, len);
}
finalBytes = os.toByteArray();
} finally {
inStream.close()
if (!client.completePendingCommand()) {
// error
}
}
Btw, there are better ways for copying from InputStream to OutputStream:
Easy way to write contents of a Java InputStream to an OutputStream
According to the documentation of the FTPClient.retrieveFileStream() method,
You must close the InputStream when you finish reading from it. The InputStream itself will take care of closing the parent data connection socket upon being closed.
When you close the stream, your client connection will be closed too. So instead of using the same client over and over, you need to create a new client connection for each file.
I didn't see the output stream is not properly closed.
finalBytes is o bytes?
where you defined the buffer variable?
please log the path so that we can see the path is correct or not. I guess the stream which is not properly closed makes the issue
I am trying to download a file that I have hosted on mega. The code is as follows:
AsyncTask.execute(new Runnable() {
#Override
public void run() {
LoggerDebug.d("Prakhar", "Inside run");
try {
URL downloadUrl = new URL(url);
HttpURLConnection httpURLConnection = (HttpURLConnection) downloadUrl.openConnection();
httpURLConnection.setRequestMethod("GET");
httpURLConnection.setDoOutput(true);
httpURLConnection.connect();
File SDCardRoot = Environment.getExternalStorageDirectory();
File downloadFile = new File(SDCardRoot, "downloaded.zip");
FileOutputStream fileOutputStream = new FileOutputStream(downloadFile);
InputStream inputStream = httpURLConnection.getInputStream();
int totalSize = httpURLConnection.getContentLength();
int downloadedSize = 0;
byte[] buffer = new byte[1024];
int bufferLength = 0;
LoggerDebug.d("Prakhar", String.valueOf(totalSize));
while ((bufferLength = inputStream.read(buffer)) > 0 ) {
LoggerDebug.d("Prakhar", String.valueOf(downloadedSize));
fileOutputStream.write(buffer, 0, bufferLength);
downloadedSize += bufferLength;
}
} catch (Exception e) {
e.printStackTrace();
}
}
});
}
Now after I run this code sample I get the following:
Total size is : -1
Downloaded size : 1024
On the mobile I can see the downloaded.zip file, its size is 1.97KB and the original size of the upload is 87.1KB
Where am I going wrong?
So I don't know if you solved your problem yet, but I can see a couple potential issues.
setDoOutput(true) indicates you're going to be uploading according to the documentation here. Specifically, it says:
To upload data to a web server, configure the connection for output using setDoOutput(true).
You're not uploading from what I'm seeing, so this isn't needed, and it's unclear how the server would respond since it appears you're requesting the ability to upload.
A return of -1 indicates an EOF. You're not accessing the file you're intending to, likely due to #3.
You probably can't just download from their site willy-nilly. You need to authenticate somehow. This could be an API key, session id, yadda yadda. They have their own API, and according to this document there is logging in via e-mail in their API. The page doesn't render correctly in GitHub, but you'll need to figure out how to log in. They have an Android app example, but it looks like the API is all native code.
Have you tried changing:
while ((bufferLength = inputStream.read(buffer)) > 0 ) {
to
while ((bufferLength = inputStream.read(buffer)) != null) {
This was originally a part 2 of a different thread, but another use suggested that I separate the part 2 into it's own topic, so here we go. Original thread is here (Original Thread)
I am using Jackcess to create a V2010 mdb file that I need to transfer to a client that will use Access 2013 to open it. Jackcess itself works - V2010 creates a file that Access 2013 can open when the file is FTP'd to the client by a third party software, such as FAR. However, when I try to upload this file to the client through the servlet (as is the goal of this project), Access on the client says "Unrecognized database format "...file name...". This is the code used for upload. Code itself works, file is transferred, has a non-zero size if it's saved - but Access cannot open it.
Note, for content type I also tried vnd.msassess and octed-stream, with same unsuccessful results. Also, I tried closing the db and creating the FileInputStream from the file name, and, as in the example, tried to create FileInputStream by calling mydb.getFile(). No difference.
response.setContentType("application/vnd.ms-access");
String fileName = "SomeFileName.mdb";
response.setHeader("Content-Disposition", "attachment; filename="+fileName);
Database mydb = generateMDBFile();
FileInputStream fis = new FileInputStream(mydb.getFile());
OutputStream os = response.getOutputStream();
byte[] buffer = new byte[1024];
try {
int byteRead = 0;
while ((byteRead = fis.read()) != -1) {
os.write(buffer, 0, byteRead);
}
os.flush();
} catch (Exception excp) {
excp.printStackTrace();
} finally {
os.close();
fis.close();
}
Why does this code corrupt the mdb file? This happens every time, regardless of the size (I tried a tiny 2 column/1 row file, and a huge file with 40 columns and 80000 rows)
Thank you!
You forgot to fill the buffer.
Use
// ...
while ((byteRead = fis.read(buffer)) != -1) {
os.write(buffer, 0, byteRead);
}
// ...
I want to download a HTTP query with java, but the file I download has an undetermined length when downloading.
I thought this would be quite standard, so I searched and found a code snippet for it: http://snipplr.com/view/33805/
But it has a problem with the contentLength variable. As the length is unknown, I get -1 back. This creates an error. When I omit the entire check about contentLength, that means I always have to use the maximum buffer.
But the problem is that the file is not ready yet. So the flush gets only partially filled, and parts of the file get lost.
If you try downloading a link like http://overpass-api.de/api/interpreter?data=area%5Bname%3D%22Hoogstade%22%5D%3B%0A%28%0A++node%28area%29%3B%0A++%3C%3B%0A%29+%3B%0Aout+meta+qt%3B with that snippet, you'll notice the error, and when you always download the maximum buffer to omit the error, you end up with a corrupt XML file.
Is there some way to only download the ready part of the file? I would like if this could download big files (up to a few GB).
This should work, i tested it and it works for me:
void downloadFromUrl(URL url, String localFilename) throws IOException {
InputStream is = null;
FileOutputStream fos = null;
try {
URLConnection urlConn = url.openConnection();//connect
is = urlConn.getInputStream(); //get connection inputstream
fos = new FileOutputStream(localFilename); //open outputstream to local file
byte[] buffer = new byte[4096]; //declare 4KB buffer
int len;
//while we have availble data, continue downloading and storing to local file
while ((len = is.read(buffer)) > 0) {
fos.write(buffer, 0, len);
}
} finally {
try {
if (is != null) {
is.close();
}
} finally {
if (fos != null) {
fos.close();
}
}
}
}
If you want this to run in background, simply call it in a Thread:
Thread download = new Thread(){
public void run(){
URL url= new URL("http://overpass-api.de/api/interpreter?data=area%5Bname%3D%22Hoogstade%22%5D%3B%0A%28%0A++node%28area%29%3B%0A++%3C%3B%0A%29+%3B%0Aout+meta+qt%3B");
String localFilename="mylocalfile"; //needs to be replaced with local file path
downloadFromUrl(url, localFilename);
}
};
download.start();//start the thread
I'm currently using this code to send a database over ftp (Using apache commons)
File file = getDatabasePath("database");
FTPClient ftp = new FTPClient();
try {
ftp.connect(InetAddress.getByName(domain));
ftp.login(username, password);
ftp.setFileType(FTP.BINARY_FILE_TYPE);
FileInputStream is = new FileInputStream(file);
BufferedInputStream buffIn = new BufferedInputStream(is);
ftp.enterLocalPassiveMode();
ftp.storeFile("database", buffIn);
buffIn.close();
ftp.logout();
ftp.disconnect();
} catch (Exception e) {
// TODO: handle exception
}
I've used it to send a text file and it works. However, I've tried it with my database and a file of the same size is put on the server but SQLite browser displays nothing when I open it. It works on a really small database but as soon as the database is larger I get this problem.
I was wondering if it could be to do with the buffer size? Could anyone shed some light on why this is happening?
Thanks
The code you posted will not work for files which don't fit to the buffered input stream's buffer. What you need to do is read repeatedly from the input stream until its end:
ftp.enterLocalPassiveMode();
ftp.storeFile("database", buffIn);
byte[] buffer = new byte[8192];
OutputStream os = ftp.storeFileStream("database");
int readCount = 0;
while ((readCount = buffIn.read(buffer)) > 0) {
os.write(buffer, 0, readCount);
}
os.close();
buffIn.close();
The important thing is the use of storeFileStream() instead of storeFile(). Also, as commenters suggest, you need to check return codes from the server and do proper error handling.
SQLite Database Copy Appears Corrupted When Generated by Device and not Emulator
Opening the database file in a different program seems to work. See above link.