I'm currently using this code to send a database over ftp (Using apache commons)
File file = getDatabasePath("database");
FTPClient ftp = new FTPClient();
try {
ftp.connect(InetAddress.getByName(domain));
ftp.login(username, password);
ftp.setFileType(FTP.BINARY_FILE_TYPE);
FileInputStream is = new FileInputStream(file);
BufferedInputStream buffIn = new BufferedInputStream(is);
ftp.enterLocalPassiveMode();
ftp.storeFile("database", buffIn);
buffIn.close();
ftp.logout();
ftp.disconnect();
} catch (Exception e) {
// TODO: handle exception
}
I've used it to send a text file and it works. However, I've tried it with my database and a file of the same size is put on the server but SQLite browser displays nothing when I open it. It works on a really small database but as soon as the database is larger I get this problem.
I was wondering if it could be to do with the buffer size? Could anyone shed some light on why this is happening?
Thanks
The code you posted will not work for files which don't fit to the buffered input stream's buffer. What you need to do is read repeatedly from the input stream until its end:
ftp.enterLocalPassiveMode();
ftp.storeFile("database", buffIn);
byte[] buffer = new byte[8192];
OutputStream os = ftp.storeFileStream("database");
int readCount = 0;
while ((readCount = buffIn.read(buffer)) > 0) {
os.write(buffer, 0, readCount);
}
os.close();
buffIn.close();
The important thing is the use of storeFileStream() instead of storeFile(). Also, as commenters suggest, you need to check return codes from the server and do proper error handling.
SQLite Database Copy Appears Corrupted When Generated by Device and not Emulator
Opening the database file in a different program seems to work. See above link.
Related
I have list of files that needs to be read from FTP server.
I have a method readFile(String path, FTPClient client) which reads and prints the file.
public byte[] readFile(String path,FTPClient client){
InputStream inStream = null;
ByteArrayOutputStream os = null;
byte[] finalBytes = new byte[0];
int reply;
int len;
byte[] buffer = new byte[1024];
try{
os = new ByteArrayOutputStream();
inStream = client.retrieveFileStream(path);
reply = client.getReplyCode();
log.warn("In getFTPfilebytes() :: Reply code -"+reply);
while ((len = inStream.read(buffer)) != -1) {
// write bytes from the buffer into output stream
os.write(buffer, 0, len);
}
finalBytes = os.toByteArray();
if(inStream == null){
throw new Exception("File not found");
}
inStream.close();
}catch(Exception e){
}finally{
try{ inStream.close();} catch(Exception e){}
}
return finalBytes;
}
I am calling above method in loop of list which contains strings of file path.
Issue - In loop only first file is getting read properly. Afterwards, it does not read file and throws an exception. inStream gives NULL for second iteration/second file. Also while iterating first file reply code after retrieveFileStream is "125(Data connection already open; transfer starting.)"
In second iteration it gives "200 (The requested action has been successfully completed.)"
I am not able to understand what is wrong here.
Have not closing inputstream connection properly?
You have to call FTPClient.completePendingCommand and close the input stream, as the documentation for FTPClient.retrieveFileStream says:
Returns an InputStream from which a named file from the server
can be read. If the current file type is ASCII, the returned
InputStream will convert line separators in the file to
the local representation. You must close the InputStream when you
finish reading from it. The InputStream itself will take care of
closing the parent data connection socket upon being closed.
To finalize the file transfer you must call completePendingCommand and
check its return value to verify success.
If this is not done, subsequent commands may behave unexpectedly.
inStream = client.retrieveFileStream(path);
try {
while ((len = inStream.read(buffer)) != -1) {
// write bytes from the buffer into output stream
os.write(buffer, 0, len);
}
finalBytes = os.toByteArray();
} finally {
inStream.close()
if (!client.completePendingCommand()) {
// error
}
}
Btw, there are better ways for copying from InputStream to OutputStream:
Easy way to write contents of a Java InputStream to an OutputStream
According to the documentation of the FTPClient.retrieveFileStream() method,
You must close the InputStream when you finish reading from it. The InputStream itself will take care of closing the parent data connection socket upon being closed.
When you close the stream, your client connection will be closed too. So instead of using the same client over and over, you need to create a new client connection for each file.
I didn't see the output stream is not properly closed.
finalBytes is o bytes?
where you defined the buffer variable?
please log the path so that we can see the path is correct or not. I guess the stream which is not properly closed makes the issue
This was originally a part 2 of a different thread, but another use suggested that I separate the part 2 into it's own topic, so here we go. Original thread is here (Original Thread)
I am using Jackcess to create a V2010 mdb file that I need to transfer to a client that will use Access 2013 to open it. Jackcess itself works - V2010 creates a file that Access 2013 can open when the file is FTP'd to the client by a third party software, such as FAR. However, when I try to upload this file to the client through the servlet (as is the goal of this project), Access on the client says "Unrecognized database format "...file name...". This is the code used for upload. Code itself works, file is transferred, has a non-zero size if it's saved - but Access cannot open it.
Note, for content type I also tried vnd.msassess and octed-stream, with same unsuccessful results. Also, I tried closing the db and creating the FileInputStream from the file name, and, as in the example, tried to create FileInputStream by calling mydb.getFile(). No difference.
response.setContentType("application/vnd.ms-access");
String fileName = "SomeFileName.mdb";
response.setHeader("Content-Disposition", "attachment; filename="+fileName);
Database mydb = generateMDBFile();
FileInputStream fis = new FileInputStream(mydb.getFile());
OutputStream os = response.getOutputStream();
byte[] buffer = new byte[1024];
try {
int byteRead = 0;
while ((byteRead = fis.read()) != -1) {
os.write(buffer, 0, byteRead);
}
os.flush();
} catch (Exception excp) {
excp.printStackTrace();
} finally {
os.close();
fis.close();
}
Why does this code corrupt the mdb file? This happens every time, regardless of the size (I tried a tiny 2 column/1 row file, and a huge file with 40 columns and 80000 rows)
Thank you!
You forgot to fill the buffer.
Use
// ...
while ((byteRead = fis.read(buffer)) != -1) {
os.write(buffer, 0, byteRead);
}
// ...
So i am building a program which needs an auto-updating feature built in to it, as i was finished up and tested it out, it seems when i send the jar file over the socket and write it to the newly made jar file it is missing 5KB (everytime... even when the size changes) size from it and becomes corrupt.
Here is my code:
package server.update;
import java.io.*;
import java.net.Socket;
public class UpdateThread extends Thread
{
BufferedInputStream input; //not used
BufferedInputStream fileInput;
BufferedOutputStream output;
public UpdateThread(Socket client) throws IOException
{
super("UpdateThread");
output = new BufferedOutputStream(client.getOutputStream());
input = new BufferedInputStream(client.getInputStream());
}
public void run()
{
try
{
File perm = new File(System.getProperty("user.dir")+"/GameClient.jar");
//fileInput = new BufferedInputStream(new FileInputStream(perm));
fileInput = new BufferedInputStream(new FileInputStream(perm));
byte[] buffer = new byte[1024];
int numRead;
while((numRead = fileInput.read(buffer)) != -1)
output.write(buffer, 0, numRead);
fileInput.close();
input.close();
output.close();
this.interrupt();
}
catch(Exception e)
{e.printStackTrace();}
}
}
This is the class that will wait for a connection from the client and then push the update to them as soon as it connects. File Perm is the jar file that i want to send over and for whatever reason it seems to either miss the last 5 bytes or the client doesn't read the last 5 (i don't know which). Here is the client's class of receiving the information here:
public void getUpdate(String ip) throws UnknownHostException, IOException
{
System.out.println("Connecting to update socket");
update = new Socket(ip,10004);
BufferedInputStream is = new BufferedInputStream(update.getInputStream());
BufferedOutputStream os = new BufferedOutputStream(update.getOutputStream());
System.out.println("Cleaning GameClient.jar file");
File updated = new File(System.getProperty("user.dir")+"/GameClient.jar");
if(updated.exists())
updated.delete();
updated.createNewFile();
BufferedOutputStream osf = new BufferedOutputStream(new FileOutputStream(updated));
System.out.println("Writing to GameClient.jar");
byte[] buffer = new byte[1024];
int numRead = 0;
while((numRead = is.read(buffer)) != -1)
osf.write(buffer, 0, numRead);
System.out.println("Finished updating...");
is.close();
os.close();
update.close();
osf.close();
}
Any help is appreciated. Thanks!
You have too many closes. Remove update.close() and is.close(). These both close the socket, which prevents the buffered stream 'osf' from being auto-flushed when closed. Closing either the input stream or the output stream or a socket closes the other stream and the socket. You should therefore only close the outermost output stream you have wrapped around the socket, in this case osf, and maybe the socket itself in a finally block to be sure.
Thanks to MDR for the answer, it worked!!
I had to change the following lines of code in the UpdateThread class:
Before:
fileInput.close();
input.close();
output.close();
this.interrupt();
After:
fileInput.close();
output.flush();
output.close();
input.close();
this.interrupt();
You must flush the stream before closing, also i switched the order because if you closed the inputstream attached to the socket it will close the socket and then will not move on to closing the outputstream or flushing it.
Thanks again!
Have you considered using an http library to delegate all of the connection handling and reading/writing to known working code? You're reinventing a lot of wheels here. Additionally at some point you're going to want to ensure the content you're receiving is authentic and undamaged (you're doing that by loading the class, which is somewhat dangerous, especially when you're exchanging data in cleartext!) Again, using a library and its methods would allow you to choose HTTPS, allowing TLS to do much of your work.
I'd also suggest that your server tell the client some metadata in advance, regardless- perhaps the content length and possibly a hash or checksum so the client can detect failures in the transfer.
This question seems to have answers relevant to your situation as well. Good luck!
This question already has answers here:
Java multiple file transfer over socket
(3 answers)
Closed 7 years ago.
Presently My Server program can able to receive the file from client socket and able to save that received file in server machine.
But I am need to receive many files from client socket to server socket without closing and opening the socket connection every time.
I have written the code, this is working fine. But in this I am closing and opening the server and client socket connection in every iteration. But I need to do this without connecting and disconnecting both the sockets every time.
Please guide me seniors...
My Server code:
int img_count=1;
int bytesRead;
int current = 0;
byte [] mybytearray = new byte [100000];
InputStream is = null;
FileOutputStream fos = null;
BufferedOutputStream bos = null;
Socket sock=null;
// create socket
ServerSocket servsock = new ServerSocket(6668);
System.out.println("Waiting... for client req");
int i=0;
for ( i=0; i<9; i++)
{
sock = servsock.accept(); // Waiting for Client
String fname = "Image000"+(img_count++)+".JPG";
String fpath = "C:/RX_images/"+fname; // Image saving path
File myFile = new File (fpath);
is = sock.getInputStream();
fos = new FileOutputStream(myFile);
bos = new BufferedOutputStream(fos);
bytesRead = is.read(mybytearray,0,mybytearray.length);
current = bytesRead;
do {
bytesRead =
is.read(mybytearray, current, (mybytearray.length-current));
if(bytesRead >= 0) current += bytesRead;
} while(bytesRead > -1);
bos.write(mybytearray, 0 , current);
bos.flush();
fos.flush();
fos.close();
bos.close();
is.close();
sock.close();
} // End of for loop
servsock.close();
System.out.println("Received : "+ (i++)+ " Images");
My Client Code:
int i=0;
int img_count=1;
FileInputStream fis=null;
BufferedInputStream bis=null;
OutputStream os=null;
Socket client=null;
System.out.println("Sending...");
for ( i=0; i<9; i++)
{
client = new Socket("192.168.1.54",6668);
String fname = "Image000"+(img_count++)+".JPG";
String fpath = "C:/Tx_Images/"+fname; // Image path
File myFile = new File (fpath);
byte [] mybytearray = new byte [(int)myFile.length()];
fis = new FileInputStream(myFile);
bis = new BufferedInputStream(fis);
bis.read(mybytearray,0,mybytearray.length);
os = client.getOutputStream();
os.write(mybytearray,0,mybytearray.length);
bis.close();
fis.close();
os.flush();
os.close();
client.close();
Thread.sleep(2000);
} // End of for loop
System.out.println("\n Sent : "+(i++)+" Images");
I am very new to java,
Help me please....
Since the socket is just a stream of bytes, in order to handle more than one file you are going to have to construct a simple protocol of some sort. In other words, the sender will have to send bytes that differentiate between the bytes in one file and the bytes in another. Since you are sending binary data, there is no series of bytes you can send to "mark" the beginning and/or/ending -- for example if you send 4 zero bytes at the end, that might be data and so the receiver cannot be sure if it's a marker or data. Two ways to handle it come to mind offhand -- break your file up into sections that are a maximum of N bytes, and send the sections one at a time. You will have to have a count of the bytes in each section, since at least one section will not have the same number of bytes as all other sections. Alternately,y you could count the bytes in the file and start with bytes that give that count, so the receiver knows how many bytes to expect. While you are giving the count, you could also give information such as the name and the type of file, if you wanted. Good luck.
This question really depends on whether you need the client to keep the connection open, or not. Typically you just need to keep the server side listening, and it's ok for the client to reconnect each time it needs to send a file.
Use an ExecutorService to keep the server side going and handle multiple connections with separate threads. Then just have the client connect and send what it needs to send and disconnect. See this question for a quick example: Multithreading Socket communication Client/Server
Also, look at how they close resources (finally) and stop the server in that example too. That is not related to your question, but you'll want to make your I/O and error handling more robust as well.
If you really do require that the server and client stay connected and send multiple files (or whatever data) then you'll need to implement some sort of a protocol as rcook notes, and you'll need to go deeper into networking and have a heartbeat and such. And, even if you do that, the client still needs to be smart enough to try to reconnect if the socket is closed, etc.
Just make simple protocol like:
File Name\r\n
File Size\r\n
File Data\r\n
File Name\r\n
File Size\r\n
File Data\r\n
....
I hope you will understand this. You can send file information initially then server will parse this file information, and make your server to read number bytes as you specified in file information. These will enable you to see file end marker and when to begin new file. BUT you must know file size before.
This will not work for data streams which have unknown length.
Make your server to read number of bytes you will be specifying, so server can know when to end file writing and begin new file or whether file is fully received before socket closes...
I would like to implement a function where you send a URL of a photo and my server will automatically download and store it in a specified folder.
I studied some use cases, but as a beginner in this area of the web, I was a bit lost. I thought about FTP but is not exactly what I want.
Like that, function on my webservice (using Java + Tomcat + AXIS2)
void getPhoto(URL url){
//receive a photo and store at folder /photos
}
but, I don't know what use, I was looking for some httppost or httpget, should I still looking for in this way? Has a dummie sample, to show me the basic way?
I would like to implement a function where you send a URL of a photo and my server will automatically download and store it in a specified folder.
That's not exactly "uploading", but just "downloading".
Just call openStream() on the URL and you've an InputStream which you can do anything with. Writing to a FileOutputStream for example.
InputStream input = url.openStream();
// ...
hey use this code to download.
try {
URL url = new URL(url of file );
URLConnection conection = url.openConnection();
conection.connect();
InputStream input = new BufferedInputStream(url.openStream());
String downloadloc = "D:\"; // or anything
OutputStream output = new FileOutputStream(downloadloc
+ "\name of file.ext");
byte data[] = new byte[1024];
long total = 0;
while ((count = input.read(data)) != -1) {
total += count;
output.write(data, 0, count);
}
output.flush();
output.close();
input.close();
} catch (Exception e) {
}
You want to look at using an HttpURLConnection, call it's 'connect' and 'getInputStream' methods, continually reading from that stream and writing that data to a file with e.g. a FileOutputStream.
To download a file using a URL, as an alternative to what suggested by others, you can take a look to Apache Commons HttpClient.
There is also a well written tutorial.