I would like to use org.apache.commons.net.ftp.FTPClient in my JSF application. How client side (Web Browser) upload to my web application server for large file. Even if I use RichFaces File Upload or PrimeFaces File Upload, client browser can use HTTP Protocol. How can I support FTP Protocol to client browser? Could you provide the better way?
Cause : the application user cannot direct access to our Repository Server(SVN). Firstly, they have to upload the files to our application on Web AS. And then, the application checkin/chekout to RepositoryServer. The application user can upload the file which has 500M to 2G at least. That's why, I am thinking, how can I support FTP Protocol to browser client' to be faster. Otherwise, am I thinking wrong?
In order to be able to send a file to a FTP server, you obviously need a FTP client.
However, a webbrowser is a HTTP client, not a FTP client. This is a natural functional design limitation of the webbrowser. JSF look like a magician, but here it really can't do anything for you. It intercepts on HTTP requests/responses only.
Indeed, you're thinking wrong. Just stick to uploading the file the usual HTTP way. If you're absolutely positive that you need FTP for this for some reason, then your best bet is most likely homebrewing a Java Applet for this, but this would after all be plain clumsy.
First do HTTP upload through primefaces to a temporary directory. then through org.apache.commons.net.ftp.FTPClient or through sun.net.ftp.FtpClient upload to the required FTP Server.
Below is an example;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.OutputStream;
import sun.net.ftp.FtpClient;
/**
*
* #author fali
*/
public class FtpUtil {
public String server, username,password, remote, remotedir, local;
FtpClient ftp;
public static int BUFFER_SIZE = 10240;
public FtpUtil(){
server = "localhost";
username = "anonymous";
password = " ";
remotedir = "/incoming";
remote = "dvs.txt";
local = "C:\\dvs.txt";
}
protected void putFile() {
if (local.length() == 0) {
System.out.println("Please enter file name");
}
byte[] buffer = new byte[BUFFER_SIZE];
try {
File f = new File(local);
int size = (int) f.length();
System.out.println("File " + local + ": " + size + " bytes");
System.out.println(size);
FileInputStream in = new FileInputStream(local);
OutputStream out = ftp.put(remote);
int counter = 0;
while (true) {
int bytes = in.read(buffer);
if (bytes < 0)
break;
out.write(buffer, 0, bytes);
counter += bytes;
System.out.println(counter);
}
out.close();
in.close();
} catch (Exception ex) {
System.out.println("Error: " + ex.toString());
}
}
public String Upload(){
String result="";
try{
ftp = new FtpClient(server);
ftp.login(username, password);
System.out.println(ftp.welcomeMsg);
ftp.cd(remotedir);
putFile();
disconnect();
}catch(Exception ex){
System.out.println(ex);
result = "Error : "+ex;
}
return "";
}
protected void disconnect() {
if (ftp != null) {
try {
ftp.closeServer();
} catch (IOException ex) {
}
ftp = null;
}
}
}
In your managedbean/controller;
public String create() {
System.out.println("Request Button Clicked");
try {
// generate reference number
//current.setReferenceno(genReferenceNo());
// add to database
//getFacade().persist(current);
// upload to ftp
FtpUtil fu = new FtpUtil();
fu.Upload();
// show reference number
JsfUtil.addSuccessMessage(ResourceBundle.getBundle("/Bundle").getString("QueueCreated"));
JsfUtil.addSuccessMessage("Your Reference No. is :" + current.referenceno);
current = null;
// try {
// System.out.println("Redirecting");
// FacesContext.getCurrentInstance().getExternalContext().dispatch("/");
// } catch (Exception ex) {
// System.out.println(ex);
// }
return "";
} catch (Exception e) {
JsfUtil.addErrorMessage(e, ResourceBundle.getBundle("/Bundle").getString("PersistenceErrorOccured"));
return null;
}
}
and some thing like this in your page;
<br />
<ppctu:commandButton action="#{appointmentController.create}" type="Submit" value="Request" />
Related
I am trying to create a web proxy client in java. I stole this guys code and modified it to create a server and now a client so that the user can just download and run the client to connect to the server. I would just like this to run on the terminal but my future plans are to add a gui.
This is the server. This works straight out of the box as in you enter the ip in firefox and it will run just fine. TLDR: It parses the URL and forwards the data from the client to the server and vice versa.
Request Handler.java
/*
-----------------------------------------------------------------------------------------------
STOLEN FROM THIS MAN ON GITHUB
https://github.com/stefano-lupo/Java-Proxy-Server/blob/master/src/RequestHandler.java
-----------------------------------------------------------------------------------------------
*/
import java.awt.*;
import java.io.*;
import java.net.*;
import javax.imageio.*;
public class RequestHandler implements Runnable {
/**
* Socket connected to client passed by Proxy server
*/
Socket clientSocket;
/**
* Read data client sends to proxy
*/
BufferedReader proxyToClientBr;
/**
* Send data from proxy to client
*/
BufferedWriter proxyToClientBw;
/**
* Thread that is used to transmit data read from client to server when using HTTPS
* Reference to this is required so it can be closed once completed.
*/
private Thread httpsClientToServer;
/**
* Creates a RequestHandler object capable of servicing HTTP(S) GET requests
* #param clientSocket socket connected to the client
*/
public RequestHandler(Socket clientSocket){
this.clientSocket = clientSocket;
try
{
this.clientSocket.setSoTimeout(2000);
proxyToClientBr = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
proxyToClientBw = new BufferedWriter(new OutputStreamWriter(clientSocket.getOutputStream()));
}
catch (IOException e)
{
System.out.println("Error on IO Exception");
e.printStackTrace();
}
}
/**
* Reads and examines the requestString and calls the appropriate method based
* on the request type.
*/
#Override
public void run() {
// Get Request from client
String requestString;
try{
requestString = proxyToClientBr.readLine();
} catch (IOException e) {
e.printStackTrace();
System.out.println("Error reading request from client");
return;
}
// Parse out URL
System.out.println("Request Received " + requestString);
// Get the Request type
String request = requestString.substring(0,requestString.indexOf(' '));
// remove request type and space
String urlString = requestString.substring(requestString.indexOf(' ')+1);
// Remove everything past next space
urlString = urlString.substring(0, urlString.indexOf(' '));
// Prepend http:// if necessary to create correct URL
if(!urlString.substring(0,4).equals("http")){
String temp = "http://";
urlString = temp + urlString;
}
// Check request type
if(request.equals("CONNECT")){
System.out.println("HTTPS Request for : " + urlString + "\n");
handleHTTPSRequest(urlString);
}
else
{
System.out.println("HTTP GET for : " + urlString + "\n");
sendNonCachedToClient(urlString);
}
}
/**
* Sends the contents of the file specified by the urlString to the client
* #param urlString URL ofthe file requested
*/
private void sendNonCachedToClient(String urlString){
try{
// Compute a logical file name as per schema
// This allows the files on stored on disk to resemble that of the URL it was taken from
int fileExtensionIndex = urlString.lastIndexOf(".");
String fileExtension;
// Get the type of file
fileExtension = urlString.substring(fileExtensionIndex, urlString.length());
// Get the initial file name
String fileName = urlString.substring(0,fileExtensionIndex);
// Trim off http://www. as no need for it in file name
fileName = fileName.substring(fileName.indexOf('.')+1);
// Remove any illegal characters from file name
fileName = fileName.replace("/", "__");
fileName = fileName.replace('.','_');
// Trailing / result in index.html of that directory being fetched
if(fileExtension.contains("/")){
fileExtension = fileExtension.replace("/", "__");
fileExtension = fileExtension.replace('.','_');
fileExtension += ".html";
}
fileName = fileName + fileExtension;
// Attempt to create File to cache to
boolean caching = true;
File fileToCache = null;
BufferedWriter fileToCacheBW = null;
try{
// Create File to cache
fileToCache = new File("cached/" + fileName);
if(!fileToCache.exists()){
fileToCache.createNewFile();
}
// Create Buffered output stream to write to cached copy of file
fileToCacheBW = new BufferedWriter(new FileWriter(fileToCache));
}
catch (IOException e){
System.out.println("Couldn't cache: " + fileName);
caching = false;
e.printStackTrace();
} catch (NullPointerException e) {
System.out.println("NPE opening file");
}
// Check if file is an image
if((fileExtension.contains(".png")) || fileExtension.contains(".jpg") ||
fileExtension.contains(".jpeg") || fileExtension.contains(".gif")){
// Create the URL
URL remoteURL = new URL(urlString);
BufferedImage image = ImageIO.read(remoteURL);
if(image != null) {
// Cache the image to disk
ImageIO.write(image, fileExtension.substring(1), fileToCache);
// Send response code to client
String line = "HTTP/1.0 200 OK\n" +
"Proxy-agent: ProxyServer/1.0\n" +
"\r\n";
proxyToClientBw.write(line);
proxyToClientBw.flush();
// Send them the image data
ImageIO.write(image, fileExtension.substring(1), clientSocket.getOutputStream());
// No image received from remote server
} else {
System.out.println("Sending 404 to client as image wasn't received from server"
+ fileName);
String error = "HTTP/1.0 404 NOT FOUND\n" +
"Proxy-agent: ProxyServer/1.0\n" +
"\r\n";
proxyToClientBw.write(error);
proxyToClientBw.flush();
return;
}
}
// File is a text file
else {
// Create the URL
URL remoteURL = new URL(urlString);
// Create a connection to remote server
HttpURLConnection proxyToServerCon = (HttpURLConnection)remoteURL.openConnection();
proxyToServerCon.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded");
proxyToServerCon.setRequestProperty("Content-Language", "en-US");
proxyToServerCon.setUseCaches(false);
proxyToServerCon.setDoOutput(true);
// Create Buffered Reader from remote Server
BufferedReader proxyToServerBR = new BufferedReader(new InputStreamReader(proxyToServerCon.getInputStream()));
// Send success code to client
String line = "HTTP/1.0 200 OK\n" +
"Proxy-agent: ProxyServer/1.0\n" +
"\r\n";
proxyToClientBw.write(line);
// Read from input stream between proxy and remote server
while((line = proxyToServerBR.readLine()) != null){
// Send on data to client
proxyToClientBw.write(line);
// Write to our cached copy of the file
if(caching){
fileToCacheBW.write(line);
}
}
// Ensure all data is sent by this point
proxyToClientBw.flush();
// Close Down Resources
if(proxyToServerBR != null){
proxyToServerBR.close();
}
}
// Close down resources
if(fileToCacheBW != null){
fileToCacheBW.close();
}
if(proxyToClientBw != null){
proxyToClientBw.close();
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
/**
* Handles HTTPS requests between client and remote server
* #param urlString desired file to be transmitted over https
*/
private void handleHTTPSRequest(String urlString){
// Extract the URL and port of remote
String url = urlString.substring(7);
String pieces[] = url.split(":");
url = pieces[0];
int port = Integer.valueOf(pieces[1]);
try{
// Only first line of HTTPS request has been read at this point (CONNECT *)
// Read (and throw away) the rest of the initial data on the stream
for(int i=0;i<5;i++){
proxyToClientBr.readLine();
}
// Get actual IP associated with this URL through DNS
InetAddress address = InetAddress.getByName(url);
// Open a socket to the remote server
Socket proxyToServerSocket = new Socket(address, port);
proxyToServerSocket.setSoTimeout(5000);
// Send Connection established to the client
String line = "HTTP/1.0 200 Connection established\r\n" +
"Proxy-Agent: ProxyServer/1.0\r\n" +
"\r\n";
proxyToClientBw.write(line);
proxyToClientBw.flush();
// Client and Remote will both start sending data to proxy at this point
// Proxy needs to asynchronously read data from each party and send it to the other party
//Create a Buffered Writer betwen proxy and remote
BufferedWriter proxyToServerBW = new BufferedWriter(new OutputStreamWriter(proxyToServerSocket.getOutputStream()));
// Create Buffered Reader from proxy and remote
BufferedReader proxyToServerBR = new BufferedReader(new InputStreamReader(proxyToServerSocket.getInputStream()));
// Create a new thread to listen to client and transmit to server
ClientToServerHttpsTransmit clientToServerHttps =
new ClientToServerHttpsTransmit(clientSocket.getInputStream(), proxyToServerSocket.getOutputStream());
httpsClientToServer = new Thread(clientToServerHttps);
httpsClientToServer.start();
// Listen to remote server and relay to client
try {
byte[] buffer = new byte[4096];
int read;
do {
read = proxyToServerSocket.getInputStream().read(buffer);
if (read > 0) {
clientSocket.getOutputStream().write(buffer, 0, read);
if (proxyToServerSocket.getInputStream().available() < 1) {
clientSocket.getOutputStream().flush();
}
}
} while (read >= 0);
}
catch (SocketTimeoutException e) {
}
catch (IOException e) {
e.printStackTrace();
}
// Close Down Resources
if(proxyToServerSocket != null){
proxyToServerSocket.close();
}
if(proxyToServerBR != null){
proxyToServerBR.close();
}
if(proxyToServerBW != null){
proxyToServerBW.close();
}
if(proxyToClientBw != null){
proxyToClientBw.close();
}
} catch (SocketTimeoutException e) {
String line = "HTTP/1.0 504 Timeout Occured after 10s\n" +
"User-Agent: ProxyServer/1.0\n" +
"\r\n";
try{
proxyToClientBw.write(line);
proxyToClientBw.flush();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
catch (Exception e){
System.out.println("Error on HTTPS : " + urlString );
e.printStackTrace();
}
}
/**
* Listen to data from client and transmits it to server.
* This is done on a separate thread as must be done
* asynchronously to reading data from server and transmitting
* that data to the client.
*/
class ClientToServerHttpsTransmit implements Runnable{
InputStream proxyToClientIS;
OutputStream proxyToServerOS;
/**
* Creates Object to Listen to Client and Transmit that data to the server
* #param proxyToClientIS Stream that proxy uses to receive data from client
* #param proxyToServerOS Stream that proxy uses to transmit data to remote server
*/
public ClientToServerHttpsTransmit(InputStream proxyToClientIS, OutputStream proxyToServerOS) {
this.proxyToClientIS = proxyToClientIS;
this.proxyToServerOS = proxyToServerOS;
}
#Override
public void run(){
try {
// Read byte by byte from client and send directly to server
byte[] buffer = new byte[4096];
int read;
do {
read = proxyToClientIS.read(buffer);
if (read > 0) {
proxyToServerOS.write(buffer, 0, read);
if (proxyToClientIS.available() < 1) {
proxyToServerOS.flush();
}
}
} while (read >= 0);
}
catch (SocketTimeoutException ste)
{
ste.printStackTrace();
}
catch (IOException e) {
System.out.println("Proxy to client HTTPS read timed out");
e.printStackTrace();
}
}
}
}
This is the client. It doesn't seem to work and I dont really understand why as it is just a modified version of the server.
Request Handler.java
/*
-----------------------------------------------------------------------------------------------
STOLEN FROM THIS MAN ON GITHUB
https://github.com/stefano-lupo/Java-Proxy-Server/blob/master/src/RequestHandler.java
-----------------------------------------------------------------------------------------------
*/
import java.awt.*;
import java.io.*;
import java.net.*;
import javax.imageio.*;
public class RequestHandler implements Runnable
{
/**
* Socket connected to client passed by Proxy server
*/
Socket clientSocket;
/**
* Read data client sends to proxy
*/
BufferedReader clientToClientBr;
/**
* Send data from proxy to client
*/
BufferedWriter clientToClientBw;
/**
* Thread that is used to transmit data read from client to server when using HTTPS
* Reference to this is required so it can be closed once completed.
*/
private Thread threads;
/**
* Creates a RequestHandler object capable of servicing HTTP(S) GET requests
* #param clientSocket socket connected to the client
*/
public RequestHandler(Socket clientSocket)
{
this.clientSocket = clientSocket;
try
{
this.clientSocket.setSoTimeout(2000);
clientToClientBr = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
clientToClientBw = new BufferedWriter(new OutputStreamWriter(clientSocket.getOutputStream()));
}
catch (IOException e)
{
System.out.println("Error on IO Exception");
e.printStackTrace();
}
}
/**
* Reads and examines the requestString and calls the appropriate method based
* on the request type.
*/
#Override
public void run()
{
// Get Request from client
String requestString;
try
{
requestString = clientToClientBr.readLine();
System.out.println(requestString);
sendToServer(requestString);
}
catch (IOException e)
{
e.printStackTrace();
System.out.println("Error reading request from client");
return;
}
}
private void sendToServer(String urlString) throws IOException
{
// Open a socket to the remote server
Socket proxyToServerSocket = new Socket("192.168.0.226", 8080);
proxyToServerSocket.setSoTimeout(5000);
// Client and Remote will both start sending data to proxy at this point
// Proxy needs to asynchronously read data from each party and send it to the other party
//Create a Buffered Writer betwen proxy and remote
BufferedWriter proxyToServerBW = new BufferedWriter(new OutputStreamWriter(proxyToServerSocket.getOutputStream()));
// Create Buffered Reader from proxy and remote
BufferedReader proxyToServerBR = new BufferedReader(new InputStreamReader(proxyToServerSocket.getInputStream()));
// Create a new thread to listen to client and transmit to server
ClientToServerHttpsTransmit clientToServerHttps = new ClientToServerHttpsTransmit(clientSocket.getInputStream(), proxyToServerSocket.getOutputStream());
threads = new Thread(clientToServerHttps);
threads.start();
// Listen to remote server and relay to client
try {
byte[] buffer = new byte[4096];
int read;
do
{
read = proxyToServerSocket.getInputStream().read(buffer);
if (read > 0)
{
clientSocket.getOutputStream().write(buffer, 0, read);
if (proxyToServerSocket.getInputStream().available() < 1)
{
clientSocket.getOutputStream().flush();
}
}
} while (read >= 0);
}
catch (SocketTimeoutException e) {
}
catch (IOException e) {
e.printStackTrace();
}
// Close Down Resources
if(proxyToServerSocket != null){
proxyToServerSocket.close();
}
if(proxyToServerBR != null){
proxyToServerBR.close();
}
if(proxyToServerBW != null){
proxyToServerBW.close();
}
if(clientToClientBw != null){
clientToClientBw.close();
}
}
}
/**
* Listen to data from client and transmits it to server.
* This is done on a separate thread as must be done
* asynchronously to reading data from server and transmitting
* that data to the client.
*/
class ClientToServerHttpsTransmit implements Runnable
{
InputStream proxyToClientIS;
OutputStream proxyToServerOS;
/**
* Creates Object to Listen to Client and Transmit that data to the server
* #param proxyToClientIS Stream that proxy uses to receive data from client
* #param proxyToServerOS Stream that proxy uses to transmit data to remote server
*/
public ClientToServerHttpsTransmit(InputStream proxyToClientIS, OutputStream proxyToServerOS)
{
this.proxyToClientIS = proxyToClientIS;
this.proxyToServerOS = proxyToServerOS;
}
#Override
public void run()
{
try
{
// Read byte by byte from client and send directly to server
byte[] buffer = new byte[4096];
int read;
do
{
read = proxyToClientIS.read(buffer);
if (read > 0)
{
proxyToServerOS.write(buffer, 0, read);
if (proxyToClientIS.available() < 1)
{
proxyToServerOS.flush();
}
}
} while (read >= 0);
}
catch (SocketTimeoutException ste)
{
ste.printStackTrace();
}
catch (IOException e)
{
System.out.println("Proxy to client HTTPS read timed out");
e.printStackTrace();
}
}
}
I think that the error is in the Request Handler for the Client as the error message I get on the client terminal is:
Wating for client to connect on port 8080
Got CONNECT www.youtube.com:443 HTTP/1.1
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.net.SocketInputStream.read(SocketInputStream.java:127)
at ClientToServerHttpsTransmit.run(RequestHandler.java:186)
at java.lang.Thread.run(Thread.java:748)
and this is the error for the terminal on the server:
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:161)
at java.io.BufferedReader.readLine(BufferedReader.java:324)
at java.io.BufferedReader.readLine(BufferedReader.java:389)
at RequestHandler.run(RequestHandler.java:75)
at java.lang.Thread.run(Thread.java:748)
Error reading request from client
I save user uploaded images in FTP.
FTP service is running on server Server-A. The actual problem is when I want to see the uploaded image from the web application running in my local host everything works, but when I deploy the local application to Tomcat running on the same server Server-A, images are not displayed correctly.
The picture when I run the web application in local Tomcat:
The same picture when I run the web application in the remote Tomcat:
You can see that the second image is not displayed correctly. Also want to mention that the FTP is the same one.
I am using Spring with Apache FtpClient library for image upload/download functionality.
Controller source code:
#RequestMapping(value = "/{id:\\d+}/image", method = RequestMethod.GET, produces = MediaType.IMAGE_JPEG_VALUE)
protected byte[] getUserImage(BaseForm form,
#PathVariable("id") int userId) {
try {
User user = checkToken(form.getToken());
log.info("/users/{id}/image [GET]. User: " + user + ", form: " + form + ", User id: " + userId);
FileWrapper image = service.getUserImage(userId);
if(image != null) {
return ftpService.downloadFtpFile(image.getName());
}
}
catch(Exception e) {
log.error(e.getMessage(), e);
}
return null;
}
FtpService source code:
public byte[] downloadFtpFile(String filePath) throws IOException {
FTPClient client = new FTPClient();
try {
client.connect(host, port);
if(!client.login(username, password)) {
throw new AdminException("Invalid ftp username/password");
}
client.enterLocalPassiveMode();
try(ByteArrayOutputStream outputStream = new ByteArrayOutputStream()) {
client.retrieveFile(filePath, outputStream);
return outputStream.toByteArray();
}
}
catch(Exception e) {
log.error(e.getMessage(), e);
}
finally {
if(client.isConnected()) {
client.logout();
client.disconnect();
}
}
return null;
}
Thanks in advance!
If you've not set the FTP transfer to be binary (as opposed to ASCII) it will "convert the line endings" (or what it thinks are line endings) which will corrupt the picture.
I implemented java code to upload files to server with org.apache.commons.net.ftp.FTPClient
For multiple files the ftp upload speed is very slow.
How can I improve the speed.
-Change library?
What is the powerful FTP client class library for uploading multiple files?
-Use multiple threads?
How can I implement ftp upload function with multiple thread?
Could someone show me an example?
I am a new for multiple threading programming.
After I read all answer, I try to change my code and test it.
Following is a sample FTPClient code:
// create instance of FTPClient
FTPClient ftp = new FTPClient();
ftp.setControlEncoding("UTF-8");
ftp.setDefaultTimeout(30000);
// connect to server
try
{
ftp.connect("10.1.1.1", 990);
}
catch(Exception e)
{
System.out.println("Cannot connect to server");
return;
}
// login to server
if (!ftp.login("username", "password"))
{
ftp.logout();
System.out.println("Cannot login to server");
return;
}
try
{
ftp.setFileTransferMode(FTP.BINARY_FILE_TYPE);
ftp.enterLocalPassiveMode();
// ftp.setBufferSize(0); <-- someone suggest me to set buffer size to 0, but it throw error sometime.
}
catch(Exception e)
{
}
// create directory on server
// dirs is list of required directories on server
for (String dir : dirs)
{
try
{
ftp.makeDirectory(dir);
}
catch(IOException e)
{
}
}
// files is a map of local file and string of remote file
// such as
// file on client is "C://test/a.txt"
// location on server is "/test/a.txt"
for (Map.Entry<File, String> entry : files.entrySet())
{
File localFile = entry.getKey();
String remoteFile = entry.getValue();
FileInputStream input = null;
try
{
input= new FileInputStream(localFile);
ftp.storeFile(remoteFile, input);
}
catch (Exception e)
{
try
{
ftp.deleteFile(remoteFile);
}
catch (IOException e1)
{
}
}
finally
{
if (input != null)
{
try
{
input.close();
}
catch (IOException e)
{
}
}
}
}
// disconnect
if (ftp != null && ftp.isConnected())
{
try
{
ftp.disconnect();
}
catch (IOException f)
{
// do nothing
}
}
When I uploaded 1050 files (each file is about 1-20 KB), it took about 49406 - 51000 millisec (This is upload time only).
I would like to improve the speed.
Some people suggest me to use ftp4j, but when I test the library with 1050 files, the upload speed of ftp4j is slowly than FTPClient about 10000 millisec.
it took about 60000 millisec.
Following is sample ftp4j code:
// create instance of FTPClient
FTPClient ftp = new FTPClient();
ftp.setCharset("UTF-8");
// connect to server
try
{
ftp.connect("10.1.1.1", 990);
}
catch(Exception e)
{
System.out.println("Cannot connect to server")
return;
}
// login to server
try
{
ftp.login("username", "password");
}
catch (Exception e)
{
try
{
ftp.logout();
}
catch (Exception e1)
{
}
System.out.println("Cannot login to server")
return;
}
try
{
ftp.setType(FTPClient.TYPE_BINARY);
ftp.setPassive(true);
}
catch(Exception e)
{
}
// create directory on server
// dirs is list of required directories on server
for (String dir : dirs)
{
try
{
ftp.createDirectory(dir);
}
catch (Exception e)
{
}
}
// files is a map of local file and string of remote file
// such as
// file on client is "C://test/a.txt"
// location on server is "/test/a.txt"
for (Map.Entry<File, String> entry : files.entrySet())
{
final File localFile = entry.getKey();
final String remoteFile = entry.getValue();
BufferedInputStream input = null;
boolean success = false;
try
{
input = new BufferedInputStream(new FileInputStream(localFile));
// ftp.upload(localFile); <-- if I use ftp.upload(File), it will took more time.
ftp.upload(remoteFile, input, 0, 2048, new MyTransferListener());
success = true;
}
catch (Exception e)
{
}
finally
{
if (input != null)
{
try
{
input.close();
}
catch (IOException e)
{
}
}
if (!success)
{
try
{
ftp.deleteFile(remoteFile);
}
catch (Exception e)
{
}
}
}
}
// disconnect
if (ftp != null && ftp.isConnected())
{
try
{
ftp.disconnect();
}
catch (IOException f)
{
// do nothing
}
}
I try to use multiple threads.
Following is multiple threading code:
final CountDownLatch latch = new CountDownLatch(files.size());
ExecutorService pool = Executors.newFixedThreadPool(10);
for (Map.Entry<File, String> entry : files.entrySet())
{
final File localFile = entry.getKey();
final String remoteFile = entry.getValue();
pool.execute(new Runnable() {
public void run()
{
FileInputStream input = null;
try
{
input= new FileInputStream(localFile);
ftp.storeFile(remoteFile, input);
}
catch (Exception e)
{
try
{
ftp.deleteFile(remoteFile);
}
catch (IOException e1)
{
}
}
finally
{
if (input != null)
{
try
{
input.close();
}
catch (IOException e)
{
}
}
latch.countDown();
}
}
});
}
try
{
// waiting for all threads finish
// see: http://stackoverflow.com/questions/1250643/how-to-wait-for-all-threads-to-finish-using-executorservice
latch.await();
}
catch(Exception e)
{
}
Is it correct? It work correctly but it cannot improve speed.
it took about 49000 - 51000 millisec same as the code without thread.
I test the speed with intranet. It will take more time for internet.
How should I do for improve upload speed?
I don't know why, but Apache Commons FTP is pretty slow in uploading, i had the same problem and i couldn't solve it.
Now i use FTP4j, it's pretty similar to apache commons ftp but uploads are really fast.
This is an example:
FTPClient client = new FTPClient();
client.connect("www.yoursite.com");
client.login("login", "password");
client.setPassive(true);
client.setType(FTPClient.TYPE_BINARY);
client.changeDirectory("a");
File f = new File("path/to/your/file");
client.upload(f);
client.disconnect(true);
With this library i uplodaded a 340KB file in less than one second, while with Apache Commons FTP it took about 1 minute.
If you want to transfer different files with threads, try to put every client.upload(f) into a different thread, but i'm not sure it will boost up the transfer.
Quoting #fge previous answer:
Basically, chances are, you can't.
Don't forget that FTP has two types of channels: the command channel and the data channels. One upload is initiated by sending the instructions over the command channel to open a data channel for the upload proper.
Now:
most FTP servers are configured so that one command channel can only open one data channel at anyone time;
there are bandwidth limits: your upstream bandwidth, and the server's downstream bandwidth.
Were it possible to upload several files in parallel, ie, opening more than one data channel, you'd have the problem that the overhead of TCP itself would in fact slow down the upload process in general.
Basically: keep one data channel open at any time. Trying and opening more than one is just not worth it. It may work in ~1% of cases in general. This is just not worth the hassle.
This Q&A some possible explanations of what is going on: why is ftp upload slow in java 7
And furthermore, it offers a couple of workarounds:
Upgrade to the 3.3 snapshot which can (currently) be found here
Call FTPClient.setBufferSize(0).
Apparently there is also a regression in Java 7 for Windows where the firewall application filter for the FTP client is blocking the client from using PASV mode FTP. It is not clear what the best solution to that is, but you could try the following:
Change your Windows firewall to disable the firewall application filter (as described in the Microsoft KB page.
Change your FTP application to use "active" mode ... though this requires that the FTP server can initiate connections to the machine running your clients.
Note: that there seem to be more than one explanation for the problem ... or maybe more than one possible problems.
I'm using ftp4j as FTP client.
FTPClient client = new FTPClient();
client.connect("86.22.11.178");
client.login("usr", "pwd");
client.changeDirectory("/dir");
client.upload(file);
It works fine at localhost, but it does not work when enclosed in a JSF web application deployed on a web server. I succeeded to do connect and login, when the code reaches to the upload command, it just skips on that and does nothing. No exception is been thrown.
There is full conectivity to the FTP server, it can't be a problem. I have also set chmod 777 permission on the files and they belong to the same owner.
This code worked on a Windows machine, could it be that machines running on Linux have different "rules"?
Your code seems to be correct. Try to find out the FTP error which its throws. Sometimes timeout may happens, which i faced!!!
import org.apache.commons.net.ftp.;
import java.io.;
/**
* This class is used to demonstrate the usage of the Jakarta Commons Net package
*/
public class TestFTP {
/** Creates a new instance of TestFTP */
public TestFTP() {
}
/**
* main - Unit test program
* #param args Command line arguments
*
*/
public static void main(String args[]) {
try {
String ftpHost = "157.227.38.131";
String ftpUserName = "firebird";
String ftpPassword = "tcs#12345";
String ftpRemoteDirectory = "/etc/vlp/uploaded_files";
String fileToTransmit = "c:\\temp\\VLPDYN18022010174439.an";
//Create a Jakarta Commons Net FTP Client object
FTPClient ftp = new FTPClient();
//A datatype to store responses from the FTP server
int reply;
//
//Connect to the server
//
ftp.connect(ftpHost);
//
// After connection attempt, you should check the reply code to verify success.
//
reply = ftp.getReplyCode();
if(!FTPReply.isPositiveCompletion(reply)) {
try {
ftp.disconnect();
} catch (Exception e) {
System.err.println("Unable to disconnect from FTP server " +
"after server refused connection. "+e.toString());
}
throw new Exception ("FTP server refused connection.");
}
System.out.println("Connected to " + ftpHost + ". "+ftp.getReplyString());
//
//Try to login
//
if (!ftp.login(ftpUserName, ftpPassword)) {
throw new Exception ("Unable to login to FTP server " +
"using username "+ftpUserName+" " +
"and password "+ftpPassword);
}
System.out.println(ftp.getReplyString());
System.out.println("Remote system is " + ftp.getSystemName());
//
//Set our file transfer mode to either ASCII or Binary
//
//ftp.setFileType(FTP.ASCII_FILE_TYPE);
ftp.setFileType(FTP.BINARY_FILE_TYPE);
//
//Change the remote directory
//
if (ftpRemoteDirectory != null && ftpRemoteDirectory.trim().length() > 0) {
System.out.println("Changing to FTP remote dir: " + ftpRemoteDirectory);
ftp.changeWorkingDirectory(ftpRemoteDirectory);
reply = ftp.getReplyCode();
if(!FTPReply.isPositiveCompletion(reply)) {
throw new Exception ("Unable to change working directory " +
"to:"+ftpRemoteDirectory);
}
}
//
//Get the file that we will transfer and send it.
//
File f = new File(fileToTransmit);
System.out.println("Storing file as remote filename: " + f.getName());
boolean retValue=true;
try{
retValue = ftp.storeFile(f.getName(), new FileInputStream(f));
}catch(Exception e){e.printStackTrace();}
if (!retValue) {
throw new Exception ("Storing of remote file failed. ftp.storeFile() returned false.");
}
//Disconnect from the FTP server
//
try {
//ftp.logout();
ftp.disconnect();
} catch (Exception exc) {
System.err.println("Unable to disconnect from FTP server. " + exc.toString());
}
} catch (Exception e) {
System.err.println("Error: "+e.toString());
}
System.out.println("Process Complete.");
System.exit(0);
}
}
How can I parse URL queries with a system like this.
For Example something like get these URL arguments in variables.
http://localhost?format=json&apikey=838439873473kjdhfkhdf
http://tutorials.jenkov.com/java-multithreaded-servers/multithreaded-server.html
I made these files
WorkerRunnable.java
package servers;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.IOException;
import java.net.Socket;
/**
*/
public class WorkerRunnable implements Runnable{
protected Socket clientSocket = null;
protected String serverText = null;
public WorkerRunnable(Socket clientSocket, String serverText) {
this.clientSocket = clientSocket;
this.serverText = serverText;
}
public void run() {
try {
InputStream input = clientSocket.getInputStream();
OutputStream output = clientSocket.getOutputStream();
long time = System.currentTimeMillis();
output.write(("HTTP/1.1 200 OK\n\nWorkerRunnable: " +
this.serverText + " - " +
time +
"").getBytes());
output.close();
input.close();
System.out.println("Request processed: " + time);
} catch (IOException e) {
//report exception somewhere.
e.printStackTrace();
}
}
}
MultiThreadedServer.java
package servers;
import java.net.ServerSocket;
import java.net.Socket;
import java.io.IOException;
public class MultiThreadedServer implements Runnable{
protected int serverPort = 8080;
protected ServerSocket serverSocket = null;
protected boolean isStopped = false;
protected Thread runningThread= null;
public MultiThreadedServer(int port){
this.serverPort = port;
}
public void run(){
synchronized(this){
this.runningThread = Thread.currentThread();
}
openServerSocket();
while(! isStopped()){
Socket clientSocket = null;
try {
clientSocket = this.serverSocket.accept();
} catch (IOException e) {
if(isStopped()) {
System.out.println("Server Stopped.") ;
return;
}
throw new RuntimeException(
"Error accepting client connection", e);
}
new Thread(
new WorkerRunnable(
clientSocket, "Multithreaded Server")
).start();
}
System.out.println("Server Stopped.") ;
}
private synchronized boolean isStopped() {
return this.isStopped;
}
public synchronized void stop(){
this.isStopped = true;
try {
this.serverSocket.close();
} catch (IOException e) {
throw new RuntimeException("Error closing server", e);
}
}
private void openServerSocket() {
try {
this.serverSocket = new ServerSocket(this.serverPort);
} catch (IOException e) {
throw new RuntimeException("Cannot open port 8080", e);
}
}
}
Dispatch.java
package servers;
public class Dispatch {
/**
* #param args
*/
public static void main(String[] args) {
MultiThreadedServer server = new MultiThreadedServer(9000);
new Thread(server).start();
try {
Thread.sleep(20 * 1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Stopping Server");
server.stop();
}
}
You're doing fine so far.
Read the data off of the InputStream (BufferedReader might help) one line at a time.
Read and learn the HTTP Protocol (see Request Message section here: http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol).
The first line that the client sends is going to follow that format: GET /foo.html?x=y&a=b HTTP/1.1 followed by \n\n that's the Method, URL (with query parameters) and Protocol. Split that line (on the spaces...) and then break the URL up according to the specs.
Everything you need can be found in the String class for parsing the data.
You have forgotten to read what the clients sends. In http the clients opens the connection and than sends the request and waits for the server to reply.
To read the request you have two options. Use a BufferedReader or read it byte by byte.
The BufferedReader is easier. You get a String for every line and can easily split it or replace characters, or whatever ;)
Reading every byte is a little bit faster, but it will only be relevant if you need to serve a huge amount of request per seconds. Than this can really make a difference. I just put this information just so you know ;)
I have included the necessary part for reading in your WorkerRunnable.java.
This reads and prints out the whole client request.
Start your server, open your browser and type: http://127.0.0.1:9000/hello?one=1&two=2&three=3
The First line on the Console will read: GET /hello?one=1&two=2&three=3 HTTP/1.1
Before closing an OutputStream, be sure to call the flush() method. This will force any buffered bytes to be written out. If you don't do it, than there might be some bytes/characters missing and you might be spending a long time looking for the error.
try {
InputStream input = clientSocket.getInputStream();
// Reading line by line with a BufferedReader
java.io.BufferedReader in = new java.io.BufferedReader(
new java.io.InputStreamReader(input));
String line;
while ( !(line=in.readLine()).equals("") ){
System.out.println(line);
}
OutputStream output = clientSocket.getOutputStream();
long time = System.currentTimeMillis();
output.write(("HTTP/1.1 200 OK\n\nWorkerRunnable: " +
this.serverText + " - " +
time +
"").getBytes());
output.flush();
//Flushes this output stream and forces any buffered output bytes to be written out.
output.close();
input.close();
System.out.println("Request processed: " + time);
I don't know exactly what you are doing there. You just told us you need to parse the URL, but maybe a better way is to use the simpleframework (http://www.simpleframework.org)
It is like an embedded HTTP-Server, you can look at the tutorial. It will give you a request object, from there you can easily fetch the parameters in the url.
Technically speaking, you can, but it would leave you with implementing the http protocol on your own.
A much better option would be to use the Java Http Server from Oracle. See the following article for tips http://alistairisrael.wordpress.com/2009/09/02/functional-http-testing-with-sun-java-6-httpserver/