I have to find a solution for uploading large content (10GB+) to a remote server. The uploader should be managed by a web page.
I didn't find any reliable solutions (or at least concepts) while searching the internet, so I came up with these scenarios. Can you provide any kind of feedback?
At the moment the common solutions in the market:
Http upload with post form (deprecated by speed and content size);
Flash uploader - create a ftp client, and deliver the content to the server;
Flash uploader - socket based transfer - treat the file as an InputStream and sent it through a socket opened with flash.
Java applet uploader - use Apache's ftp client, and send the content with ftp.
Java network socket - use the JRE to open a socket and send the content through a socket.
There is this other solution of using a c++ ftp client, that will be called from the Java Applet and use the c++ speed advantage (compared to java) to deliver the content.
Which one should be more efficient to deliver the content? Is there any other technology for doing what I'm looking for?
The limiting factor on the speed of upload will almost inevitably be the network - not the language you implement it in.
Do not use FTP if you can possibly avoid it.
Invoking an ftp client from an applet regardless of which client it is nor what language you imlpement the applet in is going to have to work outside the sandbox (which is not a trivial problem to work around). Why does it have to be implemented via a web page?
If you've got any control over the client being used, then websockets are probably the most sensible solution - but support is still patchy in browsers. There are lots of flash uploaders already written - and a few java ones.
Related
im Trying to Write a Windows client-server Version control application .
i've created two servers , one works with java socket ( java.net library ) to handling the requests ( login, signup and ... ) coming from client . and the other server is running on FTP protocol (using apache common net library) for serving my files .
and client is able to communicate with the first server over tcp socket and download or upload files to the second server using FTP .
but recently someone just told me that i should use HTTP instead of both . because HTTP in java in really easy to use for both communicating and FILE serving and most importantly is its able to traverse the NAT which now what im using is not able to do .
now im wondering is he right ? should i change my servers to use HTTP instead of TCP socket and FTP ? whats the benefit ?
There's some advantages of changing your protocol stack to HTTP:
You can easily add security later (only a matter of a single 's')
You don't have to do two servers, you can do all-in-one.
At some point you could offer a browser-based access of you don't have the client installed / work of a decive where you cannot install it
HTTP webapps (even those in java) are proven to scale very, very well. So once you have a lot of users, you're still good to go.
There's a lot of helpful frameworks out there that can help you focus on the what instead of the how
Most companies that allow outside-access at all will have HTTP/HTTPS open. FTP is more limited in most places.
NAT traversal / Proxy traversal
and that's only the ones I came up with while typing :-)
Downsides:
You have to start over. But: If you run into trouble, Stack Overflow is there to help you out.
Is it possible to read datastream sent from C++ server program to C++ client over socket connection in java? I have details like port number and server IP.
Or do I need decompile the whole C++ client into Assembly and then somehow translate it into java to do that?
I'm really not sure what kind of data it's transforming, though.. Somebody told me to code HTTP server and run it on my Router but I'm not really sure if that would work?
Here’s the diagrammatic way to look at it.
Server generates data.
it puts it in a packet.
it encrypts the packet.
and sends it over the wire.
It gets to a user’s Computer (= client). (I should be in the control now on..)
(If I could somehow read data at this part?)
The client reads the encrypted packet.
(If I could somehow read data at this part?) (The later, the better :D)
The client decrypts the packet.
(If I could somehow read data at this part?) (The later, the better :D)
The client does something.
As said, the client is .exe file and it's coded using C++. And I don't have source code of it.
All you have to do is define well your application protocol. This is, the format of your data stream. As long as you are using the same format in both ends, it doesn't really matter what language or program you are using. Imagine your browser and the web server. They are both using the same application protocol (HTTP) but they are completely different programs. Even more, there exists different web servers and different browsers.
Then, all you need to do is use the java sockets to listen to some specific port, and use your c++ sockets to write to the specific port. Just make sure you know how the information is "organized".
I am practicing some web-apps technology. I setup Apache HTTP server which provides HTTP page with JNLP/FlashFX file with simple data-form to fill by the user. My first idea was to send/receive the data from JNLP with help of UDP (I just serialize object inside datagrams). To be more specific:
Apache provides a static HTTP and JNLP/FlashFX (HTTP is just to deploy JNLP)
JNLP communicates via UDP with server
server runs simple Java program to send/receive UDP packets to JNLP
My problem is when I access the page from 'client' browser machine a firewall asking if I want to allow/deny the access to network from 'java'. No doubts this is normal, but I think no one is expecting this from a web page.... I want to change this approach and use existing HTTP protocol.
QUESTION UPDATE
As far as I understood with HTTP protocol we have couple of methods which are used to communicate with the server (GET, PUT, POST .... ) and provided by Apache service.
I would like to use this for data exchange like this:
JNLP sends some serialized data with HTTP methods
Apache will redirect some (or full) traffic to my Java program
My Java program will answers via Apache to my JNLP
How can I do that?
I believe build a small home-made program to make ourselves more comfortable is quite common nowadays. Just few days before, I really tired to get the same named log files from different remote devices through FTP connection again and again so that I started to build one Java web application.
The purpose of the Java web application is simple as once the user filled in the absolute path of source file in remote device and selected corresponding remote devices he or she want to connect to, the web application will finally store those same named log files in user's local computer with well organized folder structure. You can simply understand that this Java servlet is a proxy sits between client and remote devices.
Currently, I have already done and tested the downloading function from remote devices to the server in Java servlet by using Apache common net FTPClient library. It worked fine and provided me the copies of same named log files in a well organized folder structure.
However, when I moved on, I realized that the "pushing" function maybe the killer. Following are few queries I want to discuss with you all:
Even I could get IP address or host name from client's requests, is it possible or suitable for me to auto establish a FTP connection from servlet to client?
If an auto FTP connection is achievable, what are the security concerns I should pay attention?
If an auto FTP connection is not achievable, is it possible or suitable for me to return those files in the response to the client?
I appreciate your comments or suggestions. Hope you all also enjoy the open discussion here.
Can I write a socket programming to provide services to the web clients?
I did it using servlet, but I want to do it using java.net API.
Please give me a sample code of some program, such that I can access that simply by mentioning URL at the address bar of any web browser.
I suggest you look at the source for jetty. It is the simplest web server I can think of. If you want an ultra basic web server, you can do this with plain sockets, however the HTTP protocol is quite complex and using a web server library to handle all the details is likely to be the best approach.
If you want to be able to receive requests typed into a web browser, you need to do a couple things.
-Set the socket to listen on port 80
-Receive/parse/process HTML requests
-Return an HTML response across the socket
Rather than write the code for you, here is some pseudocode
//setup socket on port 80
socket.lisen();
while(true)
{
newsocket = socket.accept();
new thread(process(newsocket));
}
The most complicated part, I think, will be handling the HTML, processing the request, and generating a response. After that, just send it back over the socket.
Considering how many libraries are out there for this sort of thing, I wouldn't reccommend writing one from scratch.
The problem is that "web client" is just a browser, so you don't have direct access to TCP/IP.
Few options:
HTML5 WebSockets (only modern browsers)
flash helper (there are javascript wrappers)
java applet helper (there are javascript wrappers)
some tricks based on ajax pooling