I am currently beginning the development of a (UI?) backup from a Webplatform. It is not our platform and I don't have access to the source.
I just have the HTML-rendered view of the Form-Data of the elements I entered.
So the task is to browse to the HTML, store the data (XML/JSON) and then login to the site to fill out the forms again to resubmit the data...
At the moment I'm prototyping with C++ QtWebEngine.
What' the best way to do such a task? What are good frameworks for "browsing" the web and analysing HTML?
Solutions in c++/java/javascript (or a firefox-addon?) are preferred.
Thanks for your help!
same as DSL language interpreter use "Document Object Model (DOM)"
my advice : C# webform app and webbrowser control:
webbrowser.navigate([url])
WebBrowser.DocumentCompleted Event
WebBrowser.Document (read document and help about "System.Windows.Forms.HtmlDocument" )
maybe need inject some java script in
/*
please don't use this info for hack and attack
*/
You could definitely do something like this using Firefox's Addon SDK. In particular you should look into the PageWorker module that allows you to load and run JS code against web pages without showing the page - everything happens in the background.
I need to write some code in Java to upload a file(.tz format) to a URL(prqs#vks.com) by using HTTP PUT/POST and SMTP.I have no idea how to do this and my application is not a web based application i have to write a pure java client to do this. I searched the net but could not find a suitable tutorial on how to do this also i found a lot of stuff where in they are uploading file using a HTML form to a server. but that is not what i want.
Can you please let me know how this is to be done. If you can direct me to a tutorial or provide any sample code then it would be very helpful. I am scratching my head on this for a long time now.
Thanks
vikeng21
I do this all the time from a server side Java Application.
If what your asking for is creating an e-mail with an attached file in .tz format then there are some good resources out there.
I personally used the JavaMail API. It comes with attached JavaDoc and examples. Take a look at the samples in the demo directory within. Hopefully it should help you out.
Java Mail API
You can try out Apache Commons Fileupload for uploading files.
Have a look at this example as well. This might work for you
I am developing a new report engine for my company, our main web application is writing by PHP 5 and running on an Apache server. The new report server which I designed to use Apache FOP 1.0 ruing on Tomcat7,so far I created My own XSLT stylesheets and I can manage to generate a PDF report from the URL and let it display in my web browser.My goal on this project is to generate dynamic report from xml file which export from the PHP application.
Could anyone explain to me the logic behind How I can give those two talk to each others. I understand I could POST the XML and XSLT as string to the report server and POST it back the result I want(like PDF).
I know this will need Java involved,and as a PHP dev. I really don't have much background about Java,if you could show me some examples or links, it would be much appreciate.
I am using Saxon-B with Fop 1.0 on Tomcat 7 for the report server.
if you need more info from me,please also leave a comment. I will add it soon.
Thanks
There are two things your going to need to study.
http://www.w3schools.com/php/php_ajax_intro.asp This is a tutorial on using the XHR object from php. You should go through that. Also, if you aren't familiar with the XHR object, I would read this as well: http://www.w3schools.com/XML/xml_http.asp
You're going to have to create a servlet capable of responding to these requests. There are various tutorials on how to create a servlet.
Here is a link to a youtube video I fall back on when I forget how to do servlets: http://www.youtube.com/watch?v=EOkN5IPoJVs
In the video, he just uses straight up javascript; so you're php code is going to be doing what the javascript did in the video (item 1 on my list).
The video uses an older version of eclipse (I think ganymede), but it is pretty close to the same process in the newest version of eclipse.
Just to be blunt, this isn't something you're going to be able to figure out and make work properly. You've probably got at least a day to a weeks worth of learning here depending on your experience and aptitude; and this will only get you something basic. The servlet you make won't be secure, it won't have user authentication...my point is don't take anything for granted.
Using Matlab, I'm trying to download a file from a website that requires you to login first. The file I'm trying to download is here (http://www.standardandpoors.com/prot/spf/docs/indices/SPUSA-500-USDUF--P-US-L--Constituents.xls)
I'm guessing this involves cookies and hacking into the java code in the urlwrite function. Has anybody done this or does anybody have ideas on how I get started? Is there some standard way to login and get a cookie, and then pass the cookie back with future requests?
I would think system, rather than tool. I agree with Alex - Perl to download, maybe Pentaho/Kettle to parse/process. Especially if you're going to do more of this.
I'm working on a web application. There is one place where the user can upload files with the HTTP protocol. There is a choice between the classic HTML file upload control and a Java applet to upload the files.
The classic HTML file upload isn't great because you can only select one file at a time, and it's quite hard to get any progress indication during the actual upload (I finally got it using a timer refreshing a progress indicator with data fetched from the server via an AJAX call). The advantage: it's always working.
With the Java applet I can do more things: select multiple files at once (even a folder), compress the files, get a real progress bar, drag'n'drop files on the applet, etc...
BUT there are a few drawbacks:
it's a nightmare to get it to work properly on Mac Safari and Mac Firefox (Thanks Liveconnect)
the UI isn't exactly the native UI and some people notice that
the applet isn't as responsive as it should (could be my fault, but everything looks ok to me)
there are bugs in the Java UrlConnection class with HTTPS, so I use the Apache common HTTP client to do the actual HTTP upload. It's quite big a package and slows down the download of the .jar file
the Apache common HTTP client has sometimes trouble going through proxies
the Java runtime is quite big
I've been maintaining this Java applet for a while but now I'm fed up with all the drawbacks, and considering writing/buying a completely new component to upload theses files.
Question
If you had the following requirements:
upload multiple files easily from a browser, through HTTP or HTTPS
compress the files to reduce the upload time
upload should work on any platform, with native UI
must be able to upload huge files, up to 2gb at least
you have carte blanche on the technology
What technology/compontent would you use?
Edit :
Drag'n'Drop of files on the component would be a great plus.
It looks like there are a lot of issues related to bugs with the Flash Player (swfupload known issues). Proper Mac support and upload through proxies with authentication are options I can not do without. This would probably rule out all Flash-based options :-( .
I rule out all HTML/Javascript-only options because you can't select more than one file at a time with the classic HTML control. It's a pain to click n-times the "browse" button when you want to select multiple files in a folder.
I implemented something very recently in Silverlight.
Basically uses HttpWebRequest to send a chunk of data to a GenericHandler.
On the first post, 4KB of data is sent. On the 2nd chunk, I send another 4K chunk.
When the 2nd chunk is received, I calculate the round trip it took between first and 2nd chunk and so now
the 3rd chunk when sent will know to increase speed.
Using this method I can upload files of ANY size and I can resume.
Each post I send along this info:
[PARAMETERS]
[FILEDATA]
Here, parameters contain the following:
[Chunk #]
[Filename]
[Session ID]
After each chunk is received, I send a response back to my Silverlight saying how fast it took so that it can now send a larger
chunk.
Hard to put my explaination without code but that's basically how I did it.
At some point I will put together a quick writeup on how I did this.
I've never used it with files of 2GB in size, but the YUI File Uploader worked pretty well on a previous project. You may also be interested in this jQuery Plugin.
That said, I still think the Java Applet is the way to go. I think you'll end up with less portability and UI issues than you expect and Drag/Drop works great. For the record, Box.net uses a Java Applet for their multi-file quick uploads.
OK this is my take on this
I did some testing with swfupload, and I have my previous experience with Java, and my conclusion is that whatever technology is used there is no perfect solution to do uploads on the browser : you'll always end up with bugs when uploading huge files, going through proxies, with ssl, etc...
BUT :
a flash uploader (a la swfupload) is really lightweight, doesn't need authorization from the user and has a native interface which is REALLY cool, me thinks
a java uploader needs authorization but you can do whatever you want with the files selected by the user (aka compression if needed), and drag and drop works well. Be prepared for some epic bugs debuggin' though.
I didn't get a change to play with Silverlight as long as I'd like maybe that's the real answer, though the technology is still quite young so ... I'll edit this post if I get a chance to fiddle a bit with Silverlight
Thanks for all the answers !!
There are a number of free flash components that exist with nice multiple file upload capability. They make use of ActionScripts FileReference class with a PHP (or whatever) receiver on the server side. Some have recently broken with the launch of FP10 but I know for certain that swfupload will work :)
Hope this helps!
What about these two
Jupload
http://jupload.sourceforge.net/
and
jumploader
http://jumploader.com/
Both are java applets but they are also both really easy to use and implement.
what about google gears?
There are HTTP/HTTPS upload controls that allow multi-file upload. Here is one from Telerik, which I have found to be solid and reliable. The latest version looks to have most if not all of your feature requirements.
You can upload multiple files with HTTP forms as well, as Dave already pointed out, but if you're set on using something beyond what HTTP and Javascript offers I would heavily consider Flash. There are even some pre-existing solutions for it such as MultiPowUpload and it offers many of the features you're looking for. It's also easier to obtain progress information using a Flash client than with AJAX calls from Javascript since you have a little more flexibility.
You may check the Apache Commons FileUpload package. It allows you to upload multiple files, monitor the progress of the upload, and more. You can find more information here:
http://commons.apache.org/fileupload/
http://commons.apache.org/fileupload/using.html
Good luck