Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I want to write a program to save web site when we enter web site link. What is the easiest programming language to do that. I want to save entire web site to my computer. I know there is way to write a program to save web page. But my requirement is to save entire web site. How can i do it. I just need some tips. Then i can do some research and find out a solution. Please help me to get start with my work. Thanx.
What you are trying to create is actually a download manager. It is easier to create a simple download manager in java but quite tedious to create a full fledged one.
The idea behind it is simple. Say you have a webpage with url www.example.com/index.html. to download just index.html is easy. But to download all pages of a domain or website. You have to download index.html. Then parse index.html for links that are inside domain (ie within www.example.com).You need to download all the links, and then go through all pages downloaded from links and find more links. This goes on till you have parsed all links once. So essentially you would need to read a webpage,grab links and then download those links.You need to search info on web crawler,web page parsing etc.
If you are just trying to download a website please try softwares like flashget,internet download manager etc. There are some opensource once so you could get source as well.
Please go through the links below for more info
http://www.9code.in/java-download-manager-with-full-source-code/
http://www.javaworld.com/article/2076095/core-java/download-a-website-for-offline-browsing.html
http://www.programcreek.com/2012/12/how-to-make-a-web-crawler-using-java/
How to get a web page's source code from Java
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I want to scrape my website and then use the data from the website to populate elements in my app, my website has login pages and certain pages only open after the login has been done.
I started working with HtmlUnit as it is a headless browser and completed the custom api in a java IDE, later i tried to use the jar i generated from the java IDE and found that there are incompatibility issues with HtmlUnit and Android.
Can anyone propose a solution to this problem?
Edit :
Since no one actually answered this question I am currently going with a work around using android's native WebView, settings its Visibility to invisible and then using javascript interfacing to a Java object, I can inject JS code to scrape any data.
Use Jsoup library for such purpose. Very handy and easy to use.
Start with this answer and follow documents and other examples.
Either you contribute to HtmlUnit to produce a version of HtmlUnit not using the missing dependencies from Android.
Or you can use an alternative method like this one, as this seems to be the path someone else go before you.
If a real headless browser able to manage any recent web features, would exist, it would mean a team would have developed it and then invest much effort in it (in supporting existing and coming features) consistently.
Apart from Opera, Chrome, IE, and Firefox browsers, there is no such team.
I would point out Chromium (CEF) as the most open and actively supported cross language wise. Try Cef for java
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I'm trying to find the best way to add help to a Java EE web application. I am reading that .chm files are obsolete and shouldn't be used due to security issues in Windows. The site will only be accessed on a private LAN through a web browser.
It sounds kind of old school, but I was thinking that I would just write the HTML and have hyperlinks to jump to the section of the page where the relevant content is.
I'm wondering if just a regular old HTML page will work? Is this okay for enterprise? Are there any standards, or software programs that would help with a table of contents and list of help items?
The users will obviously need to be able to have quick and easy access to help, if they have questions on a particular page. Could I implement some type of tool-tips written in Java EE or otherwise that would prompt quicker than an HTML page would load?
You can have a HELP link throughout your application. When clicked it opens a plain browser window showing the help content. Plain browser window means no navigation buttons, this way users won't be using this window to continue their work which can be problematic. This will force them to simply close the window when they are done with it.
Help content can be plain HTML or dynamic (.jsp) depending on how complex the help structure is. Advantage of jsp is the content can be served dynamically from a database, and you can build the table of contents from server side, so more suited for larger help content.
The help link can be customised depending on what page you're on, so that users are taken directly to the relevant help. This can be done with query strings, for example help.html#abc or help.jsp?topic=abc to take users directly to the abc topic.
In addition to the full-blown help page(s), you could also have inline tooltips for individual fields for example. Add a small question mark icon next to the field label that when hovered or clicked will display a short information about that particular field.
A good example of this tooltip is what you can find next to the CVV field on various credit card payment pages.
Hope that helps.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
This is a problem I've faced recently.
I have a Java application which give the facility to create user accounts and then the user can upload files to the remote server which the application is hosted.
At some point the user can do the followings to edit that file.
Download the file
do the editing
upload the file (and it will replace the existing file or keep both)
The problem is the user have to upload it to the server again. How can I avoid that.
I mean to simplify the user experience.
I think below will be a good solution.
User click the file and it will open in the relevant file editor (MS Word, Note Pad, MS PowerPoint - assume the editors are installed already )
User do the editing and just click the save in the relevant editor. (say in MS word user edit the file and click save) - then the file in the remote server will automatically updated to the edited one.
I know this is not a programming code error or something , I'm just wondering how can we achieve this ? Is there any library or a technology for this.
I heard about webDav , and its implementation like Milton.
Is these can solve this ? or any of you know a way to achieve this. I need some ideas and thoughts how i can do this. please give your kind help.
Yes, you can do this by implementing webdav support in your webapp, and Milton.io is designed specifically to allow that.
There are tutorials on the milton website showing how to do it. Takes about 20 mins
http://milton.io/
To integrate click to edit into the web browser you need to use IE. Its possible to do the same in FF and Chrome, but requires addons to be installed.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I'm using JSP/Java and Spring MVC as framework. I'm going to support uploading PDF files to my site. The site will be uploaded to a free web hosting site.
I want to know which is the best way to support the uploading the PDF files:
Save it into database?
Save it to the web server ( if this is possible )
Save it to your recommendation ( please share what you have in mind )
And also give me a link or tutorial of how may I do this ( if you suggest other than option 1 and 2 )
Thank you in advance.
Saving file into database is not recommended and i am not sure if anyone would like to do that also.
Uploading file into web server is also not a good choice because it will increase the load unnecessary.
The best option is use a dedicated space (it may be cloud or anywhere), and secure it.
You can refer to spring file upload for reference.
If you are working in localhost for development then i would suggest just use the drive other than where your sever is running. And you can mention the path in web.xml or where ever you want.
Little is achieved by putting "large blobs of binary data" (such as PDFs) into a database. The only situation where this would be justified is when the data has to be handled transactionally.
If there is nothing to be achieved by putting the PDFs into the database, then don't do it. You should be able to upload and download large documents faster if they are stored as files in the file system.
How do you do it? Well there are a variety of ways. But WebDAV offers a simple "off-the-shelf" solution ... if that is what you are after.
Recommended way is always to use a File System storage service like Amazon S3.
This application on GitHub shows you exactly what you need using AmazonS3 but uploading an image instead of a .pdf. But I am sure that you will find the example really useful.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
What I want to do is, given a url to a csv file, I need build a web service that takes the url as a parameter, then download the file, parse data in the file and finally visualize the data in the file, The web service is supposed to be working on any given url pointing to a csv file. I'm trying to build the web service using JAX-RS right now. Any hints on how a little more detailed architecture that could work for this purpose? Thanks in advance!
This is a very broad question, so I will answer how you would do it. I've done this before with XML and NOAA Weather Prediction data.
Make sure the URL you are parsing from contains pure .csv/.xml data, e.g. NOAA data. For testing purposes, it's good idea to download the .csv directly from the site and write code doing IO on the .csv file, but for once you're done with that, it's a much better idea to just read directly from the URL. I'm not sure what it's like for REST protocols, but for SOAP, the URL contains input parameters, so you can specify everything from the longitude latitude of a location, to date ranges, etc.
Use a CSV parsing library, or find a tutorial, like such.
Store the parsed items. You could use a multi-dimensional array for testing, and transition to a database for long-term storage and processing.
Process/visualize the data. Once you have the data structure all set, you can then get creative with visualizing the data.
Go and read some tutorials, this is a very general question. Don't expect people to write all code for your service. here is a tutorial link http://docs.oracle.com/cd/E17802_01/webservices/webservices/docs/1.6/tutorial/doc/
and here an explanation about how to download a cvs file and save in java Programatically Downloading CSV Files with Java
here a nice tut about how to parse csv via csv http://www.mkyong.com/java/how-to-read-and-parse-csv-file-in-java/