Develop Reference

java httprequest fetch the content of page to wait until javascript is done executing

I have a piece of Java code for getting the content of a webpage. The partial content of the webpage will be loaded/retrieved by a javascript once the page is loaded. That is, the page contains a javascript which will load the rest of the content. In Java, I want to wait until that partial content is loaded before start the rest of process. Is there anyway to do that?

Java is typically a server-side technology while JavaScript is a client-side technology. So if you want Java to wait for something on the client, it can't, because typically Java's job is to send something static (e.g. HTML) to the client. That being said, Java can render JavaScript code that will be run by the browser, and that JavaScript code can continue the process.

You can use AJAX to request the rest of the content from the server once the javascript processing is done.

How PHP script can be called using Java Applet

I want to access user's scanner using PHP script. I know this can be done by using PHP script on Linux based machine.
I want to put this script on the server and wanted to run it via Java Applet I don't know Java much also I am not sure if this task can be done by this way.
Please suggest if this is correct way and How to proceed on it.
Thanks for your time!

Can the scanner PHP code be operated purely via parameters in the URL? If so, then a Java applet could connect to those URLs passing parameters as necessary, and if the PHP script returns information in the HTML output stream (as if it were returning a web page), then the applet can parse the page it 'downloads' from that URL to get the information out. Not elegant, but do-able.

how to download 'generated source' from web developer toolbar in firefox

I want my java program to see the 'generated source' of a webpage, in Web Developer Toolbar: https://addons.mozilla.org/en-US/firefox/addon/web-developer/
in FireFox, found under the 'view source' menu, as opposed to simply the actual html source which regularly returns itself through java networking:
HttpURLconnection.getInputStream();
Can a java program do this, or at least delegate the task to another application on the same computer, written in something else (javascript) which gets embedded in the browser?

selenium should be able to do that. i used it a long time ago so i don't remember how exactly. but it's basically a browser plugin and some server code which communicates with the plugin. you can communicate via a java driver with the server and control the browser content and also get all the data from the DOM.
EDIT:
depending if a "real" browser is not necessary you can also use htmlunit which is basically a gui less browser in java.

If by "generated source", you mean the full DOM of a working web page, including elements that have been added, removed or modified by javascript in that page, then there is no way to do this without using a full browser engine to first render the page and then some sort of communication with that page or engine to give you the HTML for the generated page.
You could not do this with java alone.
You could put javascript in the web page itself which would fetch the innerHTML of the whole web page after it was fully generated and then use an ajax call to send that to your server. You would have to stay within the limitations of the same-origin-policy (which doesn't allow you to make ajax calls to domains other than where the host web page came from).
You could also find some server-side rendering engine that could do the same on the server side that your java application could use/communicate with.

How to use wkhtmltopdf in Java web application?

I am newbie in wkhtmltopdf. I am wondering how to use wkhtmltopdf with my Dynamic Web Project in Eclipse? How to integrate wkhtmltopdf with my Java dynamic web application?
Is there any tutorials available for beginners of wkhtmltopdf ?
(Basically, I would like to use wkhtmltopdf in my web application so that when user click a save button , the current page will be saved to PDF file).

First, a technical note: Because you want to use wkhtmltopdf in a web project, if and when you deploy to a Linux server machine that you access via ssh (i.e. over the network), you will need to either use the patched Qt version, or run an X server, e.g. the dummy X server xvfb. (I don't know what happens if you deploy to a server running an operating system other than Linux.)
Second, it should be really quite simple to use wkhtmltopdf from any language in a web project.
If you just want to save the server-generated version of the current page, i.e. without any changes which might have been made like the user filling on forms, or Javascript adding new DOM elements, you just need to have an extra optional argument like ?generate=pdf on the end of your URL, which will cause that page to be generated as a PDF, and then the PDF button will link to that URL. This may be a lot of work to add to each page manually if you are just using simple JSP or something, but depending on which web framework you are using, the web framework may offer some help to implement the same action on every page, if you need to implement that.
To implement this approach, you would probably want to capture the response by wrapping the response object and overridding its getWriter() and getOutputStream() methods.
Another approach is to have a button "submit and generate PDF" which will generate the next page as a PDF. This might make more sense if you have a form the user needs to fill in - I don't know. It's a design decision really.
A third approach is to use Javascript to upload the current state of the page back to the server, and process that using wkhtmltopdf. This will work on any page. (This can even be used on any site, not just yours, if you make it a bookmarklet. Just an idea that occurred to me - it may not be a good idea.)
A fourth approach is, because wkhtmltopdf can fetch URLs, to pass the URL of your page instead of the contents of the page (which will only work if the request was a HTTP GET, or if it's equivalent to a HTTP GET on the same URL). This has some small amount of overhead over capturing your own response output, but it will probably be negligible. You will also very likely need to copy the cookie(s) into a cookie jar with this approach, since presumably your user might be logged in or have an implicit session.
So as you can see there are quite a lot of choices!
Now, the question remains: when your server has the necessary HTML, from any of the above approaches, how to feed it into wkhtmltopdf? This is pretty simple. You will need to spawn an external process using either Runtime.getRuntime().exec(), or the newer API called ProcessBuilder - see http://www.java-tips.org/java-se-tips/java.util/from-runtime.exec-to-processbuilder.html for a comparison. If you are smart about it you should be able to do this without needing to create any temporary files.
One of the wkhtmltopdf websites is currently down, but the main README is available here, which explains the command line arguments.
This is merely an outline answer which gives some pointers. If you need more details, let us know what specifically you need to know.

Additional info:
If you do end up trying to call wkhtmltopdf in an external process from java (or for that matter, any language), please note that the "normal" output that you see when using wkhtmltopdf from the command line (i.e. what you would expect to see in STDOUT) is not not in STDOUT but in STDERR. I raised this issue in the project page
http://code.google.com/p/wkhtmltopdf/issues/detail?id=825
and was replied that this is by design because wkhtmltopdf supports giving the actual pdf output in STDOUT. Please see the link for more details and java code.

java-wkhtmltopdf-wrapper provides an easy API for using wkhtmltopdf in Java.
It also works out-of-the-box on a headless server with xvfb.
E.g., on a Ubuntu or Debian server:
aptitude install wkhtmltopdf xvfb
Then in Java:
Pdf pdf = new Pdf();
pdf.addPage("http://www.google.com", PageType.url);
pdf.saveAs("output.pdf");
See the examples on their Github page for more options.

How can I open a webpage within a Java app and run my own javascript code

I would like to open a webpage and run a javascript code from within a java app.
For example I would like to open the page www.mytestpage.com and run the following javascript code:
document.getElementById("txtEmail").value="test#hotmail.com";
submit();
void(0);
This works in a browser...how can I do it programatically within a java app?
Thanks!

You can use Rhino to execute JavaScript but you won't have a DOM available - i.e. document.getElementById() would work.
You can use HTML Unit (headless) or WebDriver/Selenium (Driving a browser) to execute JavaScript in an environment that has a DOM available.

I'm not sure what you are looking for but I assume that you want to write automated POST request. This can be done in with Http Client library. Only you have to set appropriate request (POST or GET) parameters.
Look at examples - with this library you can do basic authentication or post files too.

Your question is a bit ambiguous, as we don't know the position of the Java program.
If that's a Java applet inside your page, you should look at Java<->JavaScript interaction, it works well.
If you need a separate Java program to control a browser, like sending a bookmarklet in the address bar (as one of your tags suggests), it is a bit harder (depends on target browser), perhaps look at the Robot class.

There's Rhino JS engine written in Java that you can run on app server such as Tomcat and feed JS to, however - it's not clear what are you trying to do with this?
There's also Envjs simulated browser environment which is based on Rhino but complete enough to run jQuery and/or Prototype

DWR (and other frameworks) now support "reverse ajax." The general idea is that you use one of three methods to communicate back to the client:
Comet (long-lived https session)
Polling
opportunistic / piggy-back (i.e. next time a request comes from the client, append your js call)
Regardless of method (which is typically a configuration-time decision and not a coding issue), you will have full access to any/all js calls you want to make.
Check out the reference page from DWR to get a pretty good explanation.