Develop Reference

Parse JavaScript from html string in C#/Java

I am trying to do some WebScraping of a site and the data is in dynamically loaded containers. It appears that the loading is done via JavaScript. Therefore I would need to execute it, in order to get it. I have written 2 versions of the program, one in Java and one in C#, so helping me with one would be nice enough.
I am currently using WebResponse/WebRequest in C# and HttpURLConnection in Java. I have to login into the site first, which already works like a charm. Now I need to parse the content, so the data gets filled in and the containers loaded. Is there an easy way to run the html through a browser control or an already included library?

In C# you could just use a hidden WebControl. With that you could execute everything you need.

read data from a java applet (preferably with js)

Here's my problem. I have an application that shows some KPIs... the problem is that it displays those KPIs through a Java applet (and I don't have access to it's source). I need only a few data from that applet. Is there a way to parse data from a Java applet? I would like to parse it with Javascript. Would that be possible?

It's possible to call Java applet methods from JavaScript (read), but it would have to be set up like this already. If you don't have access to the source, perhaps inspect Java console output and/or use 'jar' and 'javap' commands to inspect the classes to see if there is something you can call.

Execute javascript function from java

I want to know if it's possible to execute a javascript function from java. Basically I have a javascript function to which I want to pass a string and then get the output from that function.
If it's possible, how do I accomplish it?
EDIT -
I couldn't find a solution to this, what I did was re-write all the javascript code in Java. But I am guessing, Rhino ported on Android should do the trick. If someone has tried it out, please post a solution.

You probably want to take a look at the ScriptEngine. There are plenty of samples out there on how to use it. Works on anything but Mac where they for some reason selected to include AppleScript instead of JavaScript by default.
Edit: Take a look at this page, there seems to be a Rhino port for Android out there.

Javascript is not natively supported in java. If you need it, you may implement the Rhino javascript engine to do this.
"Rhino is an open-source implementation of JavaScript written entirely in Java. It is typically embedded into Java applications to provide scripting to end users."

When JEditorPane will not do. JRex, JRex wherefore are thou JRex?

IDEA: Implement a recent web browser into a java application (for saved offline, non server content).
The question is this: can I have a java application implement a webbrowser with jquery / html / css support within a java program?
So I am asking anyone who has played with JRex for advice: I want to know how complicated will it be to integrate an open source webbrowser into java. I am not all that keen on the idea of compiling Mozilla from source build. Is there a ready made compiled version?
Is there a simplified method to have latest compiled version (most current in terms of support for HTML css & javascript), and integrate that into an application?
Also: I appreciate the amount of work required to support for HTML4 nevermind 5, and CSS2 compliance. How close is JRex to that?
Application: My intention with the webbrowser is to render a webpage from offline content. It will not need to be online content, and will simply be for file based displays = e.g. file:///C:...
Does the webbrowser have to be wrapped into a server to function, e.g. to pass files to the browser to render is how complicated? I am not keen to have to implement Jetty or another server type application just for this.
If JRex is not the solution... what then? Is it possible to start a browser implementation within Java and can Java interact with the information and traverse the Dom?
Or alternatively is there .hta equivalent in recent browsers like firefox?

If you need to have the embedded browser interact with your application code, you could try the SWT Browser control, it's actually maintained as opposed to JRex. Browser uses either WebKit or Gecko or embedded IE as appropriate, or lets you choose which one you want, so it should run jQuery and familiar Javascript. And since SWT is a JNI library to begin with they probably already have guidance on how to deploy an app that uses JNI.
You can feed HTML into the control from a string (example) or a java Url - which can point to local files or resource files in your JAR, which I assume will let you split your app into different files.
To call Java code, you need to expose it as Javascript functions. example
To manipulate the HTML from Java code, you need to call Javascript functions from Java. example
To make the previous two tasks easier, you might want to look into a JSON library to simplify passing around complex data.

Does it have to be implemented within a Java program? Could you let the user use the default browser on their machine (ie does it matter what browser)?
If not would use the Java Desktop API.
if (desktop.isSupported(Desktop.Action.BROWSE)) {
txtBrowserURI.setEnabled(true);
btnLaunchBrowser.setEnabled(true);
}
If you are using Java 1.5 try http://javadesktop.org/articles/jdic/

How can I open a webpage within a Java app and run my own javascript code

I would like to open a webpage and run a javascript code from within a java app.
For example I would like to open the page www.mytestpage.com and run the following javascript code:
document.getElementById("txtEmail").value="test#hotmail.com";
submit();
void(0);
This works in a browser...how can I do it programatically within a java app?
Thanks!

You can use Rhino to execute JavaScript but you won't have a DOM available - i.e. document.getElementById() would work.
You can use HTML Unit (headless) or WebDriver/Selenium (Driving a browser) to execute JavaScript in an environment that has a DOM available.

I'm not sure what you are looking for but I assume that you want to write automated POST request. This can be done in with Http Client library. Only you have to set appropriate request (POST or GET) parameters.
Look at examples - with this library you can do basic authentication or post files too.

Your question is a bit ambiguous, as we don't know the position of the Java program.
If that's a Java applet inside your page, you should look at Java<->JavaScript interaction, it works well.
If you need a separate Java program to control a browser, like sending a bookmarklet in the address bar (as one of your tags suggests), it is a bit harder (depends on target browser), perhaps look at the Robot class.

There's Rhino JS engine written in Java that you can run on app server such as Tomcat and feed JS to, however - it's not clear what are you trying to do with this?
There's also Envjs simulated browser environment which is based on Rhino but complete enough to run jQuery and/or Prototype

DWR (and other frameworks) now support "reverse ajax." The general idea is that you use one of three methods to communicate back to the client:
Comet (long-lived https session)
Polling
opportunistic / piggy-back (i.e. next time a request comes from the client, append your js call)
Regardless of method (which is typically a configuration-time decision and not a coding issue), you will have full access to any/all js calls you want to make.
Check out the reference page from DWR to get a pretty good explanation.