I want to run JavaScript code at the server side. I want to manipulate result returned by JavaScript inside my Java code. How can it be done?
The start is clearly to look into rhino.
I think you will find this 3 links very useful
JavaScript EE, Part 1: Run JavaScript files on the server side
JavaScript EE, Part 2: Call remote JavaScript functions with Ajax
JavaScript EE, Part 3: Use Java scripting API with JSP
You can also have a look to helma
Helma is a server-side Javascript environment and web application framework for fast and efficient scripting and serving of your websites and Internet applications.
Helma is written in Java and employs Javascript for its server-side scripting environment ...
The other answers are correct that if you want to execute Javascript on the server side, you'd need to evaluate it in the context of a JS runtime.
However, I'm not convinced that this is exactly what you're asking. I think there may be a chance that you want to run some "typical" JS functionality that relates to how the page is displayed on the client's machine or interacted with on the client - and that will not be possible to run on the server side.
As a concrete examples:
If you want to run some kind of algorithm in JS without porting it to Java - say, you have some opaque Javascript code that generates a particular sequence given a seed - this would work fine if you run it on Rhino on the server.
If you want to invoke a Javascript function while creating the page, rather than while it's running - say, to get the user's colour depth/screen resolution and change how the page is generated - then this will not be possible from the server, as there is no client at this point to query.
Broadly speaking, any Javascript that involves document or navigator or even any elements of the page itself, is likely to fall into the latter category.
If you really need to get information about the client environment in order to control how the page is rendered, this must be extracted from the client on the previous page, and encoded into the request (as query parameters or form data). These parameters can then be read directly on the server and used to control the output.
Remember that when your code is running on the server side, you're creating a page (ultimately a bunch of HTML, CSS and JS) that will be sent to the client once it's done - at this point there is no client yet and so you can't interact with them.
Apologies if I've got the wrong end of the stick on this one, but this type of question is typically asked by people who haven't grasped the client/server separation.
You need a JS runtime inside of a Java runtime. One way to do this is Rhino
You execute the JavaScript with Rhino, a JavaScript library for Java.
You can use RHINO or NASHORN.
public class RhinoApp {
private String simpleAdd = "var z=9; z*=9";
public void runJavaScript() {
Context jsCx = Context.enter();
Context.getCurrentContext().setOptimizationLevel(-1);
ScriptableObject scope = jsCx.initStandardObjects();
Object result = jsCx.evaluateString(scope, simpleAdd , "formula", 0, null);
Context.exit();
System.out.println(result);
}
This example should clearly state how to load, evaluate and execute a Javascript function in Java:
ScriptEngineManager factory = new ScriptEngineManager();
ScriptEngine engine = factory.getEngineByName("JavaScript");
URI source_js = JavascriptExecutor.class.getResource("/file.js").toURI();
String source_text = Files.readAllLines(Paths.get(source_js)).stream().collect(Collectors.joining("\n"));
engine.eval(source_text);
Invocable inv = (Invocable) engine;
Object returnValue = inv.invokeFunction("functionJsName", "functionJsParameter");
System.out.println(returnValue.toString());
Related
I want to run JavaScript code at the server side. I want to manipulate result returned by JavaScript inside my Java code. How can it be done?
The start is clearly to look into rhino.
I think you will find this 3 links very useful
JavaScript EE, Part 1: Run JavaScript files on the server side
JavaScript EE, Part 2: Call remote JavaScript functions with Ajax
JavaScript EE, Part 3: Use Java scripting API with JSP
You can also have a look to helma
Helma is a server-side Javascript environment and web application framework for fast and efficient scripting and serving of your websites and Internet applications.
Helma is written in Java and employs Javascript for its server-side scripting environment ...
The other answers are correct that if you want to execute Javascript on the server side, you'd need to evaluate it in the context of a JS runtime.
However, I'm not convinced that this is exactly what you're asking. I think there may be a chance that you want to run some "typical" JS functionality that relates to how the page is displayed on the client's machine or interacted with on the client - and that will not be possible to run on the server side.
As a concrete examples:
If you want to run some kind of algorithm in JS without porting it to Java - say, you have some opaque Javascript code that generates a particular sequence given a seed - this would work fine if you run it on Rhino on the server.
If you want to invoke a Javascript function while creating the page, rather than while it's running - say, to get the user's colour depth/screen resolution and change how the page is generated - then this will not be possible from the server, as there is no client at this point to query.
Broadly speaking, any Javascript that involves document or navigator or even any elements of the page itself, is likely to fall into the latter category.
If you really need to get information about the client environment in order to control how the page is rendered, this must be extracted from the client on the previous page, and encoded into the request (as query parameters or form data). These parameters can then be read directly on the server and used to control the output.
Remember that when your code is running on the server side, you're creating a page (ultimately a bunch of HTML, CSS and JS) that will be sent to the client once it's done - at this point there is no client yet and so you can't interact with them.
Apologies if I've got the wrong end of the stick on this one, but this type of question is typically asked by people who haven't grasped the client/server separation.
You need a JS runtime inside of a Java runtime. One way to do this is Rhino
You execute the JavaScript with Rhino, a JavaScript library for Java.
You can use RHINO or NASHORN.
public class RhinoApp {
private String simpleAdd = "var z=9; z*=9";
public void runJavaScript() {
Context jsCx = Context.enter();
Context.getCurrentContext().setOptimizationLevel(-1);
ScriptableObject scope = jsCx.initStandardObjects();
Object result = jsCx.evaluateString(scope, simpleAdd , "formula", 0, null);
Context.exit();
System.out.println(result);
}
This example should clearly state how to load, evaluate and execute a Javascript function in Java:
ScriptEngineManager factory = new ScriptEngineManager();
ScriptEngine engine = factory.getEngineByName("JavaScript");
URI source_js = JavascriptExecutor.class.getResource("/file.js").toURI();
String source_text = Files.readAllLines(Paths.get(source_js)).stream().collect(Collectors.joining("\n"));
engine.eval(source_text);
Invocable inv = (Invocable) engine;
Object returnValue = inv.invokeFunction("functionJsName", "functionJsParameter");
System.out.println(returnValue.toString());
I am using JSOUP to fetch the documents from a website.
Below is my code
webPageUrl = https://mwcc.ms.gov/#/electronicDataInterchange
Document doc = Jsoup.connect(webPageUrl).get();
Elements links = doc.getElementsByAttribute("a[href]");
Below line of code is not working. It is supposed to return an element but doesn't:
doc.getElementsByAttribute("a[href]")
Can someone please point out the mistake in my code?
That page seems to be an Angular application, which means it loads some (probably all or most) of its content via JavaScript scripts.
The fact that the URL contains the fragment separator # is already a strong indicator of that fact, because if you do a HTTP request, then everything after that indicator is cut off (i.e. not sent to the server), so the actual request will just be of https://mwcc.ms.gov/.
As far as I know JSoup does not support running JavaScript, so you might need to look into a more involved scraping tool (possibly running a full browser engine).
The java API for CICS is here. Does anyone know if there any method to put a couple of radio buttons to a web form using this API?
Here's my code to create radio button
HttpRequest req = HttpRequest.getHttpRequestInstance();
String msg = "ZEUSBANK ANTI-FRAUD CHECK BY SHE0008.<br> "
+ "When investigation is complete. Tick the check box and submit.<br>";
String template = "<form><input type=\"radio\"> YES<br><input type=\"radio\"> NO<br></form>";
HttpResponse resp = new HttpResponse();
Document doc = new Document();
doc.createText(msg);
doc.appendFromTemplate(template);
resp.setMediaType("text/plain");
resp.sendDocument(doc, (short)200, "OK", ASCII);
But when I run it on a browser, it print plain text and doesn't convert html tag.
Fixed it, I just change media type from text/plain to text/html and it works.
As you've already discovered, you needed to send the request with the text/html content type.
If you're planning to do more Java web-based work through CICS Java, you might want to investigate the embedded WebSphere Liberty. It adds support for Java EE features, which includes JSF, JSP and Servlets, which can make web development in Java a lot easier.
Tri,
I haven't used CICS for 15 years, so I doubt I'm an expert anymore. But looking quickly at the API, it seems like all the presentation logic would be in your regular Java code. You would then format appropriate messages and invoke the CICS API to update the server & get a response.
There doesn't seem to be any 'BMS-related' methods at all (which is a good thing).
The only 'field' method I see is com.ibm.cics.server.FormField but that only has get() methods, not set().
Are you just starting with Java CICS, or are you just stuck on this particular issue? If you have some sample code of what you are trying, post it so we can see if anyone has any ideas.
HTH, Jim
I'm connecting to a webserver with a specific JavaScript. (Using HttpURLConnection atm)
What i need is a connection that makes it possible to manipulate a JavaScript function.
Afterwards i want to run the whole JavaScript again.
I want the following function always to return "new FlashSocketBackend()"
function createBackend() {
if (flashSocketsWork) {
return new FlashSocketBackend()
} else {
return new COMETBackend()
}
}
Do i have to use HtmlUnit for this?
Whats the easiest way to connect, manipulate and re-run the script?
Thanks.
With HtmlUnit you indeed can do it.
Even though you can not manipulate an existing JS function, you can however execute what JavaScript code you wish on an existing page.
Example:
WebClient htmlunit = new WebClient();
HtmlPage page = htmlunit.getPage("http://www.google.com");
page = page.executeJavaScript("<JS code here>").getNewPage();
//manipulate the JS code and re-excute
page = page.executeJavaScript("<manipulated JS code here>").getNewPage();
//manipulate the JS code and re-excute
page = page.executeJavaScript("<manipulated JS code here>").getNewPage();
more:
http://www.aviyehuda.com/2011/05/htmlunit-a-quick-introduction/
Your best shot is probably to use Rhino — an open-source implementation of JavaScript written entirely in Java. Loading your page with a window.location and hopefully running your JavaScript function. I read sometime before Bringing the Browser to the Server and seemed possible.
What are the best Java libraries to "fully download any webpage and render the built-in JavaScript(s) and then access the rendered webpage (that is the DOM-Tree !) programmatically and get the DOM Tree as an "HTML-Source"?
(Something similarly what firebug does in the end, it renders the page and I get access to the fully rendered DOM Tree, as the page looks like in the browser! In contrast, if I click "show source" I only get the JavaScript source code. This is not what I want. I need to have access to the rendered page...)
(With rendering I mean only rendering the DOM Tree not a visual rendering...)
This does not have to be one single library, it's ok to have several libraries that can accomplish this together (one will download, one render...), but due to the dynamic nature of JavaScript most likely the JavaScript library will also have to have some kind of downloader to fully render any asynchronous JS...
Background:
In the "good old days" HttpClient (Apache Library) was everything required to build your own very simple crawler. (A lot of cralwers like Nutch or Heretrix are still built around this core princible, mainly focussing on Standard HTML parsing, so I can't learn from them)
My problem is that I need to crawl some websites that rely heavily on JavaScript and that I can't parse with HttpClient as I defenitely need to execute the JavaScripts before...
You can use JavaFX 2 WebEngine. Download JavaFX SDK (you may already have it if you installed JDK7u2 or later) and try code below.
It will print html with processed javascript.
You can uncomment lines in the middle to see rendering as well.
public class WebLauncher extends Application {
#Override
public void start(Stage stage) {
final WebView webView = new WebView();
final WebEngine webEngine = webView.getEngine();
webEngine.load("http://stackoverflow.com");
//stage.setScene(new Scene(webView));
//stage.show();
webEngine.getLoadWorker().workDoneProperty().addListener(new ChangeListener<Number>() {
#Override
public void changed(ObservableValue<? extends Number> observable, Number oldValue, Number newValue) {
if (newValue.intValue() == 100 /*percents*/) {
try {
org.w3c.dom.Document doc = webEngine.getDocument();
new XMLSerializer(System.out, new OutputFormat(doc, "UTF-8", true)).serialize(doc);
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
});
}
public static void main(String[] args) {
launch();
}
}
This is a bit outside of the box, but if you are planning on running your code in a server where you have complete control over your environment, it might work...
Install Firefox (or XulRunner, if you want to keep things lightweight) on your machine.
Using the Firefox plugins system, write a small plugin which takes loads a given URL, waits a few seconds, then copies the page's DOM into a String.
From this plugin, use the Java LiveConnect API (see http://jdk6.java.net/plugin2/liveconnect/ and https://developer.mozilla.org/en/LiveConnect ) to push that string across to a public static function in some embedded Java code, which can either do the required processing itself or farm it out to some more complicated code.
Benefits: You are using a browser that most application developers target, so the observed behavior should be comparable. You can also upgrade the browser along the normal upgrade path, so your library won't become out-of-date as HTML standards change.
Disadvantages: You will need to have permission to start a non-headless application on your server. You'll also have the complexity of inter-process communication to worry about.
I have used the plugin API to call Java before, and it's quite achievable. If you'd like some sample code, you should take a look at the XQuery plugin - it loads XQuery code from the DOM, passes it across to the Java Saxon library for processing, then pushes the result back into the browser. There are some details about it here:
https://developer.mozilla.org/en/XQuery
The Selenium library is normally used for testing, but does give you remote control of most standard browsers (IE, Firefox, etc) as well as a headless, browser free mode (using HtmlUnit). Because it is intended for UI verification by page scraping, it may well serve your purposes.
In my experience it can sometimes struggle with very slow JavaScript, but with careful use of "wait" commands you can get quite reliable results.
It also has the benefit that you can actually drive the page, not just scrape it. That means that if you perform some actions on the page before you get to the data you want (click the search button, click next, now scrape) then you can code that into the process.
I don't know if you'll be able to get the full DOM in a navigable form from Selenium, but it does provide XPath retrieval for the various parts of the page, which is what you'd normally need for a scraping application.
You can use Java, Groovy with or without Grails. Then use Webdriver, Selenium, Spock and Geb these are for testing purposes, but the libraries are useful for your case.
You can implement a Crawler that won't open a new window but just a runtime of these either browser.
Selenium : http://code.google.com/p/selenium/
Webdriver : http://seleniumhq.org/projects/webdriver/
Spock : http://code.google.com/p/spock/
Geb : http://www.gebish.org/manual/current/testing.html
MozSwing could help http://confluence.concord.org/display/MZSW/Home.
You can try JExplorer.
For more information see http://www.teamdev.com/downloads/jexplorer/docs/JExplorer-PGuide.html
You can also try Cobra, see http://lobobrowser.org/cobra.jsp
I haven't tried this project, but I have seen several implementations for node.js that include javascript dom manipulation.
https://github.com/tmpvar/jsdom