Develop Reference

Selenium4: "The method newWindow(WindowType) is undefined for the Type WebDriver.TargetLocator"

I'm trying to play with the Selenium 4 features in eclipse with java but can't seem to get them to work, which i'm assuming must be my mistake in configuration, so I'd appreciate if anyone can explain where i'm going wrong and how I can correct it?
Code below: I get an error warning on the last line (containing the newWindow() method) and the error message is as per the Title of this post.
I've downloaded the selenium-java-4.0.0-alpha-4 from here https://selenium-release.storage.googleapis.com/index.html?path=4.0-alpha4/
i've created a new eclipse java project, and unzipped and included all of the jar files in my java build path, but it doesn't seem to recognise / or be able to find the newWindow() method
package practice;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.WindowType;
import org.openqa.selenium.chrome.ChromeDriver;
import static org.openqa.selenium.support.locators.RelativeLocator.withTagName;
import org.openqa.selenium.By;
public class Sel4alpha4 {
public static void main(String[] args) {
System.setProperty("webdriver.chrome.driver","C:\\Users\\me\\Downloads\\chromedriver_win32\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("http://www.google.com");
driver.switchTo().newWindow(WindowType.TAB); // this line has the error
}
}

There might be two problems:
1) Incorrect Import- You should import the following:
org.openqa.selenium.WindowType;
2) Issue with the version you are using. Try to upgrade the selenium version.
Version:
selenium 4.0.0-beta-4
Please try to download the updated jar from the below link:
https://www.selenium.dev/downloads/
or
https://selenium-release.storage.googleapis.com/index.html?path=4.0-beta-4/

GraalVM - embedding python multi-file project in java

I couldn't find a solution create a polyglot source out of multiple files in GraalVM.
What exactly I want to achieve:
I have a python project:
my-project:
.venv/
...libs
__main__.py
src/
__init__.py
Service.py
Example sourcecode:
# __main__.py
from src.Service import Service
lambda url: Service(url)
# src/Service.py
import requests
class Service:
def __init__(self, url):
self.url = url
def invoke(self):
return requests.get(self.url)
This is very simple example, where we've got an entry-point script, project is structured in packages and there is one external library (requests).
It works, when I run it from command-line with python3 __main__.py, but I can't get it work, when embedding it in Java (it can't resolve imports).
Example usage in java:
import org.graalvm.polyglot.Context;
import org.graalvm.polyglot.Source;
import org.graalvm.polyglot.Value;
import java.io.File;
import java.io.IOException;
public class Runner {
public static void main(String[] args) throws IOException {
Context context = Context.newBuilder("python")
.allowExperimentalOptions(true)
.allowAllAccess(true)
.allowIO(true)
.build();
try (context) {
// load lambda reference:
Value reference = context.eval(Source.newBuilder("python", new File("/path/to/my-project/__main__.py")).build());
// invoke lambda with `url` argument (returns `Service` object)
Value service = reference.execute("http://google.com");
// invoke `invoke` method of `Service` object and print response
System.out.println("Response: " + service.getMember("invoke").execute());
}
}
}
It fails with Exception in thread "main" ModuleNotFoundError: No module named 'src'.
The solution works for javascript project (having similar index.js to __main__.py, its able to resolve imports - GraalVM "sees" other project's files, but somehow it doesn't, when using python.
I found out, that python is able to run zip package with project inside, but this also doesn't work with GraalVM.
Is there any chance to accomplish it? If not, maybe there is a similar tool to webpack for python (if I could create a single-file bundle, it should also work).
Btw, I don't know python at all, so I may missing something.
Thanks for any help!

How can I work around YouTube API embed restrictions like other websites?

I am building a java program that has the option to play YouTube videos in an embedded player.
The problem is that most of the music videos won't play and I get the following error:
"This video contains content from (Media Corporation Name). It is restricted from playback on certain sites."
I tried loading the same URL in Chrome and got the same results.
https://www.youtube.com/embed/TMZi25Pq3T8
However, after some research, I quickly got it fixed by installing a Chrome Extension that allows me to add HTTP Request Headers and added a Referer header that follows this structure "https://www..com" and got it working.
So I thought that must be it.
I added the following code in order to add request headers to my JavaFX WebView / WebEngine:
URI uri = URI.create("https://www.youtube.com/embed/TMZi25Pq3T8");
List<String> cookies = new ArrayList<>();
cookies.add("User-Agent=BDM/v0.92");
cookies.add("Referer=https://www.youtube.com");
Map<String, List<String>> headers = new LinkedHashMap<String, List<String>>();
headers.put("Set-Cookie", cookies);
try {
CookieHandler.getDefault().put(uri, headers);
} catch (IOException ex) {
ex.printStackTrace();
}
System.out.println(webView.getEngine().getUserAgent());
webView.getEngine().load(uri.toString());
Still, no success, the same error message.
The website that I'm using to extract data about releases through their API, Discogs, is able to play "restricted" videos as well. What am I missing here?
LATER EDIT:
Further clarifications:
I would like to apologize for the mistakes I made:
The line System.out.println(webView.getEngine().getUserAgent()); doesn't print "BDM/v0.92" as I first stated, it prints the default JavaFX user agent, "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/538.19 (KHTML, like Gecko) JavaFX/8.0 Safari/538.19". And this leads to number 2
As Roman Nazarenko pointed out, I was confusing cookies with request headers.
This leads to the real question, how can I send HTTP Request headers for JavaFX WebEngine? The only option is to set the user agent by calling webView.getEngine().setUserAgent("myUserAgent");
I found a hack here but this didin't work for me: https://twitter.com/codingfabian/status/524942996748652544
Thanks!

I managed to solve the issue by using javassist and this tutorial on how to instrument Java code.
As I stated in my question, the YouTube player needs a Referer header to play some videos (like music videos owned by VEVO, Sony Music Enternatinment, etc.).
What I did is I intercepted prepareConnection method from the URLLoader class that is used by JavaFX's WebEngine and inserted my instruction at the top of the method body:
c.setRequestProperty("Referer", "https://www.discogs.com");
(Again, please follow the tutorial for all the instructions)
(Note: Even though the tutorial above is explains very well the concepts, it doesn't really touch much on the role and structure of a MANIFEST.MF file, so please check this link for more info about this aspect)
These are my two classes:
MyJavaAgent.java
package com.busytrack.discographymanager.headerfixagent;
import java.lang.instrument.Instrumentation;
public class MyJavaAgent {
public static void premain(String agentArgument, Instrumentation instrumentation) {
ClassTransformer transformer = new ClassTransformer();
instrumentation.addTransformer(transformer);
}
}
ClassTransformer.java
package com.busytrack.discographymanager.headerfixagent;
import java.io.ByteArrayInputStream;
import java.lang.instrument.ClassFileTransformer;
import java.lang.instrument.IllegalClassFormatException;
import java.security.ProtectionDomain;
import javassist.ClassPool;
import javassist.CtClass;
import javassist.CtMethod;
public class ClassTransformer implements ClassFileTransformer {
public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {
byte[] byteCode = classfileBuffer;
if (className.equals("com/sun/webkit/network/URLLoader")) {
try {
ClassPool classPool = new ClassPool(true);
CtClass ctClass = classPool.makeClass(new ByteArrayInputStream(classfileBuffer));
CtMethod method = ctClass.getDeclaredMethod("prepareConnection");
String src = "$1.setRequestProperty(\"Referer\", \"https://www.discogs.com\");"; // Confused about there being "$1" instead of "c"? Please read below
method.insertBefore(src);
byteCode = ctClass.toBytecode();
ctClass.detach();
} catch (Exception e) {
e.printStackTrace();
}
}
return byteCode;
}
}
This is why I used "$1" to access the method parameter, instead of "c":
The statement and the block can refer to fields and methods. They can also refer to the parameters to the method that they are inserted into if that method was compiled with the -g option (to include a local variable attribute in the class file). Otherwise, they must access the method parameters through the special variables $0, $1, $2, ... described below. Accessing local variables declared in the method is not allowed although declaring a new local variable in the block is allowed.
The entire javassist tutorial can be found here.
After packing the two classes and the MANIFEST.MF file in a separate JAR, import it in your IDE (I used Eclipse) and add the following VM argument:
-javaagent:./(your-jar-name).jar
In Eclipse, you can add VM arguments like this:
right click on your project -> Run As -> Run Configurations... -> open the Arguments tab -> insert your VM argument -> Apply
I hope this helps someone out there. I know I spent a few days on this issue.
I don't know if this is the best approach but it does the job for me.
Still, it makes me wonder why isn't there a straightforward way of setting Request Headers for JavaFX's WebEngine...
Later edit:
I found a much cleaner and easier approach for loading Java Agents, dynamically, without the need to create a separate JAR, manifest file, importing them, passing the -javaagent VM parameter at startup, etc.
I used the ea-agent-loader (JAR download link).
Import the JAR in your IDE and change the MyJavaAgent class (the one that had the premain method) to this:
package com.busytrack.discographymanager.headerfixagent;
import java.lang.instrument.Instrumentation;
public class MyJavaAgent {
public static void agentmain(String agentArgument, Instrumentation instrumentation) {
ClassTransformer transformer = new ClassTransformer();
instrumentation.addTransformer(transformer);
}
}
My main method from the MainClass looks like this:
public static void main(String[] args) {
AgentLoader.loadAgentClass(MyJavaAgent.class.getName(), null); // Load the MyJavaAgent class
launch(args); // Start the JavaFX application
}
I wanted to be able to load the Agent dynamically because, using the static method required me to create separate launchers for all platforms and pass the -javaagent parameter on startup. Now, I can export a runnable JAR from eclipse like I usually do and the agent will load automatically (no VM parameters required). Thanks, BioWare for this tool! :D

Calling A Java Library From Pentaho Kettle

I have developed a java class exported into a .jar library that will be called by a Pentaho Kettle 'modified java script'. The .jar is compiled in Eclipse with JDC Compliance level 1.7.
When I try to use this class inside a 'modified java script', I get the error: ReferenceError: “xeroCallPackage” is not defined. I have tried lots of things without much luck so far.
My file xeroCallPackage.jar is in the path with the other *.jar files in Pentaho (..\data-integration\lib)
For info:
The stripped down (for simplicity) java library code is here:
package xeroCallPackage;
import java.io.IOException;
import java.net.URISyntaxException;
import java.util.Collections;
import java.util.Map;
public class xeroURLCall {
public String getResponse(String CONSUMER_KEY, String CONSUMER_PRIVATE_KEY, String URL) throws IOException, OAuthException, URISyntaxException {
// stripped out code here
return response.readBodyAsString();
}
}
The stripped down Pentaho 'modified java script' is here:
var CONSUMER_KEY = "ffffff";
var CONSUMER_PRIVATE_KEY = "aaaaa";
var URL = "https://gggggggg.rrrrr.wwww";
var ResponseAsString;
ResponseAsString = new xeroCallPackage.xeroURLCall.getResponse(CONSUMER_KEY,CONSUMER_PRIVATE_KEY,URL);

You will either have to have org as top-most package or prefix the fully qualified name of your class with Packages.
So in your case the fully qualified name of your class that can be used for calling from Spoon will be Packages.xeroCallPackage.xeroURLCall
Apart from that the supplied JavaScript code won't work (but maybe that's just because of the stripped down code). You'd have to create a xeroURLCall object first and then call the getResponse method on that object:
var call = new Packages.xeroCallPackage.xeroURLCall(...);
var responseAsString = call.getResponse(CONSUMER_KEY,CONSUMER_PRIVATE_KEY,URL);

using selenium in a web service throws java.lang.NoClassDefFoundError exception

I tried to implemented a web service from a dynamic web project. I added the selenium-server-standalone-2.32.0.jar file to the buildpath, then I also added it to the WEB-INF/lib folder. Then I used the web service wizard to generate the web service from the project. At the start of the wizard it displayed a pop-up warning that read:
The service class "test.eko3.TestEko3" does not comply to one or more requirements of the JAX-RPC 1.1 specification, and may not deploy or function correctly.
The value type "org.openqa.selenium.WebDriver" used via the service class "test.eko3.TestEko3" does not have a public default constructor. Chapter 5.4 of the JAX-RPC 1.1 specification requires a value type to have a public default constructor, otherwise a JAX-RPC 1.1 compliant Web service engine may be unable to construct an instance of the value type during deserialization.
The field or property "windowHandles" on the value type "org.openqa.selenium.WebDriver" used via the service class "test.eko3.TestEko3" has a data type, "java.util.Set", that is not supported by the JAX-RPC 1.1 specification. Instances of the type may not serialize or deserialize correctly. Loss of data or complete failure of the Web service may result.
I clicked ok and it generated the web server and the client and displayed the client in the eclipse's browser. But when I entered the parameters and clicked invoke it displayed this exception in the result section:
Exception: java.lang.NoClassDefFoundError: org/openqa/selenium/WebDriver; nested exception is: java.lang.NoClassDefFoundError: org/openqa/selenium/WebDriver Message: java.lang.NoClassDefFoundError: org/openqa/selenium/WebDriver; nested exception is: java.lang.NoClassDefFoundError: org/openqa/selenium/WebDriver
Since I added the selenium jar into both the buildpath and the WEB-INF/lib folder I'm not sure why it can't find the class. The code for the server is below:
package test.eko3;
import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.ui.ExpectedCondition;
import org.openqa.selenium.support.ui.Wait;
import org.openqa.selenium.support.ui.WebDriverWait;
public class TestEko3 {
public String Ekobilet(String from, String to, String date) {
//Firefox browser instantiation
WebDriver driver = new FirefoxDriver();
//Loading the URL
driver.get("http://www.amadeusepower.com/trek/portals/trek/default.aspx?Culture=en-US");
WebElement radioOneway = driver.findElement(By.id("ctl00_ctl00_ctl00_cph1_cph1_QuickSearchAll1_QuickFlightSearchControl1_rbFlightType_1"));
radioOneway.click();
waitForPageLoaded(driver);
WebElement fromText = driver.findElement(By.id("ctl00_ctl00_ctl00_cph1_cph1_QuickSearchAll1_QuickFlightSearchControl1_txtSearch_txtFrom"));
fromText.clear();
fromText.sendKeys(from);
WebElement toText = driver.findElement(By.id("ctl00_ctl00_ctl00_cph1_cph1_QuickSearchAll1_QuickFlightSearchControl1_txtSearch_txtTo"));
toText.sendKeys(to);
WebElement dateText = driver.findElement(By.id("ctl00_ctl00_ctl00_cph1_cph1_QuickSearchAll1_QuickFlightSearchControl1_txtDepartureDate_txtDate"));
dateText.sendKeys(date);
//Sign in button identification and click it
WebElement searchbutton = driver.findElement(By.id("ctl00_ctl00_ctl00_cph1_cph1_QuickSearchAll1_QuickFlightSearchControl1_btnSearch"));
searchbutton.click();
String page = driver.getPageSource();
// Writer out = new BufferedWriter(new OutputStreamWriter(
// new FileOutputStream("ekobiletselenium.html"), "UTF-8"));
// try {
// out.write(page);
// } finally {
// out.close();
// }
//Closing the browser
driver.close();
return page;
}
public static void waitForPageLoaded(WebDriver driver) {
ExpectedCondition<Boolean> expectation = new
ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver driver) {
return ((JavascriptExecutor)driver).executeScript("return document.readyState").equals("complete");
}
};
Wait<WebDriver> wait = new WebDriverWait(driver,30);
try {
wait.until(expectation);
} catch(Throwable error) {
System.out.println("exception yavrum");
}
}
}
Can someone please tell me the cause of this? Am I missing a jar file that selenium depends on? Any help would be appreciated.

Maybe you are using the standalone jar which is not good for web projects.
See http://me-ol-blog.blogspot.co.il/2013/07/using-selenium-in-java-dynamic-web.html

This exception appears when selenium is not fully referenced in the project.
Here are the steps you can take to resolve the issue:
Download the Selenium Client & WebDriver Java Bindings here.
Unzip the downloaded file.
Copy the main selenium '.jar' (example: selenium-java-2.42.2.jar) to the web project's WebContent/WEB-INF/lib dir.
Also copy All the .jar files the Unzipped selenium libs dir to the web project's WebContent/WEB-INF/lib dir.
You should now be able to run the code without getting the java.lang.NoClassDefFoundError: org/openqa/selenium/WebDriver exception.

Please check this link definitely you can get the idea which is answered by #thedrs
http://me-ol-blog.blogspot.co.il/2013/07/using-selenium-in-java-dynamic-web.html