Develop Reference

Why is a blank Java Icon appearing when I parse a PDF file using Tabula?

I am working on an integration with Apache Drill which enables users to query PDF files directly using SQL. I'm about 80% done and really impressed with how well Tabula works for this.
However, when I execute the first Drill query that uses the Tabula libraries a Java icon pops up and I get the following text in the command line:
2020-10-25 15:06:55.770 java[71188:7121498] Persistent UI failed to open file file://localhost/Users/******/Saved%20Application%20State/net.java.openjdk.cmd.savedState/window_1.data: Permission denied (13)
I changed the permissions on that directory but I'm still getting the Java popup.
This is not normal behavior for Drill and my goal here was to integrate Tabula programmatically. Is Tabula trying to open a window or something like that and if so, is there a way to disable this? I noted that this does not occur in my unit tests.
Here are some relevant code snippets:
public static List<Table> extractTablesFromPDF(PDDocument document, ExtractionAlgorithm algorithm) {
NurminenDetectionAlgorithm detectionAlgorithm = new NurminenDetectionAlgorithm();
ExtractionAlgorithm algExtractor;
SpreadsheetExtractionAlgorithm extractor=new SpreadsheetExtractionAlgorithm();
ObjectExtractor objectExtractor = new ObjectExtractor(document);
PageIterator pages = objectExtractor.extract();
List<Table> tables= new ArrayList<>();
while (pages.hasNext()) {
Page page = pages.next();
algExtractor = algorithm;
/*if (extractor.isTabular(page)) {
algExtractor=new SpreadsheetExtractionAlgorithm();
}
else {
algExtractor = new BasicExtractionAlgorithm();
}*/
List<Rectangle> tablesOnPage = detectionAlgorithm.detect(page);
for (Rectangle guessRect : tablesOnPage) {
Page guess = page.getArea(guessRect);
tables.addAll(algExtractor.extract(guess));
}
}
return tables;
}
This doesn't happen in my unit tests.
Thanks in advance for your help!

Because some code is executed that does an operation that is usually, but technically not necessarily, involved in things that require so-called 'headful' mode (well, that's perhaps not really a term, but the opposite, 'headless' certainly is). This causes a few things to happen, including that icon showing up.
One easy way out of this is to force headless mode. But note that when you do this, any of these 'usually but technically not neccessarily headful' operations may either [1] work fine and no longer show that icon, or, [2] crash with a HeadlessException. Which one you end up with is not just dependent on which operation you're doing, but also which VM you are doing it on - as a rule once one of these ops works fine and no longer throws, later versions won't revert back to throwing (in other words, newer versions of java offer more things that work in headless mode).
To force headless mode, run java with java -Djava.awt.headless=true.
If you must do it from within java code, run System.setProperty("java.awt.headless", "true"); at least once, and before you do any of these 'usually causes headful mode' operations.
Presumably, the thing that is causes headful mode to occur is something graphics involved, such as rendering a JPG or PNG into an ImageBuffer. It's not surprising that Apache Drill is doing this to 'read' images, for example.
Another option is to just upgrade your VM, maybe that helps. As a general rule, features 'move downwards' on this line:
Requires headful mode; running it makes the VM go headful (icon appears); if java.awt.headless is set, the operation fails with a HeadlessException.
Causes headful mode; running it makes the VM go headful. However, if headless is set, it works fine and won't do that.
Completely freed. Running the code works fine and does not cause the VM to go headful. the headless flag has no bearing whatsoever on how the code operates.

Jenkins displays "Build Success" even when tests are failed due to error handling in scripts (try and catch block)

Note: I am unable to paste the exact framework and code as the server I work on is not accessible from outside. Hence, I will try to explain my problem in simple words and examples.
Overview - I have created a Selenium automation framework which includes TestNG, Maven (POM.XML), Test data files, Scripts and few reusable functions.
Issue I am facing - I use Jenkins to execute my scripts. Jenkins calls POM.XML file which inturn calls testng.xml file (In testng.xml file, I have mentioned the scripts to be executed)
Let's say, I have to perform login action
Main Script
#Test
Public void maintest ()
{
//I use Extent reports for reporting purpose and hence I have created extent
//reporting reusable function which is called in the below fashion.
//If Login method returns pass, ExtentReportingFunc passes Pass to its
//function and displays Pass for that particular Login step in the report.
ExtentReportingFunc (Login(UserName, Password));
}
Reusable functions
Public String Login (String UN, String Pass)
{
//Sendkeys and set UN
driver.findelement (By.id("username")).sendkeys(UN);
//Sendkeys and set Password
driver.findelement (By.id("password")).sendkeys(pass);
//Click Login
driver.findelement (By.id("login")).click ();
//Verifying the message "Welcome User" is displayed after login
try
{
if (driver.findlement (By.id("welcomemessage")).isdisplayed ();
{
return pass;
}
} catch (Exception e)
{
//If welcome message is not found then return fail to the Reporting function
return "fail";
//Below code will be unreachable which is correct as per Java but that is the challenge.
// I tried several ways to find a work around to return fail as above as
// well throw exception but no luck till now.
// Upon throwing exception in the below way, Jenkins displays build as
//failed. If it is not done, though the script has failed,
//Jenkins displays "BUILD PASSED"
throw e;
}
}
//Extent Reporting function
ExtentReportingFunc (String status)
{
log.report (status);
}
Here, the challenge is - In the catch block, If I DO NOT mention "throw e", Jenkins won't understand that the failure has occurred and display "BUILD PASSED" in it's console output. I want it to display "BUILD FAILURE" in Jenkins console. Reason why I want it to display "BUILD FAILED" is - I have integrated JIRA with Jenkins. Only when jenkins show BUILD FAILED, it logs automatically bugs to JIRA. If it is "BUILD PASSED" and though finished status is UNSTABLE, no failures will be displayed in the test result section of Jenkins NOR it logs any bugs in JIRA.
However, that time I won't be able to pass return "fail" to the main reporting function so that it can display login step as failure in the report.
I understand, as per JAVA we can either throw or return in the catch block but not both. Is there any other way we can make this work?
I already had created end to end framework but later realised this problem when I started integrating with Jenkins (or else everything was fine till then).

Why Don't you add assert fail inside you catch statement, that way you are forcing testng test to fail when its inside the catch statement
org.testng.Assert.fail("Im failing here due to ... you can add your e message here");
Just add the line of code before
return "fail";
and keep the rest of the function as is

You can solve this problem by using asserts, so whenever your condition doesn't met the assert will fail and so will your test case and jenkins will show the status of the build as "UNSTABLE" instead of "PASS".
For example, in the above example, instead of using try catch and if condition in the try, it can be solved by a single line of assert which will also give you your desired build status as well.
You can replace the above code with:
Assert.assertTrue(driver.findElement(By.id("welcomemessage")).isDisplayed(), "Element is not present on the page");
So in this, if the element is not displayed on the page, assert will be failed as it is expecting true value but will get false and your jenkins build status will be shown as unstable.

Selenium + Java error listener ext.js

How can I catch an error when I don't know when it happens?
I'm using Selenium + Java. Creating tests for a JS web page.
When an error occurs, the test continues clicking on the elements.
The error is visible in that moment, so it is not a messagebox or something like that. An errorbox just becomes visible. Then some time passes, and the test crashes saying that it cannot click some element.
How can I listen for any random element to be visible (or clickable) while my test is running? That would allow me to catch the error and the test would not fail.
Should I put my test into two threads, one listening for errors, and another running my test case?

I would suggest to use the following
List<WebElement> Elements = getDriver().findElements(method);
First, when you entry to the page you just prepare the list of the page elements.
Then you include this procedure several time in your code. You compare your list with the new list and see if the error element is appears, means you will have one more element in the new list. Then you catch it.

The best option is to understand and resolve the underlying error condition.
If the error is only something that occurs exclusively in testing, or otherwise cannot be resolved I would suggest looking at your driver configuration and insure that the dialog you're seeing wouldn't be covered by the Driver setting for Unexpected_Alert_Behavior. If you've got that turned off, I'd try turning it on and see how it impacts your behavior.
How to handle an Alert with "UnexpectedAlertBehaviour" capability in Selenium?
https://selenium.googlecode.com/git/docs/api/java/org/openqa/selenium/ie/InternetExplorerDriver.html
I'm not entirely certain that will solve the listed problem bc the dialog you mention is part of the dom and the 'unexpected' alert behavior is typically not visible in that context from what I've seen. I believe this is also very specific to the IE implementation.
Finally, I think I would resort to using a method in my class to perform all findbys for me. In that method, I would use an explicit wait to check for the random addition and have it fall-through to resolve the actual requested object if the wait fails to resolve the 'error reference'.
/** By identifier for the error dialog addition.*/
private static final By ERR_DLG = By.className("error");
/**
* Delegate method that will attempt to resolve the occasional page error condition prior to performing the delegate lookup on the WebDriver.
* <p/>
* This method will fail on Assert behavior if the error dialog is present on the page.
*
* #param driver WebDriver referenced for the test instance.
* #param lookup By reference to resolve the desired DOM reference.
* #return WebElement of the specified DOM.
*/
private final WebElement safeResolve(WebDriver driver, By lookup) {
WebDriverWait wait = new WebDriverWait(driver, 1);
WebElement errDlgRef = null;
try {
errDlgRef = wait.until(ExpectedConditions.visibilityOfElementLocated(ERR_DLG));
} catch (TimeoutException te) {
//This is actually OK, in that in the majority of cases this is the expected behavior.
//Granted this is bad form for Unit-Testing, but Selenium at an Integration-Test level changes the rules.
}
Assert.assertNull("Unexpected Error Dialog exists in DOM", errDlgRef);
return driver.findElement(lookup);
}
This solution is kind of a hammer, but I think it would work. All lookups would need to go through this method. You'd probably also need an analogous method if you use the List WebDriver.findElements(By) function in your test, or a way to abstract it from this one.

selenium.waitForPageToLoad doesn't seem to be waiting

I'm updating a Selenium program I wrote a while back and part of it has stopped working. I want to go through a whole series of links on a page, click on each, making sure that some expected text is present. But sometimes a log-in page (https://library.med.nyu.edu/sso/ezproxy_form.php) appears before the desired page, in which case I need to log in before checking the page. The problem is, no matter what string I put in to check whether I've landed on the log in page, Selenium concludes it's not present and skips logging in, obviously causing everything else to fail. See below--I'm not sure that was actually the problem. It seems to be instead that it's rushing through the "if we need to sign in" code without actually signing in, then obviously failing the main part of the test because it's not on the right page.
Here's the code--does anyone see my mistake?
for (int i = 0; i < Resources.size(); i++) {
try {
selenium.open("/");
selenium.click("link=" + Resources.get(i).link);
selenium.waitForPageToLoad("100000");
if (selenium.isTextPresent("Please sign in to access NYUHSL e-resources")) {
selenium.type("sso_user", kid);
selenium.type("sso_pass", password);
selenium.click("name=SignIn");
selenium.waitForPageToLoad("100000");
}
if (!selenium.isTextPresent(Resources.get(i).text)) {
outfile.println(Resources.get(i).name + " failed");
}
} catch (Exception e) {
outfile.println(Resources.get(i).name + " could not be found--link removed?");
}
}

Does the login page have a page title? If yes, try validating the page title using selenium.getTitle() method to check if you are headed to login page. If not, proceed clicking on the link without logging in.
I think page title validation can help resolve this issue

Try putting:
selenium.setSpeed("1000");
Right after the selenium.open this will inject 1 second delay (1000ms) between selenium commands. You should make it a standard practice to add this, especially if you're not running headless browsers.
Also you might consider using, since you know the url you are expecting to be on if on the login page, the selenium command getLocation. This will return the absolute URL of the current page. Might be more effective than trying to look for elements that can change at any time within the page.
So to use getLocation in your code above:
if (selenium.getLocation() == "your reference url"){
do your login stuff here
}
Again this is just a sample to illustrate what I'm saying. Hope it helps you out.

JFreeChart Exceptionless Forever-blocking error

I am not sure if this is a permissions problem, or an error in JFreeChart (latest version 1.0.13) with my web application that is running on Tomcat in CentOS. But I have a very odd situation where I in my application an event fires, eventually causing the below method to be executed with the supplied parameters.
I've checked the documentation and it appears that these static methods don't throw exceptions, so I can only assume they return null series or similar if they cannot execute properly. However, in my case, I will execute the use case that causes this code to be run, and as I'm looking at the Tomcat log catalina.out, I can see the line "===================5" appear, but "===================6" never does. And that's where I am stumped. And clearly since "chart" never get's made, the image file can never be generated, leaving an ugly error on my webpage.
Can anyone shed some light on why ChartFactory.createTimeSeriesChart seems to hang? Wouldn't bad input cause the method to return a null series or something, certainly this very mature product wouldn't just sit there and block forever right?
The other detail is that this worked in the GWT servlet container, and also another Tomcat servlet container on Windows...which kind of makes me think there could be some permissions issue. Except that for my last test I made everything root...
Finally, perhaps I missed something huge and JFree methods do throw exceptions? Perhaps my catch block errors out and the message never goes to my error logs?
EDIT: The class files in my .war were compiled the Windows machine that they function correctly on. bytecode is bytecode, right? Or is there some potential problem?
EDIT 2: The project is headless and configured as such.
Code:
public LineChart(final String title, List<GraphData> graphxy[],
String url, String sensorName[], String unit[], float critHigh,
float critLow, Double percent, String historic, String clickZoomIn,
String BaseUrl,Date[][] dateDifference)
throws IOException
{
try {
...
System.out.println("===================5");
//All the parameters are built defined in excised code:
final JFreeChart chart = ChartFactory.createTimeSeriesChart(ShowsensorName, "Date", readUnit, dataset, true, true, false);
System.out.println("===================6");
...
}
catch (Exception e)
{
System.out.println("Exception in line chart demo is========="+e);
}
}

Here's a few things to try:
If headless, verify correct setup.
Try related demos on the affected server.
Try evoking your chart as an application, e.g. TimeSeriesChartDemo1.
Walk through the source by clicking the createTimeSeriesChart() link.