Develop Reference

Java Swing based windows desktop application is not detected by AccessibilityInsights

We have developed a Windows based Desktop application using Java Swing and build a exe of it using javapackager available in Java 8.
Now we are trying to automate the testing of it using Karate-Robot but Robot could not detect the fields on the GUI of Application. To further dig we have installed AccessibilityInsights suggested by Karate Community to figure out how to access elements on the GUI but it seems that our application is not getting detcetd in it. See attached screenshot.
Can someone help us on how to fix the application?
Will Java Accessibility Bridge help here? As per documentation https://docs.oracle.com/en/java/javase/11/access/java-accessibility-guide.pdf, Application needs to implement Java Accessibility APIs when it needs to be available to Windows Assistive technologies.
Edit 1: As per https://docs.oracle.com/javase/tutorial/uiswing/misc/access.html, "Because support for the Accessibility API is built into the Swing components, your Swing program will probably work just fine with assistive technologies, even if you do nothing special."

Ashish--
There have been multiple iterations of Windows-based assistive technologies over the years. The current iteration has existed since 2005 and is called UI Automation. This is the technology that is required for OS-provided tools like Magnifier and Narrator. It's also the technology that is required by Accessibility Insights for Windows. From your description, it might also be the technology that is required by Karate-Robot.
Java Accessibility Bridge uses an older iteration of assistive technology that is no longer recognized by OS-provided accessibility tools. It is recognized by some third-party accessibility tools (JAWS and NVDA, for example), but it is not recognized by OS-provided accessibility tools. To the best of my knowledge, Oracle has no plans to support UI Automation in the future. When their docs say that your apps will "work just fine", they're referring to users who rely on these third-party tools.
Accessibility Insights for Windows has made a conscious decision to support only UI Automation, since supporting 2 platforms would make things much more difficult. We chose to go with UI Automation because it is the only technology that is:
Actively supported by the operating system, with new features being regularly added and documented.
Supported by native Windows controls and most actively supported language frameworks.
The technology that we would expect new assistive technology tools to implement first.
I'm sorry to be the bearer of bad news on this.
DaveTryon (Accessibility Insights team)

IntelliJ IDEA: What qualifies as an SDK?

What exactly is an SDK, in the context of IntelliJ?
For the education of non-Java developers of PHP, Ruby or JavaScript applications who prefer IntelliJ IDEA for its Polyglot capabilities, do any knowledgeable IntelliJ IDEA users please have a moment to explain:
2. Why is the concept of an SDK (in the context of InteliJ) essential only to Java?
As Ryan J writes in his answer below:
An SDK (such as Java) can be very big, containing a lot of information and capability. Other SDK's may simply be a set of API's or libraries, and as such, may not be as conducive to being used with an IDE, and could be deemed as "convenient" from the perspective of a user that is not familiar with the concept of an SDK in the "Java" sense.
Update: 1 Year Later. I've edited this question to focus on the simplest most useful answer I've found, and am awarding the correct answer to Ryan J. Simply put: Java is a platform. Platforms have SDKs. PHP, JavaScript, and Ruby are languages, with various available runtime environments not mentioned in this Q&A. This programmer now uses Sublime Text 3 for developing in those languages. At the expense of much processor overhead there were neat features in IntelliJ for refactoring, etc.. but IMO they are no match for having tight regex game.
2. What is the "Ruby SDK" option if not an SDK?
JetBrains support writes:
Ruby has SDK for convenience as users may need multiple Ruby/RVM versions. The same applies to Python, multiple different virtualenvs can be defined and a user may need to switch between them (including Remote SDK on the server via SSH).
3. What is the "PHP Interpreter" option if not an SDK?
JetBrains support writes:
PHP has no SDK in IDEA/PhpStorm.
See http://youtrack.jetbrains.com/issue/WI-22627#comment=27-720859
PhpStorm Project Lead Alexey Gopachenko is against the PHP SDK configuration. It would be probably removed or will be available only in IDEA, not sure.
In my opinion, this makes a lot of sense. PHP is a language. We can use IDE extensions to better understand that language, but its API is completely exposed by text scripts.
Shameless big up my personal favorite PHP workflow, Phundament. This programmer no longer endorses PHP.

A Node.js-oriented answer comes from JetBrains at long last. From devnet.jetbrains.com:
No, configuring Node.js SDK isn't on our roadmap, because of IDE design limitation: a project can have only one SDK associated.
Thus, projects that already have SDK (e.g. python, java, ruby) wouldn't be able to configure Node.js SDK.
However I'm still very interested in proposing enhancements to IntelliJ to benefit those of use who choose to use it as a JavaScript full-stack IDE.
For now, I've gotten my IntelliJ IDEA setup pretty tight for a full-stack JavaScript application built with Daftmonk's angular-fullstack Yeoman generator.
I've elected to post this as a new self-answered question here.

In an attempt to answer the question as asked, an SDK is a Software Development Kit. It is a collection of tools, libraries, api's, source code, licenses, examples, documentation, and/or anything else required to develop software applications in a specific programming language or framework.
The integration of an SDK into an IDE is essentially the ability for an external application to leverage the capability provided by the SDK, and make programming and application writing easier for the user.
The SDK configured in an IDE might provide the capability for the IDE to allow API auto-complete, object inspection, source code compile/debugging, syntax highlighting, integrated code inspection and warning/error information, just-in-time compiling, unit testing, etc.
An SDK (such as Java) can be very big, containing a lot of information and capability. Other SDK's may simply be a set of API's or libraries, and as such, may not be as conducive to being used with an IDE, and could be deemed as "convenient" from the perspective of a user that is not familiar with the concept of an SDK in the "Java" sense.
The IDE's capability to leverage what is provided by the SDK may be determined by a couple things:
1. The SDK doesn't provide a very comprehensive set of tools and capabilities that allow an IDE to do what it does best, and thus appear "useless"
2. The IDE just didn't implement the SDK capability all that well
Hopefully this provides some insight...

Portable web browser embedded in java app?

Question: embed a portable web browser without cache into a java app? Using Java SE.
Can this be done? Are there any JNI supports for e.g. Chrome or Firefox?
Maybe Qt? QtJambi would allow interfacing with the Webbrowser? From what I understand QtJambi is now maintained by the community and LPGL, is QtWebbrowser embeddable for this purpose in the same licence?

Examining the available documentation and over the past few days this is what has emerged:
The open source browsers are Webkit (which is used for e.g. Safari), and the Mozilla flavoured Firefox. There is also Chromium which is the open source version of the Google flavoured Chrome which uses a lesser javascript engine (squirrel as opposed to v8) and does not have the multi-separate-thread modularisation of Google Chrome.
Additional information on Firefox: Mozilla Firefox is based on an application called xulrunner which uses xml and javascript to provide the additional functionality of firefox (other than the 'not so basic' browser - such things as the html parser, css renderer and javascript parser). Xulrunner is the implementation of the browser.
Firefox implements its own private xulrunner, however additionally there is xulrunner as a standalone install.
Xulrunner can be used to create standalone applications, that are based on a web-browser, and has support for javascript. Xulrunner is also used to create the plugins for firefox providing additional functionality.
In order to connect the front end to something a bit more sustained and powerful than javascript (which has improved in leaps and bounds in recent years), but is still not the strongest sustained engine, there is a C-library called xpcom.
Xulrunner uses xpcom (written in C) to expose the javascript variables and parse the dom, and link javascript variables, and do other 'browsery' things like read headers etc. More on Xulrunner and xpcom in a moment.
If you want a browser that is recent and functional (html 4 or 5, good support for css2 at least, and some css3, and solid rendering of javascript), your best bet other than compiling and embedding one of these open source browsers is to use SWT.
SWT is a gui interface, similar to swing, except that it implements a webbrowser type interface, and can be used to implement a browser. SWT also has some developed interfaces to interact both with the dom and the like. SWT was developed by IBM and is actively maintained.
What SWT does is implement the OS system browser, and has support for every major os. On the upside, this means that there is no need to embed your own browser, on the downside you are stuck with whatever browser / parsers exist on the operating systems, at whichever version. The default behaviour on a windows os for SWT is to use Internet Explorer.
It is also possible to implement non-os linked browsers like firefox by installing xulrunner, but this is an additional 20mb (not extensive but something to note), as the more recent versions of firefox are not recognised.
It is a requirement to have javaxpcom support and xulrunner to implement a firefox style browser in SWT. The easiest way to go about getting the last version supported by Firefox (xulrunner 1.9.) for javaxpcom is to download an application called prism. Prism uses xulrunner as its base to create browsers that are standalone to specific urls. There is a known hacked xulrunner 1.9. which has some issues - see wikipedia for information on this, if you aim to download xulrunner 1.9.* compiled already from source.
Xulrunner is developed alongside each and every new edition of the Firefox browser. Until v 1.9.* (firefox 3.* if I am not mistaken), there was additional support for a function called javaxpcom which was an interface written to link the c library Xpcom's functions to java.
Firefox's initial announcement of how easy and simple it was to embed and style your own flavour of xulrunner, and that it was easy to port to many different languages (support for C, initially support for Java, Python amongst others), but obviously there is some overhead to accessing and adding libraries.
Xulrunner is being promoted as a potential software solution to compete in the same playing field as java, but suffers the same limitations of javascript / html applications. Within those parameters it functions.
Since version 2 of xulrunner, support for javaxpcom has withered, as there is no active maintainer of that code. It is open source solution so presumably interest from someone with requisite skills might pick that up. Xulrunner is now recently in version 6 at the time of writing this answer. Xpcom however is still going strong. Presumably there was insufficient interest / activity in the community, amongst other potential issues.
Since javaxpcom has been reported to have broken, and requires someone to maintain an interest in the code; that work would have to come from someone who has a vested understanding of Xulrunners xpcom, or at least a collaboration from someone who has an understanding of changes made in xpcom. Comments from what I have read suggest that documentation on xulrunner may not be a detailed as it could be, nor on javaxpcom, but I am not in a position to evaluate these comments.
I had hoped that the basic core functionality could simply have been maintained, keeping the hard work involved in recent browser improvements in functionality 'embeddable'. Firefox's amendments have resulted in extensions breaking in newer versions, a function of keeping up with the times.
A non-open source solution that embeds a web-browser and the like is called webrenderer.com. It is maintained, but there is a price tag attached, which is not insignificant.
Another possible route to implement a web-browser in a java application is to go the route of a cross platform development platform called Qt. QtJambi is a LPGL licensed (previously commercial) webbrowser that has extensive documentation and support for Java implementation. There is obviously learning curve attached to using any new platform. Reasons for the move from commercial to LPGL? Presumably viability and commercial demand for java supported browsers has .... waned? A function of the times we live in no doubt.
GWT is the java / webbrowser google solution. GWT works by allowing developers to write java solutions, and then ports and converts this solution into a stable javascript that can be obfuscated. However GWT is not suited to developing a desktop type application, in the sense that it requires a 'server' implementation for the backend. In this sense it is most suited or similar to a PHP or ASP style client / server solution.
Whilst it is not impossible to write GWT as a desktop application, presumably you would have to implement Apache Derby, or Jetty or similar as an in-house server to do all the to-ing and fro-ing. Its not neat, elegant or specifically designed for this, but madder solutions have been tried. There have been php style desktop applications.
For a number of political and financially motivated reasons it appears that support for porting / embedding java and open source browsers is not on the up and up, a trend which began with apple's lack of interest in supporting java applications. Mind you, the browsers are open source and presumably with enough time and energy, javaxpcom etc. could be sustained, but who is going to do this for free, I do not know.
Java promised a java browser implementation but this was not released with the Java 7, and requires like most other things, an amount of work. Perhaps in 8? It is unclear even there if it will be ready by then, I can't quite work out if its currently on the roadmap or not, and doesn't appear to be their highest priority include.
SWT requires some additional libraries and installation to work on various platforms.
Some other open source solutions which do at least in part appear to work, but the maintenance, support and current interest in appears mixed to none, include JRex, MozSwing. Rhino is a javascript engine, flying saucer an html4 renderer.

If you are talking about Java SE based application I would highly recommend swt 'Browser' component. I have used it myself within Eclipse RCP based app and it works like a charm (flash, javascript support)

Which features does JavaFX 2 provide and how mature is the implementation compared to related technology like Java Applets or Silverlight?

Which functionality will be shipping with version 2 and are there any additions known to be planned for later versions?
Will it reuse parts of the Applet/WebStart technology which are a bit more mature now or is the browser integration technique completely new? Will it be possible to access the whole JavaSE library or is it limited to a subset like Silverlight?
What about the overall rendering speed and architecture? Is it comparable to Swing/AWT?

It's just a JavaSE library, so you can use the whole Java SE API.
You can use java webstart or addtional integration features too (javascript).
In the end its really a full featured java SE application or Applet, but not on swing, but on JavaFX.
But if you want, you can mix Swing and JavaFX too.

To JavaFX [1.3] or not?

We are starting a new Desktop application and considering using javafx but we are a bit confused given the recent announcement by oracle that javafx scripting is not going to be developed further.
Since we cannot wait for javafx 2.0, is it a good idea to use the version 1.3 (which) or should we look at a different technology?
Your opinion will be very much appreciated.

JavaFX 2.0 has been released General Availability http://javafx.com
and http://javafx.com/downloads/all.jsp
and the roadmap has been updated http://javafx.com/roadmap/

You should have a close look at the license for JavaFX, particularly the provisions for distribution if you are developing a desktop application.
My understanding of the current 1.3 license is that distribution of the run-time is prohibited, except through Java Web Start. This is pretty much a deal-breaker for us on the desktop.

There has been an ever increasing trend away from desktop applications to web applications recently, and the advantages are large. Browsers are becoming more sophisticated, so you can do many desktop like features via a browser (and if you are considering JavaFX, then HTML based technologies may give you everything you need).
Also, a web application gives you the flexibility to update your code without having the user update it for you. Most browsers also allow you to launch in application mode (e.g the Chrome -app switch), which locks down the browser to specifically only point at your application.
If you truely want a real desktop experience though, then Swing or SWT should be sufficient for your needs.

Another alternative is Pivot.

Rumors of JavaFX script's death have been greatly exaggerated. It lives on, and is now called Visage.
While development is not incredibly active, over the last 30 days there have been 9 check-ins from 4 different people.
"Wounded"? Sure. "Dead"? Nope.

You could also use adobe flex. This is the technology that appears to be winning this war right now. I really liked the idea of using JavaFX with existing java work. I scrapped the idea because it would be really irresponsible to a) use a dead technology that looks like Oracle will not support in the future b) waiting on vaporware is not a good strategy (their JavaFX 2.0 future api).
Someone also mentioned apache pivot. Not sure how mainstream that is or what it's future support will be. According to their about it is a mainstream project:
"Pivot joined the Apache Incubator in January 2009 and graduated as a top-level Apache project in December 2009. It is currently driven entirely by the software development community."
I would run, not walk, away from JavaFX 1.3...
JMHO

Java FX2 allows you to leverage existing Java technology. I don't know how you would manage file access etc with html5-javascript. I am using JavaFX2 currently for my project and am happy with it.

No doubt that the javafx is improving further and oracle is extending functionality of javafx.
recently oracle announced early release of java 8 in which javafx is inbuilt and is having excellent controls.