We have developed a Windows based Desktop application using Java Swing and build a exe of it using javapackager available in Java 8.
Now we are trying to automate the testing of it using Karate-Robot but Robot could not detect the fields on the GUI of Application. To further dig we have installed AccessibilityInsights suggested by Karate Community to figure out how to access elements on the GUI but it seems that our application is not getting detcetd in it. See attached screenshot.
Can someone help us on how to fix the application?
Will Java Accessibility Bridge help here? As per documentation https://docs.oracle.com/en/java/javase/11/access/java-accessibility-guide.pdf, Application needs to implement Java Accessibility APIs when it needs to be available to Windows Assistive technologies.
Edit 1: As per https://docs.oracle.com/javase/tutorial/uiswing/misc/access.html, "Because support for the Accessibility API is built into the Swing components, your Swing program will probably work just fine with assistive technologies, even if you do nothing special."
Ashish--
There have been multiple iterations of Windows-based assistive technologies over the years. The current iteration has existed since 2005 and is called UI Automation. This is the technology that is required for OS-provided tools like Magnifier and Narrator. It's also the technology that is required by Accessibility Insights for Windows. From your description, it might also be the technology that is required by Karate-Robot.
Java Accessibility Bridge uses an older iteration of assistive technology that is no longer recognized by OS-provided accessibility tools. It is recognized by some third-party accessibility tools (JAWS and NVDA, for example), but it is not recognized by OS-provided accessibility tools. To the best of my knowledge, Oracle has no plans to support UI Automation in the future. When their docs say that your apps will "work just fine", they're referring to users who rely on these third-party tools.
Accessibility Insights for Windows has made a conscious decision to support only UI Automation, since supporting 2 platforms would make things much more difficult. We chose to go with UI Automation because it is the only technology that is:
Actively supported by the operating system, with new features being regularly added and documented.
Supported by native Windows controls and most actively supported language frameworks.
The technology that we would expect new assistive technology tools to implement first.
I'm sorry to be the bearer of bad news on this.
DaveTryon (Accessibility Insights team)
Related
I've found exactly the same question I just made but it is 7 years old; so I'd like to have an "updated" answer if it is possible. Thanks.
Old days
As commented by TrogDor, there were previously two ways to deploy a Swing app through the web:
Java Applet technologyYour app would appear within a rectangle on the web page, within the browser.
Java Web Start technologyClicking a link on a web page would download a copy of your Swing app to the user’s local machine, where your app would then be launched locally using a locally-installed JVM. So your app runs separate from the web browser. This click-to-download-and-run process is defined by Java Network Launching Protocol (JNLP).
Both of these are being phased out.
➥ For details, see the white paper Java Client Roadmap Update published by Oracle, updated 2020-05-11.
Nowadays
The modern approach is to build a Swing and/or JavaFX app, then deploy by using a packaging tool to include a JVM. You end up with a complete self-contained self-launchable application.
This means you need multiple builds, one app for each platform your users may deploy on (macOS, Linux, BSD, Windows, and so on). While that is an additional burden to you, the flip-side is that you control exactly what version of Java is being used to run your app.
Because of the Java Platform Module System (JSR 376) in Java 9 and later, you can now strip down the bundled JVM and libraries to include only the parts actually used by your particular app.
The build tools for packaging your app have been rapidly evolving in recent years. So be sure to do your research to find the most robust and modern tooling.
Alternatives
You might consider any of these alternatives:
Remote execution
OpenWebStart A re-implementation of Java Web Start
GraalVM Ahead-of-time native-code compilation of your Java app
Vaadin Flow Using Java to build desktop-style apps delivered as web apps
Remote executions
Some vendors may offer a product or service to execute your Swing app remotely while displaying the user-interface within a web browser.
One such company is WEBSWING Ltd. with their Webswing product.
OpenWebStart
You might be interested in a separate implementation of Java Web Start technology.
While Oracle is phasing out Java Web Start, there is an open-source implementation of JSR 56: Java Network Launching Protocol and API called OpenWebStart. See GitHub. This project is currently maintained by the company Karakun, based on the IcedTea-Web core functionality developed at Adoptium (née AdoptOpenJDK).
GraalVM
A cutting-edge alternative is to build an entirely native-code ahead-of-time compiled version of your app using GraalVM.
Vaadin Flow
An entirely different way to build a web app by using Java is the Vaadin Flow framework.
You specify your user-interface layouts with widgets in a manner quite similar to Swing, specifying an arrangement of widgets (buttons, fields, labels, etc.). You can do so using your choice of straight Java code, an XML-based description language, or a visual design tool.
At run-time, Vaadin automatically automatically generates the HTML, CSS, and JavaScript necessary to render your app remotely on the client user’s machine within a web browser. So, you have pure Java on the server-side, and no Java on the client-side, just Web standards technologies built into all modern browsers.
More info
All of this has been covered many times already on Stack Overflow. So search to learn more.
For tips on obtaining a JVM to bundle with your app, see How to get java 11 run-time environment working since there is no more jre 11 for download?.
Nowadays you can run Swing app on server with UI in the browser. You can find example here: https://github.com/JetBrains/projector-demo.
Update: JetBrains ceased development of Projector as a separate product. See their notice. They have incorporated the technology for their own use in their Gateway product for remote execution of their IDE products.
The possibility of connection to JAWS and NVDA through Java Access Bridge is needed. As far as I understand there is no support for Assistive Technologies in JavaFX (unlike in Swing).
I found the project called fxaccessible, but it is not well suited in my case.
How can I enable support for assistive technologies in a JavaFX program?
At least some of the controls have source code for accessibility related implementations.
The great majority of JavaFX code is open source, so you can download the source repository and check the extent of accessibility integration in the current builds and also contribute any modifications you make for accessibility support back to the JavaFX project if you desire.
The JavaFX roadmap states:
Oracle is committed to providing accessibility support in its product offerings, and JavaFX is planned to be fully accessibility compliant as part of the JDK 8 release.
So one course of action is to wait until JDK 8.
That is rather risky though as the roadmap also includes a disclaimer:
It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.
I'd advise posting your question to the openjfx-dev mailing lists where the Oracle developers and project management can advise on the current status (and possibly rough future timeline) of the accessibility integration into JavaFX as well as the current technologies and strategies used for the accessibility implementation in the code base.
Also check the javafx issue tracker for issues related to accessibility.
The fxaccessible project you reference in your question (as well as being not well suited to your case) is not really applicable for anybody anymore as it is based on the obsolete JavaFX 1.x scripting technology and there is no way it can function with modern JavaFX 2.x systems which don't include the ability to run JavaFX 1.x script.
Question: embed a portable web browser without cache into a java app? Using Java SE.
Can this be done? Are there any JNI supports for e.g. Chrome or Firefox?
Maybe Qt? QtJambi would allow interfacing with the Webbrowser? From what I understand QtJambi is now maintained by the community and LPGL, is QtWebbrowser embeddable for this purpose in the same licence?
Examining the available documentation and over the past few days this is what has emerged:
The open source browsers are Webkit (which is used for e.g. Safari), and the Mozilla flavoured Firefox. There is also Chromium which is the open source version of the Google flavoured Chrome which uses a lesser javascript engine (squirrel as opposed to v8) and does not have the multi-separate-thread modularisation of Google Chrome.
Additional information on Firefox: Mozilla Firefox is based on an application called xulrunner which uses xml and javascript to provide the additional functionality of firefox (other than the 'not so basic' browser - such things as the html parser, css renderer and javascript parser). Xulrunner is the implementation of the browser.
Firefox implements its own private xulrunner, however additionally there is xulrunner as a standalone install.
Xulrunner can be used to create standalone applications, that are based on a web-browser, and has support for javascript. Xulrunner is also used to create the plugins for firefox providing additional functionality.
In order to connect the front end to something a bit more sustained and powerful than javascript (which has improved in leaps and bounds in recent years), but is still not the strongest sustained engine, there is a C-library called xpcom.
Xulrunner uses xpcom (written in C) to expose the javascript variables and parse the dom, and link javascript variables, and do other 'browsery' things like read headers etc. More on Xulrunner and xpcom in a moment.
If you want a browser that is recent and functional (html 4 or 5, good support for css2 at least, and some css3, and solid rendering of javascript), your best bet other than compiling and embedding one of these open source browsers is to use SWT.
SWT is a gui interface, similar to swing, except that it implements a webbrowser type interface, and can be used to implement a browser. SWT also has some developed interfaces to interact both with the dom and the like. SWT was developed by IBM and is actively maintained.
What SWT does is implement the OS system browser, and has support for every major os. On the upside, this means that there is no need to embed your own browser, on the downside you are stuck with whatever browser / parsers exist on the operating systems, at whichever version. The default behaviour on a windows os for SWT is to use Internet Explorer.
It is also possible to implement non-os linked browsers like firefox by installing xulrunner, but this is an additional 20mb (not extensive but something to note), as the more recent versions of firefox are not recognised.
It is a requirement to have javaxpcom support and xulrunner to implement a firefox style browser in SWT. The easiest way to go about getting the last version supported by Firefox (xulrunner 1.9.) for javaxpcom is to download an application called prism. Prism uses xulrunner as its base to create browsers that are standalone to specific urls. There is a known hacked xulrunner 1.9. which has some issues - see wikipedia for information on this, if you aim to download xulrunner 1.9.* compiled already from source.
Xulrunner is developed alongside each and every new edition of the Firefox browser. Until v 1.9.* (firefox 3.* if I am not mistaken), there was additional support for a function called javaxpcom which was an interface written to link the c library Xpcom's functions to java.
Firefox's initial announcement of how easy and simple it was to embed and style your own flavour of xulrunner, and that it was easy to port to many different languages (support for C, initially support for Java, Python amongst others), but obviously there is some overhead to accessing and adding libraries.
Xulrunner is being promoted as a potential software solution to compete in the same playing field as java, but suffers the same limitations of javascript / html applications. Within those parameters it functions.
Since version 2 of xulrunner, support for javaxpcom has withered, as there is no active maintainer of that code. It is open source solution so presumably interest from someone with requisite skills might pick that up. Xulrunner is now recently in version 6 at the time of writing this answer. Xpcom however is still going strong. Presumably there was insufficient interest / activity in the community, amongst other potential issues.
Since javaxpcom has been reported to have broken, and requires someone to maintain an interest in the code; that work would have to come from someone who has a vested understanding of Xulrunners xpcom, or at least a collaboration from someone who has an understanding of changes made in xpcom. Comments from what I have read suggest that documentation on xulrunner may not be a detailed as it could be, nor on javaxpcom, but I am not in a position to evaluate these comments.
I had hoped that the basic core functionality could simply have been maintained, keeping the hard work involved in recent browser improvements in functionality 'embeddable'. Firefox's amendments have resulted in extensions breaking in newer versions, a function of keeping up with the times.
A non-open source solution that embeds a web-browser and the like is called webrenderer.com. It is maintained, but there is a price tag attached, which is not insignificant.
Another possible route to implement a web-browser in a java application is to go the route of a cross platform development platform called Qt. QtJambi is a LPGL licensed (previously commercial) webbrowser that has extensive documentation and support for Java implementation. There is obviously learning curve attached to using any new platform. Reasons for the move from commercial to LPGL? Presumably viability and commercial demand for java supported browsers has .... waned? A function of the times we live in no doubt.
GWT is the java / webbrowser google solution. GWT works by allowing developers to write java solutions, and then ports and converts this solution into a stable javascript that can be obfuscated. However GWT is not suited to developing a desktop type application, in the sense that it requires a 'server' implementation for the backend. In this sense it is most suited or similar to a PHP or ASP style client / server solution.
Whilst it is not impossible to write GWT as a desktop application, presumably you would have to implement Apache Derby, or Jetty or similar as an in-house server to do all the to-ing and fro-ing. Its not neat, elegant or specifically designed for this, but madder solutions have been tried. There have been php style desktop applications.
For a number of political and financially motivated reasons it appears that support for porting / embedding java and open source browsers is not on the up and up, a trend which began with apple's lack of interest in supporting java applications. Mind you, the browsers are open source and presumably with enough time and energy, javaxpcom etc. could be sustained, but who is going to do this for free, I do not know.
Java promised a java browser implementation but this was not released with the Java 7, and requires like most other things, an amount of work. Perhaps in 8? It is unclear even there if it will be ready by then, I can't quite work out if its currently on the roadmap or not, and doesn't appear to be their highest priority include.
SWT requires some additional libraries and installation to work on various platforms.
Some other open source solutions which do at least in part appear to work, but the maintenance, support and current interest in appears mixed to none, include JRex, MozSwing. Rhino is a javascript engine, flying saucer an html4 renderer.
If you are talking about Java SE based application I would highly recommend swt 'Browser' component. I have used it myself within Eclipse RCP based app and it works like a charm (flash, javascript support)
We are starting a new Desktop application and considering using javafx but we are a bit confused given the recent announcement by oracle that javafx scripting is not going to be developed further.
Since we cannot wait for javafx 2.0, is it a good idea to use the version 1.3 (which) or should we look at a different technology?
Your opinion will be very much appreciated.
JavaFX 2.0 has been released General Availability http://javafx.com
and http://javafx.com/downloads/all.jsp
and the roadmap has been updated http://javafx.com/roadmap/
You should have a close look at the license for JavaFX, particularly the provisions for distribution if you are developing a desktop application.
My understanding of the current 1.3 license is that distribution of the run-time is prohibited, except through Java Web Start. This is pretty much a deal-breaker for us on the desktop.
There has been an ever increasing trend away from desktop applications to web applications recently, and the advantages are large. Browsers are becoming more sophisticated, so you can do many desktop like features via a browser (and if you are considering JavaFX, then HTML based technologies may give you everything you need).
Also, a web application gives you the flexibility to update your code without having the user update it for you. Most browsers also allow you to launch in application mode (e.g the Chrome -app switch), which locks down the browser to specifically only point at your application.
If you truely want a real desktop experience though, then Swing or SWT should be sufficient for your needs.
Another alternative is Pivot.
Rumors of JavaFX script's death have been greatly exaggerated. It lives on, and is now called Visage.
While development is not incredibly active, over the last 30 days there have been 9 check-ins from 4 different people.
"Wounded"? Sure. "Dead"? Nope.
You could also use adobe flex. This is the technology that appears to be winning this war right now. I really liked the idea of using JavaFX with existing java work. I scrapped the idea because it would be really irresponsible to a) use a dead technology that looks like Oracle will not support in the future b) waiting on vaporware is not a good strategy (their JavaFX 2.0 future api).
Someone also mentioned apache pivot. Not sure how mainstream that is or what it's future support will be. According to their about it is a mainstream project:
"Pivot joined the Apache Incubator in January 2009 and graduated as a top-level Apache project in December 2009. It is currently driven entirely by the software development community."
I would run, not walk, away from JavaFX 1.3...
JMHO
Java FX2 allows you to leverage existing Java technology. I don't know how you would manage file access etc with html5-javascript. I am using JavaFX2 currently for my project and am happy with it.
No doubt that the javafx is improving further and oracle is extending functionality of javafx.
recently oracle announced early release of java 8 in which javafx is inbuilt and is having excellent controls.
Does anyone have any experience with a really good RIA platform. I'm going to begin developing an application, and I intend to use Netbeans RCP for the thick client. I'd also like to create a RIA version of the same app. I've looked around for a RIA framework that rivals what you can get with the Netbeans RCP, and really haven't found one. My requirements are as follows.
Modular - I would like to be able to add and remove features without recompiling
Out of the box features - I'd like to have feature out of the box that are similar to Netbeans RCP (or most any other RCP), such as a menu system built in, a windowing system (allowing docking and remembering where windows were between sessions), Authentication and Authorization support out of the box, as well as the "Options' pane support (I really like how I can add amodule, and have that module expose options through the standard "options" dialog).
Actions and commands - I'd like to be able to handle actions from other modules without needing to couple the action to the other module.
I would like to deploy the app through GlassFish, but that's not a deal breaker.
Modern - the framework should use the latest standards, and I'm planning on moving to J2EE7 as soon as I can, so the platform must be actively updated to support such a migration.
I'm willing to pay, but the license must not be "per server" or "per instance"
Java isn't a must but I'm planning on using java for the rest of the app (services layer) and the Netbeans RCP front end.
I've been a .net developer for the last 6 years, and have just recently looked into the world of java, and I'm really amazed at the rich community, I thought .net had a large open source community, but the java one is orders of magnitude bigger. I don't know of anything even close to the Netbeans RCP in the .net community, let alone a RIA version.
I've looked at a few frameworks, but most of them seem to be mostly libraries that you can use to build applications, I really don't want to spend the time and maintanance cost to build the infrastructure listed above. Things I've looked at are....
Tepestry
"SmartClient"
GWT
Wicket
These all seem to be good building block, but I want an out of the box solution. The application will be mostly CRUD operations, but I will also be streaming video through the app, as well as handling alerts and events from the server.
Thanks for any advice
Joshua
Eclipse with the GWT plugin is fairly easy to get running out of the box. Step through the GWT tutorial first and you'll be ready to go. I realize you've already looked at GWT but the Eclipse plugin simplifies a lot of grunt work.
Indeed. GWT Eclipse plugin simplifies compilation, depoloying and running them (Jetty). It also has App Engine support.