I have a single Swing JFrame which contains a couple JLabels, a few read-only JEditorPanes, and a couple buttons. JAWS users are requesting that the application be read with less user interaction required. Right now, the application starts with a certain element in focus, and the user has to press tab to move focus and continue to the next element. Whereas with a web page, for example, JAWS will just go through and read everything without requiring the user to press tab. Especially annoying for our JAWS users, the read-only JEditorPanes only read one line of text at a time requiring the user to press the up and down arrow keys to move between them.
How can I make this JFrame be read to JAWS version 9 users more fluently?
You may be able to do this but it would require using the Jaws COM api that isn't really supported. Jaws interacts with desktop applications in a much different manner then with websites. Jaws renders web pages into a virtual buffer and does all kinds of magic to make them easier to use. Jaws doesn't do this for desktop applications. I assume part of the reason it doesn't do this is desktop applications don't have all the information available to create an alternative presentation mode that is available with HTML. For what ever it's worth even utilities included with Jaws don't automatically have there read only edit fields spoken. Saying all that if you really need to do this your best bet would probably be to speak the contents of read only edit fields through the jaws api when the user gives them focus. This would require using COM from with in your Java application to access the api. I know there are java libraries that will allow you to use COM in some form but I've never done this. You can find a copy of the api in your standard Jaws installation directory, in my case it's
c:\program files\Freedom Scientific\jaws\jfwapi.dll
You'll have to use your favorite application for looking at COM libraries to determine what methods are available since there's no official documentation that I can find. You may also want to take a look at the following code sample, it's in AutoIt but gives a general idea of how to speak text using multiple screen readers. http://www.scribd.com/doc/19371/speak-with-autoit
Related
I'm using IntelliJ IDEA, I have coded a frame which I want to put into an HTML file so I can run it in my browser, how do I do this now that I cannot use JApplet? I have found this documentation: http://docs.oracle.com/javase/tutorial/deployment/webstart/deploying.html and this http://docs.oracle.com/javase/tutorial/deployment/webstart/deploying.html but I am new to programming and find this difficult to follow. I don't know for instance how I would go about putting my class files and the image that I used in a separate directory nor do I know how I go about signing my application so that it will run in a browser.
I want to put into an HTML file so I can run it in my browser,
You can use the Desktop class. This class allows you to access default applications from your desktop.
Read the section from the Swing tutorial o How to Integrate With the Desktop class for more information and working examples.
See Java Plugin support deprecated and Moving to a Plugin-Free Web.
Note that is one of my 'copy/paste comments' that does not explicitly mention JFrame based apps., however the links are still relevant in that Oracle & browser makers would not be phasing out support for applets if they wanted programmers to keep trying to shove rich client apps (e.g. Swing GUIs) into thin client web pages.
OTOH you can offer a JFrame (or a JApplet) to be launched from a link in a web page to end up free floating on the desktop of the user by using Java Web Start.
Even then, it is not a simple matter for the programmer or the end user. The programmer needs to ensure the app is digitally signed using a code signing certificate issued by a CA (usually they are expensive). The end user used to just be able to click the link, 'OK' the prompts produced by the Java virtual machine, and see the app appear on-screen. But now most browsers will download the launch file to the local file system rather than directly hand it to the JVM to be launched. So the user faces an extra step in explicitly finding the downloaded launch file and double clicking it.
This is all due to security concerns related to bugs in the plug-ins that run things in web pages. So if you were to find a way around all these hoops, please let us know. It is a security bug that requires urgent fixing.
How do I check (using Java or JavaScript) if a browser has support for blind or partially-sighted people?
If you are looking to determine whether an assistive technology (like JAWS or Window-Eyes) are installed, no, you cannot with Javascript. These technologies use OS-level APIs, and current browsers don't do anything that would cause their presence to be detectable. There is a flash-based hack that can detect newer versions of screen readers see http://webaim.org/techniques/flash/techniques
Perhaps you want to provide a different Flash interface or options if a user is using a screen reader. For instance, you may want to provide additional buttons or turn on self-voicing features when the person watching your movie is using a screen reader. You can detect screen readers using ActionScript. The Accessibility.isActive() function will return "true" if a screen reader that is capable of accessing Flash content is detected (currently only up-to-date versions of Window-Eyes, JAWS, and IBM Home Page Reader). For instance, you might add:
if (Accessibility.isActive()) {
_root.selfVoicing.play();
}
Important
Because there is a slight delay between when your movie begins and when the screen reader is detected, Accessibility.isActive() may incorrectly return false during the first few moments that your movie is playing. The Flash movie must also have focus in order for this to work. This can be solved by associating the ActionScript with a button within the Flash movie.
Finally, this method of detecting a screen reader, will not detect all screen readers. It will only detect the most recent versions of screen readers that support MSAA and have the Flash 6+ player installed.
This method is the only way of automatically detecting screen readers on the web. If you want to provide alternative content for screen reader users, you could use a Flash movie to detect the presence of a screen reader and redirect appropriately. However, this also assumes that the user has the Flash player installed (which most users do). If the user does not have Flash, then the detection will not work and the user might be presented with additional accessibility issues. Care must also be taken in providing alternative content for screen reader users. The alternative content must be kept up-to-date and provide the same information and functionality as the non-screen reader content. Here is the code to redirect based on screen reader presence:
if (Accessibility.isActive()) {
getURL(screenreaderpage.htm);
} else {
getURL(normalpage.htm);
}
The short answer is there's no bulletproof way to do this. While the flash solution may work in some situations it isn't a good solution. I'm a totally blind programmer and since flash is generally so completely inaccessible I don't have it enabled by default in my browser. iPhones are used by a lot of blind individuals and for obvious reasons the flash solution won't work on them either. Even if the browser does support assistive technology that does not mean your site will be accessible. If your entire site is JavaScript or WebGL that draws pictures onto a canvas it doesn’t matter how good the screen reader supporter in a browser is, there’s no way to provide information on dynamically created graphics unless the author provides textual descriptions of what is drawn to the screen. Ideally you would want to code your site in such a way that it can be used by both blind and sited individuals with no changes. If that doesn't work you could always put an invisible link to an alternative version of your site or other invisible content specific to screen readers that won't be displayed to sited users. For a discussion of creating content that is only visible to screen reader users see this link For a general intro to web accessibility see http://webaim.org/intro/
I want to automate an external application, but I have several problems:
How can I recognize a button or other field of an external application in Java?
I use the Robot class in Java for making notepad automation where I open notepad, select file menu, and save or exit, etc.
The problem is, it needs X,Y coordinates for the mouse pointer to go to the proper location.
I want to make it more dynamic, i.e. it should recognize the file menu of a running notepad anywhere on the desktop.
How can this be done in Java? Is there any class in Java I can use to do this?
Thanks everyone to give me response, I want to be more specific i want to know how can i make ui automation by using any tool if it is not possible in java or using any api of java.automation tool must be freeware.....i am searching net for that i found AutoIt is like that.But if any one do this type of things please share his/her experiance means is it possible to do that in AutoIt or not possible if not then which tool do that kind of things.
It is easy to integrate Sikuli into a Java-application since it is written in Java. Sikuli uses image recognition to find elements visible on the screen like buttons and such. It is very easy to use and provides an alternative for tasks that are difficult to handle with static positioning, like finding moving windows and such.
Take a look at this: http://sikuli.org/docx/faq/030-java-dev.html
Hope this helps!
You should have a look at Sikuli. It takes as inputs images of the ui elements to select an area in the targeted app. It's a UI Automation Application
That's a bit difficult to install (at least on Debian/Ubuntu, where I tested it), as you'll need a recent version of OpenCV, a particular version of JXGrabKey but the quality of the program worth the trip. Good Luck
Java doesn't have an API to examine the UI of another application; that would be a very big security risk.
Which is why the Robot class can only record events (key presses, mouse movements and clicks) but not which UI element was involved in most cases.
It would be possible to do more if the external application was written in Java because then, you could analyze the objects in memory but for obvious reasons, this isn't possible for C++ or .NET applications.
I have a windows application which has a complex GUI that I would like to hide from users. In order to do this, I would like to create a wrapper with an extremely simple interface that overlays this application and automates a number of actions when a user clicks a single button on the wrapper. (I hope "wrapper" is the proper term.) Is it possible to use Java to block input to the underlying application so that users cannot inadvertently mess up the automation? How would I go about this? Also, how can I automate key presses and clicks to the application without hijacking the mouse? Is this possible in Java?
I have looked at java.awt.Robot, but it appears to hijack the mouse.
I have also looked at AutoIT, but it too hijacks the mouse and does not integrate with Java.
Neither of these options seem powerful enough for what I need, but I do not know how else to proceed.
I recommend that automation via the GUI only as the last resort if you really have no other alternative.
If there is an API that your application exposes, I would try to use that. For example, if the GUI is implemented in one DLL and the logic in another, then you can use JNA to load your application logic DLL and invoke the application functions directly from java. Even better would be if your application exposes a COM/OLE interface - there are plenty of Java<>COM briges, that will alow you to call this interface directly, e.g. Jacob.
If you really have no choice but to automate via the GUI, then here's how to go about doing that:
Use JNA to access the windows shell API. You can then use ShellExecute to launch your wrapped application. Specifically, passing SW_HIDE as the window mode should help ensure that the application does not appear.
Use JNA to access the windows API FindWindow to find your application window. You can also make it invisible using the ShowWindow API, just in case step 1 did not work (not all applications are written to use the nCmdShow parameter.)
You can now post messages to the application window using PostMessage. You can send keystrokes and mouse events using windows messages. E.g. See WM_KEYUP, WM_LBUTTONDOWN.
Because the wrapped application window is made invisible, you don't need to "block" that application, The user simply cannot access it's GUI. But you can still programmatically send input to it.
I've seen a lot of examples which use
getAccessibleContext().setAccessibleDescription(...)
to set some "magic" description.
What is the use of this description? Where can it be seen and
how should it support accessibility?
Why setDescription(...) is not used?
Additionally, what is your opinion / experience with Java accessibility
stuff?
I haven't actually used Swing's accessibility facilities in my applications (and I probably should), but I presume that it will aid in the use of screen readers and other technologies which to improve the accessibility of an application.
From the Accessibility and the Swing Set article:
If an application fully supports the
Java Accessibility API, it can be
compatible with, and friendly toward,
screen readers, screen magnifiers, and
other kinds of assistive technologies.
The linked article goes in to some depth about the accessibility features of Swing and the Accessibily API.
As pointed out the accessible information is used to expose information to assistive technologies such as screen readers (in short as a blind computer user I use a screen reader to gather useful information about what control has focus and other useful things happening in a application and then for it to speak it out to me. Examples of screen readers are NVDA www.nvda-project.org for windows, Orca http://live.gnome.org/Orca for the gnome desktop on unix platforms and voiceover included by default in MacOSX). For a number of standard controls which contain text you may be able to get away without having to explicitly set accessible information (eg. if you have a button with the text word "OK" then this probably will be spoken fine by a screen reader. The importance of setting accessible information comes in when you have no text showing (eg. a button with a graphic label) or when you are developing a custom control (from memory without going back through some of the Java swing docs, I think the accessible name is to identify the control and accessible description is to provide extra information, may be a clue as how to use it if its a custom control).
Now if you wish to see how this works, NVDA and Orca are opensource projects and voiceover is included in MacOSX 10.4 (I think) and higher, so you can try one of these at no cost (unlike some of the commercial offerings which can be very expensive).