Develop Reference

how can i make UI automation in java by which i can capture button or menu of any external application dynamically

I want to automate an external application, but I have several problems:
How can I recognize a button or other field of an external application in Java?
I use the Robot class in Java for making notepad automation where I open notepad, select file menu, and save or exit, etc.
The problem is, it needs X,Y coordinates for the mouse pointer to go to the proper location.
I want to make it more dynamic, i.e. it should recognize the file menu of a running notepad anywhere on the desktop.
How can this be done in Java? Is there any class in Java I can use to do this?
Thanks everyone to give me response, I want to be more specific i want to know how can i make ui automation by using any tool if it is not possible in java or using any api of java.automation tool must be freeware.....i am searching net for that i found AutoIt is like that.But if any one do this type of things please share his/her experiance means is it possible to do that in AutoIt or not possible if not then which tool do that kind of things.

It is easy to integrate Sikuli into a Java-application since it is written in Java. Sikuli uses image recognition to find elements visible on the screen like buttons and such. It is very easy to use and provides an alternative for tasks that are difficult to handle with static positioning, like finding moving windows and such.
Take a look at this: http://sikuli.org/docx/faq/030-java-dev.html
Hope this helps!

You should have a look at Sikuli. It takes as inputs images of the ui elements to select an area in the targeted app. It's a UI Automation Application
That's a bit difficult to install (at least on Debian/Ubuntu, where I tested it), as you'll need a recent version of OpenCV, a particular version of JXGrabKey but the quality of the program worth the trip. Good Luck

Java doesn't have an API to examine the UI of another application; that would be a very big security risk.
Which is why the Robot class can only record events (key presses, mouse movements and clicks) but not which UI element was involved in most cases.
It would be possible to do more if the external application was written in Java because then, you could analyze the objects in memory but for obvious reasons, this isn't possible for C++ or .NET applications.

mouse event, not on Jframe

i want to make my program to work after a mouse button was clicked,
but not necessarily on a Jframe of some sort.
for example, i want my program to start working after the PLAY button was clicked
on windows media player.
is there a way of doing that?

Yes, but not in standard Java. You want to listen for an event in the OS not in your Java GUI which Java can't do by itself. This can work, but in order for Java to interact closely with the operating system you will need to use the Java Native Interface (JNI) and C/C++ code or Java Native Access (JNA) (which is a bit easier to use in my opinion). I've also done similar stuff in Windows using a Windows scripting language such as AutoIt, and then connect this to the Java application either via sockets or via standard input and output.
Java Native Interface
Java Native Access
AutoIt Windows Scripting Language

Block all other input to an application and control it from a wrapper in Java

I have a windows application which has a complex GUI that I would like to hide from users. In order to do this, I would like to create a wrapper with an extremely simple interface that overlays this application and automates a number of actions when a user clicks a single button on the wrapper. (I hope "wrapper" is the proper term.) Is it possible to use Java to block input to the underlying application so that users cannot inadvertently mess up the automation? How would I go about this? Also, how can I automate key presses and clicks to the application without hijacking the mouse? Is this possible in Java?
I have looked at java.awt.Robot, but it appears to hijack the mouse.
I have also looked at AutoIT, but it too hijacks the mouse and does not integrate with Java.
Neither of these options seem powerful enough for what I need, but I do not know how else to proceed.

I recommend that automation via the GUI only as the last resort if you really have no other alternative.
If there is an API that your application exposes, I would try to use that. For example, if the GUI is implemented in one DLL and the logic in another, then you can use JNA to load your application logic DLL and invoke the application functions directly from java. Even better would be if your application exposes a COM/OLE interface - there are plenty of Java<>COM briges, that will alow you to call this interface directly, e.g. Jacob.
If you really have no choice but to automate via the GUI, then here's how to go about doing that:
Use JNA to access the windows shell API. You can then use ShellExecute to launch your wrapped application. Specifically, passing SW_HIDE as the window mode should help ensure that the application does not appear.
Use JNA to access the windows API FindWindow to find your application window. You can also make it invisible using the ShowWindow API, just in case step 1 did not work (not all applications are written to use the nCmdShow parameter.)
You can now post messages to the application window using PostMessage. You can send keystrokes and mouse events using windows messages. E.g. See WM_KEYUP, WM_LBUTTONDOWN.
Because the wrapped application window is made invisible, you don't need to "block" that application, The user simply cannot access it's GUI. But you can still programmatically send input to it.

Getting Callback from OS X When Application Foucs Changes?

I am trying to find a way how to write a "software sensor" in Java, running on Mac OS X (10.5), which records each application focus change.
My first idea was to write a "pull sensor" that uses an Applescript, which just returns the name of the app. that is in focus. Obviously this approach is not very good.
Therefore, I was wondering if there is the possibility to get a callback or some kind of notification directly from the operating system, whenever another application gets into the focus.
Thanks in advance!
Cheers Julian

java recieves no events from other applications, so there would be nothing to intercept and record. im sure OSX's SDK has something you could use though. it wouldn't be java based (unless you used JNI or JNA).

Copying selected text to a Swing Java app?

My goal is assign a global hotkey (JIntellitype, JXGrabKey) that would pass an arbitrary selected text to a java app.
The initial plan is to utilize the java.awt.Robot to emulate Ctrl-C keypress and then get the value from clipboard.
Probably there's a more elegant solution?
EXAMPLE: Open Notepad, type in some text, select that text. Now, that text needs to be copied into a Java app.

I guess you want to implement a global input monitor, Java is not so straightforward to do the job. You may have to write an API hook and pack it in a DLL, then invoke it via JNI.

The Robot only sends events inside your JVM. I don't know of anything to send events out to the operating system. Though there is plenty of examples out there of making JNI calls to the windows API, it would then be platform specific.

I've gone with with Robot and that works just fine.