Develop Reference

How to detect a pressed key globally, in a window without focus in java? [duplicate]

I'm making a small program in Java using the Robot class. The program takes over the mouse. while in the course of debugging if it starts acting in a way that I don't want it's hard to quit the program, since I can't move the mouse over to the terminate button in eclipse, and I can't use hotkeys to hit it because the mouse is constant clicking in another window, giving that window focus instead.
What I'd like to do is just hook up a keylistener so that when I hit q I can quit the program, but the only way I know how to do this involves making a window, and that window needs focus to capture the input. Is there a way to listen for keyboard or mouse input from anywhere, regardless of what has focus?

There is a library that does the hard work for you:
https://github.com/kwhat/jnativehook

This is not a trivial problem and Java doesn't give you a way to do it elegantly. You can use a solution like banjollity suggested but even that won't work all the time if your errant mouse clicks open another fullsized window currently open in your taskbar for example.
The fact is, Java by default gives developers very little control over the OS. This is due to 2 main reasons: security (as citied by java documentation) and the fact that different operating systems handle events completely differently and making one unified model to represent all of these would probably not make a whole lot of sense.
So to answer your question, I imagine what you want is some kind of behaviour for your program where it listens for keypresses globally, not just in your application. Something like this will require that you access the functionality offered by your OS of choice, and to access it in Java you are going to need to do it through a Java Native Interface (JNI) layer.
So what you want to do is:
Implement a program in C that will listen for global keypresses on your OS, if this OS is Windows than look for documentation on windows hooks which is well docuemented by Microsoft and MSDN on the web and other places. If your OS is Linux or Mac OS X then you will need to listen for global keypresses using the X11 development libraries. This can be done on an ubunutu linux distro according to a Howto that I wrote at http://ubuntuforums.org/showthread.php?t=864566
Hook up your C code to your Java code through JNI. This step is actually the easier step. Follow the procedure that I use in my tutorial at http://ubuntuforums.org/showthread.php?t=864566 under both windows and linux as the procedure for hooking up your C code to your Java code will be identical on both OSes.
The important thing to remember is that its much easier to get your JNI code working if you first code and debug your C/C++ code and make sure that it is working. Then integrating it with Java is easy.

Had same problem. In my case, robot just controlled a single Windows App, that was maximized. I placed these lines at top of main loop driving the robot:
Color iconCenterColor = new Color(255,0,0); // if program icon is red
if (iconCenterColor.equals(robot.getPixelColor(10,15)))
throw new IllegalStateException("robot not interacting with the right app.");
To cancel the robot, just alt-tab to another app. Works great for a simple one app driving robot.

Start the program from a command line in a terminal and use Ctrl-C to terminate it.

(As mentioned by #MasterID and shown on JNativeHook's documentation for native keyboard input detection {main GitHub project here}),
This code should be enough to listen to any key without app focus (press and/or release):
>>Remember to add the jnativehook library in your project to be able to use all its utilities.<<
public class yourClass implements NativeKeyListener {//<-- Remember to add the jnativehook library
public void nativeKeyPressed(NativeKeyEvent e) {
System.out.println("Key Pressed: " + NativeKeyEvent.getKeyText(e.getKeyCode()));
}
public void nativeKeyReleased(NativeKeyEvent e) {
System.out.println("Key Released: " + NativeKeyEvent.getKeyText(e.getKeyCode()));
}
public void nativeKeyTyped(NativeKeyEvent e) {
System.out.println("Key Typed: " + NativeKeyEvent.getKeyText(e.getKeyCode()));
}
public static void main(String args[]){
//Just put this into your main:
try {
GlobalScreen.registerNativeHook();
}
catch (NativeHookException ex) {
System.err.println("There was a problem registering the native hook.");
System.err.println(ex.getMessage());
System.exit(1);
}
GlobalScreen.addNativeKeyListener(new yourClass());
//Remember to include this^ ^- Your class
}
}
For this particular problem, use the nativeKeyPressed method like this:
public void nativeKeyPressed(NativeKeyEvent e) {
System.out.println("Key Pressed: " + NativeKeyEvent.getKeyText(e.getKeyCode()));
if (e.getKeyCode() == NativeKeyEvent.VC_Q){
System.exit(1);
}
}
Note that JNativeHook by default shows a lot of stuff in your console that you might not want, to change that, just add this right before the try-catch that you used in the main function as shown (this is also going to turn off warning and error messages, more info here):
//(From here)
Logger logger = Logger.getLogger(GlobalScreen.class.getPackage().getName());
logger.setLevel(Level.OFF);
logger.setUseParentHandlers(false);
//(To there-^)
try {
GlobalScreen.registerNativeHook();
}
catch (NativeHookException ex) {
System.err.println("There was a problem registering the native hook.");
System.err.println(ex.getMessage());
System.exit(1);
}
Disclaimer: I know this question was solved years ago, I just hope someone finds this a little easier to find/use.

Have your program open a second window which displays underneath your main window but is maximised, then your errant mouse clicks will all be received by the maximised window, and it can receive your keyboard input.

Here's a pure Java way to do it to solve the problem you've described (not the KeyListener problem... the quit test early when using robot problem):
Throughout your test, compare the mouse position with one that your test has recently set it to. If it doesn't match, quit the test. Note: the important part of this code is the testPosition method. Here's code that I used recently:
public void testSomething() throws Exception {
try {
// snip
// you can even extract this into a method "clickAndTest" or something
robot.mouseMove(x2, y2);
click();
testPosition(x2, y2);
// snip
} catch (ExitEarlyException e) {
// handle early exit
}
}
private static void click() throws InterruptedException {
r.mousePress(InputEvent.BUTTON1_DOWN_MASK);
Thread.sleep(30 + rand.nextInt(50));
r.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
Thread.sleep(30 + rand.nextInt(50));
}
private static void testPosition(int x2, int y2) throws ExitEarlyException {
Point p = MouseInfo.getPointerInfo().getLocation();
if(p.x != x2 || p.y != y2) throw new ExitEarlyException();
}

Alternate ways to handle Robot Class Steps in Selenium Test

My application supports only IE currently. In one case, it opens a pop up window with some fields which are not inspect-able. I am using Robot class features (tab sequences, mouse clicks, sendkeys) to enter data, do a search and other steps on that.
This scenarios works fine in my local but due to large volume of my scenarios we are running them in bulk on cloud machines where the Robot keys functions are not supporting.
Is there any alternate ways to handle this case to continue my scenarios running on virtual machines.
Have anyone faced such cases & can you share your experience handling it in selenium script
Robot class was the option tried which is working in local, need solution to run them on virtual machines.
Tried Switch to pop up window, frame - those are not working as well, the pop window is not getting identified itself (In the window its displayed as Search -- WebPage Dialog)
Below is a sample code snippet - i am using for a search function in the pop up using Robot Class
try {
Thread.sleep(5000);
sendTab(34);
sendRobotKey("enter");
String name ="ABC";
copyToClipboard(name);
Thread.sleep(2000);
cntrolVRobotKey();
Thread.sleep(2000);
sendRobotKey("enter");
Thread.sleep(2000);
sendTab(4);
Thread.sleep(2000);
sendTab(1);
sendRobotKey("enter");
}
catch (Exception e) {
e.printStackTrace();
}
}

Use AutoIt instead of robot class

How to open search box on web page using keyboard key (ctrl-f) in selenium web driver

How to open search box using windows keys ctrl+f in selenium webdriver:

Selenium doesn't provide a possibility to simulate keyboard actions. But to be able to do it, you can use a Robot class in Java.
You have not specified on which language are you developing. Thats why I have stored some useful links to different analogues of Robot:
For Python
For C#

FIRST INSTALL PYWINAUTO BY run pip install pywinauto in your python ide
from selenium import webdriver
from pywinauto.keyboard import SendKeys
import time
driver = webdriver.Chrome(executable_path="PATH OF CHROMEDRIVER")
driver.maximize_window()
driver.get("https://www.google.com")
time.sleep(5)
SendKeys("^F")

Finally, guys, I have found a solution, It is working fine for me, Thanks Andrei Suvorkov to suggest me read about robot class.
try {
Robot robot = new Robot();
// Simulate a mouse click
robot.mousePress(InputEvent.BUTTON1_MASK);
robot.mouseRelease(InputEvent.BUTTON1_MASK);
// ctrl + F
robot.keyPress(KeyEvent.VK_CONTROL);
robot.keyPress(KeyEvent.VK_F);
// CTRL+F is now pressed
robot.keyRelease(KeyEvent.VK_F);
robot.keyRelease(KeyEvent.VK_CONTROL);
} catch (AWTException e) {
e.printStackTrace();
}

How to simulate a real mouse click using java?

I'm attempting to perform a mouse click in Java, to click something in an external program. To do this, I'm using java.awt.robot, and the following code:
Robot bot = new Robot();
int mask = InputEvent.MOUSE_BUTTON1_DOWN;
bot.mouseMove(x, y);
bot.mousePress(mask);
bot.mouseRelease(mask);
Here's the problem. The external program is able to detect that this click is computer-generated and not human-generated, and hence, its rejecting this click.
I have already tried moving the mouse there naturally and that didn't have any effect. So my guess is, that it must be listening to the keyboard state or such, and telling from that, that the click is computer generated.
What do I have to do to set all keyboard / mouse states to act in the same way as a normal mouse click would?

Well I had the same exact requirement, and Robot class is perfectly fine for me. It works on windows 7 and XP (tried java 6 & 7).
public static void click(int x, int y) throws AWTException{
Robot bot = new Robot();
bot.mouseMove(x, y);
bot.mousePress(InputEvent.BUTTON1_DOWN_MASK);
bot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
}
May be you could share the name of the program that is rejecting your click?

FYI, in newer versions of Windows, there's a new setting where if a program is running in Adminstrator mode, then another program not in administrator mode, cannot send any clicks or other input events to it. Check your source program to which you are trying to send the click (right click -> properties), and see if the 'run as administrator' checkbox is selected.

it works in Linux. perhaps there are system settings which can be changed in Windows to allow it.
jcomeau#aspire:/tmp$ cat test.java; javac test.java; java test
import java.awt.event.*;
import java.awt.Robot;
public class test {
public static void main(String args[]) {
Robot bot = null;
try {
bot = new Robot();
} catch (Exception failed) {
System.err.println("Failed instantiating Robot: " + failed);
}
int mask = InputEvent.BUTTON1_DOWN_MASK;
bot.mouseMove(100, 100);
bot.mousePress(mask);
bot.mouseRelease(mask);
}
}
I'm assuming InputEvent.MOUSE_BUTTON1_DOWN in your version of Java is the same thing as InputEvent.BUTTON1_DOWN_MASK in mine; I'm using 1.6.
otherwise, that could be your problem.
I can tell it worked because my Chrome browser was open to http://docs.oracle.com/javase/7/docs/api/java/awt/Robot.html when I ran the program, and it changed to Debian.org because that was the link in the bookmarks bar at (100, 100).
[added later after cogitating on it today]
it might be necessary to trick the listening program by simulating a smoother mouse movement. see the answer here: How to move a mouse smoothly throughout the screen by using java?

With all respect the most likely thing is that you are mistaken about why the click is being 'rejected'. Why do you think some program is trying to determine if it's human or not? The Robot class (have used it a lot) should send messages that the operating system has no way to distinguish from a user doing the click.

Some applications may detect click source at low OS level. If you really need that kind of hack, you may just run target app in virtual machine's window, and run cliker in host OS, it can help.

You could create a simple AutoIt Script that does the job for you, compile it as an executable and perform a system call there.
in au3 Script:
; how to use: MouseClick ( "button" [, x, y [, clicks = 1 [, speed = 10]]] )
MouseClick ( "left" , $CmdLine[1], $CmdLine[1] )
Now find aut2exe in your au3 Folder or find 'Compile Script to .exe' in your Start Menu and create an executable.
in your Java class call:
Runtime.getRuntime().exec(
new String[]{
"yourscript.exe",
String.valueOf(mypoint.x),
String.valueOf(mypoint.y)}
);
AutoIt will behave as if it was a human and won't be detected as a machine.
Find AutoIt here: https://www.autoitscript.com/

Control a Windows apps using Java

I would like to know if there is any way I can control a Windows application using Java code. I have already googled it, and found that it can be done using JNI or a library called NewJawin.
I want to control Windows Media Player using Java code, e.g. play, pause, and change songs, but could find no relevant example to get me started so far. Do you guys have any suggestion?

As no one has answered this question, I thought I would.
public void firePlay() {
//CTRL + P
//import java.awt.Robot
//import java.awt.KeyEvent
try {
Robot robot = new Robot();
robot.keyPress(KeyEvent.VK_CONTROL);
robot.keyPress(KeyEvent.VK_P);
robot.keyRelease(KeyEvent.VK_P);
robot.keyRelease(KeyEvent.VK_CONTROL);
} catch (AWTException ex) {
Logger.getLogger(atest.class.getName()).log(Level.SEVERE, null, ex);
}
}
This would play/pause the video. You can see other shortcuts here(http://windows.microsoft.com/en-AU/windows-vista/Windows-Media-Player-keyboard-shortcuts)