Develop Reference

How to use applescript to close upload dialog window from chromedriver instance?

I have a selenium test (selenide to be precise) where the scenario requires a file upload.
The element to which I'm uploading the file is a hidden input field which is located at the end of DOM;
<input type="file" style="height: 0; width: 0; visibility: hidden;" tabindex="-1" accept="*">
and appears only after clicking on the area where the file is supposed to be "drag&dropped" or loaded from the system;
<a class="browse" ref="fileBrowse" href="#">select files...</a>
that means I am unable to use any method I've known until now without the need to click the element first - e.g., sendKeys, uploadFile, uploadFromClassPath, etc. However, the moment I click the element, a dialog window appears. After loading the file, the window won't close and I have yet to find a robust solution to close that window.
Situation how the dialog window looks within the macOS and chrome setup
I am using macOS and chrome, which means I cannot use "autoIT", and I was not able to run "sikuliX" either to create a simple screenshot script.
I was able, however, to scramble up an applescript using Automator which worked fine provided we omit the web driver's instance existence. Meaning; if I run the script from the console, setting the website exactly as the automated test would find it - it works... Unfortunately, it does not work once the test instantiates and runs within the webdriver.
I have two questions I hope someone with more experience could answer:
1) How to make the applescript use the webdriver's instance and not the regular chrome window - should this be solved somehow, it's a pretty neat solution
2) Any other idea on how to close the upload dialog window?
The applescript
on run {input, parameters}
-- Click “Google Chrome” in the Dock.
delay 6.006100
set timeoutSeconds to 2.000000
set uiScript to "click UI Element \"Google Chrome\" of list 1 of application process \"Dock\""
my doWithTimeout( uiScript, timeoutSeconds )
return input
-- Click the ÒCancelÓ button.
delay 3.763318
set timeoutSeconds to 2.0
set uiScript to "click UI Element \"Cancel\" of sheet 1 of window \"PowerFLOW portal - Google Chrome\" of application process \"Chrome\""
my doWithTimeout(uiScript, timeoutSeconds)
return input
end run
on doWithTimeout(uiScript, timeoutSeconds)
set endDate to (current date) + timeoutSeconds
repeat
try
run script "tell application \"System Events\"
" & uiScript & "
end tell"
exit repeat
on error errorMessage
if ((current date) > endDate) then
error "Can not " & uiScript
end if
end try
end repeat
end doWithTimeout
the code used to run the script within the test
try {
new ProcessBuilder("/path/to/the/script").start();
} catch (IOException e) {
e.printStackTrace();
}
}
Besides trying to use the applescript, I've tried "java robot class" but I wasn't able to close the dialog window.
Using the snippet below, the uncommented part escapes the entire chrome window (the window goes "grey"/inactive) and not the dialog window, which honestly surprised me, as I have thought the dialog window was the main working window at that moment.
The part that is commented works, but as you can imagine, it is useless, should the test be run on any other machine as the coordinates are specific to my machine only.
try {
Robot robot = new Robot();
//robot.mouseMove(906, 526);
//robot.mousePress(InputEvent.BUTTON1_DOWN_MASK);
//robot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
robot.keyPress(KeyEvent.VK_ESCAPE);
robot.keyRelease(KeyEvent.VK_ESCAPE);
} catch (AWTException e) {
e.printStackTrace();
}
The method itself looks just about like this
$$x("#ElementsCollection")
.findBy("text1")
.scrollIntoView(true)
.find(byXpath("#xpath")).val("text2")
.find(byXpath("#xpath") //this is the location of the <a> element mentioned above that needs to be clicked in order for <input type file> element to appear
.click();
$x("//input[#type=\"file\"]").sendKeys("/path/to/the/uploadedFile");

As I see, the original complexity on the way to achieve the goal is
the file is a hidden input field
and appears only after clicking on the area where the file is supposed to be "drag&dropped" or loaded from the system;
I.e. – the hidden file. Correct me if am wrong:)
But this should not be the problem, because the Selenium WebDriver's sendKeys command works with hidden elements of input tag with type=file. So just simple sendKeys should pass. The Selenide's uploadFromClassPath command is based on the original sendKeys - so it should pass too.
Here is a simple test that shows that uploading file does not depend on visibility of input element:
import org.junit.jupiter.api.Test;
import static com.codeborne.selenide.Condition.hidden;
import static com.codeborne.selenide.Condition.text;
import static com.codeborne.selenide.Selenide.*;
import static com.codeborne.selenide.Selenide.$;
public class TheInternetTest {
#Test
void fileUpload() {
// GIVEN
open("https://the-internet.herokuapp.com/upload");
executeJavaScript(
"document.getElementById('file-upload').style.display = 'none'"
);
$("#file-upload").shouldBe(hidden);
// WHEN
$("#file-upload").uploadFromClasspath("temp.txt");
$("#file-submit").click();
// THEN
$("#uploaded-files").shouldHave(text("temp.txt"));
}
}
Check the full working project with this code here: https://github.com/yashaka/selenide-file-upload-demo/blob/main/src/test/java/TheInternetTest.java
P.S.
The common best practice when writing Web UI Tests is "find the simple way to reach the goal instead of the best way in context of real user simulation". That's why we try to bypass all windows that are out of control for application under test. So my recommendation would be - forget apple script, and work with the input file directly through selenium webdriver.

How to detect a pressed key globally, in a window without focus in java? [duplicate]

I'm making a small program in Java using the Robot class. The program takes over the mouse. while in the course of debugging if it starts acting in a way that I don't want it's hard to quit the program, since I can't move the mouse over to the terminate button in eclipse, and I can't use hotkeys to hit it because the mouse is constant clicking in another window, giving that window focus instead.
What I'd like to do is just hook up a keylistener so that when I hit q I can quit the program, but the only way I know how to do this involves making a window, and that window needs focus to capture the input. Is there a way to listen for keyboard or mouse input from anywhere, regardless of what has focus?

There is a library that does the hard work for you:
https://github.com/kwhat/jnativehook

This is not a trivial problem and Java doesn't give you a way to do it elegantly. You can use a solution like banjollity suggested but even that won't work all the time if your errant mouse clicks open another fullsized window currently open in your taskbar for example.
The fact is, Java by default gives developers very little control over the OS. This is due to 2 main reasons: security (as citied by java documentation) and the fact that different operating systems handle events completely differently and making one unified model to represent all of these would probably not make a whole lot of sense.
So to answer your question, I imagine what you want is some kind of behaviour for your program where it listens for keypresses globally, not just in your application. Something like this will require that you access the functionality offered by your OS of choice, and to access it in Java you are going to need to do it through a Java Native Interface (JNI) layer.
So what you want to do is:
Implement a program in C that will listen for global keypresses on your OS, if this OS is Windows than look for documentation on windows hooks which is well docuemented by Microsoft and MSDN on the web and other places. If your OS is Linux or Mac OS X then you will need to listen for global keypresses using the X11 development libraries. This can be done on an ubunutu linux distro according to a Howto that I wrote at http://ubuntuforums.org/showthread.php?t=864566
Hook up your C code to your Java code through JNI. This step is actually the easier step. Follow the procedure that I use in my tutorial at http://ubuntuforums.org/showthread.php?t=864566 under both windows and linux as the procedure for hooking up your C code to your Java code will be identical on both OSes.
The important thing to remember is that its much easier to get your JNI code working if you first code and debug your C/C++ code and make sure that it is working. Then integrating it with Java is easy.

Had same problem. In my case, robot just controlled a single Windows App, that was maximized. I placed these lines at top of main loop driving the robot:
Color iconCenterColor = new Color(255,0,0); // if program icon is red
if (iconCenterColor.equals(robot.getPixelColor(10,15)))
throw new IllegalStateException("robot not interacting with the right app.");
To cancel the robot, just alt-tab to another app. Works great for a simple one app driving robot.

Start the program from a command line in a terminal and use Ctrl-C to terminate it.

(As mentioned by #MasterID and shown on JNativeHook's documentation for native keyboard input detection {main GitHub project here}),
This code should be enough to listen to any key without app focus (press and/or release):
>>Remember to add the jnativehook library in your project to be able to use all its utilities.<<
public class yourClass implements NativeKeyListener {//<-- Remember to add the jnativehook library
public void nativeKeyPressed(NativeKeyEvent e) {
System.out.println("Key Pressed: " + NativeKeyEvent.getKeyText(e.getKeyCode()));
}
public void nativeKeyReleased(NativeKeyEvent e) {
System.out.println("Key Released: " + NativeKeyEvent.getKeyText(e.getKeyCode()));
}
public void nativeKeyTyped(NativeKeyEvent e) {
System.out.println("Key Typed: " + NativeKeyEvent.getKeyText(e.getKeyCode()));
}
public static void main(String args[]){
//Just put this into your main:
try {
GlobalScreen.registerNativeHook();
}
catch (NativeHookException ex) {
System.err.println("There was a problem registering the native hook.");
System.err.println(ex.getMessage());
System.exit(1);
}
GlobalScreen.addNativeKeyListener(new yourClass());
//Remember to include this^ ^- Your class
}
}
For this particular problem, use the nativeKeyPressed method like this:
public void nativeKeyPressed(NativeKeyEvent e) {
System.out.println("Key Pressed: " + NativeKeyEvent.getKeyText(e.getKeyCode()));
if (e.getKeyCode() == NativeKeyEvent.VC_Q){
System.exit(1);
}
}
Note that JNativeHook by default shows a lot of stuff in your console that you might not want, to change that, just add this right before the try-catch that you used in the main function as shown (this is also going to turn off warning and error messages, more info here):
//(From here)
Logger logger = Logger.getLogger(GlobalScreen.class.getPackage().getName());
logger.setLevel(Level.OFF);
logger.setUseParentHandlers(false);
//(To there-^)
try {
GlobalScreen.registerNativeHook();
}
catch (NativeHookException ex) {
System.err.println("There was a problem registering the native hook.");
System.err.println(ex.getMessage());
System.exit(1);
}
Disclaimer: I know this question was solved years ago, I just hope someone finds this a little easier to find/use.

Have your program open a second window which displays underneath your main window but is maximised, then your errant mouse clicks will all be received by the maximised window, and it can receive your keyboard input.

Here's a pure Java way to do it to solve the problem you've described (not the KeyListener problem... the quit test early when using robot problem):
Throughout your test, compare the mouse position with one that your test has recently set it to. If it doesn't match, quit the test. Note: the important part of this code is the testPosition method. Here's code that I used recently:
public void testSomething() throws Exception {
try {
// snip
// you can even extract this into a method "clickAndTest" or something
robot.mouseMove(x2, y2);
click();
testPosition(x2, y2);
// snip
} catch (ExitEarlyException e) {
// handle early exit
}
}
private static void click() throws InterruptedException {
r.mousePress(InputEvent.BUTTON1_DOWN_MASK);
Thread.sleep(30 + rand.nextInt(50));
r.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
Thread.sleep(30 + rand.nextInt(50));
}
private static void testPosition(int x2, int y2) throws ExitEarlyException {
Point p = MouseInfo.getPointerInfo().getLocation();
if(p.x != x2 || p.y != y2) throw new ExitEarlyException();
}

How to press a keyboard key without a physical keyboard in selenium

I'm trying to build an automation script that will install a chrome extension.
On my local system (windows 10) all works fine while using Robot class with java, since I have a physical keyboard connected to my computer.
The problem is - when I try to run this automation on a virtual machine(Amazon EC2, windows server), the Robot class is not working because it doesn't detect a physical connection of a keyboard.
Is there any other way to simulate a keyboard stroke without a keyboard attached?
FYI, I have to use the keyboard because google install box is not part of the page and selenium wont recognize it.
I've tried the sendKeys function but it didn't work because it will affect only the webpage itself and not pop outside of the page

I believe you can use java robot functions to mimic the keyboard interactions.
Example:
package org.kodejava.example.awt;
import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.KeyEvent;
public class CreatingKeyboardEvent {
public static void main(String[] args) {
try {
Robot robot = new Robot();
// Create a three seconds delay.
robot.delay(3000);
// Generating key press event for writing the QWERTY letters
robot.keyPress(KeyEvent.VK_Q);
robot.keyPress(KeyEvent.VK_W);
robot.keyPress(KeyEvent.VK_E);
robot.keyPress(KeyEvent.VK_R);
robot.keyPress(KeyEvent.VK_T);
robot.keyPress(KeyEvent.VK_Y);
} catch (AWTException e) {
e.printStackTrace();
}
}
}

I don't think you can do this with Selenium, cause it is meant to test webpages, not to automate a human-computer interaction.
If you want to automate a complex scheme like this, you may try a more complete solution, like UiPath :
https://www.uipath.com/
This is a solution meant for automation, so it will give you more tools to achieve your goal. It has a community edition which is free, and an active forum, so you should be able to handle it quickly !

How can I prevent the Charm bar from showing up in my Java application?

I've got a Java swing program that runs in full screen mode. It's effectively a kiosk program in that I want it to lock out everything else while it's running. This is running on a Windows 8.1 tablet, so of course the tablet is touchscreen, and therefore if you do an "edge swipe" (drag your finger from the right) the charms bar pops up and you can get to the Start screen from there. Is there some way to disable this from happening in Java? (Or is there some third-party solution not involving Java that can work in tandem to achieve the same result?)

You can disable edge gestures while your app is active and full screen by setting the System.EdgeGesture.DisableTouchWhenFullScreen property on the window.
I don't know if Java provides a direct way to set this (probably not), but you should be able to set this from a JNI.
Here's a C++ snippet from the DisableTouchWhenFullScreen docs:
HRESULT SetTouchDisableProperty(HWND hwnd, BOOL fDisableTouch)
{
IPropertyStore* pPropStore;
HRESULT hrReturnValue = SHGetPropertyStoreForWindow(hwnd, IID_PPV_ARGS(&pPropStore));
if (SUCCEEDED(hrReturnValue))
{
PROPVARIANT var;
var.vt = VT_BOOL;
var.boolVal = fDisableTouch ? VARIANT_TRUE : VARIANT_FALSE;
hrReturnValue = pPropStore->SetValue(PKEY_EdgeGesture_DisableTouchWhenFullscreen, var);
pPropStore->Release();
}
return hrReturnValue;
}

What I ended up doing was to write a batch script that kills explorer.exe and then re-spawns it after the app exits, based on this answer on Super User.

java mouse capture

How do I capture the mouse in a Java application so that all mouse events (even ones that happen if the mouse is moved outside the app window) are seen by the Java app? This is like the Windows SetCapture function.

You don't; the JVM, or more specifically AWT, only generates input events when Windows sends it input events, and the JVM only registers for those events which occur within it's window.
You might be able to pull it off using JNI, but then again you might not - it will depend if you can get your hands on the information required by the underlying API. Since that's likely to be a window handle, you won't have what you need to invoke the API, even from JNI.

You have to hook the mouse at the operating system level. Windows(Swing, AWT, MFC, etc....) are only aware of mouse movements within their bounds. If you need a way to access the current position of the mouse regardless of where the mouse is on the screen, you need to write an Input Hook: Input Hooks. You can then use JNI or read the STDOUT from a win32 console application designed to use the Input Hook to forward mouse events/positions to your Java code. I use the latter method in some of my user interface test cases with success.

I needed to do that too!
I after searching the web I found that its possible to use the moveMouse in java.awt.Robot.
Basically use Robot to move the mouse into center of your frame. If user moves it: check how much and move it back to center.
No additional packets or JNI are needed for this (my demo uses JOGL and vecmath but that's for the graphics). Is it good enough? Try the demo, its here:
http://www.eit.se/hb/misc/java/examples/FirstPersonJavaProtoGame/
If the above solution is not good enough then perhaps lwjgl is what you need:
http://www.lwjgl.org/javadoc/org/lwjgl/input/Mouse.html
/Henrik Björkman

Just use the system-hook library available on gitHub https://github.com/kristian/system-hook
This only apply to windows-based systems but really simple to implement.
Sample usage
import lc.kra.system.keyboard.GlobalKeyboardHook;
import lc.kra.system.keyboard.event.GlobalKeyAdapter;
import lc.kra.system.keyboard.event.GlobalKeyEvent;
public class GlobalKeyboardExample {
private static boolean run = true;
public static void main(String[] args) {
// might throw a UnsatisfiedLinkError if the native library fails to load or a RuntimeException if hooking fails
GlobalKeyboardHook keyboardHook = new GlobalKeyboardHook();
System.out.println("Global keyboard hook successfully started, press [escape] key to shutdown.");
keyboardHook.addKeyListener(new GlobalKeyAdapter() {
#Override public void keyPressed(GlobalKeyEvent event) {
System.out.println(event);
if(event.getVirtualKeyCode()==GlobalKeyEvent.VK_ESCAPE)
run = false;
}
#Override public void keyReleased(GlobalKeyEvent event) {
System.out.println(event); }
});
try {
while(run) Thread.sleep(128);
} catch(InterruptedException e) { /* nothing to do here */ }
finally { keyboardHook.shutdownHook(); }
}
}