AutoIt seems to stop working when focus is given to a java application.
For this code:
MouseMove(61, 121, 10);
WinActivate("Main # paperMoney [build 1800.20]");
MsgBox(0, "got it", "got it");
Results are --> Mouse moves, java application gets focus and message appears
For this code:
WinActivate("Main # paperMoney [build 1800.20]");
MouseMove(61, 121, 10);
MsgBox(0, "got it", "got it");
Results are --> java application gets focus, mouse DOES NOT move and message appears.
Is there anything I can do to make the mouse and keyboard commands work after the java application has focus?
Considering what paperMoney does, it is probably purposely resisting automation. Especially when the Window is activated, it will hook into low-level mouse events and check all hardware mouse movement with actual movement. Using that it will prevent any simulated mouse movement. Can you use Send to type keys in the Window?
I looked to do the same thing. Mandanar is probably right. I also tried Autohotkey with no success. Perhaps TOS is prohibititng it deep in their application. :( Boo!
EDIT:
Some playing around I've discovered that TOS on Mac OSX can be controlled via scripting with Keyboard Maestro. It's a ugly, hacked solution, but it works. You can edit text boxes and click stuff if you know the X,Y position of elements.
Keyboard Maestro can be run via scrips (AppleScript, Python, etc.) so maybe you can build some elaborate rube goldberg.
Happy trading!
My solution was tested for AutoHotkey, but I believe this should work for AutoIt also. I ran the script with administrator privileges since TOS is ran as an administrator and my scripts work fine after that.
Related
Ok, so this is a really, REALLY weird issue that we're having with the application at my company. I'm going to try and describe this as best I can.
First, this is a legacy application with a Swing UI.
Second, the issue only occurs when it is compiled & run using Java 8. With Java 7 and below, this does not occur.
So, the issue: When a dialog is displayed (modal or non-modal, doesn't matter), the UI becomes unresponsive to mouse clicks. What's truly crazy, though, is the UI is NOT frozen. Hovering the mouse over something produces the hover highlight as normal. Keyboard commands are received perfectly. Mouse clicks, however, do not work.
This also only occurs on OSX. Windows and Linux don't have this problem. I'm running this on OSX El Capitan.
As far as code samples, it's affecting all dialogs across the application. JOptionPanes & JDialogs, doesn't seem to matter. Here's a simple JOptionPane declaration:
int n = JOptionPane.showOptionDialog(mcContext.getMapperView(), "xPath of dropping target can't be evaluated" +
"\nPlease, select xPath for dropped node",
"xPath calculation for dropped node",
JOptionPane.YES_NO_CANCEL_OPTION, JOptionPane.QUESTION_MESSAGE, null, options, options[0]);
This code IS being called on the EventThread, so I don't believe it's a threading issue.
I'm totally stumped as to the cause of this. So far we've ignored it by compiling and running it in Java 7, but at some point as versions progress we will need to deal with this issue more directly.
Anyone have any ideas?
Edit: Thanks for the SSCCE Idea. Now I'm more confused than ever. When I put together a quick frame/dialog demonstration, it worked perfectly. No issues. So I don't know what it is in the application that could be causing this. Any good places to start looking?
Edit 2: Wrapped one of the declarations in SwingUtilities.invokeLater, and it worked. Ok... now what could the culprit be? It's still got to be something that Java 8 is compiling different than Java 7.
Edit 3: More weird behavior. I moved my IDE into a separate "desktop" than the application it was running, and when the buggy dialog is displayed, i cant switch to that desktop. I can switch to any app on the current desktop, but not to another desktop.
Edit 4: The dialogs in question are being triggered by drag and drop actions. Not sure if that helps or not, but I do see that there is a thread in the background in the thread dump using the sun.lwawt.macosx.CDragSourceContextPeer class.
Alright, it took a while, but I think I found it. I found an OpenJDK post that seemed to very clearly describe this issue.
https://bugs.openjdk.java.net/browse/JDK-8139393
Somehow, Drag N Drop in Swing on OSX in java 8 isn't releasing its hold on the MouseEvent or Listener, so when a modal dialog is displayed the mouse isn't able to get new events through to it. Which is a crazy stupid bug, but it's there.
The solution was wrapping my code in SwingUtilities.invokeLater(...). By having the code that displays the dialogs executed asynchronously from the drag and drop code, the drag and drop operation is able to complete and release its hold on the mouse connectivity (for lack of a better description). And viola! Problem solved.
Thanks to everyone who tried to help. Hope this post helps out anyone else dealing with this issue.
I am learning Slick2D, which perfectly serves my purposes of 2D Java game programming. Unfortunately, I have a problem. I don't think it's Slick2D specific, since I think the same happens with Java Graphics 2D.
In Ubuntu (I'm using Ubuntu 12.04), pressing the Alt key shows the "Dash Home", the same as clicking on the top-left icon in the screen (Windows users, "Dash Home" is sort of "Start" menu).
When my application runs, Alt key still has this behavior and shows the Dash Home. When my game runs in windowed mode, that's not a big deal, since I can go to a paused state. However, when in fullscreen mode, it is a big problem because the system leaves fullscreen and switching between fullscreen and windowed is time consuming and, roughly speaking, ugly.
So, the question is: how can I prevent the Alt keypress to reach the OS, preventing Dash Home to open? In my code I handle keyPresses through the listener provided by Game2D and I don't call any superclass method, so, suposedly the keypress should not be informed to the OS. But... it is.
I am not attaching any code as I don't think this is related to Slick2D, but a general Java issue. Any idea?
Thanks!
Edit: The problem still persists. I have posted the whole source code and binaries of the project here. If it doesn't help to find a solution to this, it may serve as a basic tutorial about Java game programming to other people.
If you are not trying to post this java program to loads of people on Ubuntu computers then you could try this:
Open the System Settings application either by going to Session Indicator in Unity panel, or by searching for System Settings using the HUD.
Then go to Keyboard > Shortcuts > Launchers. You can redefine the HUD key with the Key to show the HUD option. Pressing Backspace will disable the HUD shortcut altogether.
If you set it to Alt manually, the setting distinguishes between the left and right Alt key. So if you want to maintain general behaviour and still use Alt combinations, this may be an alternate solution for you.
And just so you know I got this answer from this website: https://askubuntu.com/questions/122209/how-do-i-modify-or-disable-the-huds-use-of-the-alt-key
I don't know of anyway to temporarily disable the Alt key features but if you don't mind this then I hope this helps
Is it possible to get image contents of an obstructed window without bringing it to the front? Also, is it possible to send mouse clicks to a specific locations of such window? I want to do this in Java, using JNA, running Windows XP (if it is possible, would it also work on Windows 7?). If that can be done, would you mind telling me what functions will be needed and where can I read about it, because I have never worked with JNA yet. Thank you.
I'm trying to find a concrete yes for some assumptions I have about developing a Java Swing app to be run on a Windows tablet device (I believe it'll be running Windows XP). We have a demo we are writing and we're leveraging previous experience with Java Swing to write the UI and I just want to make sure that touch events in Windows are treated like mouse clicks, is this true? Does anyone have any direct experience writing Swing apps for Windows tablets? The target device is http://www.amazon.com/Promotion-X70EX-3GP-Accessory-Standard/dp/B002MZZX76.
Also, wondering if the soft keyboard will automatically be activated when focus is given to a JTextField and other Swing data entry fields.
Thanks!
We developed an application in Swing which runs well on Windows XP tablets. Yes, touches are treated just like mouse clicks. However, I can only confirm this for basic touch events, e.g. clicking on a button. I can neither confirm nor deny whether they work for more complex gestures.
As for the soft keyboard, I cannot remember whether this works or not. In the end our users ended up using the tablet just like a regular laptop, i.e. with the keyboard not the touch screen. This is mostly due to the amount of text they need to enter in the application.
I want to be able to record mouse movements, clicks and keyboard input from a user. It would be great if it was a cross platform solution.
I'd like to get back something like this (pseudo code):
mouse moved to 500, 500
mouse double clicked
mouse moved to 800, 300
mouse left clicked
keyboard typed "Hello World"
Does either C++ or Java have any classes that can do this? If I was using C++, I would probably working with the QT framework.
Edit:
I should have said this originally, but I want to record the movements and clicks outside of the applications gui, so on the desktop too.
GLUT does this, but it's tied to OpenGL which might be overkill for your project.
OpenGL is cross-platform.
I don't believe there's a cross-platform toolkit specifically for grabbing input from a window and nothing more, but most toolkits provide this capability. Two good options are:
Use SDL, as it's fairly lightweight and can handle simple input.
Implement the functionality natively per platform, as it should be trivial in X11, Windows, Mac OS X, etc.
On Windows, this is called a Journal Record Hook. You should write the hook part in C or C++, it might be technically possible to do in java, but it's not a good idea, you want your hook procedure to have as few dependencies as possible, and to be a quick as possible. System wide hooks, especially journal add a lot of overhead to keyboard and mouse input, you want to minized your impact as much as possible.
You Install Windows hooks by using SetWindowsHookEx passing WH_JOURNALRECORD to get a Journal Record Hook.
You could also (maybe) get this working by installing both WH_KEYBOARD_LL and WH_MOUSE_LL, but your two hook procedures would be called separately, and you would have to write your own code to put the events in order.
I doubt you will find a cross-platform solution.
It sounds like Qt might allow you to implement event filters that extend beyond the application to the window system. See also Qt - top level widget with keyboard and mouse event transparency?
If you want to trap events across the whole GUI system, not just one app, there's not much likelihood of a cross platform solution. However, the event hooking part could easily be separated from the recording part, so you could make most of the program cross-platform.
For Windows, you need this 17 year old (!) document. (Man, I'm getting old!)