Recently I made an autoclicker for a game, which would allow you to use F1 and F2 to turn it on or off, once I finished the application I thought to myself, "oh great, I forgot Java doesn't let you record outside keystrokes and mouse movements", is there any simple non-native ways to do this?
Thanks.
Nope. You can do application wide keyboard and mouse handling, but you have very few options if your application is not the foreground application. The best you can do is discover the mouse location:
java.awt.MouseInfo.getPointerInfo().getLocation()
This will always work. You are out of luck for keyboard input though. For that you will need to write native code.
Related
I would like to know how I would kind of like run a online browser game, like tetris, as a Java application so I could control it (like in tetris using the wasd or arrow keys to move blocks) from within the application I program for the game. Not sure if that makes sense but don't know how else to explain it. The purpose is to hook it up to an AI program that plays the game, but yeah I don't know how to allow the program to control the game using the games allowed inputs.
Java.awt.Robot seems to be what you need
It should be able to simulate any keyboard or mouse input, but if you need to do anything too complicated I would suggest something other than Java - it doesn't get too close to the operating system, so something like c++ may be more accurate.
Using Java, is it possible to detect user actions, such as key-presses, mouse-button-presses, and/or mouse's screen location, without the use of GUI elements?
If it is, how could I implement it?
Otherwise, what are the alternatives?
The only StackOverflow source i found on this matter is this question, which is unanswered.
And the only sources I found outside StackOverflow on this matter point to an "Invisible GUI" solution, wish is something i really want to avoid.
It can be implemented using JNI and/or JNA but this cannot be really called "java implementation" because you will have to write platform specific native code.
Alternative solution I tried is to use full screen transparent widow that is listening to all events of mouse and keyboard and "forwards" them to the real application using class Robot. I tried this approach. It works well with one limitation: there is a problem to support "mouse over" events of applications: the mouse is not moving over the real application. It is moving over the transparent java window.
Use the java.awt.event.KeyListener class. You will have to write three methods in which you can write which key you want to be detected, and whatever you want to have happen when the key is pressed.
I want to show the form when someone presses a key in my java swing application when it is in system tray. I have tried adding key listeners, but it didn't work. I know this can be done by using Popup menu. But I want to know can java swing app can detect keystrokes when it is in system tray? can anyone help me to do this. Thank you.
You might look at JIntelliType (http://melloware.com/products/jintellitype/index.html) but this will only be a windows solution. There may be something similar for other OSes. This library provides global hotkey registration which is what you need.
Update: looks like they reference a similar API for linux also.
You can use the JNI to hook into Window's Window-Event-API, i think, but this is complicated. A more simple but slightly hacky way to do it is create a link in the Start>>Programs menu, set a key-combination there, then with that link call your program into windowed-mode or fullscreen-mode.
Short answer, no, it's not possible.
See Is it Possible to show a previously hidden JFrame using a keylistener
I'm writing a Java application to automate character actions in an online game overnight (specifically, it catches fish in Final Fantasy XI). The app makes heavy use of java's Robot class both for emulating user keyboard input and for detecting color changes on certain parts of the screen. It also uses multithreading and a swing GUI.
The application seems to work perfectly when I test it without the game running, just using screenshots to trigger the apps responses into notepad. But for some reason, when I actually launch FFXI and start the program, all of my keyboard and mouse manipulations just stop working altogether. The program is still running, and the Robot class is still able to read pixel colors. But Robot.keyPress, Robot.keyRelease, Robot.mouseMove, Robot.mousePress and Robot.mouseRelease all do nothing. It's the strangest thing-- to test it, I wrote a simple loop that just keeps typing letters, and focused notepad. I'd then start the game, refocus notepad, and it would do nothing. Then I'd exit the game, and it'd start working again immediately.
Has anyone else come across something like this, where specific software will stop certain functions of java from working?
Also, to make this more interesting-- Last year I wrote a very similar program using the same classes and programming techniques to automate healing a party in the game as they fight. Last year, this program worked perfectly. After running into these problems I dug up that old program, ran it without making any changes, and found that it too was having the same problems. The only differences between now and when it was working: I was running Windows Vista and now I'm running Windows 7, and several new Java versions as well as FFXI versions have been released.
What the hell is going on? (if anyone needs to see my source code, email me at mikejturley#gmail.com. I'm trying to keep it to myself.)
FFXI has code to prevent cheating. Quite effectively, it would seem.
If possible, try it in WinXP. I myself have also written a bot for an online game that uses much of the same concepts (i.e. using Java Robot to read pixel colors and simulate key-presses and mouse-clicks).
Under WinXP:
Bot works as intended in all cases.
Under Win7:
Outside of the game, bot works as intended. Ingame, simulated input failed (pixel reads were okay, I think).
I want to be able to record mouse movements, clicks and keyboard input from a user. It would be great if it was a cross platform solution.
I'd like to get back something like this (pseudo code):
mouse moved to 500, 500
mouse double clicked
mouse moved to 800, 300
mouse left clicked
keyboard typed "Hello World"
Does either C++ or Java have any classes that can do this? If I was using C++, I would probably working with the QT framework.
Edit:
I should have said this originally, but I want to record the movements and clicks outside of the applications gui, so on the desktop too.
GLUT does this, but it's tied to OpenGL which might be overkill for your project.
OpenGL is cross-platform.
I don't believe there's a cross-platform toolkit specifically for grabbing input from a window and nothing more, but most toolkits provide this capability. Two good options are:
Use SDL, as it's fairly lightweight and can handle simple input.
Implement the functionality natively per platform, as it should be trivial in X11, Windows, Mac OS X, etc.
On Windows, this is called a Journal Record Hook. You should write the hook part in C or C++, it might be technically possible to do in java, but it's not a good idea, you want your hook procedure to have as few dependencies as possible, and to be a quick as possible. System wide hooks, especially journal add a lot of overhead to keyboard and mouse input, you want to minized your impact as much as possible.
You Install Windows hooks by using SetWindowsHookEx passing WH_JOURNALRECORD to get a Journal Record Hook.
You could also (maybe) get this working by installing both WH_KEYBOARD_LL and WH_MOUSE_LL, but your two hook procedures would be called separately, and you would have to write your own code to put the events in order.
I doubt you will find a cross-platform solution.
It sounds like Qt might allow you to implement event filters that extend beyond the application to the window system. See also Qt - top level widget with keyboard and mouse event transparency?
If you want to trap events across the whole GUI system, not just one app, there's not much likelihood of a cross platform solution. However, the event hooking part could easily be separated from the recording part, so you could make most of the program cross-platform.
For Windows, you need this 17 year old (!) document. (Man, I'm getting old!)