Using Java, is it possible to detect user actions, such as key-presses, mouse-button-presses, and/or mouse's screen location, without the use of GUI elements?
If it is, how could I implement it?
Otherwise, what are the alternatives?
The only StackOverflow source i found on this matter is this question, which is unanswered.
And the only sources I found outside StackOverflow on this matter point to an "Invisible GUI" solution, wish is something i really want to avoid.
It can be implemented using JNI and/or JNA but this cannot be really called "java implementation" because you will have to write platform specific native code.
Alternative solution I tried is to use full screen transparent widow that is listening to all events of mouse and keyboard and "forwards" them to the real application using class Robot. I tried this approach. It works well with one limitation: there is a problem to support "mouse over" events of applications: the mouse is not moving over the real application. It is moving over the transparent java window.
Use the java.awt.event.KeyListener class. You will have to write three methods in which you can write which key you want to be detected, and whatever you want to have happen when the key is pressed.
Related
I am a beginner and is given a project by my professor. I am in a process of making a console based game (Tetris) and want to implement controls to it for performing specific tasks i.e for movement and rotation. I have found out that KeyListener can be used but it is strictly GUI based. So I am looking for an alternate way to get the work done. If you have a solution or an article or web link related to it, it will be appreciated. Thanks!
Take a look at this thread, there are some solutions here: How to read a single char from the console in Java (as the user types it)?
To accomplish this you need to set your console into raw mode. Apparently there is no built-in way to do so with Java, but there are some libraries you could use to get there, like jCurses or jLine3.
Is there a way to detect key input when the window is not active? That is, another application is running but the program is triggered when say the F9 key is pressed or something along those lines.
Is that possible or is java not compatible for such functions? From what I found java can't get input unless it's the active window.
Note:
I typically use keylistener, which seems to stop working (with good reason) when I am not actively using the program.
It sounds like you want low level keyboard hooks.
Get global keyboard input with Java
It's not directly possible in Java (i.e. with pure Java code), but you could reference other libraries (or make your own) to acheieve this. A quick google search gives a lot of references and free/open libraries that can give you keyboard and mouse hooks (both very handy).
Hope that helps.
I want to automate an external application, but I have several problems:
How can I recognize a button or other field of an external application in Java?
I use the Robot class in Java for making notepad automation where I open notepad, select file menu, and save or exit, etc.
The problem is, it needs X,Y coordinates for the mouse pointer to go to the proper location.
I want to make it more dynamic, i.e. it should recognize the file menu of a running notepad anywhere on the desktop.
How can this be done in Java? Is there any class in Java I can use to do this?
Thanks everyone to give me response, I want to be more specific i want to know how can i make ui automation by using any tool if it is not possible in java or using any api of java.automation tool must be freeware.....i am searching net for that i found AutoIt is like that.But if any one do this type of things please share his/her experiance means is it possible to do that in AutoIt or not possible if not then which tool do that kind of things.
It is easy to integrate Sikuli into a Java-application since it is written in Java. Sikuli uses image recognition to find elements visible on the screen like buttons and such. It is very easy to use and provides an alternative for tasks that are difficult to handle with static positioning, like finding moving windows and such.
Take a look at this: http://sikuli.org/docx/faq/030-java-dev.html
Hope this helps!
You should have a look at Sikuli. It takes as inputs images of the ui elements to select an area in the targeted app. It's a UI Automation Application
That's a bit difficult to install (at least on Debian/Ubuntu, where I tested it), as you'll need a recent version of OpenCV, a particular version of JXGrabKey but the quality of the program worth the trip. Good Luck
Java doesn't have an API to examine the UI of another application; that would be a very big security risk.
Which is why the Robot class can only record events (key presses, mouse movements and clicks) but not which UI element was involved in most cases.
It would be possible to do more if the external application was written in Java because then, you could analyze the objects in memory but for obvious reasons, this isn't possible for C++ or .NET applications.
I want to be able to record mouse movements, clicks and keyboard input from a user. It would be great if it was a cross platform solution.
I'd like to get back something like this (pseudo code):
mouse moved to 500, 500
mouse double clicked
mouse moved to 800, 300
mouse left clicked
keyboard typed "Hello World"
Does either C++ or Java have any classes that can do this? If I was using C++, I would probably working with the QT framework.
Edit:
I should have said this originally, but I want to record the movements and clicks outside of the applications gui, so on the desktop too.
GLUT does this, but it's tied to OpenGL which might be overkill for your project.
OpenGL is cross-platform.
I don't believe there's a cross-platform toolkit specifically for grabbing input from a window and nothing more, but most toolkits provide this capability. Two good options are:
Use SDL, as it's fairly lightweight and can handle simple input.
Implement the functionality natively per platform, as it should be trivial in X11, Windows, Mac OS X, etc.
On Windows, this is called a Journal Record Hook. You should write the hook part in C or C++, it might be technically possible to do in java, but it's not a good idea, you want your hook procedure to have as few dependencies as possible, and to be a quick as possible. System wide hooks, especially journal add a lot of overhead to keyboard and mouse input, you want to minized your impact as much as possible.
You Install Windows hooks by using SetWindowsHookEx passing WH_JOURNALRECORD to get a Journal Record Hook.
You could also (maybe) get this working by installing both WH_KEYBOARD_LL and WH_MOUSE_LL, but your two hook procedures would be called separately, and you would have to write your own code to put the events in order.
I doubt you will find a cross-platform solution.
It sounds like Qt might allow you to implement event filters that extend beyond the application to the window system. See also Qt - top level widget with keyboard and mouse event transparency?
If you want to trap events across the whole GUI system, not just one app, there's not much likelihood of a cross platform solution. However, the event hooking part could easily be separated from the recording part, so you could make most of the program cross-platform.
For Windows, you need this 17 year old (!) document. (Man, I'm getting old!)
If I have Java program and I need to alter it to an interface and include icons,
is there any easy I can do this and is there a good application that can help me to do it ?
or do I have to code it in myself?
Nop, /me thinks ur need 1337 mad Java programin' skillz!
Translation for the rest of the world: Sorry, you'll need to program in Java.
Added: Hey, what's with the downvotes? He started it! :P Besides - no matter if he wants to add or modify (the original text wasn't clear on this) the UI of a Java program, he will need to program in Java to bring his UI together with the code. There is no miracle tool that can allow you to draw an UI and it will suddenly do what you do.
Netbeans has a Swing GUI Builder. Quoting from their website. Let's hope this doesn't count has hidden advertising :)
Design Swing GUIs by dragging and
positioning GUI components from a
palette onto a canvas. The GUI builder
automatically takes care of the
correct spacing and alignment. Click
into JLabels, JButtons, ButtonGroups,
JTrees, JTextFields, ComboBoxes and
edit their properties directly in
place. You can use the GUI builder to
prototype GUIs right in front of
customers.
If you want to add a UI to your Java program there are tools to help you, such as the Swing GUI Builder inside of IntelliJ Idea. However, you're still going to have to write the appropriate code to hook into the UI.
It's just a website? Well depending on whether it uses CSS you might be able to just modify a .css file. This will only let you modify how the site looks as opposed to works.
See here for an example of how this technology works. However this depends on how css-dependent the website is and it's possible you may still run into some difficulties.
You want to use a Java framework to help you with the UI. For example, you can use JSF (Javaserver faces), which allows you to drag and drop components for a UI onto the site. Otherwise, you can use web programs such as Dreamweaver to design the UI, before coding the backend logic yourself in java.