I would like to have my Java applet monitor a fast moving rectangle in another non-Java program and press a certain key on the keyboard when it reaches a specific location. Can someone point me in the right direction as how to detect on-screen shapes?
The API you're looking for is the Robot API.
If the background is simple enough, detecting the rectangle should be trivial. If it isn't, you're looking at an object tracking problem. I understand ImageJ has plugins that can help with that.
Related
I have a macro tool that runs using pascal. I need to track the movements of the mouse while the program is running over very long periods of time. I wrote a basic mouse recorder in java that draws the movements and clicks in slow motion so they are easier to analyze, using jnativehook. It works fine when I move the mouse manually, but it does not pick up mouse movements from pascal. It does however pick up mouse clicks from pascal.
How can I pick up mouse movements made by a pascal macro tool in Java?
The issue is how the events are generated in Pascal and are going to depend on the platform you are working with. I would suggest using the event posting functionality in JNativeHook as it will always be able to pick up the events it generates. You can still send events that the library can pick up outside of the library, but on some platforms it requires using a specific API as some methods cannot be picked up by the listener on specific platforms.
I have a paint program written in java with SWT. I am testing with SWTBot. My test case is to draw a picture on the canvas, capture the image of the canvas, and compare to the expected image.
The problem is that I can't find any way to move the mouse using the SWTBot. Apparently it only allows me to click the mouse. I want to
move to an x,y location
mouse down
move to another x,y location
mouse up
Any advice?
I've decided to stop using SWTBot. The functionality is very weak. It is designed for Eclipse apps, so it doesn't really support plain SWT apps very well. Although having direct access to the widgets is somewhat appealing, the requirement of running the test code in the app process is awkward.
I've decided to use Sikuli instead. It has pretty good API for both Java and Python. It seems to have more function and better support than SWTBot.
I have a cross platform application built with libgdx. The software is often used in conjunction with large touchscreens (like the microsoft coffee table thingy) with Windows (7 & 8) and MacOS. These screens are multitouch capable but I've only implemented single-touch, and would like to take the leap to multi.
I'm asking a few questions here in case someone knowledgeable could spare me a bunch of hours testing hypothesis that are undocumented and un-googleable AFAIK:
-does libgdx support the win7 multitouch interface? win8?
-if I must resort to mt4j, can it be used WITHOUT the 3d engine layer? Heck I don't even need gesture recognition, just the low level state of the fingers. There is no way I'm replacing 10000+ lines of libgdx with mt4j!
-has anyone tried sparsh-ui as an alternative to mt4j? Does it support win7 multitouch?
Thanks for any bit of information that can be provided on these matters!
Cheers!
As far as i know there shouldn't be a problem with multitouch. For libgdx mouse presses and touches are the same thing, so there shouldn't be a problem for win7 and win8.
The touchDown, touchUp and touchDragged methods give you an int pointer, which describes the number of the touch. So if you touch with 4 fingers, the touchDown with pointer = 4 describes the touch of the 4th finger.
You can read something about that here.
Note that i did not test this, as i don't have a win7/8 touch device. So you have to test it yourself, but as i said it should work. Let me know if it worked. Thanks
Using Java, is it possible to detect user actions, such as key-presses, mouse-button-presses, and/or mouse's screen location, without the use of GUI elements?
If it is, how could I implement it?
Otherwise, what are the alternatives?
The only StackOverflow source i found on this matter is this question, which is unanswered.
And the only sources I found outside StackOverflow on this matter point to an "Invisible GUI" solution, wish is something i really want to avoid.
It can be implemented using JNI and/or JNA but this cannot be really called "java implementation" because you will have to write platform specific native code.
Alternative solution I tried is to use full screen transparent widow that is listening to all events of mouse and keyboard and "forwards" them to the real application using class Robot. I tried this approach. It works well with one limitation: there is a problem to support "mouse over" events of applications: the mouse is not moving over the real application. It is moving over the transparent java window.
Use the java.awt.event.KeyListener class. You will have to write three methods in which you can write which key you want to be detected, and whatever you want to have happen when the key is pressed.
I'm toying with an idea for creating a Java application to automate a process that I have to do regularly and before I start any coding I thought I would seek advice as to the best way to approach it.
Basically, the application I use has a large number of images present on the screen at any one time, and what I would like to know is if there is a way to have Java identify if any of these two images are the same. If they are, I would like to automate mouse movement and button clicks.
After a bit of reading, I'm thinking that the PixelGrabber and Robot classes might be the right way to start, but like I said, I'm looking for any information on this that can be offered.
What are your suggestions?
I believe the Robot class and a Pixel Grabber would be sufficient. If you are inclined to program the solution yourself, maybe for educational purpose, by all means please do. If you, however, don't want to reinvent the wheel, you may take a look at this project:
http://sikuli.org/
I, for example, use it to do stuff that would be hard to achieve with Selenium alone. If you still can't achieve your goal after some scripting, Sikuli provides a nice API which you can use from inside your java program.
The Robot class would be sufficient to take images and being able to inspect pixels. But it seems to make more sense, to recreate your desktop with images inside of a java application (a very simple gallery application). Then operations are simplier. An other way of realizing operations I do not see.