I am building an application like in which the user can use his Android tablet as a graphics tablet like https://play.google.com/store/search?q=gfxtablet
My server program is in Java and I was successfully able to send stylus coordinates and pressure values to the server from my tablet through UDP and able to draw in applications like Paint, Photoshop etc.
The problem:
I am not able to find a way to inject this pressure values in Windows (to draw thick and thin lines depending upon stylus pressure). I know it can be easily done in Linux, but I am not able to find a way for Windows.
I even tried JNA library for Java , but it also does not seem to have the appropriate method to input pressure values. I know the existence of JPEN & JTABLET libraries but these don't allow you to draw in Paint, Photoshop, GIMP etc.
You could try using the Robot class in java to simulate native system input events. Then store local variables to test input against and fire those events through the Robot class.
Basically, for Windows Paint, whenever your pressure is greater than the previous pressure, fire a Robot event that sends the key combination CTRL and + to increase brush width.
If you're also testing what application is open, then you can have a separate section of code set up to determine which key combination to send(since in photoshop the keybind is [ and ]), or have a section that allows users to specify what their keybinding is.
While the current iteration uses awt, Robot is being added to JavaFX/Lombard in 3.0.
I'm sure there are better solutions, but this is what I would do until I had something better. :)
Related
I'm looking to write a relatively simple key macro for my own edification and also for my own use.
I intend for it to be able to run in the background when I run other programs, and at the press of a button will enter certain text into the active window. To be more specific, I want to go in a video game and use it to automatically send messages instead of having to type them.
I Googled around a bit, but apparently I'm not using the right keywords because I'm not really sure where to start. The closest I came was finding the Keystroke class, but that appears to be used for receiving keystrokes, not sending them.
I would appreciate and info regarding, or directing me to a resource for, these issues (how to send keystrokes to windows and anything about targeting which window etc).
To send key strokes you can use java.awt.Robot
To Choose which window to activate you can look around for ws script (windows script) or old VB6 code and use it to make VB script (simple text file of extn .vbs)
Or you can junk all that and use http://www.autohotkey.com/ which has window activation, sending keys, doing things on press of certain keys (like Windows Key + B) or macros.
To get a window to activate I had made an exe long back, but no longer use it, can get it from http://sourceforge.net/projects/win-utils/files/Window-Position/rel%2001/ (but only get this if the others do not work as need to seperately get COMCTL32.ocx and install that
If you do not want to use autohotkey you can use Jini to call platform specific functions, with a wrapper to call correst OSes functions. Never done it my self, when i had to use it i would make a process to call a exe that made the window come the front.
I wrote a Java application that uses the java.awt.Robot to take screen captures of the screen and was wondering if capturing a program using DirectX/OpenGL would be possible?
Each time I try to do this, I get a black screen
Don't know if this is really a solution but seems there's more possible ways to get the screenshot information using a "FrameGrabber" or some related class from JavaCV possibly: http://code.google.com/p/javacv/
Some of the final output shown on the screen could be calculated on a graphics card that has shaders set to act on the image data before it gets put into a display buffer for what's shown so it's possible some effects would be impossible to capture without an analog loopback (go from video out to video in on a capture card).
Related post How to take screenshots fast in Java?
OpenGL is a drawing API, not an all purpose interface to the graphics system. There were times, when taking screenshots with OpenGL was indeed possible through some dirty hacks. However recently I tried to re-implement this on modern OS to see if it still works, and no, it doesn't anymore.
Yes it is possible. But maybe only in limited circiumstances.
I've successfully captured the contents on OpenGL (jogl) windows on linux and windows using the Robot createScreenCapture.
Some specific information about the implementation that may be different for you:
The call to createScreenCapture was done from within the OpenGL
application.
The application used heavyweight GLCanvas
It used Java 6
Make sure you pass it the correct coordinates. You can get the screen coordinates to use from the GraphicsEnvironment
Uhh... a hard one ;o)
I had a try with some other screenshot utils and got a black screen, too. Looks like DirectX is passing the graphics directly to the monitor output. I am wondering if that could be accessed within a java application.
I want to automate an external application, but I have several problems:
How can I recognize a button or other field of an external application in Java?
I use the Robot class in Java for making notepad automation where I open notepad, select file menu, and save or exit, etc.
The problem is, it needs X,Y coordinates for the mouse pointer to go to the proper location.
I want to make it more dynamic, i.e. it should recognize the file menu of a running notepad anywhere on the desktop.
How can this be done in Java? Is there any class in Java I can use to do this?
Thanks everyone to give me response, I want to be more specific i want to know how can i make ui automation by using any tool if it is not possible in java or using any api of java.automation tool must be freeware.....i am searching net for that i found AutoIt is like that.But if any one do this type of things please share his/her experiance means is it possible to do that in AutoIt or not possible if not then which tool do that kind of things.
It is easy to integrate Sikuli into a Java-application since it is written in Java. Sikuli uses image recognition to find elements visible on the screen like buttons and such. It is very easy to use and provides an alternative for tasks that are difficult to handle with static positioning, like finding moving windows and such.
Take a look at this: http://sikuli.org/docx/faq/030-java-dev.html
Hope this helps!
You should have a look at Sikuli. It takes as inputs images of the ui elements to select an area in the targeted app. It's a UI Automation Application
That's a bit difficult to install (at least on Debian/Ubuntu, where I tested it), as you'll need a recent version of OpenCV, a particular version of JXGrabKey but the quality of the program worth the trip. Good Luck
Java doesn't have an API to examine the UI of another application; that would be a very big security risk.
Which is why the Robot class can only record events (key presses, mouse movements and clicks) but not which UI element was involved in most cases.
It would be possible to do more if the external application was written in Java because then, you could analyze the objects in memory but for obvious reasons, this isn't possible for C++ or .NET applications.
I'm trying to capture a video of a currently running swing application from within the application (the user presses a record button to start recording). I'm not sure how to go about creating a video that records "live." I've looked into using Xuggler, but that isn't available for 64-bit windows (on a 64-bit jvm), and that is important for this application. I don't think it would be feasible to save each screen off as images and then stitch them together because the video could run for several minutes resulting in a very large number of images.
Does anyone have any experience with this and can point me to some ideas on how to do this?
thanks,
Jeff
I was also going to suggest using Robot to take screenshots, but as you said, you would need a way of limiting what is captured. Detecting input events like keys and mouse movement could hint at when is a good time to take another screenshot, and perhaps limiting it to 2 frames per second. When stitching the images back together, the only way you could determine the timing is if you named the files using a timestamp format (with milliseconds).
I want to be able to record mouse movements, clicks and keyboard input from a user. It would be great if it was a cross platform solution.
I'd like to get back something like this (pseudo code):
mouse moved to 500, 500
mouse double clicked
mouse moved to 800, 300
mouse left clicked
keyboard typed "Hello World"
Does either C++ or Java have any classes that can do this? If I was using C++, I would probably working with the QT framework.
Edit:
I should have said this originally, but I want to record the movements and clicks outside of the applications gui, so on the desktop too.
GLUT does this, but it's tied to OpenGL which might be overkill for your project.
OpenGL is cross-platform.
I don't believe there's a cross-platform toolkit specifically for grabbing input from a window and nothing more, but most toolkits provide this capability. Two good options are:
Use SDL, as it's fairly lightweight and can handle simple input.
Implement the functionality natively per platform, as it should be trivial in X11, Windows, Mac OS X, etc.
On Windows, this is called a Journal Record Hook. You should write the hook part in C or C++, it might be technically possible to do in java, but it's not a good idea, you want your hook procedure to have as few dependencies as possible, and to be a quick as possible. System wide hooks, especially journal add a lot of overhead to keyboard and mouse input, you want to minized your impact as much as possible.
You Install Windows hooks by using SetWindowsHookEx passing WH_JOURNALRECORD to get a Journal Record Hook.
You could also (maybe) get this working by installing both WH_KEYBOARD_LL and WH_MOUSE_LL, but your two hook procedures would be called separately, and you would have to write your own code to put the events in order.
I doubt you will find a cross-platform solution.
It sounds like Qt might allow you to implement event filters that extend beyond the application to the window system. See also Qt - top level widget with keyboard and mouse event transparency?
If you want to trap events across the whole GUI system, not just one app, there's not much likelihood of a cross platform solution. However, the event hooking part could easily be separated from the recording part, so you could make most of the program cross-platform.
For Windows, you need this 17 year old (!) document. (Man, I'm getting old!)