I'm a rookie java learner. I'm trying to develop a bot (or trainer, whatever) for a simple mini-game, I need to analyze the screen for images and press the corresponding action. Therefore: -
I first tried to use sikuli ScreenRegion for this, but it didn't quite go as expected. What I tried was something like this: -
if(arrowSet.find(oneDown)!=null)
{
r.keyPress(KeyEvent.VK_DOWN);
r.keyRelease(KeyEvent.VK_DOWN);
r.delay(20);
}
But it just jumped to the action, even if the condition was false. Is my application of ScreenRegion wrong in this situation? Or should I use something different than sikuli?
If using Java is not required and you work under Windows, you might consider using Automa - Python tool/library for UI automation. It allows operations on images.
For example, to find out whether an image exists on screen:
Image("arrow_screenshot1.png").exists() # returns True or False
To click on an image:
click(Image("arrow_screenshot1.png"))
To find out image coordinates/center:
Image("arrow_screenshot1.png").x # returns x-coordinate
Image("arrow_screenshot1.png").y # returns y-coordinate
Image("arrow_screenshot1.png").center # returns Point object
To wait until an image appears on the screen:
wait_until(Image("arrow_screenshot1.png").exists)
etc.
I think that using Automa you can quite easily achieve what you need!
Disclaimer: I'm one of Automa's developers
Figured it out, guys! Seems like the image recognition of sikuli is fuzzy. It wont work with targetImage.setMinRank(1.00) wont work, but when I tried running the program with targetImage.setMinRank(0.99), it all worked fluidly. Thank you for you help.
Related
Is there a way to render graphical objects in console using standard jdk API?
For instance I want to render moving image in console (not via awt/swing).
The console itself can only render characters..
But if you mean you want to render an image from a command line app. (that is later displayed in other apps.) then yes, though the 'moving' part does not make much sense unless you mean an animated GIF.
No you cannot. The console renders characters so ASCII-art is your best choice.
Also this has been answered before: displaying-images-in-a-console-application
I'm using the Sikuli API to develop a repetitive task that involves some GUI components. So far I've been able to recognize images, but when I try use the type command it does not pay attention to the case of my string and simply types everything in lower case. The paste function does not work either.
I've tried using Java's Robot class too, but it doesn't simulate key presses correctly. Does anyone know what I'm doing wrong -- or perhaps have any clues that might lead me in the right direction?
Screen s = new Screen();
s.click("img/searchbar.png");
s.wait(2.5);
s.type("v",KeyModifier.CMD);
Thanks in advance!
I am working on a research project.The scenario is this.
I am taking the screenshot of my desktop and then I process it using an API to get the position of a certain text on my Desktop.e.g , say I have the browser open on my desktop and I am on stackoverflow.Now I want to search the position of the logo stackoverflow on the screenshot taken.Then I want to simulate a click on it.I am using Java platform.
Now I have 2 questions:
1)Is there any free API(OCR) which I can use to process the screenshot to fetch text position (or can be done by some trick) and gives good results.
Or Any way you can suggest that I can use (instead of taking screenshot and processing it) to get the position of any text on the screen.
2)How can I simulate the click on the screen using the code by a background program running(I mean I have done it in Swing and other language UIs but this time its different as Now I want to click on the screen.
If I understood you right you want to move your mouse and click on the screen. That not that hard you could use the robot class from Java!
For example:
Robot rob = new Robot();
rob.keyPress( KeyEvent.VK_ENTER );
or what ever, there are so much bottons and movements you could with it. A list of all methods you find here.
And your other question I can't answer. I think there is no API that is able to search a text and give you the position. But what I know is that the robot class is able to capture the screen and put it into a BufferedImage. With it you could compare two pictures.
Maybe you could get use of this but I don't know if it is what you search.
I want to write an address in the address bar of a browser as well as click on a link using java Robot class. How can I track the different objects in a certain window?
Just giving a look at the API http://docs.oracle.com/javase/7/docs/api/java/awt/Robot.html, anyone who do this should know that via Robot Class there is no "trackComponent(Component specificComponent)" method, you got 2 things that may help you:
1-getPixelColor (more than help, seems useless for you by now, maybe i'm wrong).
2-createScreenCapture.
the second method is maybe the answer for your problem, you could take a picture of the screen and with some image processor (javaCV could help you on this: https://code.google.com/p/javacv/) you could then track the components on the screen you took (for instance: from pixels xxx to pixels yyy is the Address bar of browser), of course you need to read some documentation about javaCV (OpenCV) for get this done, after that just use the method for move cursor and enter keys for fill the components, hope someone give a simpler way to do this, but i think this way you learn a bit of JavaCV a really powerful tool.
I will explain my question clearly.
I need to zoom in/zoom out the world map.
When I click on the particular country in map, control should redirected to new page with respective the country.
I dont have any idea about this in java. Please explain the steps to acheive the above task.
As the question is quite general, here is a general answer: Zooming often means, that you want to display a certain percentage of somethin, and not the whole, where your size of the displayed will not change.
But in your case it seems more like a "find a mouse click in a polygon" thing. So you have to add a selection/click listener to whatever widgets you use (Swt? swing? ....?) where you change what your program renders.
It sounds like you may be trying to reinvent the wheel. Google etc have already solved this problem rather well. It might be better to incorporate an existing solution into your application. Have a look at GoogleEarth inside Java Swing.