Develop Reference

Using Robot class causing windows to get stuck

```import javax.imageio.ImageIO;
import javax.swing.*;
import java.awt.*;
import java.awt.event.InputEvent;
import java.awt.image.BufferedImage;
import java.io.File;
import java.awt.Graphics;
import java.awt.Image;
import java.io.IOException;
public class Movement {
public static void main(String args[]) throws AWTException, IOException {
Robot mineMove = new Robot();
mineMove.delay(4000);
moveMouse(mineMove, 500, 0);
placeBlock(mineMove);
moveMouse(mineMove, -500, 0);
mineMove.delay(500);
moveMouse(mineMove, 0, -600);
interactWithVill(mineMove);
moveMouse(mineMove, 0, 600);
// moveMouse(mineMove, 500, 0);
// breakAndPickBlock(mineMove);
// moveMouse(mineMove, -500, 0);
}```
public void saveScreen (Robot mineMove) throws IOException {
Rectangle screen = new Rectangle();
screen.x = 0;
screen.y = 0;
screen.height = 1200;
screen.width = 800;
BufferedImage i = mineMove.createScreenCapture(screen);
// System.out.println(i.getType());
// Graphics g = i.getGraphics();
// JFrame frame = new JFrame();
// Image img = i;
// g.drawImage(img, 20,20,null);
File file = new File("Tester-Screen-Grab.png");
ImageIO.write(i, "png", file);
}
``` ** public static void interactWithVill(Robot mineMove) {
mineMove.mousePress(InputEvent.BUTTON3_DOWN_MASK);
mineMove.delay(100);
mineMove.mouseMove(731, 302);
//sreenshot and look
mineMove.delay(1000);
mineMove.mouseMove(731, 384);
//screenshot and check enchant
mineMove.delay(1000);
mineMove.keyPress(27); // keycode : 27 = ESC
mineMove.delay(10);
mineMove.keyRelease(27);
mineMove.delay(1000);
}**```
This is the method where I think the problem lies.
``` public static void moveMouse(Robot mineMove, int mouseMoveCordY, int mouseMoveCordX) {
Point mousePos = MouseInfo.getPointerInfo().getLocation();
int currPosX = (int) mousePos.getX();
int currPosY = (int) mousePos.getY();
currPosY += mouseMoveCordY;
currPosX += mouseMoveCordX;
mineMove.mouseMove(currPosX, currPosY);
mineMove.delay(50);
}```
``` public static void placeBlock (Robot mineMove) {
mineMove.keyPress(50); // keycode : 50 = 2
mineMove.delay(10);
mineMove.keyRelease(50);
mineMove.delay(10);
mineMove.mousePress(InputEvent.BUTTON3_DOWN_MASK);
mineMove.delay(100);
mineMove.mouseRelease(InputEvent.BUTTON3_DOWN_MASK);
mineMove.delay(1000);
}```
``` public static void breakAndPickBlock (Robot mineMove) {
mineMove.mousePress(InputEvent.BUTTON1_DOWN_MASK);
mineMove.delay(500);
mineMove.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
mineMove.keyPress(87);
mineMove.delay(1000);
mineMove.keyRelease(87);
mineMove.keyPress(83);
mineMove.delay(1000);
mineMove.keyRelease(83);
}```
}
I am trying to make a simple bot to practice making my own projects and I wanted to use the Robot class to make mouse movements and keystrokes in the game.
Everything worked until the "interactWithVill" method. When this runs it will work fine but then the computer will be stuck on whatever screen I am on, whether that be in the game or in the IDE, and no matter what I do I cannot interact with other windows but the window that it was stuck on works fine. The only fix is closing the window I was stuck on. Any help would be great if there were any errors I did working with the Robot class. If there is a better approach or class to use any help would be great as I am new to this.
Another smaller issue is that when I try to use Robot.moveMouse(int x, int y) it won't work unless I wiggle the mouse a bit before that command runs, why would this happen?
Thank you!

How to Click a desired pixels on the screen?

I'm trying to make a program that clicks the specified pixel on the screen.
I can do a screen capture like this
Dimension screenSize = Toolkit.getDefaultToolkit().getScreenSize();
Rectangle screenRectangle = new Rectangle(screenSize);
Robot robot = new Robot();
BufferedImage image = robot.createScreenCapture(screenRectangle)
but I don't know how to make a procedure on this image.
What can I use to do this? Thanks :)

You can click on a specific Pixel with the mouse using Robot class in java
import java.awt.Robot;
import java.awt.AWTException;
import java.awt.event.InputEvent;
public class MouseClicker {
static Robot rb;
public static void main(String[] args) {
try {
rb = new Robot();
} catch (AWTException e) {
System.out.println("Error while creating object" + e);
}
rb.mouseMove(1360, 768); //this method takes two parameters (Height, Width)
rb.mousePress(InputEvent.BUTTON1_DOWN_MASK);//this method takes one parameter (BUTTON) to press it
rb.delay(10); //this is the delay between every press
rb.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);//this method takes one parameter (BUTTON) to release it
}
}
For more refrance you can see the Oracle Docs

Web Cam access using JavaFX

I've been searching for a solution to accessing a built-in webcam from a java or javaFX application. I've seen loads of other posts pointing to OpenCV and JavaCV, Sarxos's library and quite a few others.
I've run into difficulties such as newer versions of OpenCV not working with older code posted on various sites and newer code that uses OpenCV 3.0 is hard to find or doesn't do what I need, which is simply a customer application which saves an image taken from the web cam to a variable (or file).
Hope someone can point me in the right direction.
Thanks in advance

You're in luck. I toyed around with OpenCV last weekend and ran into the same problems as you. Here's an example about how to do it. The example opens the camera, uses an AnimationTimer (a bit overkill, but was a quick solution for prototyping) to grab a mat image periodically, converts the mat image to a JavaFX image, performs face detection and paints it on a canvas.
Here's what you need:
Download OpenCV, e. g. in my case the windows version. Rename the opencv-3.0.0.exe to opencv-3.0.0.exe.zip and open it. Extract the contents of build/java.
Create a new JavaFX project. Put the jar and dlls into a lib folder, e. g.:
lib/opencv-300.jar
lib/x64/opencv_java300.dll
Add the jar to your build path.
In your src folder create a path opencv/data/lbpcascades and put the file lbpcascade_frontalface.xml in there (found in etc/lbpcascades). That's only for face detection, you can uncomment the code in case you don't need it.
Create the application class, code:
import java.io.ByteArrayInputStream;
import java.lang.reflect.Field;
import java.net.URISyntaxException;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import javafx.animation.AnimationTimer;
import javafx.application.Application;
import javafx.event.EventHandler;
import javafx.geometry.Rectangle2D;
import javafx.scene.Group;
import javafx.scene.Scene;
import javafx.scene.canvas.Canvas;
import javafx.scene.canvas.GraphicsContext;
import javafx.scene.image.Image;
import javafx.scene.paint.Color;
import javafx.stage.Stage;
import javafx.stage.WindowEvent;
import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.MatOfByte;
import org.opencv.core.MatOfRect;
import org.opencv.core.Rect;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.objdetect.CascadeClassifier;
import org.opencv.videoio.VideoCapture;
public class Camera extends Application {
private static final int SCENE_W = 640;
private static final int SCENE_H = 480;
CascadeClassifier faceDetector;
VideoCapture videoCapture;
Canvas canvas;
GraphicsContext g2d;
Stage stage;
AnimationTimer timer;
#Override
public void start(Stage stage) {
this.stage = stage;
initOpenCv();
canvas = new Canvas(SCENE_W, SCENE_H);
g2d = canvas.getGraphicsContext2D();
g2d.setStroke(Color.GREEN);
Group group = new Group(canvas);
Scene scene = new Scene(group, SCENE_W, SCENE_H);
stage.setScene(scene);
stage.setResizable(false);
stage.show();
timer = new AnimationTimer() {
Mat mat = new Mat();
#Override
public void handle(long now) {
videoCapture.read(mat);
List<Rectangle2D> rectList = detectFaces(mat);
Image image = mat2Image(mat);
g2d.drawImage(image, 0, 0);
for (Rectangle2D rect : rectList) {
g2d.strokeRect(rect.getMinX(), rect.getMinY(), rect.getWidth(), rect.getHeight());
}
}
};
timer.start();
}
public List<Rectangle2D> detectFaces(Mat mat) {
MatOfRect faceDetections = new MatOfRect();
faceDetector.detectMultiScale( mat, faceDetections);
System.out.println(String.format("Detected %s faces", faceDetections.toArray().length));
List<Rectangle2D> rectList = new ArrayList<>();
for (Rect rect : faceDetections.toArray()) {
int x = rect.x;
int y = rect.y;
int w = rect.width;
int h = rect.height;
rectList.add(new Rectangle2D(x, y, w, h));
}
return rectList;
}
private void initOpenCv() {
setLibraryPath();
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
videoCapture = new VideoCapture();
videoCapture.open(0);
System.out.println("Camera open: " + videoCapture.isOpened());
stage.setOnCloseRequest(new EventHandler<WindowEvent>() {
public void handle(WindowEvent we) {
timer.stop();
videoCapture.release();
System.out.println("Camera released");
}
});
faceDetector = new CascadeClassifier(getOpenCvResource(getClass(), "/opencv/data/lbpcascades/lbpcascade_frontalface.xml"));
}
public static Image mat2Image(Mat mat) {
MatOfByte buffer = new MatOfByte();
Imgcodecs.imencode(".png", mat, buffer);
return new Image(new ByteArrayInputStream(buffer.toArray()));
}
private static void setLibraryPath() {
try {
System.setProperty("java.library.path", "lib/x64");
Field fieldSysPath = ClassLoader.class.getDeclaredField("sys_paths");
fieldSysPath.setAccessible(true);
fieldSysPath.set(null, null);
} catch (Exception ex) {
ex.printStackTrace();
throw new RuntimeException(ex);
}
}
public static String getOpenCvResource(Class<?> clazz, String path) {
try {
return Paths.get( clazz.getResource(path).toURI()).toString();
} catch (URISyntaxException e) {
throw new RuntimeException(e);
}
}
public static void main(String[] args) {
launch(args);
}
}
Of course you can do whatever you wish (e. g. saving) with the JavaFX image once you have it.

for sarxos, pseudo code, i can't publish whole class:
import com.sleepingdumpling.jvideoinput.Device;
import com.sleepingdumpling.jvideoinput.VideoFrame;
import com.sleepingdumpling.jvideoinput.VideoInput;
Device choosenDevice;
for (Device device : VideoInput.getVideoDevices()) {
// select your choosenDevice webcam here
if (isMyWebcam(device)) {
choosenDevice = device;
break;
}
}
// eg. VideoInput(640,480,25,choosenDevice );
VideoInput videoInput = new VideoInput(frameWidth, frameHeigth,
frameRate, choosenDevice );
VideoFrame vf = null;
while (grabFrames) {
vf = videoInput.getNextFrame(vf);
if (vf != null) {
frameReceived(vf.getRawData());
// or vf.getBufferedImage();
}
}
videoInput.stopSession();

Applet Java To take a screenshot of a part of my web page

i need to ask if there are a way ( applet or any think else ) to take a screenshot of the content on a div in webpage ???
i found this java code to take screenshot of the full screen :
import java.awt.AWTException;
import java.awt.Rectangle;
import java.awt.Robot;
import java.awt.Toolkit;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import javax.imageio.ImageIO;
public class ScreenShot
{
/**
method to capture screen shot
#param String uploadPath to save screen shot as image
#returns boolean true if capture successful else false
*/
boolean captureScreenShot(String uploadPath)
{
boolean isSuccesful = false;
Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
BufferedImage capture;
try
{
capture = new Robot().createScreenCapture(screenRect);
// screen shot image will be save at given path with name "screen.jpeg"
ImageIO.write(capture, "jpg", new File( uploadPath, "screen.jpeg"));
isSuccesful = true;
}
catch (AWTException awte)
{
awte.printStackTrace();
isSuccesful = false;
}
catch (IOException ioe)
{
ioe.printStackTrace();
isSuccesful = false;
}
return isSuccesful;
}
}
thanks

This is similar to the following question: Take a screenshot of a web page in Java

Get background image from PowerPoint Slide using Java

I need to get the background image from a PowerPoint slide using java. I am aware of the Apache POI project. I can find material for getting text and shapes from the slides, but not the actual background. Does anyone have any suggestions?
EDIT: I have creaked the following code using the suggested link. This code seems to grab the contents of the slide, but not exactly the background. The resulting images are white for the background.
I tried it with this PowerPoint
package PowerPointProcessing;
import Logging.LogRunner;
import java.awt.Color;
import java.awt.Dimension;
import java.awt.Graphics2D;
import java.awt.geom.Rectangle2D;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.imageio.ImageIO;
import org.apache.poi.hslf.HSLFSlideShow;
import org.apache.poi.hslf.model.Background;
import org.apache.poi.hslf.model.Fill;
import org.apache.poi.hslf.model.Shape;
import org.apache.poi.hslf.model.Slide;
import org.apache.poi.hslf.usermodel.SlideShow;
/**
*
* #author dvargo
*/
public class PPI
{
Dimension pageSize;
public Slide[] theSlides;
public PPI(String powerPointFilePath)
{
SlideShow ppt = null;
//open the presentation
try
{
ppt = new SlideShow(new HSLFSlideShow(powerPointFilePath));
}
catch(Exception e)
{
LogRunner.getLogger().severe("Could not open the powerpoint presentation");
return;
}
//get all the slides
theSlides = ppt.getSlides();
//see how many slides there are
int numberOfSlides = theSlides.length;
pageSize = ppt.getPageSize();
}
public BufferedImage getBackground(Slide theSlide)
{
Background background;
background = theSlide.getBackground();
Fill f = background.getFill();
Color color = f.getForegroundColor();
Shape[] theShapes = theSlide.getShapes();
BufferedImage img = new BufferedImage(pageSize.width, pageSize.height, BufferedImage.TYPE_INT_RGB);
Graphics2D graphics = img.createGraphics();
graphics.setPaint(color);
graphics.fill(new Rectangle2D.Float(0, 0, pageSize.width, pageSize.height));
theSlide.draw(graphics);
return img;
}
public static void main(String[] args) {
PPI ppi = new PPI("C:\\Documents and Settings\\dvargo\\Desktop\\Cludder\\a.ppt");
int count= 0;
for (Slide currSlide : ppi.theSlides)
{
BufferedImage img = ppi.getBackground(currSlide);
try
{
ImageIO.write(img, "jpeg", new File("C:\\Documents and Settings\\dvargo\\Desktop\\ppt\\" + count + ".jpeg"));
}
catch (IOException ex)
{
Logger.getLogger(PPI.class.getName()).log(Level.SEVERE, null, ex);
}
count++;
}
}
}

Looking at the code from this question:
Extracting images from pptx with apache poi
Looks like it should be something like:
Background background = slides[current].getBackground();
Fill f = background.getFill();
Color color = f.getForegroundColor();