Grab a screenshot of Chrome using Chrome Developer Tools? - java

Is it possible to grab a screenshot of an open window using the Chrome Development Tools remote debugger?
For example, I'm connecting to the remote debug port and I have this
code which pops an empty window:
private void sendWindowPop(int width, int height) throws
IOException {
hsc.send("{\"method\": \"Runtime.evaluate\", \"id\": "
+ hsc.nextInt()
+ ", \"params\": {"
+ "\"expression\":
\"window.open('about:blank','name','toolbar=0,scrollbars=0,"
+ "location=0,status=0,menubar=0,resizable=0,width="
+ width
+ ",height="
+ height
+ "');\""
+ "}}");
(hsc is my connection to the debugger at http://localhost:9222)
Then, I load up my target URL with this:
private void loadPage(String uriString) throws IOException {
hsc.send("{\"method\": \"Page.open\", \"id\": " +
hsc.nextInt() + ", \"params\": {\"url\": \"" + uriString + "\"}}");
hsc.waitFor(ChromeNotifications.PAGE_LOADEVENTFIRED, DEFAULT_TIMEOUT_MILLIS);
}
The code above works fine, and first pops a window and then loads the
URL. Ideally, the next thing I would like to do is grab a screenshot
of the loaded web page. Right now, these browser windows pop into an
Xvfb virtual desktop, and I can use ImageMagick's import tool to grab
a screenshot of the target window, but only if it's in the
foreground.
This is a problem, since this application is designed to run in
parallel with multiple windows popping into the virtual desktop. Any
window overlapping my target window will just give me a black
screenshot, since Xfvb only renders what's visible.
I also looked into the API reference, chrome.tabs.captureVisibleTab. No luck there, it doesn't capture what's not visible.
Is there a way, using the remote debugger, to grab a screenshot of an
open window?
(for reference purposes, my ImageMagick command for import is this)
DISPLAY=:0.0 import -window "Google - Chromium" screenshot.png
Where I open the URL http://www.google.com in my chromium browser using loadPage() above. It works great as long as the "Google - Chromium" window that pops is unobstructed and has focus. Drop another window over part of it, and I get a big black area that was not rendered.
Thanks!

Chrome Remote Debugging Protocol now supports the Page.captureScreenshot function
Here is an example in coffee-script
screenshot: (name, callback)=>
safeName = name.replace(/[^()^a-z0-9._-]/gi, '_') + ".png"
png_File = "./_screenshots".append_To_Process_Cwd_Path().folder_Create()
.path_Combine(safeName)
#chrome._chrome.Page.captureScreenshot (err, image)->
require('fs').writeFile png_File, image.data, 'base64',(err)->
callback()
(snippet from
https://github.com/TeamMentor/TM_4_0_Design/blob/Issue_80_Jade_Cleanup/QA/API/QA-TM_4_0_Design.coffee#L54)

Try the Aviary Screen Capture
Check this site too:
http://www.webdesignerdepot.com/2011/08/25-must-have-chrome-extensions-for-web-designers-and-developers/

If you need Java based solution use cdp4j to capture full page screen.
public static void main(String[] args) throws IOException, InterruptedException {
SessionFactory factory = new Launcher().launch();
Path file = createTempFile("screenshot", ".png");
try (Session session = factory.create()) {
session.navigate("https://webfolder.io");
session.waitDocumentReady();
byte[] data = session.captureScreenshot();
write(file, data);
}
if (isDesktopSupported()) {
getDesktop().open(file.toFile());
}
factory.close();
}
Screenshot.java

Related

Java : Determine if String is an URL(without www or http), take its screenshot

I am working on a Spring-MVC webapplication in which we are trying to get a screenshot of an URL. Currently I am using PhantomJS for that task, but it's too slow(>10seconds). Also, the URL's have to be with http/https and www for detecting that it's an URL. As this is a chat application, there can be simple URL's which users add like helloworld.com. Any help would be nice. Thank you.
Code:
String[] words = message.split(" ");
for( String item : words ){
boolean val = ResourceUtils.isUrl(item);
if(val){
urlIdentifier = calcUrl(item);
break;
}else {
System.out.println("Url false is "+item);
}
}
if(urlIdentifier!=null){
replies.setPreviewIdentifier(urlIdentifier);
input.put("preview_identifier",urlIdentifier);
}
Method to calculate screenshot :
private String calcUrl(String website){
try {
String identifier = String.valueOf(new BigInteger(130, random).toString(32));
String previewLocation = msg + "chatthumbs/" + identifier ;
Process proc = Runtime.getRuntime().exec("phantomjs --ssl-protocol=any " +
"/home/deploy/phantom/rasterizepdf.js " +" "+website+" " +previewLocation);
proc.waitFor();
BufferedImage image = ImageIO.read(new File("/home/akshay/testme.png"));
if(image!=null){
if (image.getWidth() > image.getHeight()) {
image = Scalr.resize(image, Scalr.Mode.FIT_TO_HEIGHT, 250, 250);
} else {
image = Scalr.resize(image, Scalr.Mode.FIT_TO_WIDTH, 250, 250);
}
image = Scalr.crop(image, 250, 250);
ImageIO.write(image, "png", new File(previewLocation));
}
return identifier;
}catch (Exception e){
e.printStackTrace();
}
return null;
}
Any help would be nice. Thank you.
(a) I think the process of taking a screen shot is taking time. Is this code running on the same device as the chat screen? Why not use java.awt.Robot to take the screen shot ? or just save the text why do you need a screen shot?
(b) Is the system too busy/ Use on a SSD to see if faster?
(c) But curious as to the end application, is this part of a web app? How will your code run on the client systems? Or will you java agent be installed on all user systems that monitors the web page and takes screen shots? Then why use a web page, use a java app to chat, and parse the text as typed.
Parsing the text. What if user types/ pastes a long message? Are you parsing everything once or on change? Think of ways to improve that if it seems to be a problem. Ignore if not the immediate issue now.
Also if the msg is too long the parsing can take a lot of time. maybe process after every key press or change event (if paste) keep local js copy of previous text to get diff?

IBM PCOMM Automation using Java

I am trying to automate IBM PComm Application using HACL Java library classes .
I have succeeded in establishing connection to the pcom session as well as set /get cursor position and extracting text from the current cursor position on the application's screen. But unable to put / send text at a desired cursor position on the screen. Kindly help in resolving this issue .Please find the code for establishing connection and fetch text from the screen as below :
import java.util.Properties;
import com.ibm.eNetwork.ECL.ECLConnMgr;
import com.ibm.eNetwork.ECL.ECLConnection;
import com.ibm.eNetwork.ECL.ECLErr;
import com.ibm.eNetwork.ECL.ECLField;
import com.ibm.eNetwork.ECL.ECLFieldList;
import com.ibm.eNetwork.ECL.ECLPS;
import com.ibm.eNetwork.ECL.ECLSession;
import org.ohio.iOhioScreen;
public class Pcom {
public static void main(String[] args) throws ECLErr {
try{
System.loadLibrary("pcseclj");
Properties prop = new Properties();
// prop.put("SESSION_VT_LOCAL_ECHO ", "true");
prop.put("SESSION_HOST", "C:\\Mainframe\\A.ws"); // works OK
prop.put("SESSION_WIN_STATE", "MAX");
prop.put("SESSION_VT_KEYPAD ", "SESSION_VT_KEYPAD_APPL");
prop.put("SESSION_VT_LOCAL_ECHO", "SESSION_VT_LOCAL_ECHO_ON");
ECLSession session = new ECLSession(prop);
session.StartCommunication(); //works OK
Thread.sleep(5000);
session.connect(); //works OK
ECLFieldList fieldList = session.GetPS().GetFieldList();
session.GetPS().SetCursorPos(18, 044); //works OK
/session.GetPS().SetString("some_text"); // does not work
for(int i=0;i<fieldList.size();i++){ //works OK
//System.out.println("field ======================= "+fieldList.GetFirstField(i).getAttribute());
ECLPS ps=session.GetPS();
System.out.println(session.GetName()); //works Ok
session.GetPS().SetCursorPos(17, 44); //works OK
session.GetPS().SendKeys("some_text",17,44); // does not work ,17,44 are co ordinate positions pn screen
System.out.println(session.GetConnType()); // works ok
ps.SendKeys("some_text"); //does not work
/* ------------ does not work-------------
fieldList.FindField(17, 44).SetText("some_text");
fieldList.FindField(17, 44).SetString("some_text");
fieldList.FindField(18, 44).setString("some_text");
*/
System.out.println(fieldList.FindField(17, 44).GetLength()); // works ok
System.out.println(fieldList.FindField(17, 28).getString()); //works ok
}
catch(Exception e)
{
System.out.println(e);
}
}
}
I had similar issue before while automating PCOMM with Cucumber to make some automated regression test framework "middleware" for green screen in BDD style.
Thing is that SetCursorPos does not send new cursor position to the Host System (we use IBM i). Telnet5250 protocol is quite complicated but in few words you have two separate buffers - one on the client system (Terminal Emulator) and second on the Host System (telnet server). Usually they are sychronized, but in some circumstances they are not which leads to undefined behavior.
Litle hack is to send up and down arrow keys like this:
SendKeys("<Up>");
SendKeys("<Down>");
This will force PComm to send new cursor position to server and sync screen buffers.

Elevate Java application while running

A nasty problem popped out with my software. I am making a program that interacts with another existing software (a game). User has reported that he runs the game with administrator privileges and under that circumstances, my program stops working for him.
Short investigation revealed that some people really need to run the game under administrator account and some don't. It would be great if my program would be able to detect this and warn user if the game is running under administrator account:
If the user clicks "Elevate", I'd like to ask windows to elevate the java.exe running my jar file and invoke the typical UAC dialog.
Obviously, this time the question would not be about java updater but JRE
My question is: Is this possible? Can windows elevate my java.exe instance's privilege? Does java have a way to do it? Or can I use command line command?
I want to avoid restarting the program (though it wouldn't probably be such a big deal).
Edit:
If you look in the comments, you'll see that there's no avoiding the restart of an application - process can only start elevated, not become elevated. This kinda shifts the question, unfortunately. Basically, it now sounds more like: "How to restart my application with admin rights?". Unless, of course, there's a trick like two java.exe sharing one jar...
If still of interest: In Windows 7 my JavaElevator works. It elevates a running Java process when used in the main method of the Java application. Simply add -elevate as last program parameter and use the elevator in the main method.
The elevator class:
package test;
import com.sun.jna.Native;
import com.sun.jna.platform.win32.Kernel32;
import com.sun.jna.platform.win32.Kernel32Util;
import com.sun.jna.platform.win32.ShellAPI;
import com.sun.jna.platform.win32.WinDef;
/**
* Elevates a Java process to administrator rights if requested.
*/
public class JavaElevator {
/** The program argument indicating the need of being elevated */
private static final String ELEVATE_ARG = "-elevate";
/**
* If requested, elevates the Java process started with the given arguments to administrator level.
*
* #param args The Java program arguments
* #return The cleaned program arguments
*/
public static String[] elevate(String[] args) {
String[] result = args;
// Check for elevation marker.
boolean elevate = false;
if (args.length > 0) {
elevate = args[args.length - 1].equals(ELEVATE_ARG);
}
if (elevate) {
// Get the command and remove the elevation marker.
String command = System.getProperty("sun.java.command");
command = command.replace(ELEVATE_ARG, "");
// Get class path and default java home.
String classPath = System.getProperty("java.class.path");
String javaHome = System.getProperty("java.home");
String vm = javaHome + "\\bin\\java.exe";
// Check for alternate VM for elevation. Full path to the VM may be passed with: -Delevation.vm=...
if (System.getProperties().contains("elevation.vm")) {
vm = System.getProperty("elevation.vm");
}
String parameters = "-cp " + classPath;
parameters += " " + command;
Shell32.INSTANCE.ShellExecute(null, "runas", vm, parameters, null, 0);
int lastError = Kernel32.INSTANCE.GetLastError();
if (lastError != 0) {
String errorMessage = Kernel32Util.formatMessageFromLastErrorCode(lastError);
errorMessage += "\n vm: " + vm;
errorMessage += "\n parameters: " + parameters;
throw new IllegalStateException("Error performing elevation: " + lastError + ": " + errorMessage);
}
System.exit(0);
}
return result;
}
}
Usage in the main method of the Java application:
public static void main(String[] args) {
String[] args1 = JavaElevator.elevate(args);
if (args1.length > 0) {
// Continue as intended.
...
I know, this is a very basic implementation - sufficient for one of my daily hiccups: Starting an elevated process from Eclipse. But maybe it points someone in some dicrection...
As has been pointed in comments, sadly the Java (or any other process) cannot be elevated while running. While in the case of JWM, it could be theoretically possible to move whole program context from normal user java.exe to elevated one, I don't think it's possible. I hope some day someone will come and tell me I'm wrong.
Surprisingly, even with restart in place, this was a tricky task that took me a while to figure out.
The non java part
First, how do we exactly run a program elevated from command line? There's an answer and you can see it's not simple. But we can break it to this VBS script:
Set UAC = CreateObject("Shell.Application")
UAC.ShellExecute "program name", "command line parameters", "working directory", "runas", 1
Soon, it also turns out that we won't have any success running java.exe from VBS script. In the end, I decided to run a helper batch file. Finally, here (answer to question in the last link) we have a complete set of two scripts which really run the given .jar file elevated. Here's improved version that allows quick testing by drag'n'dropping the Jar file on it:
' Require first command line parameter
if WScript.Arguments.Count = 0 then
MsgBox("Jar file name required.")
WScript.Quit 1
end if
' Get the script location, the directorry where it's running
Set objShell = CreateObject("Wscript.Shell")
strPath = Wscript.ScriptFullName
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.GetFile(strPath)
strFolder = objFSO.GetParentFolderName(objFile)
'MsgBox(strFolder)
' Create the object that serves as runnable something
Set UAC = CreateObject("Shell.Application")
' Args:
' path to executable to run
' command line parameters - first parameter of this file, which is the jar file name
' working directory (this doesn't work but I use it nevertheless)
' runas command which invokes elevation
' 0 means do not show the window. Normally, you show the window, but not this console window
' which just blinks and disappears anyway
UAC.ShellExecute "run-normally.bat", WScript.Arguments(0), strFolder, "runas", 0
WScript.Quit 0
The Java part
Java part is more straightforward. What we need to do is to open new process and execute the prepared scripts in it.
/**
* Start this very jar file elevated on Windows. It is strongly recommended to close any existing IO
* before calling this method and avoid writing anything more to files. The new instance of this same
* program will be started and simultaneous write/write or read/write would cause errors.
* #throws FileNotFoundException if the helper vbs script was not found
* #throws IOException if there was another failure inboking VBS script
*/
public void StartWithAdminRights() throws FileNotFoundException, IOException {
//The path to the helper script. This scripts takes 1 argument which is a Jar file full path
File runAsAdmin = new File("run-as-admin.vbs");;
//Our
String jarPath;
//System.out.println("Current relative path is: " + s);
try {
jarPath = "\""+new File(Main.class.getProtectionDomain().getCodeSource().getLocation().toURI().getPath()).getAbsolutePath()+"\"";
} catch (URISyntaxException ex) {
throw new FileNotFoundException("Could not fetch the path to the current jar file. Got this URISyntax exception:"+ex);
}
//If the jar path was created but doesn't contain .jar, we're (most likely) not running from jar
//typically this happens when running the program from IDE
//These 4 lines just serve as a fallback in testing, should be deleted in production
//code and replaced with another FileNotFoundException
if(!jarPath.contains(".jar")) {
Path currentRelativePath = Paths.get("");
jarPath = "\""+currentRelativePath.toAbsolutePath().toString()+"\\AutoClient.jar\"";
}
//Now we check if the path to vbs script exists, if it does we execute it
if(runAsAdmin.exists()) {
String command = "cscript \""+runAsAdmin.getAbsolutePath()+"\" "+jarPath;
System.out.println("Executing '"+command+"'");
//Note that .exec is asynchronous
//After it starts, you must terminate your program ASAP, or you'll have 2 instances running
Runtime.getRuntime().exec(command);
}
else
throw new FileNotFoundException("The VBSScript used for elevation not found at "+runAsAdmin.getAbsolutePath());
}
This is my version. It creates a VBScript script, then executes it. This only works if the program that is being run is in a jar file, so you will have to run your IDE as administrator to actually test your program.
public static void relaunchAsAdmin() throws IOException {
relaunchAsAdmin(ThisClass.class); //Change ThisClass to the class that this method is in
}
public static void relaunchAsAdmin(Class<?> clazz) throws IOException {
if(isCurrentProcessElevated()) {
return;
}
final String dir = System.getProperty("java.io.tmpdir");
final File script = new File(dir, "relaunchAsAdmin" + System.nanoTime() +
".vbs");
try {
script.createNewFile();
OutputStreamWriter osw = new OutputStreamWriter(new FileOutputStream(script));
osw.append("Set s=CreateObject(\"Shell.Application\")" + ln + "s.ShellExecute \"" +
System.getProperty("java.home") + "\\bin\\java.exe" + "\",\"-jar \"\"" +
new File(clazz.getProtectionDomain().getCodeSource(
).getLocation().toURI()).getAbsolutePath() + "\"\"\",,\"runas\",0" +
ln + "x=createObject(\"scripting.fileSystemObject\").deleteFile(" +
"WScript.scriptfullname)");
osw.close();
if(System.getenv("processor_architecture").equals("x86")) {
Runtime.getRuntime().exec("C:\\Windows\\System32\\wscript.exe \"" +
script.getAbsolutePath() + "\"");
} else {
Runtime.getRuntime().exec("C:\\Windows\\SysWoW64\\wscript.exe \"" +
script.getAbsolutePath() + "\"");
}
} catch(URISyntaxException e) {
e.printStackTrace();
}
Runtime.getRuntime().exit(0);
}
Note that it is a bit messy. I have been using this method before, so it has been line wrapped to 100 characters (except the comment I wrote for this answer). The
isCurrentProcessElevated()
method will have to be implemented in one way or another. You could try using JNI, or you could use a pure Java method, such as writing in the Program Files or System32 directory and seeing if it failed.
Obviously, this solution will only work on Windows. I never needed to elevate on Linux or Mac systems (mainly because I don't have any Mac systems, and I don't use Linux - I just play with it).

Sikuli - Click on element does not work with Java API but it works from IDE - WPF app

I am evaluating tools for testing a WPF based app. I am currently trying Sikuli with the Java API. When I try to click on an object from Java code, the mouse cursor goes to the object and the object is highlighted, however the click action does not work, because the expected menu does not open. The click() method returns status 1 though.
If I am doing a click from Sikuli IDE, it works fine.
I tried 1.0.1 version and also the nightly build. Here's my code:
#Test
public void testLogin() {
Screen s = new Screen();
try {
s.wait(Constants.overflowMenu);
System.out.println(s.click(Constants.overflowMenu));
s.wait(Constants.signInMenuOption, 5);
} catch (FindFailed e) {
Assert.fail(e.getMessage());
}
}
What am I doing wrong?
try this code, it worked for me. what it does, it checks the image, click on it and then check this image again on the screen, if it still exists, click again.
Screen screen = new Screen();
Pattern pattern = null;
try
{
pattern = new Pattern(imageLocation);
screen.wait(pattern,30);
screen.click(pattern);
System.out.println("First Attempt To Find Image.");
}
catch(FindFailed f)
{
System.out.println("Exception In First Attempt: " +f.getMessage());
System.out.println("FindFailed Exception Handled By Method: ClickObjectUsingSikuli. Please check image being used to identify the webelement. supplied image: " +imageLocation);
Assert.fail("Image wasn't found. Please use correct image.");
}
Thread.sleep(1000);
//In case image/object wasn't clicked in first attempt and cursor stays in the same screen, then do second atempt.
if(screen.exists(pattern) != null)
{
try
{
screen.getLastMatch().click(pattern);
System.out.println("Second Attempt To Find Image.");
System.out.println("Object: " +imageLocation + " is clicked successfully.");
}
catch(FindFailed f)
{
System.out.println("Exception In Second Attempt: " +f.getMessage());
System.out.println("FindFailed Exception Handled By Method: ClickObjectUsingSikuli. Please check image being used to identify the webelement. supplied image: " +imageLocation);
}
}
In my case it seems it was a problem with the fact that I have two monitors..

How to pass URL parameters from Java to local HTML file in Windows 7?

I desperately need your expertise in resolving a Windows-7 issue.
Scenario: I have a frame-based Help package that is set up for context-sensitive help calls. A Java application is able to control what page the Help packages opens to by passing a tag representing the desired HTML named anchor to an HTML file called pophelp. This file has javascripting that reads the passed tag from the end of the URL and maps it to the appropriate HTML file in the help package and opens it.
Issue: The above scenario works in Windows XP, but no longer in Windows 7.
Calling mechanism from Java application: rundll32 url.dll,FileProtocolHandler file://filepath/pophelp.html?tag
Summary of findings so far: It appears that url.dll no longer allows parameters to be passed with URLs in Windows 7. Parameters are being stripped. I also tried the same type of call using Desktop.getDesktop().browse() from Java, but it too seems to strip off all parameters after .html.
Sample code:
Original call that works in Windows XP --
Running command: rundll32 url.dll,FileProtocolHandler file://C:\Program Files\App System\App-Editor-8.0.1\help\pophelp.html?TAG=callsubflow
Result: ?TAG=callsubflow is not passed.
New code using Desktop.getDesktop().browse() --
public static void main(String[] args) {
String url = "file:///C:/Program Files/App System/App-Editor-8.0.1/help/pophelp.html?TAG=callsubflow";
try {
if (Desktop.isDesktopSupported()) {
Desktop desktop = Desktop.getDesktop();
if (desktop.isSupported(Desktop.Action.BROWSE)) {
desktop.browse(new URI(url.replace(" ", "%20")));
}
}
} catch (IOException e) {
System.out.println("Unable to open "+url+": "+e.getMessage());
} catch (URISyntaxException e) {
e.printStackTrace();
}
}
Result: ?TAG=callsubflow is not passed.
Any assistance would be appreciated!
I really can't tell why Windows removes parameters on local files. As mentioned in the comments this seams to be some kind of weird restrictions for security. But I once had a similar problem and I found a workaround that fits in this situation as well.
Simply create a local temporary HTML file (without parameters) that redirects you to the desired one (with parameters).Have a look at these two methods:
// creates w3c conform redirect HTML page
public String createRedirectPage(String url){
return "<!DOCTYPE HTML>" +
"<meta charset=\"UTF-8\">" +
"<meta http-equiv=\"refresh\" content=\"1; url=" + url + "\">" +
"<script>" +
"window.location.href = \"" + url + "\"" +
"</script>" +
"<title>Page Redirection</title>" +
"<!-- Note: don't tell people to `click` the link, just tell them that it is a link. -->" +
"If you are not redirected automatically, follow the <a href='" + url + "'>link</a>";
}
// creates temporary file with content of redirect HTML page
public URI createRedirectTempFile(String url) {
BufferedWriter writer = null;
File tmpFile = null;
try {
// creates temporary file
tmpFile = File.createTempFile("pophelp", ".html", null);
// deletes file when the virtual machine terminate
tmpFile.deleteOnExit();
// writes redirect page content to file
writer = new BufferedWriter(new FileWriter(tmpFile));
writer.write(createRedirectPage(url));
writer.close();
}
catch (IOException e) {
return null;
}
return tmpFile.toURI();
}
Now you can use these like this:
String url = "file:///C:/Program Files/App System/App-Editor-8.0.1/help/pophelp.html?TAG=callsubflow";
if (Desktop.isDesktopSupported()) {
Desktop desktop = Desktop.getDesktop();
if (desktop.isSupported(Desktop.Action.BROWSE)) {
desktop.browse(createRedirectTempFile(url.replace(" ", "%20")));
}
}
I have a solution, not a quick (or pretty) solution, but a solution nonetheless :)
rundll32 url.dll,FileProtocolHandler does pass params when using URLs with http/s protocol (try rundll32 url.dll,FileProtocolHandler http://www.google.com?q=google), so you can setup small http server (like Jetty i guess) to serve your help files and show them using
rundll32 url.dll,FileProtocolHandler http://localhost:[helpServerIp]/help/pophelp.html?TAG=callsubflow

Categories

Resources