Scraping flash using HtmlUnit or other java tool - java

I am using HtmlUnit to scrap date from a site but after login all the data is displayed using adobe flash player as swf object, I don't know any way to scrap data from such page.
Is there any way to extract data from flash page, If yes please help me out, either using HtmlUnit or any other java tool.
thanks.

There is no way that you can interact with Flash applet using HtmlUnit. You can try with selenium; I never used it, but it looks like there are plugins that enable flash communication like flash selenium
Personally, I think the only way to test user interfaces, when they have such pitfalls, is to hire a tester human being (a good one), and teach the developers to deeply test the gui every time they alter it.

Seems am not too late, there i answered to my question. what all you need to do is
Create your own browser instant
Inject "stockwave flash" plugin
Instantiate htmlunit with new browser instance.
Hope it will help you.

Related

Java - Go to new url in new window/tab

I want to be able to use Java to tell it to go to X url when X browser is open/running (my lingo is terrible). (Firefox/Chrome/IE is already up, and I want it to go from the default page to let's say Twitter.)
Most of the solutions are using java.awt.Desktop to launch native browser with a url in it, but that isn't useful if I want to change the url later on. (Already on Twitter-Home Page, but want to go to Twitter-Contact Us afterwards.)
The other solutions I've seen involve using Selenium WebDriver, but I also need to eventually learn how to basically force the Java to read a long list of URLs off an excel and simply verify that url isn't dead, and then do this on the Native Android browser, for example. So the Selenium might not be the right choice. Granted, you can also tell me this is an awesome choice for this too if it truly is. I haven't really been exploring Selenium.
Sorry for asking such a basic question. Company wants QA Automation without training/hiring an Automation QA. My end goal (aside not getting canned), is to see if I can get a bunch of urls to load on specific browsers. I can sort of (praying) be able to do stuff with it afterwards.
A simple trick would be to create an add-on( if you know javascript ) which will be quite similar in chrome and firefox (for IE I have no idea in my days it needed BHO) and send websocket commands from java to your addon. But this needs a java websocket server running where your addon will connect when the browser opens. Rest of communication can be carried upon the protocol lines of your requirements.
There are multiple parts to your question.
Read urls from excel.
Use Apache POI to do the same. Selenium code can use the same.
Check that the urls are not dead.
Use any java http client, (apache) to do that without even opening a browser. If the link is dead, it will be dead for all the browser.
Open the links in a multiple browsers.
Selenium is perfect for this. I am assuming that after the page is loaded you have way of validating that the page is correct. Selenium is very powerful here.
Target native android browser too.
I do not know of much difference between this and the previous question unless you are also testing site display based on browser size. The browser is more or less the same as chrome with webkit rendering engine.

Using Cucumber with JOptionPane

So I'm completely new to testing Cucumber, and so far I love it. I've been going through some tutorials that use Selenium with web drivers up to this point, to take input. However, my application uses a bunch of JOptionPane for user input.
I haven't found any packages or functionality to pass data through JOptionPane. If there isn't a way to do this, what is the best way to pass data around without a web interface (besides just directly passing parameters into my methods)? Am I totally missing the point of Cucumber by asking this?
JOptionPane is an awt component. That is, it has nothing to do with Selenium or WebDriver.
What you want is a tool for automating a Java gui.
I wrote some stuff about that a few years ago. It is available at http://www.thinkcode.se/blog/2012/11/01/cucumber-jvm---not-just-for-testing-guis
What I think you really are looking for is described here http://www.thinkcode.se/blog/2012/11/01/building-a-swing-gui
I used fest-swing for communication with the Java ui. It is an old tool that hasn't been updated for a long time. Not sure if there is anything newer or better out there.

How best should I interact with Flash from Selenium Java WebDriver

I am aware a bunch of similar questions such as (Interact flash elements using WebDriver) have been asked in the past however it is still not clear how best to interact with a Flash Element on a page in association with Selenium's Java WebDriver.
By default I know it doesnt support Flash, so I use it to log onto the site which is fine. Now I need to interact with a flash element as seen on this page :
http://store.nike.com/us/en_us/product/free-tr-4-id/?piid=34979&pbid=517639039
I want to be able to pick a shoe size from the Flash on the left hand side of the page, and then click the add to cart button.
Can anybody who has achieved this please offer their guidance and opinion on the best way. I have tried https://code.google.com/p/flash-selenium/ however this appears to be old, or not compatible with the new Java WebDriver.
The answer to your question is: No, there is no way you can interact with Flash from/using Selenium WebDriver. Full stop.
I know there are heaps of developers out there need to automated Flash. But this has never been a part of the Selenium project and it will never be added into. You might want to have a wander around Selenium Users group.
For projects like flash-selenium and flex-ui-selenium, they are not part of the Selenium project but created by third party, which means they can easily be discontinued due to various reasons. If you have decided to use them, you might end up developing the project itself instead of using it. Furthermore, I'm not aware of such projects compatible with Selenium WebDriver anyway.
One other possible solution might be using Sikuli, which is not a Flash automation tool though. It should support automating Flash with its unique image comparison technology. But once again, this has nothing to do with Selenium in any way.
There is a massive misconception that webdriver cannot interact with flash elements.The answer is YES you can interact with flash elements embedded inside html if you can locate the xy coordinates you can interact using selenium ACTION class like so..
Actions Action = new Actions(driver);
WebElement e = driver.findElements(By.cssSelector("button"));
Action.moveToElement(e).clickAndHold().perform();
Action.release().perform();

Firefox = Render the Web-Page to an image file

what is the current "state of art" to render a webpage in firefox into an image file (saved on disk). I want to implement something like "browsershots.org"
1.) Does firefox provide some API to interact with it?
2.) Does the new Canvas (HTML5) Element help here?
The programming language does not matter.
If you know any more information about this topic or a concrete solution, please help.
Uupdate: Yes, the Idea is to do this programatically, only using Technologies "within" Firefox. Further Ideal requirements: Multithreaded (allow rendering of several Websites in parallel on one machine)
Thanks very much, Markus
update: Yes, the Idea is to do this
programatically, only using
Technologies "within" Firefox.
See: https://developer.mozilla.org/en/Drawing_Graphics_with_Canvas#Rendering_Web_Content_Into_A_Canvas
Mozilla's canvas is extended with the
drawWindow() method. This method draws
a snapshot of the contents of a DOM
window into the canvas.
Sounds good, but note that:
This feature is only available for
code running with Chrome privileges.
It is not allowed in normal HTML
pages. Read why.
Why reinvent the wheel?
Fireshot does exactly that: https://addons.mozilla.org/en-US/firefox/addon/fireshot/
Even if you want to implement that yourself, looking at that add-on might help.
Selenium RC is a good choice.
It allows you to control Firefox (and other browsers) using Java (for instance).
For example:
http://seleniumexamples.com/blog/examples/capturing-screenshots-from-remote-selenium-rc/

Blackberry browser field implementation

I need to display any web page as a browser field,not browser session.I am building the app in 4.7 OS.Can anyone help me with sample code?
On 4.7 you are stuck with the old browser field. Also known as Browser Field 1. It renders HTML pretty close to how it was on 4.5 devices. There is no way to take advantage of the newer browser that you invoke via browser session. So, you can forget about CSS2 or or JavaScript on the pages you want to render as a browser field. But for the fairly basic stuff it is good enough.
For the sample code - you really should check out and understand all the examples there are in the development package before starting writing the code. It will really save more time than take for learning. Look at components\samples\com\rim\samples\device\blackberry\browser\BrowserFieldDemo.java
I doubt that the problem lies with the browser field per se. Rather with communications.
The people in the post that you refer to were probably suggesting ";deviceside=true" as this gets around the "BES effect" in a simulator.
You may wish to experiment with running the MDS emulator alongside the simulator session as well. There are a number of posts that discuss the impact of deviceside and simulation. You can test in code to see if you are running a simulator so that you can programmatically influence the "BES effect".

Categories

Resources