Develop Reference

How can I switch off Microsoft Defender SmartScreen using selenium-webdriver to avoid errors when downloading xml files?

I'm using Edge for automated tests with selenium-webdriver. I have faced an issue which prevents downloading of xml files.
Error message
Do you have any ideas how can I switch off Microsoft Defender SmartScreen when starting Edge browser? For chrome browser it can be done by setting option prefs.put("safebrowsing.enabled", false);. Is there anything similar for Edge?

The error message pop up is not controlled by Microsoft Defender SmartScreen so there's no use to switch it off.
I know that you want to control it with Selenium but there's no Edge option that can override the warning in Selenium. The solutions in other threads are for old versions of browsers. safe browsing is now the default behavior so it won't work either. There's not much Selenium can do with the warning.
If you want to get rid of the warning when downloading xml files, you can only set this group policy first, then use Selenium to automate.
For more information, you can also refer to this thread. In the accepted answer of the thread, it uses PowerShell script to update Windows Registry to set the group policy, you can also use it as a workaround.

Java Selenium Chromedriver webdriver as fast as possible

I am connecting a standalone program to a website, and I have to read some pages of the website. For first, I used Jsoup, but with this I discovered that some informations that I need are loaded after page load, so I searched for webdrivers. (I am not looking for images or something big, my content are all textual)
Now, i found the ChromeDriver, but it is too slow for my case because it has a lot of options and features.
In my case I need just a step more than the Jsoup possibilities.
It is possible to disable the best part of ChromeDriver options and features to reach this goal?
For example, i saw plugin disabling, but is one by one and is not for every chrome browser on every pc. I didn't found an option like "plugin.disable-all".
Furthermore,in this way I cannot open more than a few instance of chromedriver. In this moment, every instance of the chromedriver is opening a Google Chrome Helper that uses 100Mb of ram.
Hope all is clear

HtmlUnit might be enough for your needs. It does support some Javascript.
It can be used with Webdriver. But might as well be enough on its own

To make your webdriver run faster (but not that much faster), you can run the driver in Headless mode. See these articles for a tutorial on how to go into Headless mode for Chrome.
Before starting the driver, add the --headless argument to ChromeOptions.
Headless mode can speed up your automation by not rendering the browser window, but keep in mind that doing a straight HTTP GET with JSoup would always be faster.
My advice would be to reverse engineer the page a bit more, and see if you can figure out how to query directly whatever the (presumably AJAX) calls are putting on the page. If you can treat those specific requests as an API and only query for exactly what you want, you will be able to get results faster than with browser automation through Selenium.

Selenium Chrome Driver Limitations Web Scraping at Scale

I'm planning to use Selenium Chrome Driver for my project which will be used to do web scraping to multiple public websites (something like kayak or skyscanner). So there will be a REST GET endpoint where my backend would launch headless Chrome to scrape multiple websites, and eventually return a manipulated JSON.
I want to know how scalable is Chrome Driver as it sounds like a headless Chrome instance needs to be launched whenever a request comes in.
Updated: Question using Google Chrome Headless

Please find the pros and cons of phantom js which I noticed during implementation .Hope this helps.
Cons:
1)It will fail to recognize the browser elements like id,xpath,csselector
when compared to chrome driver.
2)If you have login mechanism ,redirects won't work as you expect when compared to chrome driver.
3)You need to manually implement the custom logic for screen shots for the test failures if you need it.
4)If you want to switch between multiple drivers like chrome,html etc then it is very difficult
Pros:
1)Test case execution is faster when compared to chrome driver
2)No browser is required it will run without GUI.
3)No much configurations are needed when compared to chromedriver.
You can go with html driver also which is quite faster then phantom but even it has its own limitations that you need take care of before implementation.

I am not sure that you really need to use PhantomJS.
Chrome implemented "headless" mode couple of months ago.
"Headless Chrome" does the same job that PhantomJS, and does it better.
I heard that PhantomJS authors even said that they will not support it anymore.
You can enable headless mode in Selenide with just on line:
Configuration.headless = true;

Did you think about headless chrome?
Headless Chrome

Cannot get Java Applets to work in any browser

We are trying to download something from GE that uses Java to download when logging into the site. This is a windows 7 Professional PC. I have other computers that are able to do this successfully. Here is my order of operations:
Log into the site and select the file I want to download
Click download
It takes me to the page that says it will start in a few seconds but nothing happens. It is supposed to have a box that asks for Java to run.
I have reinstalled Java fresh and still nothing. Tried with multiple user accounts. Added the site to the list of exceptions in the firewalls and Java configuration. I have tried an earlier version of Java. This happens in Mozilla, Chrome and IE. I have made sure that the Java plugin shows up and is enabled. I just cant think of what I am missing. And since we are a contractor GE is not going to help us. Can anyone here think of anything?

Are you sure you have the Java plugin enabled? e.g. in Firefox, go to Settings, Plugins, and change Java Platform SE8 'Next Generation Java Plugin' to 'always activate'?

This sounds similar to the issues I had with Cisco WebVPN, Java-style. Once you have Java installed correctly and set as a plugin for any of the browsers you would like to use, see below.
Before you even start looking at browsers - if you think it is already set up correctly
Look at your Anti-Virus programs or anything else that could prevent it from working. McAfee Host Intrusion Protection is known to cause many Java programs to fail. Kapersky had issues, a while back, with Java on Windows (Java Applets not loading in Windows 8 ).
You must have a 64-bit browser to use 64-bit Java (also mentioned in the Chrome link below).
See below for any specific things that can be modified in the browser.
Chrome 43 is the more complicated browser to set up. They have a dedicated page with instructions.
How do I use Java with the Google Chrome browser?
Firefox 38 will prompt you.
In Internet Explorer 11, it's under Internet Options->Security. I recommend adding the hostname the applet is on as a Trusted Site (Select Trusted Sites and click the Sites button, then add the first part of the url). Click the Custom level button and make sure that Scripting of Java applets is not disabled.
If you still have problems with the applet:
Verify your Java version will work with the applet you are accessing
Verify the plugin is enabled for the browser through the Java Control Panel, which is available in Windows Control Panel, or on Mac/Linux, execute it from the JDK directory ($JAVA_HOME/ControlPanel ).

I ended up fixing the issue. I had to allow their UK site on the list for Java and enable the SSL 2.0 for HTTP in Java config as well

Oracle plans for java plugin on chrome [duplicate]

I use Java applets in my application, today I surprisingly saw the redirect from my page and the next message from java
In details I've read about Chrome dropping 'NPAPI' support
Recently, Google has revised their plans and now state that they plan
to completely remove NPAPI by late 2015. As it is unclear if these
dates will be further extended or not, we strongly recommend Java
users consider alternatives to Chrome as soon as possible. Instead, we
recommend Firefox, Internet Explorer and Safari as longer-term
options. As of April 2015, starting with Chrome Version 42, Google has
added an additional step to configuring NPAPI based plugins like Java
to run — see the section Enabling NPAPI in Chrome Version 42 and later
below.
So will it be handled by Java somehow or it is the death of my applet in the next year ?

As of Chrome Version 42, an additional configuration step is required to continue using NPAPI plugins.
In your URL bar, enter:
chrome://flags/#enable-npapi
Click the Enable link for the Enable NPAPI configuration option.
Click the Relaunch button that now appears at the bottom of the configuration page.

So will it be handled by Java somehow ..
If they found a way, that would be a bug in Chrome (that would quickly be fixed).
..or it is the death of my applet in the next year ?
Yep. Not that they were really that 'healthy' for a long time now.

Java applets are not dead. There are a lot of intranet web applications that use them, for instance, for digital signature with applet-javascript-web page interaction. Other option is migration to Firefox.

I saw a demonstration that A Plugin written in Pepper API (PPAPI) can launch an executable and communicate with it. Oracle says at here " This change does not affect Web Start applications, it only impacts applets." But the problem is that browser downloads jnlp file but doesn't launch it. User has to click that jnlp file (of course .jnlp file association should be OK).
I think Oracle write a plugin to launch Web Start application and communicate with it for sending cookie etc. If It is written Applets can convert to Web Start applications with less effort.