I'm trying to add cookies to a link before I open it with webdriver but it keeps giving me this error:
org.openqa.selenium.UnableToSetCookieException: Unable to set cookie (WARNING: The server did not provide any stacktrace information)
Please find my code below:
System.setProperty("webdriver.edge.driver","C:\\Program Files\\Latest Webdriver\\MicrosoftWebDrive.exe" );
EdgeDriver = new EdgeDriver();
Thread.sleep(2000);
Cookie cookie = new Cookie("Testing", "11111");
EdgeDriver.manage().addCookie(cookie);
EdgeDriver.get("https://www.google.ca/?gws_rd=ssl"); // The link is an example
Please help with a relevant solution.
You are creating the cookie before navigating to the site. If you are trying to create a cookie on the domain www.example.com, then you would want to navigate to some page on that domain, create the cookie, and then start your test.
From my reading a while back, the best way to do this is to navigate to some page you know will not exist on the domain, e.g. www.example.com/this404page, then create the cookie. It should load a lot faster since it's an error page and shouldn't contain much content. After creating the cookie on the 404 page, start your test.
You are unable to do this because the WebDriver spec requires that you have the browser landed at the domain you are trying to set cookies for.
The work-around as mentioned is to go to the page prior to setting the cookie. But that causes some issues:
This unfortunately prevents 2 key use cases:
You want to re-use cookies from another webdriver session in a new session to avoid repeating the work it took to get the cookies. Such as having to repeat a login.
Allow a webdriver pool to be shared amongst unrelated threads where each thread might have their own cookies.
The webdriver spec clearly needs to take this into account. I have opened an issue here:
https://github.com/w3c/webdriver/issues/1238
Give it some votes. All browsers should start carrying a way to handle this.
First navigate to URL and then try to add cookies, try below code:
System.setProperty("webdriver.edge.driver","C:\\Program Files\\Latest Webdriver\\MicrosoftWebDrive.exe" );
EdgeDriver = new EdgeDriver();
Thread.sleep(2000);
Cookie cookie = new Cookie("Testing", "11111");
EdgeDriver.manage().addCookie(cookie);
EdgeDriver.get("https://www.google.ca/?gws_rd=ssl"); // The link is an example
Replace your code with this :
System.setProperty("webdriver.edge.driver","C:\\Program Files\\Latest Webdriver\\MicrosoftWebDrive.exe" );
EdgeDriver = new EdgeDriver();
Thread.sleep(2000);
EdgeDriver.get("https://www.google.ca/?gws_rd=ssl"); // The link is an example
Cookie cookie = new Cookie("Testing", "11111");
EdgeDriver.manage().addCookie(cookie);
Adding cookies before navigating to the domain are called domain-less cookies which i believe is not possible.
I have not found a way to drop a cookie before the url, but I think below scenario might help you out-
Create a factory class to create webdriver instances
Before returning the webdriver instance, navigating to any page on the domain under test, and drop the cookie, then navigate browser back.
Your test can now begin, unaware that the navigation and cookie dropping has taken place
Just in case this is simpler, I did not need to create a factory class, merely load the root page from the site (which did not demand a cookie)
First collect the cookie as described above
pickle.dump(driver.get_cookies(), open("ChromeCookies.pkl", "wb"))
Then get the web site root which does not require a cookie (in my case)
driver.get(url_root)
Then execute the cookie load
for cookie in pickle.load(open("ChromeCookies.pkl", "rb")):
driver.add_cookie(cookie)
Then go to the page that I actually wanted to access
driver.get(url_not_root_demanding_cookie_which_is_now_there)
Please do post if there is an issue with this approach
Related
I have a java web application. I implemented a login system, with user rights a while ago, and included a "remember-me" functionality with unique string ID's saved client side in cookies.
This has worked ok, except for the fact that the remember me functionality always fails on first page load whenever a new session starts. However, since most users access a non-restricted page first, complaints has been absent. Nonetheless, I'd like to fix it. Here is what I have learned.
I use implementations of javax.servlet.Filter to check if a user has rights to se a page. for example baseURL/pages/admin/*. Filter interface has a method called doFilter, which accepts a ServletRequest, and ServletResponse object as parameters. I cast these to HttpServletRequest and HttpServletResponse. The HttpServletRequest gives me access to cookies and session.
If i iterate through the cookies, I find my "remember"-cookie, with the unique ID as a value. However, this ID is wrong.
Now, in my Servlet class, which follows the front controller pattern, I also have a check for user logged in, and remember me. But because this is executed after the filter, it is not sufficient to check only here. Still, I do want to check for every page, even if it is not restricted, as it changes the layout slightly if you are logged in.
The java HttpServlet service method accepts a HttpServletRequest and HttpServletResponse object. In other words, no need for casting here. Funny thing is though, If i try to access my cookies from here, it will give me an identical id for the session cookie, but a completely different uid for my "remember"-cookie.
I have found that my system adds new remember cookies for each of my filters. And if I try to access a page in the admin path, both cookies from /webapp/pages, and cookie from /webapp/pages/admin will be present in chrome inspector. When accessing the cookies in the filter, the /webapp/pages/admin is the only one that will exist. Oppositely, the /webapp/pages is the only one that will exist in the front controller servlet service method.
I guess this is because of the mapping of said filter and servlet, which matches the path of the cookie. The problem is that I never intended there to be cookies stored hierarchically, and only want the one to be stored, at /webapp/pages. My system has now stored plenty of these deeper pathed cookies all over my client network, and whenever a user logs in and out, they might get out of sync with a new uid.
Is there a way I can force retrieving the /webapp/pages cookie over the /webapp/pages/admin cookie? Or is there a way to retrieve both? I could just check both uid's for a match if I can manage that (hence the title of my question)
For the future, I have made sure to set the path of cookie storage, so that the same path will be used, but as the cookies has a year to expire, this will not solve my problems for a long time, unless I find a way to check the correct cookie.
The answer to the title question is; you can't.
The browser will decide which cookies it deems most relevant, and there is nothing you can do to change that. When your filter is mapped to a subpath, and servlet is mapped to a higher path, you will get the best matching cookie for each path.
The specific problem in the question text is caused by a bad coding pattern. The remember me cookies should be stored at a specific path when created, in this case /webapp/pages. This will prevent the cookie from being created as multiples, in hierarchical paths.
There is still the problem of already existing cookies client side. These can be handled by adding the following javascript in a central area of your code, somewhere where you'd know that all users will encounter it:
document.cookie = 'remember=; path=/webapp/pages/user; expires=' + new Date(0).toUTCString();
document.cookie = 'remember=; path=/webapp/pages/admin; expires=' + new Date(0).toUTCString();
This will set the unwanted cookies to expire at an already past date, effectively deleting it.
Now only one cookie with name "remember" will exist for the domain, and both servlet and filters will fetch the same cookie, regardless of their mapped subpaths.
Need some help with fetching some data from a website.
Previously , we had following code in our application and it used to fetch the required data. We just used to read the required fields by forming a URL by passing username , password and search parameter (DEA number). The same URL (with parameters ) could also be hit from browser directly to see the results. It was a simple GET request:
{URL url = new URL(
"http://www.deanumber.com/Websvc/deaWebsvc.asmx/GetQuery?UserName="+getUsername()+"&Password="+getPassword()+"&DEA="
+ deaNumber
+ "&BAC=&BASC=&ExpirationDate=&Company=&Zip=&State=&PI=&MaxRows=");
Document document = parser.parse(url.toExternalForm());
// Ask the document for a list of all <sect1> tags it contains
NodeList sections = document.getElementsByTagName("DEA");
//Followed by a loop code to get each element by using sections.item(index).getFirstChild() etc.
}
Now, the website URL has got changed to following:
https://www.deanumber.com/RelId/33637/ISvars/default/Home.htm
I am able to login to the URL with credentials , go to the search page , enter the DEA number and search. The login page comes as a pop-up once I click 'Login' link on home page. Also, the final result comes as a pop-up. This is a POST request so I am unable to form the complete URL which I could use in my code.
I am not an expert in Web Services , but I think I need a web service URL like the one mentioned in the code above. Not sure how to get that !! Even if I get the URL , I am not sure how to perform the login through Java code and search the DEA number.
Also, it would be great if I could validate the URL manually before using in Java. Let me know if there is any way.
Or, in case there is any alternate approach in Java; kindly suggest.
Thanks in advance.
First of all, the previous approach provided by the website was completely wrong and insecure, because it passes the username and password as querystring parameters in plain text. I think, they would have realized this thing and changed their way of authentication.
Also, it looks like that they have restricted the direct URL based requests from the client applications like yours. For such requests from clients, they have published the web services. Check this link. They also have mentioned the rates for web service request counts.
So, you may need to open a formal communication channel to get authentication and other details to access their web services for this purpose. Depends on what they use for web service client authentication, you may code your client to access the web services.
I hope this helps.
Scenario:
I am setting Cookies in page https://www.example.com/#step1 by using method:
Cookies.setCookies("firstFileName","NewDocument");
Cookies.setCookies("firstFileExt","doc");
Users are supposed to click next and redirect themselves to https://www.example.com/#step2 . But in Case, user click to some other menu (like Home, About us, Contact us), I am deleting these Cookies by using the following method:
Cookies.removeCookie("firstFileName");
Cookies.removeCookie("firstFileExt");
But on removing I found that these two Cookies still holding the values in browser, When I do the following in https://www.example.com/#step1 before setting these two Cookies:
if(!Cookies.getCookie("firstFileName").toString().equals("undefined")){
Window.alert("firstFileName "+firstFileName);
}
I get an alert box that firstFileName NewDocument
Please let me know how can I set and remove the Cookies in my Case.
If you set a cookie without providing a path, as you do using Cookies.setCookies("firstFileName","NewDocument");, the cookie can only be removed on the same page.
A solution would be to always set the cookie for a specific path like that:
// TODO: set expiration time to something more useful
Date expires = new Date();
Cookies.setCookie("firstFileName","NewDocument", expires, null, "/", false);
Then you can remove it by calling that later on:
Cookies.removeCookie("firstFileName", "/");
As your application is based on the relative path "/", every page can set or remove the cookie and all pages access the same cookie value.
If you just set the cookie without providing a path, two pages of your application could even set different cookies.
For example in gmail login, when we consider a login test, when doing it manually for the first time we'll get the login page, from next time onwards we'll be directly getting into the inbox page.
If you try to do the same thing in webdriver(Run login test twice), in all these attempts we'll get the login page as we didn't login from this machine earlier. What is happening in behind the scenes in maintaining the session with respect to cookies or session ?
Here is the description & code snippet from selenium docs to add or remove cookies:
Before we leave these next steps, you may be interested in
understanding how to use cookies. First of all, you need to be on the
domain that the cookie will be valid for. If you are trying to preset
cookies before you start interacting with a site and your homepage is
large / takes a while to load an alternative is to find a smaller page
on the site, typically the 404 page is small
(http://example.com/some404page)
// Go to the correct domain
driver.get("http://www.example.com");
// Now set the cookie. This one's valid for the entire domain
Cookie cookie = new Cookie("key", "value");
driver.manage().addCookie(cookie);
// And now output all the available cookies for the current URL
Set<Cookie> allCookies = driver.manage().getCookies();
for (Cookie loadedCookie : allCookies) {
System.out.println(String.format("%s -> %s", loadedCookie.getName(), loadedCookie.getValue()));
}
// You can delete cookies in 3 ways
// By name
driver.manage().deleteCookieNamed("CookieName");
// By Cookie
driver.manage().deleteCookie(loadedCookie);
// Or all of them
driver.manage().deleteAllCookies();
I have a very basic question how the creation of HTTPSession works.I know you folks will fire me on looking at this question as similar kind
of questions exist.But there is reasoning why i am asking this question Here it is :-
I know httpsession is unique to web browser and server creates it when we do HttpServletRequest.getSession first time.It will maintaintain the same session till we
close the browser. But i have little bit different scenario.I Have a web application on one tomcat instance say T1.On welcome page of this web application
i have provided two links on click of which takes me to same java servlet(S1) of different web application hosted on another tomcat instance T2 (these two links
opens two seperate pop up windows). Now first i click the link1 and inspect the sessionId in S1 and find its value as 1678. Now first i click the link2 and
inspect the sessionId in S1 and find its value again as 1678. My question here is why i am getting the same session id for both the requests origintaing
from link1 and link2? what can i do to to get the different session for both of these requests?
What i tried after looking for possible solutions on net :- On click of link1, in Servlet S1 , i copied session attributes, invalidate it and create new one.
Say new session id is 8765 . Now i click the link2 and found the same session in this request too. So i further invalidate it and creates new one(say
new session id is 4897). Ideally it should expire the first browser session (generated on click of link1). To verify it,i click anywhere on pop up 1 it does not get
expired but i see again last generated session id i.e 4897. I am not getting why it attaching the same session id with both pop up windows?
Folks Thanks for your patience for taking your time out and read this long scenario?
Edit :-
Cookie[] cookies = req.getCookies();
if(cookies!=null)
for (int i = 0; i < cookies.length; i++) {
cookies[i].setMaxAge(0);
context.getResponse().getHttpServletResponse().addCookie(cookies[i]);
}
HttpSession myAppSession = req.getSession();//line 1
Assume on click of link1 i get session id as 1234,then after click of link 2 also i get the same session id. As per my understanding, after executing the code above line 1 , i should get the different session id as i am setting the MaxAge as0 before getting the session. But its not happening?
I think this is what you are looking for :
By default session tracking happens by cookies. WebServer sends the session id to the browser in the form of cookie. And, the browser send the cookie having session id for the subsequent requests.
How does the browser identifies which cookies to send for a link/request?
It is based on the these parameters. If the request matches these paramters the browser sends that particular cookie:
Domain: The domain name to which the request is made. Verify in your case if the domain name is same for two instances
Path: If the path name is same. Web Server send the context root as the path , requests under same context root share cookies.
Secure: Server sends if the given cookie is secure or not. Meaning, if the cookie can be sent on non-secure channel.
These parameters will let the browser to send the cookies to the server. And because the same cookie is sent for both the instances you are having. I think the session id is being shared.
If the request propeties such as Request URI, domain and path(i.e, context root) are same between requests, there is no way to tell the browser to use different cookies.
You have some options below:
Use different domain names.
Use different context roots.
Have a LB in front of two nodes and redirect to the correct node based on Session id