I already created a method to get the transaction ID but the element cannot be located. I cannot retrieve the ID in the web table because it shows an error that unable to locate the element. Besides, it will open another blank window.
This is the method to get the ID
public String getTransactionID(String ID) throws IOException {
File src = new File("C:\\selenium\\ExcelData\\TestData.xlsx");
FileInputStream fis = new FileInputStream(src);
wb = new XSSFWorkbook(fis);
XSSFSheet sh1 = wb.getSheetAt(0);
String getValue = null;
String TransactionID = null;
for (int i = 0; i<=1000; i++) {
if(sh1.getRow(i).getCell(0) != null)
{
getValue = sh1.getRow(i).getCell(0).getStringCellValue();
}
if(getValue != null) {
if(getValue.contains(ID)) {
System.setProperty("webdriver.gecko.driver","C:\\selenium\\geckodriver-v0.23.0-win64\\geckodriver.exe");
WebDriver driver = new FirefoxDriver();
TransactionID = driver.findElement(By.xpath("//table/tbody/tr[2]/td/table/tbody/tr/td/table/tbody/tr[1]/td/table/tbody/tr[1]/td")).getText();
FileOutputStream fout = new FileOutputStream(src);
sh1.getRow(i).createCell(1).setCellValue(TransactionID);
wb.write(fout);
fout.close();
break;
}
}
}
return TransactionID;
This is the line to call the method
GetExcel transID = new GetExcel();
transID.getTransactionID("Transaction ID");
Issue
The issue lies in the below snippet:
WebDriver driver = new FirefoxDriver();
The will open up a new Firefox browser. That's why you saw a 'blank window' open up.
TransactionID = driver.findElement(By.xpath("//table/tbody/tr[2]/td/table/tbody/tr/td/table/tbody/tr[1]/td/table/tbody/tr[1]/td")).getText();
Immediately after that you're attempting to search for the element on the website. As you can imagine, attempting to find an element on a blank page will obviously throw you an error.
You need to navigate to the website first:
driver.get(your_url_here);
Sidenotes
Your xpath isn't ideal as it is too hardcoded -- if a new element is created in the website, your xpath will probably break. You should instead use a more dynamic approach that utilizes the element's DOM properties such as id:
//table[#id='xxx']//td
Documentation: https://www.w3schools.com/xml/xml_xpath.asp
In addition, your variable name TransactionID should be in lower caps, i.e. transactionID.
Related
I am trying to merge 2 docx files which has their own bullet number, after merging of word docs the bullets are automatically updated.
E.g:
Doc A has 1 2 3
Doc B has 1 2 3
After merging the bullet numbering are updated to be 1 2 3 4 5 6
how to stop this.
I am using following code
if(counter==1)
{
FirstFileByteStream = org.apache.commons.codec.binary.Base64.decodeBase64(strFileData.getBytes());
FirstFileIS = new java.io.ByteArrayInputStream(FirstFileByteStream);
FirstWordFile = org.docx4j.openpackaging.packages.WordprocessingMLPackage.load(FirstFileIS);
main = FirstWordFile.getMainDocumentPart();
//Add page break for Table of Content
main.addObject(objBr);
if (htmlCode != null) {
main.addAltChunk(org.docx4j.openpackaging.parts.WordprocessingML.AltChunkType.Html,htmlCode.toString().getBytes());
}
//Table of contents - End
}
else
{
FileByteStream = org.apache.commons.codec.binary.Base64.decodeBase64(strFileData.getBytes());
FileIS = new java.io.ByteArrayInputStream(FileByteStream);
byte[] bytes = IOUtils.toByteArray(FileIS);
AlternativeFormatInputPart afiPart = new AlternativeFormatInputPart(new PartName("/part" + (chunkCount++) + ".docx"));
afiPart.setContentType(new ContentType(CONTENT_TYPE));
afiPart.setBinaryData(bytes);
Relationship altChunkRel = main.addTargetPart(afiPart);
CTAltChunk chunk = Context.getWmlObjectFactory().createCTAltChunk();
chunk.setId(altChunkRel.getId());
main.addObject(objBr);
htmlCode = new StringBuilder();
htmlCode.append("<html>");
htmlCode.append("<h2><br/><br/><br/><br/><br/><br/><br/><br/><br/><br/><br/><p style=\"font-family:'Arial Black'; color: #f35b1c\">"+ReqName+"</p></h2>");
htmlCode.append("</html>");
if (htmlCode != null) {
main.addAltChunk(org.docx4j.openpackaging.parts.WordprocessingML.AltChunkType.Html,htmlCode.toString().getBytes());
}
//Add Page Break before new content
main.addObject(objBr);
//Add new content
main.addObject(chunk);
}
Looking at your code, you are adding HTML altChunks to your document.
For these to display it Word, the HTML is converted to normal docx content.
An altChunk is usually converted by Word when you open the docx.
(Alternatively, docx4j-ImportXHTML can do it for an altChunk of type XHTML)
The upshot is that what happens with the bullets (when Word converts your HTML) is largely outside your control. You could experiment with CSS but I think Word will mostly ignore it.
An alternative may be to use XHTML altChunks, and have docx4j-ImportXHTML convert them. main.convertAltChunks()
If the same problem occurs when you try that, well, at least we can address it.
I was able to fix my issue using following code. I found it at (http://webapp.docx4java.org/OnlineDemo/forms/upload_MergeDocx.xhtml). You can also generate your custom code, they have a nice demo where they generate code according to your requirement :).
public final static String DIR_IN = System.getProperty("user.dir")+ "/";
public final static String DIR_OUT = System.getProperty("user.dir")+ "/";
public static void main(String[] args) throws Exception
{
String[] files = {"part1docx_20200717t173750539gmt.docx", "part1docx_20200717t173750539gmt (1).docx", "part1docx_20200717t173750539gmt.docx"};
List blockRanges = new ArrayList();
for (int i=0 ; i< files.length; i++) {
BlockRange block = new BlockRange(WordprocessingMLPackage.load(new File(DIR_IN + files[i])));
blockRanges.add( block );
block.setStyleHandler(StyleHandler.RENAME_RETAIN);
block.setNumberingHandler(NumberingHandler.ADD_NEW_LIST);
block.setRestartPageNumbering(false);
block.setHeaderBehaviour(HfBehaviour.DEFAULT);
block.setFooterBehaviour(HfBehaviour.DEFAULT);
block.setSectionBreakBefore(SectionBreakBefore.NEXT_PAGE);
}
// Perform the actual merge
DocumentBuilder documentBuilder = new DocumentBuilder();
WordprocessingMLPackage output = documentBuilder.buildOpenDocument(blockRanges);
// Save the result
SaveToZipFile saver = new SaveToZipFile(output);
saver.save(DIR_OUT+"OUT_MergeWholeDocumentsUsingBlockRange.docx");
}
I am trying to learn java and selenium by myself and creating a robot that will scan job/career pages for certain string (job name e.g. QA , developer...)
I'm trying to create JAVA code using selenium, that will read URL links from CSV file and open a new tab.
the main goal is to add several url in the CSV and assert/locate a certain string in the designated url's for example: is there "Careers" link in each URL, the test will pass for this specific url.
created a selenium project
created new chromeDriver
Created CSV built from 3 columns (ID, company's name, URL) - and added it to the project
import org.openqa.selenium.chrome.ChromeDriver;
import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;
public class URLSearch {
public static void main(String[] args) {
ChromeDriver driver = new ChromeDriver();
driver.manage().window().maximize();
String fileName = "JobURLList.csv";
File file = new File(fileName); //read from file
try {
Scanner inputStream = new Scanner(file);
while (inputStream.hasNext()) {
String data = inputStream.next();
System.out.println(data);
}
inputStream.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
}
first line in the CSV - titles: id, name, url
Read the url from the second line - e.g. https://careers.google.com/jobs/"
open browsertab and start going over the url list (from the CSV)
locate a hardcoded string (e.g. "developer" , "qa" ..) in each url
if such a string was found, write in console the url that the test turned out to be positive (such a string was found in one of the url's).
if no such string was found, skip to the next url.
To open the new tab do something like this (this assumes "driver" object is your WebDriver):
((JavascriptExecutor)driver).executeScript("window.open('about:blank', '_blank');");
Set<String> tab_handles = driver.getWindowHandles();
int number_of_tabs = tab_handles.size();
int new_tab_index = number_of_tabs-1;
driver.switchTo().window(tab_handles.toArray()[new_tab_index].toString());
You could then create a function that takes a list of key/value pairs, with URL and term to search for and loop through it. Do you want to use a hashmap for this, or maybe an ArrayList of a class (id/name/url)? The code for finding the text would be something like this (assumes you've defined a var of "Pass" to boolean):
driver.get([var for URL]);
//driver will wait for pageready state, so you may
// not need the webdriver wait used below. Depends
// on if the page populates data after pagereadystate
String xpather = "//*[contains(text(), '" + [string var for text to search for] + "')]";
try
{
wait = new WebDriverWait(driver, 10);
List<WebElement> element = wait.until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.xpath(xpather)));
this.Pass = false;
if (element.size() > 0)
{
this.Pass = true;
}
}
catch (Exception ex)
{
this.Pass = false;
System.out.println ("Exception finding text: " + ex.toString());
}
Then logic for if (this.Pass==true or false)..
i need to access a website multiple times a day and want to skip the Log-In page. This is why i want to use Cookies in Java Selenium Chromedriver, for skipping that Log-In after accessing it the first time on a day.
Selenium is saving the Cookies correctly, but does not use them and i dont get access on the following page. Can you help me?
This is my Code:
public static void main(String[] args) throws InterruptedException {
Set<Cookie> cookie = null;
Iterator<Cookie> itr = null;
while (true) {
System.setProperty("webdriver.chrome.driver", "C:\\Users\\Maxi\\Desktop\\ChromeDriver.exe");
driver = new ChromeDriver();
driver.get("https://www.xxxxxx.xxx");
while (itr != null && itr.hasNext()) {
driver.manage().addCookie(itr.next());
}
driver.navigate().refresh();
WebDriverWait wait0 = new WebDriverWait(driver, 20);
if (itr == null) {
String UserID = "LoginFieldXpath";
wait0.until(ExpectedConditions.visibilityOfElementLocated(By.xpath(UserID)));
driver.findElement(By.xpath(UserID)).sendKeys("Username");
String PW = "PasswordField Xpath";
driver.findElement(By.xpath(PW)).sendKeys("Password");
String LogIn = "LoginButtonXpath";
driver.findElement(By.xpath(LogIn)).click();
cookie = driver.manage().getCookies();
itr = cookie.iterator();
}
}
}
You can create user profile in Chrome like this
options:
addArguments("user-data-dir="+"path_to_empty_folder");
Then make sign up, so cookie will be stored in this profile. And than just copy this user data dir to another folder
FileUtils.copyDirectory(new File("path-to-dir-with-cookie"), new File("new-dir"));
options.addArguments("user-data-dir="+"new-dir");
Selenium starts a new temporary browser instance each time, so it doesn't come with any cookies stored, cache or anything like that.
I would suggest moving your open chrome driver to be outside of the while loop so you'll be able to reuse it each time you want to check a page. Maybe either move the login outside of the loop as well or check if you need to log in. Then just grab whatever page you're trying to check inside the loop each time you need to.
public static void main(String[] args) throws InterruptedException {
Set<Cookie> cookie = null;
Iterator<Cookie> itr = null;
// move this outside the loop
System.setProperty("webdriver.chrome.driver", "C:\\Users\\Maxi\\Desktop\\ChromeDriver.exe");
driver = new ChromeDriver();
while (true) {
driver.get("https://www.xxxxxx.xxx");
// not sure why you're adding a cookie?
// it should automatically accept page cookies that are set
while (itr != null && itr.hasNext()) {
driver.manage().addCookie(itr.next());
}
driver.navigate().refresh();
WebDriverWait wait0 = new WebDriverWait(driver, 20);
// check to make sure that you need to log in
if (itr == null) {
String UserID = "LoginFieldXpath";
wait0.until(ExpectedConditions.visibilityOfElementLocated(By.xpath(UserID)));
driver.findElement(By.xpath(UserID)).sendKeys("Username");
String PW = "PasswordField Xpath";
driver.findElement(By.xpath(PW)).sendKeys("Password");
String LogIn = "LoginButtonXpath";
driver.findElement(By.xpath(LogIn)).click();
cookie = driver.manage().getCookies();
itr = cookie.iterator();
}
}
}
I'm trying to build a jsoup based java app to automatically download English subtitles for films (I'm lazy, I know. It was inspired from a similar python based app). It's supposed to ask you the name of the film and then download an English subtitle for it from subscene.
I can make it reach the download link but I get an Unhandled content type error when I try to 'go' to that link. Here's my code
public static void main(String[] args) {
try {
String videoName = JOptionPane.showInputDialog("Title: ");
subscene(videoName);
}
catch (Exception e) {
System.out.println(e.getMessage());
}
}
public static void subscene(String videoName){
try {
String siteName = "http://www.subscene.com";
String[] splits = videoName.split("\\s+");
String codeName = "";
String text = "";
if(splits.length>1){
for(int i=0;i<splits.length;i++){
codeName = codeName+splits[i]+"-";
}
videoName = codeName.substring(0, videoName.length());
}
System.out.println("videoName is "+videoName);
// String url = "http://www.subscene.com/subtitles/"+videoName+"/english";
String url = "http://www.subscene.com/subtitles/title?q="+videoName+"&l=";
System.out.println("url is "+url);
Document doc = Jsoup.connect(url).get();
Element exact = doc.select("h2.exact").first();
Element yuel = exact.nextElementSibling();
Elements lis = yuel.children();
System.out.println(lis.first().children().text());
String hRef = lis.select("div.title > a").attr("href");
hRef = siteName+hRef+"/english";
System.out.println("hRef is "+hRef);
doc = Jsoup.connect(hRef).get();
Element nonHI = doc.select("td.a40").first();
Element papa = nonHI.parent();
Element link = papa.select("a").first();
text = link.text();
System.out.println("Subtitle is "+text);
hRef = link.attr("href");
hRef = siteName+hRef;
Document subDownloadPage = Jsoup.connect(hRef).get();
hRef = siteName+subDownloadPage.select("a#downloadButton").attr("href");
Jsoup.connect(hRef).get(); //<-- Here's where the problem lies
}
catch (java.io.IOException e) {
System.out.println(e.getMessage());
}
}
Can someone please help me so I don't have to manually download subs?
I just found out that using
java.awt.Desktop.getDesktop().browse(java.net.URI.create(hRef));
instead of
Jsoup.connect(hRef).get();
downloads the file after prompting me to save it. But I don't want to be prompted because this way I won't be able to read the name of the downloaded zip file (I want to unzip it after saving using java).
Assuming that your files are small, you can do it like this. Note that you can tell Jsoup to ignore the content type.
// get the file content
Connection connection = Jsoup.connect(path);
connection.timeout(5000);
Connection.Response resultImageResponse = connection.ignoreContentType(true).execute();
// save to file
FileOutputStream out = new FileOutputStream(localFile);
out.write(resultImageResponse.bodyAsBytes());
out.close();
I would recommend to verify the content before saving.
Because some servers will just return a HTML page when the file cannot be found, i.e. a broken hyperlink.
...
String body = resultImageResponse.body();
if (body == null || body.toLowerCase().contains("<body>"))
{
throw new IllegalStateException("invalid file content");
}
...
Here:
Document subDownloadPage = Jsoup.connect(hRef).get();
hRef = siteName+subDownloadPage.select("a#downloadButton").attr("href");
//specifically here
Jsoup.connect(hRef).get();
Looks like jsoup expects that the result of Jsoup.connect(hRef) should be an HTML or some text that it's able to parse, that's why the message states:
Unhandled content type. Must be text/*, application/xml, or application/xhtml+xml
I followed the execution of your code manually and the last URL you're trying to access returns a content type of application/x-zip-compressed, thus the cause of the exception.
In order to download this file, you should use a different approach. You could use the old but still useful URLConnection, URL or use a third party library like Apache HttpComponents to fire a GET request and retrieve the result as an InputStream, wrap it into a proper writer and write your file into your disk.
Here's an example about doing this using URL:
URL url = new URL(hRef);
InputStream in = url.openStream();
OutputStream out = new BufferedOutputStream(new FileOutputStream("D:\\foo.zip"));
final int BUFFER_SIZE = 1024 * 4;
byte[] buffer = new byte[BUFFER_SIZE];
BufferedInputStream bis = new BufferedInputStream(in);
int length;
while ( (length = bis.read(buffer)) > 0 ) {
out.write(buffer, 0, length);
}
out.close();
in.close();
We use Lucene to index some internal documents. Sometimes we need to remove documents. These documents have an unique id and are represented by a class DocItem as follows (ALL THE CODE IS A SIMPLIFIED VERSION WITH ONLY SIGNIFICANT (I hope) PARTS):
public final class DocItem {
public static final String fID = "id";
public static final String fTITLE = "title";
private Document doc = new Document();
private Field id = new Field(fID, "", Field.Store.YES, Field.Index.ANALYZED);
private Field title = new Field(fTITLE, "", Field.Store.YES, Field.Index.ANALYZED);
public DocItem() {
doc.add(id);
doc.add(title);
}
... getters & setters
public getDoc() {
return doc;
}
}
So, to index a document, a new DocItem is created and passed to an indexer class as follows:
public static void index(DocItem docitem) {
File file = new File("indexdir");
Directory dir= new SimpleFSDirectory(file);
IndexWriter idxWriter = new IndexWriter(dir, new StandardAnalyzer(Version.LUCENE_30), IndexWriter.MaxFieldLength.UNLIMITED);
idxWriter.addDocument(docitem.getDoc());
idxWriter.close();
}
We created an auxiliary method to iterate over the index directory:
public static void listAll() {
File file = new File("indexdir");
Directory dir = new SimpleFSDirectory(file);
IndexReader reader = IndexReader.open(dir);
for (int i = 0; i < reader.maxDoc(); i++) {
Document doc = reader.document(i);
System.out.println(doc.get(DocItem.fID));
}
}
Running the listAll, we can see that our docs are being indexed properly. At least, we can see the id and other attributes.
We retrieve the document using IndexSearcher as follows:
public static DocItem search(String id) {
File file = new File("indexdir");
Directory dir = new SimpleFSDirectory(file);
IndexSearcher searcher = new IndexSearcher(index, true);
Query q = new QueryParser(Version.LUCENE_30, DocItem.fID, new StandardAnalyzer(Version.LUCENE_30)).parse(id);
TopDocs td = searcher.search(q, 1);
ScoreDoc[] hits = td.scoreDocs;
searcher.close();
return hits[0];
}
So after retrieving it, we are trying to delete it with:
public static void Delete(DocItem docitem) {
File file = new File("indexdir");
Directory dir= new SimpleFSDirectory(file);
IndexWriter idxWriter = new IndexWriter(dir, new StandardAnalyzer(Version.LUCENE_30), IndexWriter.MaxFieldLength.UNLIMITED);
idxWriter.deleteDocuments(new Term(DocItem.fID, docitem.getId()));
idxWriter.commit();
idxWriter.close();
}
The problem is that it doesn't work. The document is never deleted. If I run the listAll() after the deletion, the document is still there. We tried to use IndexReader, with no lucky.
By this post and this post, We think that we are using it accordinlgy.
What we are doing wrong? Any advice? We are using lucene 3.0.3 and java 1.6.0_24.
TIA,
Bob
I would suggest, use IndexReader DeleteDocumets, it returns the number of documents deleted. this will help you narrow whether the deletions occur on first count.
the advantage of this over the indexwriter method, is that it returns the total document deleted, if none if shall return 0.
Also see the How do I delete documents from the index? and this post
Edit: Also i noticed you open the indexreader in readonly mode, can you change the listFiles() index reader open with false as second param, this will allow read write, perhaps the source of error
I call IndexWriterConfig#setMaxBufferedDeleteTerms(1) during IndexWriter instantiation/configuration and all delete operations go to disc immediately. Maybe it's not correct design-wise, but solves the problem explained here.
public static void Delete(DocItem docitem) {
File file = new File("indexdir");
Directory dir= new SimpleFSDirectory(file);
IndexWriter idxWriter = new IndexWriter(dir, new StandardAnalyzer(Version.LUCENE_30), IndexWriter.MaxFieldLength.UNLIMITED);
idxWriter.deleteDocuments(new Term(DocItem.fID, docitem.getId()));
idxWriter.commit();
idxWriter.close(