Currently, this program will run down a column of URLs and output the selected data to the neighboring cell. I can set which column it starts on, but that is all I can do. Right now, I only have it working on one column. How can I instruct it to go to say, column 4 (Column E) and work top down once it is through with column 0 (A)? And then perhaps another, say column J after that?
I believe my problem lies within the "while (!(cell = sheet.getCell..." line, but I am unsure of what to change there without breaking the program.
My code is as follows:
public class App {
private static final int URL_COLUMN = 0; // Column A
private static final int PRICE_COLUMN = 1; //Column B
public static void main(final String[] args) throws Exception {
Workbook originalWorkbook = Workbook.getWorkbook(new File("C:/Users/Shadow/Desktop/original.xls"));
WritableWorkbook workbook = Workbook.createWorkbook(new File("C:/Users/Shadow/Desktop/updated.xls"), originalWorkbook);
originalWorkbook.close();
WritableSheet sheet = workbook.getSheet(0);
int currentRow = 1;
Cell cell;
while (!(cell = sheet.getCell(URL_COLUMN, currentRow)).getType().equals(CellType.EMPTY)) {
String url = cell.getContents();
System.out.println("Checking URL: " + url);
if (url.contains("scrapingsite1.com")) {
String Price = ScrapingSite1(url);
System.out.println("Scraping Site1's Price: " + Price);
Label cellWithPrice = new Label(PRICE_COLUMN, currentRow, Price);
sheet.addCell(cellWithPrice);
}
currentRow++;
}
workbook.write();
workbook.close();
}
private static String ScrapingSite1 (String url) throws IOException {
Document doc = null;
for (int i=1; i <= 6; i++) {
try {
doc = Jsoup.connect(url).userAgent("Mozilla/5.0").timeout(6000).validateTLSCertificates(false).get();
break;
} catch (IOException e) {
System.out.println("Jsoup issue occurred " + i + " time(s).");
}
}
if (doc == null){
return null;
}
else{
return doc.select("p.price").text();
}
}
}
To simplify the code I made an assumption that the price comes always to the next column (+1).
Also to process few columns instead of using single value int URL_COLUMN = 0 I replaced it with array of columns to process: int[] URL_COLUMNS = { 0, 4, 9 }; // Columns A, E, J.
You can then loop over every column {0, 4, 9} and save the data to the next column {1, 5, 10}.
private static final int[] URL_COLUMNS = { 0, 4, 9 }; // Columns A, E, J
public static void main(final String[] args) throws Exception {
Workbook originalWorkbook = Workbook.getWorkbook(new File("C:/Users/Shadow/Desktop/original.xls"));
WritableWorkbook workbook = Workbook.createWorkbook(new File("C:/Users/Shadow/Desktop/updated.xls"), originalWorkbook);
originalWorkbook.close();
WritableSheet sheet = workbook.getSheet(0);
Cell cell;
// loop over every column
for (int i = 0; i < URL_COLUMNS.length; i++) {
int currentRow = 1;
while (!(cell = sheet.getCell(URL_COLUMNS[i], currentRow)).getType().equals(CellType.EMPTY)) {
String url = cell.getContents();
System.out.println("Checking URL: " + url);
if (url.contains("scrapingsite1.com")) {
String Price = ScrapingSite1(url);
System.out.println("Scraping Site1's Price: " + Price);
// save price into the next column
Label cellWithPrice = new Label(URL_COLUMNS[i] + 1, currentRow, Price);
sheet.addCell(cellWithPrice);
}
currentRow++;
}
}
workbook.write();
workbook.close();
}
Related
In the actual pipeline (locally it works) I am getting an error:
[Utils] [ERROR] [Error] java.lang.NullPointerException
at org.apache.poi.xssf.usermodel.XSSFWorkbook.getSheet(XSSFWorkbook.java:1112)
at utilities.ExcelData
See this screenshot for the full error message.
Below is the code that I've used.
#DataProvider
public Object[][] GetDataFromExcelSheet() throws Exception {
System.out.println("Enter GetDataFromExcelSheet");
// Get all the data of excel method
// Main Objective: Create data array which will contain excel data
ExcelDataProvider excelDataProviderObj = new ExcelDataProvider();
// 1. Find excel sheet name
excelDataProviderObj.GetSheetName(PropertiesfileDataProvider.GetPropertyValue("nameofthesheet"));
// 2. Find total rows of excel
int rows = excelDataProviderObj.FindTotalRows();
System.out.println("number of rows found as: " +rows);
Object[][] data = new Object[rows][6];
for (int i = 0; i < rows; i++)
{
data[i][0] = excelDataProviderObj.GetCellData(i, 0);
data[i][1] = excelDataProviderObj.GetCellData(i, 1);
data[i][2] = excelDataProviderObj.GetCellData(i, 2);
data[i][3] = excelDataProviderObj.GetCellData(i, 3);
data[i][4] = excelDataProviderObj.GetCellData(i, 4);
data[i][5] = excelDataProviderObj.GetCellData(i, 5);
}
System.out.println("Exit GetDataFromExcelSheet");
return data;
}
public class ExcelDataProvider
{
XSSFSheet sheetName;
public void GetSheetName(String SheetName) throws Exception
{
System.out.println("Enter GetSheetName");
File file=new File(".\\TestData\\TestData.xlsx");
FileInputStream fs=new FileInputStream(file);
XSSFWorkbook wb=new XSSFWorkbook(fs);
//CommonFunctions commonFunctionsObj=new CommonFunctions(driver);
if(PropertiesfileDataProvider.GetPropertyValue("URL").contains("urltag"))
{
sheetName= wb.getSheet(SheetName);
}
else
sheetName= wb.getSheet(SheetName);
System.out.println("Exit GetSheetName");
}
public int FindTotalRows()
{
int totalRows=sheetName.getLastRowNum()+1;
//System.out.println(totalRows);
//System.out.println(sheetName);
return totalRows;
}
public String GetCellData(int row,int column)
{
String data=sheetName.getRow(row).getCell(column).getStringCellValue();
//String data=sheetName.getRow(row).getCell(column).getColumnIndex();
//String data=sheetName.getRow(row).getCell(column).toString();
System.out.println(data);
return data;
}
}
After executing a test case with multiple parameters I am passing the result (Pass/Fail) into an excel sheet. there are 6 rows the 1st result fail is entering into all the 6 rows (it should enter into only 1st row), the 2nd result pass is starting from 1st row and entering into all the 6 rows
Getting Data Code
public static Object[][] getData() throws IOException{
int rowCount = ExcelUtil.getRowCount();
int colCount = ExcelUtil.getColumnCount();
Object[][] data = new Object[rowCount-1][colCount];
for(int i=1; i<rowCount; i++)
{
for(int j=0; j<colCount; j++)
{
//Check cell is empty or not
if (data[i-1][j] == null) {
data[i-1][j] = "";
}
//change values to string
data[i-1][j] = ExcelUtil.setCellDataToString(i, j);
}
}
return data;
}
Test Case
public static void signIn(String email, String password, String result) throws IOException, InterruptedException {
LoginPage lp = PageFactory.initElements(driver, LoginPage.class);
driver.navigate().to(loginUrl);
lp.email().sendKeys(email);
lp.password().sendKeys(password);
lp.submitlogin().click();
//Verify
String expectedUrl = "http://automationpractice.com/index.php?controller=my-account";
String actualUrl = driver.getCurrentUrl();
String loginmessage;
if (expectedUrl.equalsIgnoreCase(actualUrl)) {
loginmessage = "pass";
lp.logout().click();
}else {
loginmessage ="fail";
}
ExcelUtil.writeIntoExcel(filePath, loginmessage);
}
Writing into excel
public static void writeIntoExcel(String FilePath, String dataToWrite) throws IOException {
int rowCount = ExcelUtil.getRowCount();
int colCount = ExcelUtil.getColumnCount();
try {
for (int i = 1; i <= rowCount; i++) {
shFile.getRow(i).createCell(colCount-1).setCellValue(dataToWrite);
fileOut = new FileOutputStream(filePath);
wbFile.write(fileOut);
}
fileOut.close();
} catch (Exception e) {
System.out.println("Data is not entered into excel");
}
}
You should not be iterating rows for each test run. Change method
public static void writeIntoExcel
to find row number based on email and password content and then create cells of result into that row.
I am new to Apache POI.
I have written a small code for removing duplicate records from a excel file. I am successfully able to identify the duplicate records across sheets but when writing to a new file after removing records, no output is being generated.
Please help where I am goin wrong?
Am I writing properly ?? Or am missing something?
public static void main(String args[]) {
DataFormatter formatter = new DataFormatter();
HSSFWorkbook input_workbook;
HSSFWorkbook workbook_Output_Final;
HSSFSheet input_workbook_sheet;
HSSFRow row_Output;
HSSFRow row_1_index;
HSSFRow row_2_index;
String value1 = "";
String value2 = "";
int count;
//main try catch block starts
try {
FileInputStream input_file = new FileInputStream("E:\\TEST\\Output.xls"); //reading from input file
input_workbook = new HSSFWorkbook(new POIFSFileSystem(input_file));
for (int sheetnum = 0; sheetnum < input_workbook.getNumberOfSheets(); sheetnum++) { //traversing sheets
input_workbook_sheet = input_workbook.getSheetAt(sheetnum);
int input_workbook_sheet_total_row = input_workbook_sheet.getLastRowNum(); //fetching last row nmber
for (int input_workbook_sheet_row_1 = 0; input_workbook_sheet_row_1 <= input_workbook_sheet_total_row; input_workbook_sheet_row_1++) { //traversing row 1
for (int input_workbook_sheet_row_2 = 0; input_workbook_sheet_row_2 <= input_workbook_sheet_total_row; input_workbook_sheet_row_2++) {
row_1_index = input_workbook_sheet.getRow(input_workbook_sheet_row_1); //fetching one iteration row index
row_2_index = input_workbook_sheet.getRow(input_workbook_sheet_row_2); //fetching sec iteration row index
if (row_1_index != row_2_index) {
count = 0;
value1 = "";
value2 = "";
for (int row_1_index_cell = 0; row_1_index_cell < row_1_index.getLastCellNum(); row_1_index_cell++) { //traversing cell for each row
try {
value1 = value1 + formatter.formatCellValue(row_1_index.getCell(row_1_index_cell)); //fetching row cells value
value2 = value2 + formatter.formatCellValue(row_2_index.getCell(row_1_index_cell)); //fetching row cells value
} catch (NullPointerException e) {
}
count++;
if (count == row_1_index.getLastCellNum()) {
if (value1.hashCode() == value2.hashCode()) { //remove the duplicate logic
System.out.println("deleted : " + row_2_index);
System.out.println("------------------");
input_workbook_sheet.removeRow(row_2_index);
}
}
}
}
}
}
}
FileOutputStream fileOut = new FileOutputStream("E:\\TEST\\workbook.xls");
input_workbook.write(fileOut);
fileOut.close();
input_file.close();
} catch (Exception e) {
//e.printStackTrace();
}
//main try catch block ends
}
A couple of things to note:
you swallow any kind of Exception; Igotsome nullpointers with my test data, and that would prevent the workbook from being written
when removing rows, it is an old trick to move backwards through the row numbers because then you don't have to adjust for the row number you have just removed
the code empties the row, but it doesn't move all rows upwards (=there is a gap after the delete). If you want to remove that gap, you can work with shiftRows
you compare things by hashcode, which is possible (in some use cases), but I feel like .equals() is what you want to do. See also Relationship between hashCode and equals method in Java
Here's some code that worked for my test data, feel free to comment if something doesn't work with your data:
public static void main(String args[]) throws IOException {
DataFormatter formatter = new DataFormatter();
HSSFWorkbook input_workbook;
HSSFWorkbook workbook_Output_Final;
HSSFSheet input_workbook_sheet;
HSSFRow row_Output;
HSSFRow row_1_index;
HSSFRow row_2_index;
String value1 = "";
String value2 = "";
int count;
FileInputStream input_file = new FileInputStream("c:\\temp\\test.xls");
input_workbook = new HSSFWorkbook(new POIFSFileSystem(input_file));
for (int sheetnum = 0; sheetnum < input_workbook.getNumberOfSheets(); sheetnum++) {
input_workbook_sheet = input_workbook.getSheetAt(sheetnum);
int input_workbook_sheet_total_row = input_workbook_sheet.getLastRowNum();
for (int input_workbook_sheet_row_1 = input_workbook_sheet_total_row; input_workbook_sheet_row_1 >=0; input_workbook_sheet_row_1--) { // traversing
for (int input_workbook_sheet_row_2 = input_workbook_sheet_total_row; input_workbook_sheet_row_2 >= 0 ; input_workbook_sheet_row_2--) {
row_1_index = input_workbook_sheet.getRow(input_workbook_sheet_row_1);
row_2_index = input_workbook_sheet.getRow(input_workbook_sheet_row_2);
if (row_1_index != null && row_2_index != null && row_1_index != row_2_index) {
count = 0;
value1 = "";
value2 = "";
int row_1_max = row_1_index.getLastCellNum() - 1;
for (int row_1_index_cell = 0; row_1_index_cell < row_1_max; row_1_index_cell++) {
try {
value1 = value1 + formatter.formatCellValue(row_1_index.getCell(row_1_index_cell));
value2 = value2 + formatter.formatCellValue(row_2_index.getCell(row_1_index_cell));
} catch (NullPointerException e) {
e.printStackTrace();
}
count++;
if (value1.equals(value2)) {
System.out.println("deleted : " + row_2_index.getRowNum());
System.out.println("------------------");
input_workbook_sheet.removeRow(row_2_index);
input_workbook_sheet.shiftRows(
row_2_index.getRowNum() + 1,
input_workbook_sheet_total_row,
-1,
true,
true);
}
}
}
}
}
}
FileOutputStream fileOut = new FileOutputStream("c:\\temp\\workbook.xls");
input_workbook.write(fileOut);
fileOut.close();
input_file.close();
input_workbook.close();
}
I want to run selenium-webdriver-java-eclipse, using excel file contains multiple excel sheets with different name(sheet1,sheet2,sheet3,...), i need a for loop help me to do that and read from this sheets.
public class ExcelDataConfig {
XSSFWorkbook wb;
XSSFSheet sheet = null;
public ExcelDataConfig(String Excelpath) throws IOException {
// TODO Auto-generated method stub
try {
File file = new File(Excelpath);
// Create an object of FileInputStream class to read excel file
FileInputStream fis = new FileInputStream(file);
wb = new XSSFWorkbook(fis);
} catch (Exception e) {
}
}
public String GetData(int sheetNumber, int Row, int Column) {
Iterator<Row> rowIt=sheet.rowIterator();
DataFormatter formatter = new DataFormatter();
XSSFCell cell = sheet.getRow(Row).getCell(Column);
String data = formatter.formatCellValue(cell);
return data;
}
public int GetRowCount(String sheetNumber) {
int row = wb.getSheet(sheetNumber).getLastRowNum();
row = row + 1;
return row;
}
}
try something like this, it is working for me you need to add the sheet numbers and cell numbers at the places of k and j
enter code here
String filePath="C:\\Users\\USER\\Desktop\\Book1.xlsx";// file path
FileInputStream fis=new FileInputStream(filePath);
Workbook wb=WorkbookFactory.create(fis);
ArrayList<String> ls=new ArrayList<String>();
for(int k=0; k<=3;k++)//k =sheet no
{
Sheet sh=wb.getSheetAt(k);
System.out.println(sh);
// int count=0;
for(int i=0;i<=sh.getLastRowNum();i++)
{
System.out.println("row no:"+i);
for(int j=0; j<=4;j++)//j=column no
{
try {
String values=sh.getRow(i).getCell(j).getStringCellValue().trim();
System.out.println(values);
//condetions
/* if(values.contains("condtn1"))
{
System.out.println("Value of cell "+values+" ith row "+(i+1));
ls.add(values);
count++;
}
if(values.contains("condn2"))
{
System.out.println("Value of cell "+values+" ith row "+(i+1));
ls.add(values);
count++;
}*/
}catch(Exception e){
}
}
}
}
}
}
Please try writing similar to something like this:
for (int i = startRow; i < endRow + 1; i++) {
for (int j = startCol; j < endCol + 1; j++) {
testData[i - startRow][j - startCol] = ExcelWSheet.getRow(i).getCell(j).getStringCellValue();
Cell cell = ExcelWSheet.getRow(i).getCell(j);
testData[i - startRow][j - startCol] = formatter.formatCellValue(cell);
}
}
Terms used in method are pretty self explanatory. Let us know if you get stuck or need more info.
I am trying to read Excel -2*2 matrix through Apache POI. But the first value returned by 2D array is [null,null]. Please check my code and advise for suitable corrections.
public String[][] getDataArray(String sheetName)
{
String value ="";
String[][] data = null;
int rowCount = wb.getSheet(sheetName).getLastRowNum();
int colCount = wb.getSheet(sheetName).getRow(1).getLastCellNum()-1;
data = new String[rowCount][colCount];
for(int i=1; i<=rowCount;i++)
{
Row row = wb.getSheet(sheetName).getRow(i);
for(int j=0;j<colCount;j++)
{
Cell cell = row.getCell(j);
if(cell.getCellType()==Cell.CELL_TYPE_NUMERIC)
{
value = ""+cell.getStringCellValue();
}
else
{
value = cell.getStringCellValue();
}
data[i][j] = value;
}
}
return data;
}
The debug view where we can see that the first value stored in the variable data is null, null
The excel which i am trying to read. I need only the userName and password data(2*2) alone. Not the header and Run mode datas.
Of course the value in the index 0 will be null because the i starts from 1 and not 0
for (int i = 1; i <= rowCount; i++) //i starts from one
...
data[i][j] = value;
either initialize the i from 0 or do like this
data[i-1][j] = value;
public static String[][] getSheetData(final String fileName, final String workSheetName)
throws Exception {
Integer lastRow = null;
short lastCol = 0;
String[][] sheetData = null;
FileInputStream file=new FileInputStream(MettlTest.class.getClass().getResource("/" + fileName).getPath());
workbook = new XSSFWorkbook(file);
sheet = workbook.getSheet(workSheetName);
try {
XSSFRow row;
XSSFCell cell;
lastRow = sheet.getPhysicalNumberOfRows();
lastCol = sheet.getRow(1).getLastCellNum();
sheetData = new String[lastRow - 1][lastCol];
for (int r = 1; r < lastRow; r++) {
row = sheet.getRow(r);
if (row != null) {
for (int c = 0; c < lastCol; c++) {
cell = row.getCell(c);
if (cell == null) {
sheetData[r][c] = null;
} else {
sheetData[r-1][c] = new DataFormatter().formatCellValue(cell);
}
}
}
}
return sheetData;
}
catch (final Exception e) {
throw e;
}
finally {
try {
file.close();
} catch (IOException io) {
Reporter.log("Unable to close File : " + fileName);
throw io;
}
}