Develop Reference

XSSFTable setDataRowCount overwrites totals row [duplicate]

I've got an table in excel with formulae I would like to add data to.
My motivation for this is the fact that tables in excel can dynamically expand to the range of data you add to them, meaning that the formula rows automatically keep up with the amount of data rows.
I'm however having a hard time finding out if this is possible using apache-POI.
One thing I was going to try (see code below) was to expand the AreaReference of the table to cover the data, however both AreaReference(CR,CR2); (as used in this example) and AreaReference(CR,CR2, SpreadsheetVersion.EXCEL2007) (seen in the apache docs) give "constructor is undefined".
No idea what is causing that constructor error as I do have org.apache.poi.ss.util imported.
The other option on the apache docs AreaReference(java.lang.String reference) lets me compile and run but instead gives a "NoSuchMethod" error.
List<XSSFTable> tableList = spreadSheet.getTables();
CellReference CR = new CellReference(0, 0);
CellReference CR2 = new CellReference(5, 2);
AreaReference my_data_range = new AreaReference(CR,CR2);
tableList.get(0).setArea(my_data_range);
Any help will be appreciated.

The main problem using apache poi until now is that it is not ready to be used without having detailed knowledge about Microsoft Office as such and about the storage of Microsoft Office files. There are many things only half way ready and there are regressions often in new versions (bugs occur again which were solved already).
So your requirement: "Expanding an existing table in Excel using Apache POI" is not possible only simply using apache poi. One must know that Office Open XML files *.xlsx are simply ZIP archives which can be unzipped. And after unzipping we find /xl/tables/table1.xml for storage of the table. This XML we can analyzing and comparing it with XML which was created using Excel's GUI. So we can find problems which results from shortcomings of apache poi. Same is with the sheet's XML in /xl/tables/sheet1.xml.
Also we need to know that apache poi builds on the low level classes of ooxml-schemas. Partially we need using those classes because of the halfway readiness of apache poi. In the following example we need ooxml-schemas-1.4.jar additionally because apache poi's poi-ooxml-schemas-4.0.0.jar has not included org.openxmlformats.schemas.spreadsheetml.x2006.main.CTTableFormula until now. Unfortunately there is no documentation about ooxml-schemas public available. So we need downloading the sources and doing javadoc our own.
The following example works for me using apache poi 4.0.0. If you get problems while compiling or running, the reason might be that multiple different versions of apache poi jars are in class path while compile time and/or run time. Do not mix different apache poi versions. Also, as said already, my code needs the full jar of all of the schemas ooxml-schemas-1.4.jar.
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.*;
import org.apache.poi.ss.util.*;
import org.apache.poi.ss.SpreadsheetVersion;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import org.openxmlformats.schemas.spreadsheetml.x2006.main.CTTableColumn;
class ExcelExpandingTable {
static void addRowToTable(XSSFTable table) {
int lastTableRow = table.getEndCellReference().getRow();
int totalsRowCount = table.getTotalsRowCount();
int lastTableDataRow = lastTableRow - totalsRowCount;
// we will add one row in table data
lastTableRow++;
lastTableDataRow++;
// new table area plus one row
AreaReference newTableArea = new AreaReference(
table.getStartCellReference(),
new CellReference(
lastTableRow,
table.getEndCellReference().getCol()
),
SpreadsheetVersion.EXCEL2007
);
// new table data area plus one row
AreaReference newTableDataArea = new AreaReference(
table.getStartCellReference(),
new CellReference(
lastTableDataRow,
table.getEndCellReference().getCol()
),
SpreadsheetVersion.EXCEL2007
);
XSSFSheet sheet = table.getXSSFSheet();
if (totalsRowCount > 0) {
//if we have totals rows, shift totals rows down
sheet.shiftRows(lastTableDataRow, lastTableRow, 1);
// correcting bug that shiftRows does not adjusting references of the cells
// if row 3 is shifted down, then reference in the cells remain r="A3", r="B3", ...
// they must be adjusted to the new row thoug: r="A4", r="B4", ...
// apache poi 3.17 has done this properly but had have other bugs in shiftRows.
for (int r = lastTableDataRow; r < lastTableRow + 1; r++) {
XSSFRow row = sheet.getRow(r);
if (row != null) {
long rRef = row.getCTRow().getR();
for (Cell cell : row) {
String cRef = ((XSSFCell)cell).getCTCell().getR();
((XSSFCell)cell).getCTCell().setR(cRef.replaceAll("[0-9]", "") + rRef);
}
}
}
// end correcting bug
}
// if there are CalculatedColumnFormulas do filling them to the new row
XSSFRow row = sheet.getRow(lastTableDataRow); if (row == null) row = sheet.createRow(lastTableDataRow);
int firstTableCol = table.getStartCellReference().getCol();
for (CTTableColumn tableCol : table.getCTTable().getTableColumns().getTableColumnList()) {
if (tableCol.getCalculatedColumnFormula() != null) {
int id = (int)tableCol.getId();
String formula = tableCol.getCalculatedColumnFormula().getStringValue();
XSSFCell cell = row.getCell(firstTableCol + id - 1); if (cell == null) cell = row.createCell(firstTableCol + id - 1);
cell.setCellFormula(formula);
}
}
table.setArea(newTableArea);
// correcting bug that Autofilter includes possible TotalsRows after setArea new
// Autofilter must only contain data area
table.getCTTable().getAutoFilter().setRef(newTableDataArea.formatAsString());
// end correcting bug
table.updateReferences();
}
public static void main(String[] args) throws Exception {
try (Workbook workbook = WorkbookFactory.create(new FileInputStream("SAMPLE.xlsx"));
FileOutputStream out = new FileOutputStream("SAMPLE_NEW.xlsx")) {
XSSFSheet sheet = ((XSSFWorkbook)workbook).getSheetAt(0);
XSSFTable table = sheet.getTables().get(0);
addRowToTable(table);
workbook.write(out);
}
}
}
The above was 2018. Now we have 2022.
Using current apache poi 5.2.2 the code may be like follows:
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.*;
import org.apache.poi.ss.util.*;
import org.apache.poi.ss.SpreadsheetVersion;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import org.openxmlformats.schemas.spreadsheetml.x2006.main.CTTableColumn;
class ExcelExpandingTable {
static XSSFRow addRowToTable(XSSFTable table) {
int lastTableRow = table.getEndCellReference().getRow();
int totalsRowCount = table.getTotalsRowCount();
int lastTableDataRow = lastTableRow - totalsRowCount;
int firstTableCol = table.getStartCellReference().getCol();
// we will add one row in table data
lastTableRow++;
lastTableDataRow++;
// new table area plus one row
AreaReference newTableArea = new AreaReference(
table.getStartCellReference(),
new CellReference(
lastTableRow,
table.getEndCellReference().getCol()
),
SpreadsheetVersion.EXCEL2007
);
XSSFSheet sheet = table.getXSSFSheet();
if (totalsRowCount > 0) {
//if we have totals rows, shift totals rows down
sheet.shiftRows(lastTableDataRow, lastTableRow, 1);
//correct all sheet table-reference-formulas which probably got damaged after shift rows
for (CTTableColumn tableCol : table.getCTTable().getTableColumns().getTableColumnList()) {
if (tableCol.getCalculatedColumnFormula() != null) {
int id = (int)tableCol.getId();
String formula = tableCol.getCalculatedColumnFormula().getStringValue();
int rFirst = table.getStartCellReference().getRow() + table.getHeaderRowCount();
int rLast = table.getEndCellReference().getRow() - table.getTotalsRowCount();
int c = table.getStartCellReference().getCol() + id - 1;
sheet.getWorkbook().setCellFormulaValidation(false);
for (int r = rFirst; r <= rLast; r++) {
XSSFRow row = sheet.getRow(r); if (row == null) row = sheet.createRow(r);
XSSFCell cell = row.getCell(c); if (cell == null) cell = row.createCell(c);
cell.setCellFormula(formula);
}
}
}
}
// if there are CalculatedColumnFormulas do filling them to the new row
XSSFRow row = sheet.getRow(lastTableDataRow); if (row == null) row = sheet.createRow(lastTableDataRow);
for (CTTableColumn tableCol : table.getCTTable().getTableColumns().getTableColumnList()) {
if (tableCol.getCalculatedColumnFormula() != null) {
int id = (int)tableCol.getId();
String formula = tableCol.getCalculatedColumnFormula().getStringValue();
XSSFCell cell = row.getCell(firstTableCol + id - 1); if (cell == null) cell = row.createCell(firstTableCol + id - 1);
cell.getSheet().getWorkbook().setCellFormulaValidation(false); // see https://bz.apache.org/bugzilla/show_bug.cgi?id=66039
cell.setCellFormula(formula);
}
}
// copy cell styles to the new row from the row above
row = sheet.getRow(lastTableDataRow); if (row == null) row = sheet.createRow(lastTableDataRow);
XSSFRow rowAbove = sheet.getRow(lastTableDataRow - 1); if (row == null) row = sheet.createRow(lastTableDataRow - 1);
for (CTTableColumn tableCol : table.getCTTable().getTableColumns().getTableColumnList()) {
int id = (int)tableCol.getId();
XSSFCell cellAbove = rowAbove.getCell(firstTableCol + id - 1);
if (cellAbove != null) {
XSSFCellStyle styleAbove = cellAbove.getCellStyle();
XSSFCell cell = row.getCell(firstTableCol + id - 1); if (cell == null) cell = row.createCell(firstTableCol + id - 1);
cell.setCellStyle(styleAbove);
}
}
// set new table area
table.setArea(newTableArea);
// update table references
table.updateReferences();
return sheet.getRow(lastTableDataRow);
}
public static void main(String[] args) throws Exception {
try (Workbook workbook = WorkbookFactory.create(new FileInputStream("SAMPLE.xlsx"));
FileOutputStream out = new FileOutputStream("SAMPLE_NEW.xlsx")) {
XSSFSheet sheet = ((XSSFWorkbook)workbook).getSheetAt(0);
XSSFTable table = sheet.getTables().get(0);
XSSFRow row = addRowToTable(table);
workbook.write(out);
}
}
}
According to FAQ, this code needs poi-ooxml-full-5.2.2.jar instead of ooxml-schemas-1.4.jar now for run using apache poi 5.2.2.
Some bugs are fixed. But there is one new bug when using XSSFCell.setCellFormula when formula contains table references. But XSSFWorkbook.setCellFormulaValidation(false) avoids this.

HasMap inserting null values instead of string?

Below is my code which I am trying to read the excel data using Apache POI and want to store the first column and last column values in Hashmap by taking First column values as keys and last column as values. I am facing an issue while putting the values in the hashmap. My expected output should be like
Key value
UserDefined Xpath_original
seldriver xpath;//div[#id='mBody']
Selenium xpath;//table[#id='choice']/tbody/tr/td[1]/ul/li[1]
but I am getting the output as
*
null null
null
null xpath;//div[#id='mBody']
*
Below is my piece of code
package com.selenium.common;
import java.io.File;
import java.util.HashMap;
import java.util.Iterator;
import java.util.LinkedHashMap;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.ss.usermodel.WorkbookFactory;
public class ReadMacroExcel {
public static void main(String[] args) {
try {
File f=new File("C:\\Users\\Rajaram.8.Kannuri\\Downloads\\Fire-IEBrowser1.4.xlsm");
HashMap <String,String> innerMap =null;
Workbook workbook = WorkbookFactory.create(f);
System.out.println(workbook);
int numberOfSheets = workbook.getNumberOfSheets();
org.apache.poi.ss.usermodel.Sheet sheet=null;
String key = "";
String value = "";
//Get the sheet in the xlsx file
for (int i = 0; i < numberOfSheets; i++) {
sheet = workbook.getSheetAt(i);
System.out.println(sheet.getSheetName());
int firstColumn = sheet.getRow(0).getFirstCellNum();
int lastColumn = sheet.getRow(0).getLastCellNum();
Iterator<Row> rowIterator = sheet.iterator();
innerMap = new HashMap<String,String>();
while(rowIterator.hasNext())
{
Row row = rowIterator.next();
Iterator<Cell> cellIterator = row.cellIterator();
while (cellIterator.hasNext())
{
Cell cell = cellIterator.next();
if(cell.getColumnIndex()==firstColumn||cell.getColumnIndex()==lastColumn-1)
{
switch (cell.getCellType())
{
case Cell.CELL_TYPE_STRING:
key = cell.getStringCellValue();
/*System.out.print(key + "\t");*/
/* Thread.sleep(5000);*/
break;
case Cell.CELL_TYPE_FORMULA:
value = cell.getRichStringCellValue().toString();
/*System.out.print(value + "\t");*/
/*Thread.sleep(5000);*/
break;
}
System.out.print(innerMap.put(key,value));
}
}
System.out.println("");
}
/* System.out.println(innerMap.get("UserDefined"));*/
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
Can any one help me in solving this issue??

It seems that you are trying to print the inserted value with
System.out.print(innerMap.put(key,value));
However, put returns the previous value associated with the given key, or null if there was no mapping for that key.
See also https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html#put-K-V-
You should use something like
innerMap.put(key,value); // add the new mapping to the hashmap
System.out.print(innerMap); // dumps the complete map with all entries
System.out.print(innerMap.get(key)); // dumps the value for the recently inserted key

How to set default value in Apache POI pivot table report filter

I have a worksheet with data in it, I am trying to create a pivot table with report filter. I want to set default value to the report filter.
pivotTable.addReportFilter(13);
column contains 0's and 1's, I would like to set 0 as my default value in the report filter.

At first this question is not answerable in that general context as it is asked now. Creating pivot tables using apache poi is in beta state until now. So we need not only the high level apache poi API but also the underlying low level objects. And we need exactly to know which kind of data shall be in the pivot table. To be general able creating pivot tables from all kind of data, as Excel can do, there is much more effort necessary. Microsoft has programmed this in decades with big teams of programmers. From this apache poi is far away.
Until now apache poi adds as much pivot field items of type "default" (<item t="default"/>) as rows are present in the data range, if the pivot fields where used as axis fields.
This is because they don't want to have a look at the data, and so they are assuming as much different values as rows where in the data.
This is fine because Excel will rebuild its pivot cache while opening. But if we want preselect items, then this is not fine. Then we must know what items there are that can be preselected.
So we need at least as much items, as we want preselecting, as numbered items: <item x="0"/><item x="1"/><item x="2"/>...
And we need to build a cache definition which has shared elements for those items.
Example:
import org.apache.poi.xssf.usermodel.*;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.ss.util.*;
import java.util.Random;
import java.io.*;
class PivotTableTest4 {
private static void setCellData(Sheet sheet) {
Row row = sheet.createRow(0);
Cell cell = row.createCell(0);
cell.setCellValue("Name");
cell = row.createCell(1);
cell.setCellValue("Value1");
cell = row.createCell(2);
cell.setCellValue("Value2");
cell = row.createCell(3);
cell.setCellValue("City");
for (int r = 1; r < 15; r++) {
row = sheet.createRow(r);
cell = row.createCell(0);
cell.setCellValue("Name " + ((r-1) % 4 + 1));
cell = row.createCell(1);
cell.setCellValue(r * new java.util.Random().nextDouble());
cell = row.createCell(2);
cell.setCellValue(r * new java.util.Random().nextDouble());
cell = row.createCell(3);
cell.setCellValue("City " + ((r-1) % 3 + 1));
}
}
public static void main(String[] args) {
try {
XSSFWorkbook wb = new XSSFWorkbook();
XSSFSheet sheet = wb.createSheet();
//Create some data to build the pivot table on
setCellData(sheet);
XSSFPivotTable pivotTable = sheet.createPivotTable(
new AreaReference(new CellReference("A1"), new CellReference("D15")), new CellReference("H5"));
//Configure the pivot table
//Use first column as row label
pivotTable.addRowLabel(0);
//Sum up the second column
pivotTable.addColumnLabel(DataConsolidateFunction.SUM, 1);
//Avarage the third column
pivotTable.addColumnLabel(DataConsolidateFunction.AVERAGE, 2);
//Add fourth column as page filter
pivotTable.addReportFilter(3);
/*
Apache poi adds 15 pivot field items of type "default" (<item t="default"/>) here.
This is because there are 15 rows (A1:D15) and, because they don't have a look at the data,
they are assuming max 15 different values. This is fine because Excel will rebuild its pivot cache while opening.
But if we want preselect items, then this is not fine. Then we must know what items there are that can be preselected.
So we need at least as much items as we want preselecting as numbered items: <item x="0"/><item x="1"/><item x="2"/>...
And we must build a cache definition which has shared elements for those items.
*/
for (int i = 0; i < 3; i++) {
//take the first 3 items as numbered items: <item x="0"/><item x="1"/><item x="2"/>
pivotTable.getCTPivotTableDefinition().getPivotFields().getPivotFieldArray(3).getItems().getItemArray(i).unsetT();
pivotTable.getCTPivotTableDefinition().getPivotFields().getPivotFieldArray(3).getItems().getItemArray(i).setX((long)i);
//build a cache definition which has shared elements for those items
//<sharedItems><s v="City 1"/><s v="City 2"/><s v="City 3"/></sharedItems>
pivotTable.getPivotCacheDefinition().getCTPivotCacheDefinition().getCacheFields().getCacheFieldArray(3).getSharedItems().addNewS().setV("City " + (i+1));
}
//Now we can predefinite a page filter. Second item, which is "City 2", in this case.
pivotTable.getCTPivotTableDefinition().getPageFields().getPageFieldArray(0).setItem(1);
FileOutputStream fileOut = new FileOutputStream("PivotTableTest4.xlsx");
wb.write(fileOut);
fileOut.close();
wb.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
This needs the full jar of all of the schemas, ooxml-schemas-1.3.jar, as mentioned in the FAQ.

How to parse a tree structure saved in an Excel file using Apache POI

All,
Good Morning!
I have an excel file with data listed as the following, I'm trying to parse down using POI
A
B
C
D1
D2
F
G1
G2
G3
M
S1
R
T
U
L
X
Y
Z
is it possible to generate an output like the following
A
A-->B
A-->B-->C
A-->B-->C-->D1
A-->B-->C-->D2
A-->B-->F
A-->B-->F-->G1
A-->B-->F-->G2
A-->B-->F-->G3
A-->B-->M
A-->B-->M-->S1
A-->R
A-->R-->T
A-->U
L
L-->X
L-->X-->Y
L-->X-->Y-->Z
I have been trying from quite some time but havent figured out the logic
Thanks

Solution in Java, using Apache POI:
import java.io.File;
import java.io.FileInputStream;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
public class ParseTreeDemo
{
private static final int NUM_COLUMNS = 4;
public static void main(String[] args)
{
try
{
FileInputStream file = new FileInputStream(new File("Test.xlsx"));
XSSFWorkbook workbook = new XSSFWorkbook(file);
XSSFSheet sheet = workbook.getSheetAt(0);
// Use a column marker to save the 'farthest' column so far
int currColMarker = -1;
List<String> list = new ArrayList<>();
//Iterate through each rows one by one
Iterator<Row> rowIterator = sheet.iterator();
while (rowIterator.hasNext())
{
Row row = rowIterator.next();
for(int currCol = 0; currCol < NUM_COLUMNS; currCol++)
{
Cell cell = row.getCell(currCol);
if(cell == null)
continue;
if(cell.getCellType() == Cell.CELL_TYPE_STRING) {
if(currCol > currColMarker) {
// A farther column, simply append and
// update column marker
currColMarker = currCol;
list.add(cell.getStringCellValue());
}
else if (currCol == currColMarker) {
// At same level as column marker
// Remove old value at same level, before appending
list.remove(list.size() - 1);
list.add(cell.getStringCellValue());
}
else {
// At a 'nearer' column, remove those values beyond
// this level before appending
currColMarker = currCol;
list = list.subList(0, currCol);
list.add(cell.getStringCellValue());
}
}
}
// For displaying the current contents
StringBuilder sb = new StringBuilder();
for(String s : list) {
if(sb.length() != 0) {
sb.append("-->");
}
sb.append(s);
}
System.out.println(sb.toString());
}
file.close();
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
Output:
A
A-->B
A-->B-->C
A-->B-->C-->D1
A-->B-->C-->D2
A-->B-->F
A-->B-->F-->G1
A-->B-->F-->G2
A-->B-->F-->G3
A-->B-->M
A-->B-->M-->S1
A-->R
A-->R-->T
A-->U
L
L-->X
L-->X-->Y
L-->X-->Y-->Z
The idea:
Use a 'column marker' to keep track of the active column
If the new value is at a column, with a larger column value, append
If it has the same column value, remove the last value, and append
If it has a smaller column value, remove all current values beyond the new column value, before appending
Note: Test.xlsx contains the values as stated in the question.

If the listed data is in a variable called data, the following will work in Tcl:
proc merge {a b} {
set res {}
foreach ac [split $a {}] bc [split $b {}] {
if {![string is space $ac] && [string is space -strict $bc]} {
append res $ac
} else {
append res $bc
}
}
set res
}
set current {}
foreach line [split [string trim $data] \n] {
set current [merge $current [string trimright $line]]
puts [join $current -->]
}
I originally went with a pseudo-stack approach, but it seemed simpler to "merge" each new line with the accumulated line (current) such that non-blank text in the new line would overwrite text in the accumulated line, and that the accumulated line would be truncated if the new line was shorter (after trimming off trailing whitespace from it).
Once I had the merged line, I could take advantage of the fact that (most) strings in Tcl are also lists, and print it as a string formed by joining the words using "-->" tokens.
Documentation: append, foreach, if, proc, puts, set, split, string

How to get excell range name for HSSFCell apache POI API?

As title, I have a little trouble here.
I can get font from cell
HSSFFont font =
cell.getRow().getSheet().getWorkbook().
getFontAt(cell.getCellStyle().getFontIndex());
But now I need to get range name of it. Actually I need something to anchor and determine key cell and its own value cell.
Is there some method to get range name such as workBook.getName() or .getNameAt() but how to get name index from HSSFCell?

Apart from rich text strings, a cell has only one font assigned,
but it may be referenced by more than one named range.
So you'll need to iterate through the named ranges of the workbook and check if the cell is referenced. For the sake of simplicity, I have iterated over all area.getAllReferencedCells() - in case of big ranges you'll need to check if the area isContiguous() and if your cell/row-index is inside the cell/row-index of getFirstCell() and getLastCell() bounding box.
For more info check the Busy Developers' Guide to HSSF and XSSF Features.
Or search on stackoverflow ...
(in my testcase, a cell(row 4, col 3) was referenced by three different shaped named ranges)
import java.io.File;
import java.util.*;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.ss.util.*;
public class XlsRangeNames {
public static void main(String[] args) throws Exception {
Workbook wb = WorkbookFactory.create(new File("src/test/resources/name-range.xls"));
Cell cell = wb.getSheetAt(0).getRow(3).getCell(2);
for (Name n : getNamedRangesForCell(wb, cell)) {
System.out.println(n.getNameName());
}
}
static List<Name> getNamedRangesForCell(Workbook wb, Cell cell) {
int col = cell.getColumnIndex();
int row = cell.getRowIndex();
String sheetName = cell.getSheet().getSheetName();
List<Name> result = new ArrayList<Name>();
for (int i=0; i<wb.getNumberOfNames(); i++) {
Name name = wb.getNameAt(i);
if (!sheetName.equals(name.getSheetName())) continue;
AreaReference area = new AreaReference(name.getRefersToFormula());
CellReference crList[] = area.getAllReferencedCells();
for (CellReference cr : crList) {
if (cr.getCol() == col
&& cr.getRow() == row) {
result.add(name);
continue;
}
}
}
return result;
}
}