Develop Reference

How to add columns to an existing large excel file using SXSSF Apache POI?

I am working with a large excel file ( larger than 40 Mb , more than 100k rows and 50 columns ). I am successfully reading it using POI ( 3.10.1 version ) event stream and then doing some calculation and storing result into a List.
Now I have to append this List as a column in the same file. In this part I am facing issue.
I have tried to achieve this by using the below code
FileInputStream excelFile = new FileInputStream(new File(pathToFile));
Workbook workbook = new XSSFWorkbook(excelFile);
Sheet datatypeSheet = workbook.getSheetAt(0); // Get first sheet
Iterator<Row> iterator = datatypeSheet.iterator();
int i=0;
while (iterator.hasNext()) { // Loop over each row
Row currentRow = iterator.next();
Cell cell = currentRow.createCell(currentRow.getLastCellNum());
cell.setCellType(Cell.CELL_TYPE_STRING);
if(currentRow.getRowNum() == 0)
cell.setCellValue("OUTPUT-COLUMN"); // set column header for the new column
else {
cell.setCellValue(list.get(i)); // list contains the output to populate in new column
i++;
}
}
FileOutputStream fos = new FileOutputStream(new File(pathToOutput));
workbook.write(fos);
fos.close();
It is working fine with smaller files But the issue is that I am getting Out of memory for the larger files. Now I tried to modify this and use SXSSF in place of XSFF to get over the memory issue (See below code). But while testing even for smaller files I am getting output file same as the input file.
FileInputStream excelFile = new FileInputStream(new File(pathToFile));
XSSFWorkbook xwb = new XSSFWorkbook(inputStream);
inputStream.close();
SXSSFWorkbook wb = new SXSSFWorkbook(xwb,100);
wb.setCompressTempFiles(true);
SXSSFSheet sh = (SXSSFSheet) wb.getSheetAt(0);
Iterator<Row> iterator = datatypeSheet.iterator();
int i=0;
while (iterator.hasNext()) { // Loop over each row
Row currentRow = iterator.next();
Cell cell = currentRow.createCell(currentRow.getLastCellNum());
cell.setCellType(Cell.CELL_TYPE_STRING);
if(currentRow.getRowNum() == 0)
cell.setCellValue("OUTPUT-COLUMN"); // set column header for the new column
else {
cell.setCellValue(list.get(i)); // list contains the output to populate in new column
i++;
}
}
FileOutputStream fos = new FileOutputStream(new File(pathToOutput));
wb.write(fos);
fos.close();
Using a db is not suitable in my use case and i want to avoid using a temporary data structure to hold data for writing due to memory constraint.
Is there a way to write in output workbook while streaming ? Here is the code that I am using to read using POI Streaming API
private class ExcelData implements SheetContentsHandler {
LinkedHashMap<Strin, String> rowMap;
public void startRow(int rowNum) {
}
public void endRow(int rowNum) {
// Process the row
// Handle write to output workbook ??
}
public void cell(String cellReference, String formattedValue,
XSSFComment comment) {
// Save current row in rowMap ( column name => cell value )
}
public void headerFooter(String text, boolean isHeader, String tagName)
{
}
}

It is not possible to add column to existing workbook using POI SXSSF. It only allows addition of new rows.
The only solution is to read the existing workbook and write to a new workbook with the added column.
To achieve this we can store the rows in a data structure or database in the endrow() method and then use the persisted data to write a new workbook.

Fetching Data From xlsx sheet, not showing last three rows in jtable

I am trying to fetching data from excel sheet(.xlsx) file. when I use to print fetched data using
System.out.println(sheet.getRow(i).getCell(c).getStringCellValue()); I see the all the rows from excel sheet is fetched. But when I add this data in jtable it misses last three rows and shows following error:
Cannot invoke "org.apache.poi.xssf.usermodel.XSSFRow.getCell(int)" because the return value of "org.apache.poi.xssf.usermodel.XSSFSheet.getRow(int)" is null
enter code here
JFileChooser fileChooser = new JFileChooser("D:");
int returnValue = fileChooser.showOpenDialog(null);
if (returnValue == JFileChooser.APPROVE_OPTION) {
selectedFile = fileChooser.getSelectedFile();
FileName = selectedFile.getName();
String FilePath = selectedFile.getPath();
System.out.println(FileName);
System.out.println(FilePath);
File excelfile = new File(FileName);
try{
FileInputStream fis = new FileInputStream(selectedFile);
XSSFWorkbook wb = new XSSFWorkbook(fis);
XSSFSheet sheet = wb.getSheetAt(0);
int totalrows = sheet.getPhysicalNumberOfRows();
for (int i = 0; i <=totalrows; )
{
dmodel.addRow(new Object[]{"" });
System.out.println(sheet.getRow(i).getCell(0).getStringCellValue()); // this line work
String name = sheet.getRow(i).getCell(0).getStringCellValue();
jTable1.setValueAt(name , i, 0); // this line does not work
i++ ;
}
JOptionPane.showMessageDialog(null, "All rows are fetched Successfully" );
}catch(Exception fx)
{
System.out.println(fx.getMessage());
// System.out.println(fx.getCause());
}

System.out.println(sheet.getRow(r).getCell(c).getStringCellValue()); // this line work
String name = sheet.getRow(i).getCell(c).getStringCellValue();
jTable1.setValueAt(name , i, 0); // this line does not work
Don't know if this will solve your problem, but the above code is not how you use System.out.println(...) to debug your logic.
To verify the data you use:
//System.out.println(sheet.getRow(r).getCell(c).getStringCellValue()); // this line work
String name = sheet.getRow(i).getCell(c).getStringCellValue();
System.out.println( name );
jTable1.setValueAt(name , i, 0); // this line does not work
Don't attempt to read the data twice. By assigning the data to a variable you are debugging the data in the variable. Don't repeat the nested methods multiple times.
Maybe the class expects you to only read the data once, so the second read gives the null value. The simple change above will prevent this.

Apache POI createTable generates corrupted file when a header's cell contains a line break

I am using Apache POI 4.1.2 to create Excel files in Java. I have a piece of code that creates a table from existing cells and everything used to work fine, untill I had a linebreak inside a header's cell.
I tried to change the table's column name afterward but it didn't fix anything.
Below is a minimal piece of code to reproduce the problem:
public void test() throws IOException {
XSSFWorkbook wb = new XSSFWorkbook();
XSSFSheet sheet = wb.createSheet();
// headers
XSSFRow headersRow = sheet.createRow(0);
headersRow.createCell(0).setCellValue("Column1");
headersRow.createCell(1).setCellValue("Column2");
// a second row
XSSFRow row = sheet.createRow(1);
row.createCell(0).setCellValue(1);
row.createCell(1).setCellValue(2);
// create a table
AreaReference area = wb.getCreationHelper().createAreaReference(
new CellReference(sheet.getRow(0).getCell(0)),
new CellReference(sheet.getRow(1).getCell(1))
);
XSSFTable table = sheet.createTable(area);
// styling (no problem here)
sheet.setColumnWidth(0, 5000);
sheet.setColumnWidth(1, 5000);
CTTable cttable = table.getCTTable();
cttable.addNewTableStyleInfo();
XSSFTableStyleInfo style = (XSSFTableStyleInfo) table.getStyle();
style.setName("TableStyleMedium6");
style.setShowColumnStripes(false);
style.setShowRowStripes(true);
cttable.addNewAutoFilter().setRef(area.formatAsString());
CellStyle cellStyle = wb.createCellStyle();
cellStyle.setWrapText(true);
headersRow.getCell(0).setCellStyle(cellStyle);
// this file is OK
try (FileOutputStream outputStream = new FileOutputStream("C:\\tmp\\test.xlsx")) {
wb.write(outputStream);
}
// add a line break in a header's cell
headersRow.getCell(0).setCellValue("Column1\nwith a line break");
// this file has a problem
try (FileOutputStream outputStream = new FileOutputStream("C:\\tmp\\test2.xlsx")) {
wb.write(outputStream);
}
// this doesn't fix anything
table.getColumns().get(0).setName("Column1");
try (FileOutputStream outputStream = new FileOutputStream("C:\\tmp\\test3.xlsx")) {
wb.write(outputStream);
}
// neither does this
cttable.getTableColumns().getTableColumnList().get(0).setName("Column1");
try (FileOutputStream outputStream = new FileOutputStream("C:\\tmp\\test4.xlsx")) {
wb.write(outputStream);
}
}
Excel loads text.xlsx properly, but complains about all other files:
We found a problem with some content...
After Excel fixes the files, everything is OK but I would like to get rid of the warning message.
Any help will be appreciated.
Thanks

This is an inaccuracy with XSSFTable.updateHeaders. This method gets called while the table's XML gets written. This is because the table column names always must be synchronized with the cell contents. For example if the cell content is "Column1" and this cell is a column header of a table, then this tables column name also must be "Column1" (XML: <tableColumn id="1" name="Column1"/>).
But for line feeds in column headers, there is a specialty. If the cell content is "Column1\nwith a line break" and this cell is a column header of a table, then this tables column name must be XML as <tableColumn id="1" name="Column1_x000a_with a line break"/>. So "\n" is replaced by "x000a". Also "\r" would must be replaced by "x000d". This is because "\r\n" line breaks will not have the meaning of line break in XML.
So XSSFTable.java - updateHeaders would must be patched that way that "\n" gets replaced by "x000a" and "\r" gets replaced by "x000d".
...
public void updateHeaders() {
XSSFSheet sheet = (XSSFSheet)getParent();
CellReference ref = getStartCellReference();
if (ref == null) return;
int headerRow = ref.getRow();
int firstHeaderColumn = ref.getCol();
XSSFRow row = sheet.getRow(headerRow);
DataFormatter formatter = new DataFormatter();
if (row != null && row.getCTRow().validate()) {
int cellnum = firstHeaderColumn;
CTTableColumns ctTableColumns = getCTTable().getTableColumns();
if(ctTableColumns != null) {
for (CTTableColumn col : ctTableColumns.getTableColumnList()) {
XSSFCell cell = row.getCell(cellnum);
if (cell != null) {
String colName = formatter.formatCellValue(cell);
colName = colName.replace("\n", "_x000a_");
colName = colName.replace("\r", "_x000d_");
col.setName(colName);
}
cellnum++;
}
}
}
tableColumns = null;
columnMap = null;
xmlColumnPrs = null;
commonXPath = null;
}
...
Since XSSFTable.updateHeaders gets called while the table's XML gets written while XSSFWorkbook.write, there is no other way than patching this method. One does not have any chance to change table's XML while XSSFWorkbook.write.

Problems with starting a new row in Excel with Apache POI

I'm using the apache poi package to create a spreadsheet of figures which represent features of a shape (area, perimeter, centroid). The problem is that i have a method: writeDatabase() which outputs the features of the shape as they are found, the output spreadsheet looks like this:
http://s23.postimg.org/hqsfg76jv/Capture.png
All of these figures need to be in the same line, and then a new line needs to be taken for the next record. the writeDatabase method is shown below
public static void writeDatabase(int value, int cellNum){
try {
Cell cell1=null;
FileInputStream file = new FileInputStream(new File("features.xls"));
Workbook workbook = new HSSFWorkbook(file);
Sheet sheet = workbook.getSheetAt(0);
int lastRow = sheet.getPhysicalNumberOfRows();
cell1 = sheet.createRow(lastRow).createCell(cellNum);
cell1.setCellValue(value);
FileOutputStream outFile =new FileOutputStream(new File("features.xls"));
workbook.write(outFile);
outFile.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
I think the problem is with this line being called each time, but i cant think of an alternative:
int lastRow = sheet.getPhysicalNumberOfRows();
Any ideas?

You need to provide some indication of whether a new line should be started or not (unless you can tell that a new line is starting simply because cellNum is 0?)
Then, you can either create a new row, or use the existing row:
int lastRow = sheet.getPhysicalNumberOfRows();
Row row;
if (startNewRow) {
row = sheet.createRow(lastRow);
} else {
row = sheet.getRow(lastRow - 1);
}
cell1 = row.createCell(cellNum);
cell1.setCellValue(value);
Where startNewRow might be set based on cellNum, or might be an additional parameter that is passed into writeDatabase, whichever is appropriate.

Delete multiple Rows containing a specific String

I've been trying to develop a code that can delete multiple rows containing a specific String "POST". I have been doing this like:
private void processExcelFile(File f_path){
File path=f_path;
HSSFWorkbook wb = null;
try{
FileInputStream is=new FileInputStream(path);
wb= new HSSFWorkbook(is);
HSSFSheet sheet = wb.getSheetAt(0);
int j = 0;
for(Row row: sheet){
for(Cell cell : row){
count++;
if (cell.getCellType() == Cell.CELL_TYPE_STRING){
if (cell.getRichStringCellValue().getString().trim().contains("POST")){
rowcount_post=row.getRowNum();
System.out.print("ROW COUNT= "+rowcount_post);
HSSFRow removingRow = sheet.getRow(rowcount_post);
if (removingRow != null) {
sheet.removeRow(removingRow);
}
}
}
}
}
} catch(Exception e){
JOptionPane.showMessageDialog(this,e.getMessage(),"Error",JOptionPane.ERROR_MESSAGE);
}
try{
FileOutputStream fileOut = new FileOutputStream("C:/juni.xls");
wb.write(fileOut);
} catch(Exception e) {
JOptionPane.showMessageDialog(this,e.getMessage(),"SAVE Error",JOptionPane.ERROR_MESSAGE);
}
}
It is a matter of fact that it only deletes one row at a time. Is it possible that I could delete all the rows containing the String POST?

Why do you have a problem with deleting them one at a time? It doesn't seem too inefficient...
Anyway, I'm pretty sure you still have to loop over every cell to see if it contains "POST".
If you have a problem specifically with deleting one row at a time, you could save the indices of the rows you want to delete in an array; then use the array to delete the saved rows.

The reason you're deleting them one at a time is because you're finding them one at a time. As the other poster mentioned, you could keep track of the locations through out your program (whether that be during this loop), but that wouldn't make it algorithmically efficient (unless you kept track of it somewhere else and pass that info to this method).
I'm not too familiar with using Excel files in Java, but these seems to be a solid way to do it.