I've been trying to build my first web application using IntelliJ and Tomcat, and one of the tasks is being able to upload and process an Excel sheet file. So, I looked up online, and found the Apache POI library that can help me parse an Excel file. But when I downloaded all the required jars and copied and pasted some testing code, and start up the server, it shows on the webpage an error with http status 500, the root cause being: java.lang.ClassNotFoundException: org.apache.poi.openxml4j.opc.internal.marshallers.PackagePropertiesMarshaller$NamespaceImpl.
I've encountered the problem with other jars, but all solved by putting the corresponding jars inside tomcat's lib folder, just except for this one.
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import java.io.File;
import java.io.FileInputStream;
import java.util.Iterator;
public class ExcelParser {
private String pathname;
public ExcelParser(String pathname) {
this.pathname = pathname;
}
public void parse() {
try {
FileInputStream file = new FileInputStream(new File("/Users/JohnDoe/Desktop/test.xlsx"));
//Create Workbook instance holding reference to .xlsx file
XSSFWorkbook workbook = new XSSFWorkbook(file);
//Get first/desired sheet from the workbook
XSSFSheet sheet = workbook.getSheetAt(0);
//Iterate through each rows one by one
Iterator<Row> rowIterator = sheet.iterator();
while (rowIterator.hasNext()) {
Row row = rowIterator.next();
//For each row, iterate through all the columns
Iterator<Cell> cellIterator = row.cellIterator();
while (cellIterator.hasNext()) {
Cell cell = cellIterator.next();
//Check the cell type and format accordingly
switch (cell.getCellType()) {
case NUMERIC:
System.out.print(cell.getNumericCellValue() + "t");
break;
case STRING:
System.out.print(cell.getStringCellValue() + "t");
break;
}
}
System.out.println();
}
file.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
I'm just testing the functionality of Excel parsing, so don't really worry about the pathname.
Btw, I can see that this (inner) class is declared in poi-ooxml4-4.1.0.jar, which is also included in my Tomcat lib folder.
Any ideas why this is happening, and how I should fix it is appreciated.
To use Apache POI, you need the following jar files.
poi-ooxml-4.1.0.jar
poi-ooxml-schemas-4.1.0.jar
xmlbeans-3.1.0.jar
commons-compress-1.18.jar
curvesapi-1.06.jar
poi-4.1.0.jar
commons-codec-1.12.jar
commons-collections4-4.3.jar
commons-math3-3.6.1.jar
You can refer to the following link, which I have answered few things.
Unable to read Excel using Apache POI
I think I missed something when moving the jars to the lib directory, as I removed the original files and redo the cp command, everything works now. I'm closing the question with answer, thanks for the help!
Related
The application I am working on creates Excel exports using Apache POI. It was brought to our attention, through a security audit, that cells containing malicious values can spawn arbitrary processes if the user is not careful enough.
To reproduce, run the following:
import java.io.FileOutputStream;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
public class BadWorkbookCreator {
public static void main(String[] args) throws Exception {
try(
Workbook wb = new HSSFWorkbook();
FileOutputStream fos = new FileOutputStream("C:/workbook-bad.xls")
) {
Sheet sheet = wb.createSheet("Sheet");
Row row = sheet.createRow(0);
row.createCell(0).setCellValue("Aaaaaaaaaa");
row.createCell(1).setCellValue("-2+3 +cmd|'/C calc'!G20");
wb.write(fos);
}
}
}
Then open the resulting file:
And follow these steps:
Click on (A) to select the cell with malicious content
Click on (B) so that the cursor is in the formula editor
Press ENTER
You will be asked if you allow Excel to run an external application; if you answer yes, Calc is launched (or any malicious code)
One may say that the user is responsible for letting Excel run arbitrary things and the user was warned. But still, the Excel is downloaded from a trusted source and someone may fall into the trap.
Using Excel, you can place a single quote in front of the text in the formula editor to escape it. Placing the single quote in the cell content programmatically (e.g. code as below) makes the single quote visible!
String cellValue = cell.getStringCellValue();
if( cellValue != null && "=-+#".indexOf(cellValue.charAt(0)) >= 0 ) {
cell.setCellValue("'" + cellValue);
}
The question: Is there a way to keep the value escaped in the formula editor, but show the correct value, without the leading single quote, in the cell?
Thanks to the hard work investigating of Axel Richter here and Nikos Paraskevopoulos here....
From Apache POI 3.16 beta 1 onwards (or for those who live dangerously, any nightly build after 20161105), there are handy methods on CellStyle for getQuotePrefixed and setQuotePrefixed(boolean)
Your code could then become:
// Do this once for the workbook
CellStyle safeFormulaStyle = workbook.createCellStyle();
safeFormulaStyle.setQuotePrefixed(true);
// Per cell
String cellValue = cell.getStringCellValue();
if( cellValue != null && "=-+#".indexOf(cellValue.charAt(0)) >= 0 ) {
cell.setCellStyle(safeFormulaStyle);
}
Thanks to the instant (kudos) response from the POI team (see accepted answer), this solution should be obsolete. Keeping it as a reference, could be useful in cases an upgrade to POI >= 3.16 is not possible.
Thanks to the comment of Axel Richter (for which I am very-very thankful) I managed to work out a solution. It is definitely NOT as straightforward as in the case of XLSX files (XSSFWorkbook), because it involves creating the org.apache.poi.hssf.model.InternalWorkbook by hand; this class is marked as #Internal by the POI project, but is public as far as Java is concerned. Additionally, the field that is set to correct the problem, i.e. ExtendedFormatRecord.set123Prefix(true) is not documented!
Here is the solution, for what it's worth - compare it with the code in the question:
import java.io.FileOutputStream;
import org.apache.poi.hssf.model.InternalWorkbook;
import org.apache.poi.hssf.record.ExtendedFormatRecord;
import org.apache.poi.hssf.usermodel.HSSFCellStyle;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
public class GoodWorkbookCreator {
public static void main(String[] args) throws Exception {
InternalWorkbook internalWorkbook = InternalWorkbook.createWorkbook();
try(
HSSFWorkbook wb = HSSFWorkbook.create(internalWorkbook);
FileOutputStream fos = new FileOutputStream("C:/workbook-good.xls")
) {
HSSFCellStyle style = (HSSFCellStyle) wb.createCellStyle();
ExtendedFormatRecord xfr = internalWorkbook.getExFormatAt(internalWorkbook.getNumExFormats() - 1);
xfr.set123Prefix(true); // THIS IS WHAT IT IS ALL ABOUT
Sheet sheet = wb.createSheet("Sheet");
Row row = sheet.createRow(0);
row.createCell(0).setCellValue("Aaaaaaaaaa");
row.createCell(1).setCellValue("-2+3 +cmd|'/C calc'!G20");
Cell cell = row.createCell(2);
cell.setCellValue("-2+3 +cmd|'/C calc'!G20");
cell.setCellStyle(style);
wb.write(fos);
}
}
}
This coding is able to read the small data of excel file... but not reading the large data files in excel files.... how to modify the code further?
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.sql.SQLException;
import java.util.Iterator;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
/**
*
* #author Administrator
*/
public class ReadExcelNdArray {
public static void main(String[] args) throws Exception {
long start = System.currentTimeMillis();
System.out.println("Time taken: " + (System.currentTimeMillis() - start) + " ms");
File myFile = new File("D://Raghulpr/Transaction Data.xlsx");
FileInputStream fis = new FileInputStream(myFile);
// Finds the workbook instance for XLSX file
XSSFWorkbook myWorkBook = new XSSFWorkbook (fis);
// Return first sheet from the XLSX workbook
XSSFSheet mySheet = myWorkBook.getSheetAt(0);
// Get iterator to all the rows in current sheet
Iterator<Row> rowIterator = mySheet.iterator();
// Traversing over each row of XLSX file
while (rowIterator.hasNext()) {
Row row = rowIterator.next();
// For each row, iterate through each columns
Iterator<Cell> cellIterator = row.cellIterator();
while (cellIterator.hasNext()) {
Cell cell = cellIterator.next();
switch (cell.getCellType()) {
case Cell.CELL_TYPE_STRING:
System.out.print(cell.getStringCellValue() + "\t");
break;
case Cell.CELL_TYPE_NUMERIC:
System.out.print(cell.getNumericCellValue() + "\t");
break;
case Cell.CELL_TYPE_BOOLEAN:
System.out.print(cell.getBooleanCellValue() + "\t");
break;
default :
}
}
System.out.println("");
}
}
}
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.io.ByteArrayOutputStream.<init>(ByteArrayOutputStream.java:77)
at org.apache.poi.openxml4j.util.ZipInputStreamZipEntrySource$FakeZipEntry.<init>(ZipInputStreamZipEntrySource.java:121)
at org.apache.poi.openxml4j.util.ZipInputStreamZipEntrySource.<init>(ZipInputStreamZipEntrySource.java:55)
at org.apache.poi.openxml4j.opc.ZipPackage.<init>(ZipPackage.java:88)
at org.apache.poi.openxml4j.opc.OPCPackage.open(OPCPackage.java:272)
at org.apache.poi.util.PackageHelper.open(PackageHelper.java:37)
at org.apache.poi.xssf.usermodel.XSSFWorkbook.<init>(XSSFWorkbook.java:254)
at readexcelndarray.ReadExcelNdArray.main(ReadExcelNdArray.java:36)
I don't know if you still need answer to this, but I was also searching for the same and was struggling to read a large file . After spending a lot of time all over the internet I found one solution to this . You can check
Excel streaming reader
import com.monitorjbl.xlsx.StreamingReader;
InputStream is = new FileInputStream(new File("G:\\Book1.xlsx"));
Workbook workbook = StreamingReader.builder()
.rowCacheSize(100)
.bufferSize(4096)
.open(is);
Now you can use workbook to process your file further .
I was able to process xlsx file having more than 4 lac records .
Firstly you need to close all Input - output stream object like FileInputStream etc in your code. Secondly, you can also increase your JVM heap space as mention in this link: Increase heap size in Java
We have jxl api for reading, writing excel files. The problem with this api is at the max you can read and write 65535 rows while starting row is indexed at 0. But it's really flexible.
Since, number of rows are more than 65535 in your case, I would suggest you to prefer Apache POI. Virtually, there is no limit for this api.
You need to increase the heap size so as to read the large files.I suggest using 64bit machine.
I've had the same problem, if you change to the much lower level SAX parsing instead you will save a lot of memory. http://poi.apache.org/spreadsheet/how-to.html#xssf_sax_api
I think I reduced about 4.5 GB(!) memory usage (about 11MB file with a lot of formulas) down to something more manageable (don't remember exactly, but it was so low it didn't matter anymore, at least reduced by a factor of 10).
Harder to implement but worth the time if you need to reduce memory footprint
I am trying to read an excel file using poi apache library. I tried different types of code but still i am getting the same error with all of my codes. I do not know why this error is coming.
You can download POI apache library from this link:
https://poi.apache.org/download.html
Here is my code to read an excel file:
import java.io.File;
import java.io.FileInputStream;
import java.util.Iterator;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
/**
*
* #author Pacer
*/
public class ReadExcelDemo
{
public static void main(String[] args)
{
try
{
System.out.println("Working Directory = " + System.getProperty("user.dir"));
FileInputStream file = new FileInputStream(new File("book.xlsx"));
//Create Workbook instance holding reference to .xlsx file
XSSFWorkbook workbook = new XSSFWorkbook(file);
//Get first/desired sheet from the workbook
XSSFSheet sheet = workbook.getSheetAt(0);
//Iterate through each rows one by one
Iterator<Row> rowIterator = sheet.iterator();
while (rowIterator.hasNext())
{
Row row = rowIterator.next();
//For each row, iterate through all the columns
Iterator<Cell> cellIterator = row.cellIterator();
while (cellIterator.hasNext())
{
Cell cell = cellIterator.next();
//Check the cell type and format accordingly
switch (cell.getCellType())
{
case Cell.CELL_TYPE_NUMERIC:
System.out.print(cell.getNumericCellValue() + "t");
break;
case Cell.CELL_TYPE_STRING:
System.out.print(cell.getStringCellValue() + "t");
break;
}
}
System.out.println("");
}
file.close();
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
And here is the error i am getting:
Working Directory = E:\NetBeansProjects\Project24\CoverageCodetool
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/xmlbeans/XmlException
at coveragecodetool.ReadExcelDemo.main(ReadExcelDemo.java:30)
Caused by: java.lang.ClassNotFoundException: org.apache.xmlbeans.XmlException
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more
Java Result: 1
Please help!
You're missing xmlbeans apache library in your classpath.
Add xmlbeans to your classpath and everything will work.
The library itself can be downloaded here.
The general algorithm of resolving NoClassDefFoundException is the following:
search for the library that uses the class mentioned in the Exception. I prefer this service
add the library to your classpath
try to run the code and see if the problem still persists.
repeat from step one
I am trying to convert xlsx file to csv file using below code
import java.io.*;
import java.util.Iterator;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
public class XLStoCSVConvert {
static void xlsx(File inputFile, File outputFile) {
// For storing data into CSV files
StringBuffer data = new StringBuffer();
try {
FileOutputStream fos = new FileOutputStream(outputFile);
// Get the workbook object for XLSX file
System.out.println("working......1");
XSSFWorkbook wBook = new XSSFWorkbook(new FileInputStream(inputFile));
// Get first sheet from the workbook
System.out.println("working......2");
XSSFSheet sheet = wBook.getSheetAt(0);
Row row;
Cell cell;
// Iterate through each rows from first sheet
Iterator<Row> rowIterator = sheet.iterator();
while (rowIterator.hasNext()) {
row = rowIterator.next();
// For each row, iterate through each columns
Iterator<Cell> cellIterator = row.cellIterator();
while (cellIterator.hasNext()) {
cell = cellIterator.next();
switch (cell.getCellType()) {
case Cell.CELL_TYPE_BOOLEAN:
data.append(cell.getBooleanCellValue() + ",");
break;
case Cell.CELL_TYPE_NUMERIC:
data.append(cell.getNumericCellValue() + ",");
break;
case Cell.CELL_TYPE_STRING:
data.append(cell.getStringCellValue() + ",");
break;
case Cell.CELL_TYPE_BLANK:
data.append("" + ",");
break;
default:
data.append(cell + ",");
}
}
}
fos.write(data.toString().getBytes());
fos.close();
} catch (Exception ioe) {
ioe.printStackTrace();
}
}
public static void main(String[] args) {
File inputFile = new File("/home/raptorjd4/Desktop/ToConsult.xlsx");
//writing excel data to csv
File outputFile = new File("/home/raptorjd4/Desktop/RaptorTrackingSystem/ToConsult.csv");
xlsx(inputFile, outputFile);
}
}
But i am getting output,
Working......1
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/poi/UnsupportedFileFormatException
My jar in lib folder,
poi-3.5-FINAL.jar
poi-ooxml-3.11.jar
why i am getting this error when i mapped all needed jar file in lib folder.
Where am i doing mistake?
For me your code worked just fine, below are the dependencies I added:-
The problem is either with the data in your excel or some missing dependency.
Ensure that you do have all the necessary dependencies on your classpath.
Your problem is this part:
poi-3.5-FINAL.jar
poi-ooxml-3.11.jar
As explained in this Apache POI FAQ entry:
Can I mix POI jars from different versions?
No. This is not supported.
All POI jars in use must come from the same version. A combination such as poi-3.11.jar and poi-ooxml-3.9.jar is not supported, and will fail to work in unpredictable ways.
You must ensure that all of your Apache POI jars come from the same version!
Switch your jars to be from the same version, ideally the latest, and you should be good
I solved this problem by adding below jar files,
poi-3.9.jar
poi-ooxml-3.9.jar
poi-ooxml-schemas-3.9-20121203.jar
xmlbeans-2.3.0.jar
dom4j-1.6.1.jar
I'm trying to get updated cell values after use setForceFormulaRecal method. But I'm getting still old values. Which is not actual result. If I opened Original file by clicking It will asking update Links dialogue box. If I click "ok" button then Its updating all cell formula result. So I want to update excel sheet links before its open by using poi. Please help in this situation.
//Before Setting values
HSSFCell cel2=row1.getCell(2);
HSSFCell cel4=row1.getCell(5);
cel2.setCellValue(690);
cel4.setCellValue(690);
wb.setForceFormulaRecalculation(true);
wb.write(stream);
//After Evaluatting the work book formulas I'm trying as follow
HSSFWorkbook wb = HSSFReadWrite.readFile("D://workspace//ExcelProject//other.xls");
HSSFSheet sheet=wb.getSheetAt(14);
HSSFRow row11=sheet.getRow(10);
System.out.println("** cell val: "+row11.getCell(3).getNumericCellValue());
I'm Also tried with Formula Evaluator But its showing errors As follow
Could not resolve external workbook name '\Users\asus\Downloads\??? & ???? ?????_091230.xls'. Workbook environment has not been set up.
at org.apache.poi.ss.formula.OperationEvaluationContext.createExternSheetRefEvaluator(OperationEvaluationContext.java:87)
at org.apache.poi.ss.formula.OperationEvaluationContext.getArea3DEval(OperationEvaluationContext.java:273)
at org.apache.poi.ss.formula.WorkbookEvaluator.getEvalForPtg(WorkbookEvaluator.java:660)
at org.apache.poi.ss.formula.WorkbookEvaluator.evaluateFormula(WorkbookEvaluator.java:527)
at org.apache.poi.ss.formula.WorkbookEvaluator.evaluateAny(WorkbookEvaluator.java:288)
at org.apache.poi.ss.formula.WorkbookEvaluator.evaluate(WorkbookEvaluator.java:230)
at org.apache.poi.hssf.usermodel.HSSFFormulaEvaluator.evaluateFormulaCellValue(HSSFFormulaEvaluator.java:351)
at org.apache.poi.hssf.usermodel.HSSFFormulaEvaluator.evaluateFormulaCell(HSSFFormulaEvaluator.java:213)
at org.apache.poi.hssf.usermodel.HSSFFormulaEvaluator.evaluateAllFormulaCells(HSSFFormulaEvaluator.java:324)
at org.apache.poi.hssf.usermodel.HSSFFormulaEvaluator.evaluateAll(HSSFFormulaEvaluator.java:343)
at HSSFReadWrite.readSheetData(HSSFReadWrite.java:85)
at HSSFReadWrite.main(HSSFReadWrite.java:346)
Caused by: org.apache.poi.ss.formula.CollaboratingWorkbooksEnvironment$WorkbookNotFoundException: Could not resolve external workbook name '\Users\asus\Downloads\??? & ???? ?????_091230.xls'. Workbook environment has not been set up.
at org.apache.poi.ss.formula.CollaboratingWorkbooksEnvironment.getWorkbookEvaluator(CollaboratingWorkbooksEnvironment.java:161)
at org.apache.poi.ss.formula.WorkbookEvaluator.getOtherWorkbookEvaluator(WorkbookEvaluator.java:181)
at org.apache.poi.ss.formula.OperationEvaluationContext.createExternSheetRefEvaluator(OperationEvaluationContext.java:85)
... 11 more
OK, trying an answer:
First of all: Support for links to external workbooks is not included into the current stable version 3.10. So with this version it is not possible to evaluate such links directly. That's why evaluateAll() will fail for workbooks with links to external workbooks.
With Version 3.11 it will be possible to do so. But also only even if all the workbooks are opened and Evaluators for all the workbooks are present. See: http://poi.apache.org/apidocs/org/apache/poi/ss/usermodel/FormulaEvaluator.html#setupReferencedWorkbooks%28java.util.Map%29
What we can do with the stable version 3.10, is to evaluate all the cells which contains formulas which have not links to external workbooks.
Example:
The workbook "workbook.xlsx" contains a formula with a link to an external workbook in A2:
import org.apache.poi.xssf.usermodel.*;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.ss.util.*;
import org.apache.poi.openxml4j.exceptions.InvalidFormatException;
import java.io.FileOutputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.FileInputStream;
import java.io.InputStream;
import java.util.Map;
import java.util.HashMap;
class ExternalReferenceTest {
public static void main(String[] args) {
try {
InputStream inp = new FileInputStream("workbook.xlsx");
Workbook wb = WorkbookFactory.create(inp);
Sheet sheet = wb.getSheetAt(0);
Row row = sheet.getRow(0);
if (row == null) row = sheet.createRow(0);
Cell cell = row.getCell(0);
if (cell == null) cell = row.createCell(0);
cell.setCellValue(123.45);
cell = row.getCell(1);
if (cell == null) cell = row.createCell(1);
cell.setCellValue(678.90);
cell = row.getCell(2);
if (cell == null) cell = row.createCell(2);
cell.setCellFormula("A1+B1");
FormulaEvaluator evaluator = wb.getCreationHelper().createFormulaEvaluator();
//evaluator.evaluateAll(); //will not work because external workbook for formula in A2 is not accessable
System.out.println(sheet.getRow(1).getCell(0)); //[1]Sheet1!$A$1
//but we surely can evaluate single cells:
cell = wb.getSheetAt(0).getRow(0).getCell(2);
System.out.println(evaluator.evaluate(cell).getNumberValue()); //802.35
FileOutputStream fileOut = new FileOutputStream("workbook.xlsx");
wb.write(fileOut);
fileOut.flush();
fileOut.close();
} catch (InvalidFormatException ifex) {
} catch (FileNotFoundException fnfex) {
} catch (IOException ioex) {
}
}
}