My program generate some Excel using apache POI.
This part take up 50% of CPU for 1 excel..
If users generate at the same time multiple excel the server is out...
How can i reduce it ?
final Workbook workbook = new HSSFWorkbook();
initDataFormat(workbook);
EnumMap<BomXlsSheetEnum, Sheet> sheetMap = sheetMap = BomXlsSheetEnum.getSheetMap(workbook);
buildBomSheet(sheetMap, bom, productModelIds, listSizes, false);
workbook.write(response.getOutputStream());
Related
I'm using Apache Poi in java to copy an excel sheet to the same workbook This way :
FileInputStream file = new FileInputStream(new File("exemple.xlsx"));
XSSFWorkbook workbook = new XSSFWorkbook(file);
XSSFSheet sheet_copy = workbook.cloneSheet(0);
int num = workbook.getSheetIndex(sheet_copy);
workbook.setSheetName(num, "copy_file");
FileOutputStream outputStream = new FileOutputStream("exemple.xlsx");
workbook.write(outputStream);
the file "sheet_copy" is created as well , but it doesn't have the same scale as the first one ,
The first one has 59% and the second one 100% when i try to save it as pdf it's too much big, how to solve that in code ? i mean that sheet_copy gets the same format as the original one , i don't want to do it manually .
I am using Apache POI to read the data only in the first sheet of an excel file. The xlsx files that are submitted usually have only 1 sheet and are around 2.5MB (with a little more than 130k rows of data), and everything goes slow but smooth with no errors. However, if the submitted xlsx has more than one sheet, and if the other sheet(s) also have a lot of data in them, the execution throws an OutOfMemoryError: Java heap space error. Now I am trying to figure out if it somehow possible to always only read the data on the first sheet without worrying about the memory errors (i am running this with -Xmx1024m -Xms512m arguments)
EDIT: here is my code
InputStream inputStream = new FileInputStream(new File(excelfile));
XSSFWorkbook workbook = new XSSFWorkbook(inputStream);
if (workbook.getNumberOfSheets() != 1) {
throw new Exception("Make sure excel only has 1 sheet");
}
The program is throwing an error on the second line (if the excel file has a lot of data on the second sheet as well)
Apache POI usually triggers a lot of issues related to memory, I strongly recommend to use monitorjbs instead https://github.com/monitorjbl/excel-streaming-reader
InputStream is = new FileInputStream(new File(filePath));
Workbook workbook = StreamingReader.builder()
.rowCacheSize(100) // number of rows to keep in memory (defaults to 10)
.bufferSize(2048) // buffer size to use when reading InputStream to file (defaults to 1024)
.open(is)) {
Sheet sheet = workbook.getSheetAt(0);
I have an .xlsx file with multiple sheets containing different data. Of all the sheets one sheet needs to accommodate close to 100,000 rows of data, and the data needs to be written using Java with poi.
This seems quite fast and simple with SXSSFWorkbook, where I can keep only 100 rows in memory, but the disadvantage is that I can only write to a new file (or overwrite existing file).
Also, I am not allowed to 'load' an existing file, i.e
SXSSFWorkbook wb = new SXSSFWorkbook(file_input_stream) is not allowed.
I can use Workbook factory:
Workbook workbook = new SXSSFWorkbook();
workbook = WorkbookFactory.create(file_input_stream);
but when the time comes for me to flush the rows,
((SXSSFSheet)sheet).flushRows(100);
I get the error that type conversion is not allowed from XSSFSheet to SXSSFSheet.
I tried to see if there was any way to copy sheets across different workbooks, but so far it seems it has to be done cell by cell.
Any insights on how to approach this problem?
You are probably having a template to which you want to add large data. You need to use the SXSSFWorkbook(XSSFWorkbook) constructor:
XSSFWorkbook wb = new XSSFWorkbook(new File("template.xlsx"));
SXSSFWorkbook wbss = new SXSSFWorkbook(wb, 100);
Sheet sheet = wbss.createSheet("sheet1");
// now add rows to sheet
I'm generating an Excel file in Java with POI 3.2 (Latest version I can use for my client).
Here is my code. As you can see I'm using HSSF because I need to make a XLS file.
HSSFWorkbook wb = new HSSFWorkbook();
HSSFSheet sheet = wb.createSheet("Reporting");
sheet.setColumnWidth(250,250);
HSSFRow Row;
HSSFCell Cell;
//Content part (doesn't matter)
IWDResource resource = WDResourceFactory.createCachedResource(
wb.getBytes(),
"workbook.xls",
WDWebResourceType.XLS);
wdContext.currentContextElement().setXls(resource);
Now after I've downloaded the XLS file, I want to open it but the file seems to be corrupt.
Image on link: http://tinyurl.com/nop52sh
When I'm pressing 2 times on 'Don't send', de excell file opens in correct form.
Any idea why?
Don't call wb.getBytes(), it doesn't do what you want. From the getBytes() javadoc
Method getBytes - get the bytes of just the HSSF portions of the XLS file. Use this to construct a POI POIFSFileSystem yourself.
Instead, if you want the overall xls file as a byte array, do
ByteArrayOutStream baos = new ByteArrayOutStream();
wb.write(baos);
byte[] xlsBytes = baos.toByteArray();
Finally, Apache POI 3.2 is ancient, over 5 years old now! You really ought to upgrade, see the changelog for an idea of all the bugs fixed since then
I have a data of almost 100,000 records and I am trying to write the data to .xlsx file using XSSFWorkbook through Java code. I am able to fetch all the data from database to an ArrayList. By iterating the ArryList, I am writing the data to .xlsx file cell by cell.
As it reaches to 8000 rows, java code throws Out of Memory Heap Space Error.
I have read somewhere that SXSSFWorkbook will be lighter when compared to XSSFWorkbook, so I tried using SXSSFWorkbook. But still I am facing the same problem.
So is there anything that I am missing with the Workbooks or with my Java Code??
Initially, when I have 60,000 records data, I had used .xls file. The same java code is able to generate the .xls file with HSSFWorkbook.
Increasing the Java Heap Space is not at all an option as my data will be increased tremendously in future.
Any help will be greatly appreciated.
Small piece of code, the way I am writing the data to Excel.
int rowNum = sheet.getLastRowNum();
Row lastRow = null ;
Cell cell = null;
ReportingHelperVo reportingHelperVo = null;
for (ReportingVo reportingVo : reportingVos) {
rowNum++;
lastRow = sheet.createRow(rowNum);
reportingHelperVo = reportingVo.reportingHelperVo;
cell = lastRow.createCell(0);
cell.setCellValue(reportingHelperVo.getLocation());
cell.setCellStyle(style);
cell = lastRow.createCell(1);
cell.setCellValue(reportingHelperVo.getCity());
cell.setCellStyle(style);
cell = lastRow.createCell(2);
cell.setCellValue(reportingHelperVo.getCountry());
cell.setCellStyle(style);
}
SXSSFWorkbook is not like light weight,but there is a advantage with this.
If you declare as
SXSSFWorkbook workbook= new SXSSFWorkbook(200);
then for every 200 rows written on the workbook, memory will be flushed to diskspace so there will be no burden in heapspace.
XSSFWorkbook - creates an object representation for all Excel documents (should work like DOM).
SXSSFWorkbook - should require constant memory. When is OOM thrown by JMV? What type of ResultSet did you use? Try to use FORWARD_ONLY to restrict caching data by JDBC driver retrieved from DB.
BTW best weay to fix OutOfMemoryError is to analyze heap dump.
Use -XX:+HeapDumpOnOutOfMemoryError parameter and MAT to understand how your application works.
I am writing the data to .xlsx file cell by cell. As it reaches to
8000 rows, java code throws Out of Memory Heap Space Error.
Re-use exsiting java objects, instead of creating new ones each iteration.
And/or use a csv file instead of excel.
Workbook workBook = new SXSSFWorkbook();
You can export more than 1 lakh (100000) records.
Page results from the database rather than reading them all in one go.
I had similar problems long time ago attempting to write from R to an excel file (but using XLConnection).
In the end, I solved by using write.csv and then opening it with Excel and using the botton "Text to column".
It is increadibly fast and reliable.
I have got the same issue when my excel was reaching 3000 lines. In my case the main memory related issue with POI Excel generation happens with the style sheet. Following are the things which I removed from my code.
Try to use style sheet setting in a row level.
If at all you need to set the style sheet for every cell.. avoid setting border for each and every cell.
Hi Use latest Apache POI JAR//And Use SXSSF for streaming or downloading
SXSSFWorkbook workbook = new SXSSFWorkbook(100);
workbook.setCompressTempFiles(true);
Sheet sh = workbook.createSheet();
((SXSSFSheet) sh).setRandomAccessWindowSize(100);
//write your logic
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-Disposition", "attachment;
filename="+filename+".xlsx");
workbook.write(response.getOutputStream());
workbook.close();
workbook.dispose();
I had an out of memory issues writing a XSSFWorkbook to a file.
The Suggestions above helped a lot.
See http://poi.apache.org/components/spreadsheet/how-to.html#xssf_sax_api
Changing XSSFWorkbook wb = new XSSFWorkbook();
to SXSSFWorkbook wb = new SXSSFWorkbook(-1);
SXSSFSheet sh = ... to SXSSFSheet sh = ...
XSSFRow to SXSSFRow
XSSFCell to SXSSFCell
inside the for loop USE sh.flushRows(100); for every 100th row
after wb.write(out);
ADD wb.dispose();