Develop Reference

How to add columns to an existing large excel file using SXSSF Apache POI?

I am working with a large excel file ( larger than 40 Mb , more than 100k rows and 50 columns ). I am successfully reading it using POI ( 3.10.1 version ) event stream and then doing some calculation and storing result into a List.
Now I have to append this List as a column in the same file. In this part I am facing issue.
I have tried to achieve this by using the below code
FileInputStream excelFile = new FileInputStream(new File(pathToFile));
Workbook workbook = new XSSFWorkbook(excelFile);
Sheet datatypeSheet = workbook.getSheetAt(0); // Get first sheet
Iterator<Row> iterator = datatypeSheet.iterator();
int i=0;
while (iterator.hasNext()) { // Loop over each row
Row currentRow = iterator.next();
Cell cell = currentRow.createCell(currentRow.getLastCellNum());
cell.setCellType(Cell.CELL_TYPE_STRING);
if(currentRow.getRowNum() == 0)
cell.setCellValue("OUTPUT-COLUMN"); // set column header for the new column
else {
cell.setCellValue(list.get(i)); // list contains the output to populate in new column
i++;
}
}
FileOutputStream fos = new FileOutputStream(new File(pathToOutput));
workbook.write(fos);
fos.close();
It is working fine with smaller files But the issue is that I am getting Out of memory for the larger files. Now I tried to modify this and use SXSSF in place of XSFF to get over the memory issue (See below code). But while testing even for smaller files I am getting output file same as the input file.
FileInputStream excelFile = new FileInputStream(new File(pathToFile));
XSSFWorkbook xwb = new XSSFWorkbook(inputStream);
inputStream.close();
SXSSFWorkbook wb = new SXSSFWorkbook(xwb,100);
wb.setCompressTempFiles(true);
SXSSFSheet sh = (SXSSFSheet) wb.getSheetAt(0);
Iterator<Row> iterator = datatypeSheet.iterator();
int i=0;
while (iterator.hasNext()) { // Loop over each row
Row currentRow = iterator.next();
Cell cell = currentRow.createCell(currentRow.getLastCellNum());
cell.setCellType(Cell.CELL_TYPE_STRING);
if(currentRow.getRowNum() == 0)
cell.setCellValue("OUTPUT-COLUMN"); // set column header for the new column
else {
cell.setCellValue(list.get(i)); // list contains the output to populate in new column
i++;
}
}
FileOutputStream fos = new FileOutputStream(new File(pathToOutput));
wb.write(fos);
fos.close();
Using a db is not suitable in my use case and i want to avoid using a temporary data structure to hold data for writing due to memory constraint.
Is there a way to write in output workbook while streaming ? Here is the code that I am using to read using POI Streaming API
private class ExcelData implements SheetContentsHandler {
LinkedHashMap<Strin, String> rowMap;
public void startRow(int rowNum) {
}
public void endRow(int rowNum) {
// Process the row
// Handle write to output workbook ??
}
public void cell(String cellReference, String formattedValue,
XSSFComment comment) {
// Save current row in rowMap ( column name => cell value )
}
public void headerFooter(String text, boolean isHeader, String tagName)
{
}
}

It is not possible to add column to existing workbook using POI SXSSF. It only allows addition of new rows.
The only solution is to read the existing workbook and write to a new workbook with the added column.
To achieve this we can store the rows in a data structure or database in the endrow() method and then use the persisted data to write a new workbook.

How can I load CSV file into Excel sheet using Java

I have an Excel spreadsheet that has the first sheet designated for the raw data. There are 3 more sheets that are coded to transform and format the data from the raw sheet. The fifth sheet has the final output.
How can I use Java:
load the data from the CSV file into the first sheet of the excel file?
save the data from the 5th sheet into the new CSV file.
Also, if the original CSV has thousands of rows, I assume the multi-sheet transformations would take some time before the 5th sheet gets all the final data - is there a way to know?

I would follow this approach:
Load the specific .csv file and prepare to read it with Java
Load the .xlsx file and change it according to your requirements and the data that you get from the .csv file. A small example of how an excel file is changed with Apache POI can be seen below:
try
{
HashMap<Integer, ArrayList<String>> fileData; // This for example keeps the data from the csv in this form ( 0 -> [ "Column1", "Column2" ]...)
// Working with the excel file now
FileInputStream file = new FileInputStream("Data.xlsx");
XSSFWorkbook workbook = new XSSFWorkbook(file); // getting the Workbook
XSSFSheet sheet = workbook.getSheetAt(0);
Cell cell = null;
AtomicInteger row = new AtomicInteger(0);
fileData.forEach((key, csvRow) ->
{
//Update the value of the cell
//Retrieve the row and check for null
HSSFRow sheetRow = sheet.getRow(row);
if(sheetRow == null)
{
sheetRow = sheet.createRow(row);
}
for (int i = 0; i < csvRow.size(); i++)
{
//Update the value of cell
cell = sheetRow.getCell(i);
if(cell == null){
cell = sheetRow.createCell(i);
}
cell.setCellValue(csvRow.get(i));
}
});
file.close();
FileOutputStream outFile =new FileOutputStream(new File("Data.xlsx"));
workbook.write(outFile);
outFile.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
After saving the .xlsx file, you can create the .csv file by following this question.

In java eclipse, i retrieved data from the database and pasted in the already present excel. The data already present is getting deleted

In java eclipse, I ran a query to retrieve data from the database and pasted in the already present excel sheet (exceldatabase16) in the C drive. Already present excel -exceldatabase16 contains column names in the first row. And the data pasted into the exceldatabase16 is pasted from row 6 and column F. Still the row 1 data is getting deleted.
public static void retrieveData1( Connection connection) throws SQLException, IOException
{
Statement stmt = null;
ResultSet rs = null;
stmt = connection.createStatement();
rs = stmt.executeQuery("SELECT * FROM countries where region_id='3' ");
// getColumnNames(rs);
XSSFWorkbook workbook = new XSSFWorkbook();
XSSFSheet spreadsheet = workbook.createSheet("countriesdetails");
XSSFRow row = spreadsheet.createRow(5);
XSSFCell cell;
cell = row.createCell(5);
cell.setCellValue("country id");
cell = row.createCell(6);
cell.setCellValue("country name");
cell = row.createCell(7);
cell.setCellValue("region");
int i = 6;
while(rs.next()) {
row = spreadsheet.createRow(i);
cell = row.createCell(5);
cell.setCellValue(rs.getString(1));
cell = row.createCell(6);
cell.setCellValue(rs.getString(2));
cell = row.createCell(7);
cell.setCellValue(rs.getInt(3));
i++;
}
FileOutputStream out = new FileOutputStream(new File("C:\\Users\\lenovo\\workspace\\ApachePoi\\exceldatabase16.xlsx"));
workbook.write(out);
out.close();
workbook.close();
System.out.println("exceldatabase.xlsx written successfully");
}

It looks like you are overwriting your existing file with a new one whenever you run this program.
According to the XSSF documentation, there is a method signature of the XSSFWorkbook creation which can open an existing file instead of making a new one.
Try something along the lines of:
XSSFWorkbook workbook = new XSSFWorkbook(new FileInputStream("C:\\Users\\lenovo\\workspace\\ApachePoi\\exceldatabase16.xlsx"));

Export both Image file and String from SQLite database into Excel using Apache POI

I am trying to export my SQLite database into a Excel file using Apache POI. I can export all string and double type values. But there have images(Blob files) on it. How can I export them together on the same Excel file and import them back?
Here is the code what I was implementing to store String and Double type values:
String query = "select * from CarOwnerDB";
preparedStatement = connection.prepareStatement(query);
rs = preparedStatement.executeQuery();
XSSFWorkbook wb = new XSSFWorkbook();
XSSFSheet sheet = wb.createSheet("Car Owner Details");
XSSFRow header = sheet.createRow(0);
header.createCell(0).setCellValue("Name");
header.createCell(1).setCellValue("Username");
header.createCell(2).setCellValue("Password");
header.createCell(3).setCellValue("Company ID");
header.createCell(4).setCellValue("NID");
header.createCell(5).setCellValue("Bank Account Number");
header.createCell(6).setCellValue("Bank Name");
header.createCell(7).setCellValue("Permanent Address");
header.createCell(8).setCellValue("Present Address");
header.createCell(9).setCellValue("Phone");
header.createCell(10).setCellValue("Email");
header.createCell(11).setCellValue("Date Of Birth");
header.createCell(12).setCellValue("Age");
header.createCell(13).setCellValue("Gender");
header.createCell(14).setCellValue("Maritial Status");
header.createCell(15).setCellValue("Profile Picture");
header.createCell(16).setCellValue("Balance");
int index = 1;
while (rs.next())
{
XSSFRow row = sheet.createRow(index);
row.createCell(0).setCellValue(rs.getString("name"));
row.createCell(1).setCellValue(rs.getString("username"));
row.createCell(2).setCellValue(rs.getString("password"));
row.createCell(3).setCellValue(rs.getString("comID"));
row.createCell(4).setCellValue(rs.getString("nid"));
row.createCell(5).setCellValue(rs.getString("bankAccountNumber"));
row.createCell(6).setCellValue(rs.getString("bankName"));
row.createCell(7).setCellValue(rs.getString("permanentAddress"));
row.createCell(8).setCellValue(rs.getString("presentAddress"));
row.createCell(9).setCellValue(rs.getString("phone"));
row.createCell(10).setCellValue(rs.getString("email"));
row.createCell(11).setCellValue(rs.getString("dob"));
row.createCell(12).setCellValue(rs.getInt("age"));
row.createCell(13).setCellValue(rs.getString("gender"));
row.createCell(14).setCellValue(rs.getString("maritialStatus"));
row.createCell(15).setCellValue(rs.getBlob("profilePicture"));
row.createCell(16).setCellValue(rs.getDouble("balance"));
index++;
}
FileOutputStream fout = new FileOutputStream(
System.getProperty("user.home") + "/Desktop/" + "Car Owner Database.xlsx");
wb.write(fout);
fout.close();
preparedStatement.close();
rs.close();
In the line
row.createCell(15).setCellValue(rs.getBlob("profilePicture"));
I have image file. How can I store and retrieve those on the same Excel sheet?
Now I know how to get byte array with the image data from a SQL blob field. But don't know how to store them all using loop and array (To store string values here I used "XSSFRow row = sheet.createRow(index)").
Now I am able to store one image file by implementing this code:
InputStream is = new FileInputStream("a.png");
byte[] bytes = IOUtils.toByteArray(is);
int pictureIdx = wb.addPicture(bytes, Workbook.PICTURE_TYPE_PNG);
is.close();
CreationHelper helper = wb.getCreationHelper();
XSSFSheet sheet1 = wb.createSheet("Car Owner Pictures");
Drawing drawing = sheet1.createDrawingPatriarch();
ClientAnchor anchor = helper.createClientAnchor();
anchor.setCol1(3);
anchor.setRow1(2);
Picture pict = drawing.createPicture(anchor, pictureIdx);
pict.resize();
String FileName = "Car Owner Database.xlsx";
FileOutputStream fout = new FileOutputStream(Main.getDirectory() + "/" + FileName);
wb.write(fout);
fout.close();
How can I add multiple images(same number I have created row)?

Problems with starting a new row in Excel with Apache POI

I'm using the apache poi package to create a spreadsheet of figures which represent features of a shape (area, perimeter, centroid). The problem is that i have a method: writeDatabase() which outputs the features of the shape as they are found, the output spreadsheet looks like this:
http://s23.postimg.org/hqsfg76jv/Capture.png
All of these figures need to be in the same line, and then a new line needs to be taken for the next record. the writeDatabase method is shown below
public static void writeDatabase(int value, int cellNum){
try {
Cell cell1=null;
FileInputStream file = new FileInputStream(new File("features.xls"));
Workbook workbook = new HSSFWorkbook(file);
Sheet sheet = workbook.getSheetAt(0);
int lastRow = sheet.getPhysicalNumberOfRows();
cell1 = sheet.createRow(lastRow).createCell(cellNum);
cell1.setCellValue(value);
FileOutputStream outFile =new FileOutputStream(new File("features.xls"));
workbook.write(outFile);
outFile.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
I think the problem is with this line being called each time, but i cant think of an alternative:
int lastRow = sheet.getPhysicalNumberOfRows();
Any ideas?

You need to provide some indication of whether a new line should be started or not (unless you can tell that a new line is starting simply because cellNum is 0?)
Then, you can either create a new row, or use the existing row:
int lastRow = sheet.getPhysicalNumberOfRows();
Row row;
if (startNewRow) {
row = sheet.createRow(lastRow);
} else {
row = sheet.getRow(lastRow - 1);
}
cell1 = row.createCell(cellNum);
cell1.setCellValue(value);
Where startNewRow might be set based on cellNum, or might be an additional parameter that is passed into writeDatabase, whichever is appropriate.