My requirement is to read the excel dynamically and store the contents in Map. The headers should be key and the respective column should be value so that when I pass the key, I can get the cell value.
My Excel sheet contains multiple rows and columns which can be changed dynamically. My intention is to write a common function in java to read an excel and store the contents in map. The Key should be column headers and the value should be respective cell value of the header. So while retrieving when I pass the header key, I should receive the respective cell value. My plan is to pass the row number and the key in a method so that it can retrieve the respective cell value related to the row as well as the key
static Map<String> excelMap = new LinkedHashMap<String>();
public static String readWriteExcel(String sheetName) throws EncryptedDocumentException, InvalidFormatException, IOException, JSONException
{
File file = new File("File Path");
FileInputStream inputStream = new FileInputStream( file );
Workbook workbook = WorkbookFactory.create( inputStream );
Sheet sheet = workbook.getSheet( sheetName );
int rowNumber =0;
int rowCount = sheet.getLastRowNum();
for(int i=1;i<=rowCount;i++){
Row row = sheet.getRow(0);
for(int j=0;j<row.getLastCellNum();j++) {
String key=row.getCell(j).getStringCellValue();
String value=sheet.getRow(rowNumber+1).getCell(j).getStringCellValue();
excelMap.put(key, value);
}
}
}
By looking at the code, i am assuming the issues: Multiple rows present in the sheet, but you are getting data of first row only.
Solution :
First of all, you need to store all rows data in a List of Map. where list index corresponds to the row number. Also You are not incrementing the rowNumber variable anywhere. Its always 0.
Why not directly use the variable i to get a particular row from sheet?
I think this should work.
static List<Map<String, String>> excelData = new ArrayList<HashMap<String, String>>();
public static String readWriteExcel(String sheetName) throws EncryptedDocumentException, InvalidFormatException, IOException, JSONException
{
File file = new File("File Path");
FileInputStream inputStream = new FileInputStream( file );
Workbook workbook = WorkbookFactory.create( inputStream );
Sheet sheet = workbook.getSheet( "sheetName" );
int rowCount = sheet.getLastRowNum();
for(int i=1;i<=rowCount;i++){
Row row = sheet.getRow(0);
Map<String,String> rowData = new HashMap<>();
for(int j=0;j<row.getLastCellNum();j++) {
String key=row.getCell(j).getStringCellValue();
String value=sheet.getRow(i).getCell(j).getStringCellValue();
rowData.put(key, value);
}
excelData.add(rowData);
}
}
Since you want to map multiple rows you have to store multiple rows.
One way is storing a map consisting column->list containing values for each row.
Another way is storing a map consisting column+row number->value.
Sample code for second method:
static Map<String, String> excelMap = new LinkedHashMap<String, String>();
...
// storing values
excelMap.put(key + "_" + i, value); // i is row index
...
// getting values
public static String getCellValue(String cellName, int rowIndex) {
return excelMap.get(cellName + "_" + rowIndex);
}
Related
I am working with a large excel file ( larger than 40 Mb , more than 100k rows and 50 columns ). I am successfully reading it using POI ( 3.10.1 version ) event stream and then doing some calculation and storing result into a List.
Now I have to append this List as a column in the same file. In this part I am facing issue.
I have tried to achieve this by using the below code
FileInputStream excelFile = new FileInputStream(new File(pathToFile));
Workbook workbook = new XSSFWorkbook(excelFile);
Sheet datatypeSheet = workbook.getSheetAt(0); // Get first sheet
Iterator<Row> iterator = datatypeSheet.iterator();
int i=0;
while (iterator.hasNext()) { // Loop over each row
Row currentRow = iterator.next();
Cell cell = currentRow.createCell(currentRow.getLastCellNum());
cell.setCellType(Cell.CELL_TYPE_STRING);
if(currentRow.getRowNum() == 0)
cell.setCellValue("OUTPUT-COLUMN"); // set column header for the new column
else {
cell.setCellValue(list.get(i)); // list contains the output to populate in new column
i++;
}
}
FileOutputStream fos = new FileOutputStream(new File(pathToOutput));
workbook.write(fos);
fos.close();
It is working fine with smaller files But the issue is that I am getting Out of memory for the larger files. Now I tried to modify this and use SXSSF in place of XSFF to get over the memory issue (See below code). But while testing even for smaller files I am getting output file same as the input file.
FileInputStream excelFile = new FileInputStream(new File(pathToFile));
XSSFWorkbook xwb = new XSSFWorkbook(inputStream);
inputStream.close();
SXSSFWorkbook wb = new SXSSFWorkbook(xwb,100);
wb.setCompressTempFiles(true);
SXSSFSheet sh = (SXSSFSheet) wb.getSheetAt(0);
Iterator<Row> iterator = datatypeSheet.iterator();
int i=0;
while (iterator.hasNext()) { // Loop over each row
Row currentRow = iterator.next();
Cell cell = currentRow.createCell(currentRow.getLastCellNum());
cell.setCellType(Cell.CELL_TYPE_STRING);
if(currentRow.getRowNum() == 0)
cell.setCellValue("OUTPUT-COLUMN"); // set column header for the new column
else {
cell.setCellValue(list.get(i)); // list contains the output to populate in new column
i++;
}
}
FileOutputStream fos = new FileOutputStream(new File(pathToOutput));
wb.write(fos);
fos.close();
Using a db is not suitable in my use case and i want to avoid using a temporary data structure to hold data for writing due to memory constraint.
Is there a way to write in output workbook while streaming ? Here is the code that I am using to read using POI Streaming API
private class ExcelData implements SheetContentsHandler {
LinkedHashMap<Strin, String> rowMap;
public void startRow(int rowNum) {
}
public void endRow(int rowNum) {
// Process the row
// Handle write to output workbook ??
}
public void cell(String cellReference, String formattedValue,
XSSFComment comment) {
// Save current row in rowMap ( column name => cell value )
}
public void headerFooter(String text, boolean isHeader, String tagName)
{
}
}
It is not possible to add column to existing workbook using POI SXSSF. It only allows addition of new rows.
The only solution is to read the existing workbook and write to a new workbook with the added column.
To achieve this we can store the rows in a data structure or database in the endrow() method and then use the persisted data to write a new workbook.
I've the following code where fetching of data from excel sheet is done and then this data is added to list as below:
List<HashMap<String,String>> mydata = new ArrayList<>();
try
{
FileInputStream fs = new FileInputStream(filepath);
XSSFWorkbook workbook = new XSSFWorkbook(fs);
XSSFSheet sheet = workbook.getSheet(sheetName);
Row HeaderRow = sheet.getRow(0);
for(int i=1;i<sheet.getPhysicalNumberOfRows();i++)
{
Row currentRow = sheet.getRow(i);
HashMap<String,String> currentHash = new HashMap<String,String>();
for(int j=0;j<currentRow.getPhysicalNumberOfCells();j++)
{
Cell currentCell = currentRow.getCell(j);
switch (currentCell.getCellType())
{
case Cell.CELL_TYPE_STRING:
currentHash.put(HeaderRow.getCell(j).getStringCellValue(), currentCell.getStringCellValue());
break;
}
}
mydata.add(currentHash);
I've another class where we are making use Properties class and setting the property :
public List<HashMap<String,String>> datamap;
public static Properties prop;
public void read_Data_FromExcel(String arg1) throws Throwable {
datamap = DataHelper.data("C:/FINALORDER/ORDERING_Tools/ordering-tools/src/test/resources/Test451.xlsx","Sheet1");
prop=new Properties();
int index = Integer.parseInt(arg1)-2;
FileInputStream fs = new FileInputStream("C:\\FINALORDER\\ORDERING_Tools\\ordering-tools\\src\\test\\resources\\Test451.xlsx");
XSSFWorkbook workbook = new XSSFWorkbook(fs);
XSSFSheet sheet = workbook.getSheet("Sheet1");
Row HeaderRow = sheet.getRow(0);
for(int i=0;i<HeaderRow.getPhysicalNumberOfCells();i++) {
prop.setProperty(HeaderRow.getCell(i).toString(), datamap.get(index).get(HeaderRow.getCell(i).toString()));
Using the above code i m fetching the data from excel and then inputting that data into the fields present in the screen like below:
try {
String remark=ExcelSteps.prop.getProperty("Remark");
mosdpOrderSummaryEditPage.tbx_remarks.type(remark);
}catch(Exception e) {
}
The issue is if there are many fields like remark,name,address,id then i have to fetch them one by one using the above code as for remark which is sort of redundant work and increasing code unnecessarily , is there a way by which i can reduce the code or make any generic method.
You can create separate column and get those through
ExcelSteps.prop.getProperty("Here you should pass the column data");
By this code will automatically get remark,name,address,id field names dynamically.
:)
I have a problem with adding headers to the newly created .xlsx file
I would like to see the first header = 'UserName' and the second header = 'Password'. Can anyone help me on how to do this?
XSSFWorkbook new_workbook = new XSSFWorkbook(); // create a blank workbook object
XSSFSheet sheet = new_workbook.createSheet("EMP_DETAILS"); // create a worksheet with caption score_details
// Define the SQL query
String sql = "SELECT login, password FROM \"Permissions\"";
ResultSet rs = stmt.executeQuery(sql);
// Create Map for Excel Data
Map<String, Object[]> excel_data = new HashMap<String, Object[]>();
int row_counter = 0;
//Extract data from result set
while(rs.next()){
row_counter = row_counter+1;
String login = rs.getString("login");
String password = rs.getString("password");
excel_data.put(Integer.toString(row_counter), new Object[] {login, password});
}
rs.close();
// Load data into logical worksheet
Set<String> keyset = excel_data.keySet();
int rownum = 0;
for(String key : keyset){ // loop through the data and add them to the cell
Row row = sheet.createRow(rownum++);
Object [] objArr = excel_data.get(key);
int cellnum = 0;
for(Object obj : objArr){
Cell cell = row.createCell(cellnum++);
if(obj instanceof Double)
cell.setCellValue((Double)obj);
else
cell.setCellValue((String)obj);
}
}
FileOutputStream output_file = new FileOutputStream(new File("File.xlsx")); // create XLSX file
new_workbook.write(output_file); // write excel document to output stream
output_file.close(); // close the file
My $0.02... don't create an Excel file for something this simple. Instead create a CSV file. The first line you write will be
UserName, Password
for the headers and the lines after that will just be your data for each column separated by a comma. Once the file is written, you can easily open it in Excel and save it as an Excel file.
This method is way simpler and easier than trying to actually write an Excel file.
If you don't want to do that, you just need to create a row before you start looping through your keyset as you would any other row and write the strings for headers.
You can use following code after this line ResultSet rs = stmt.executeQuery(sql);:
ResultSetMetaData rsmd=rs.getMetaData();
Then to fetch column headers; use below:
String columnName1 = rsmd.getColumnName(1); // will fetch you "UserName"
String columnName2 = rsmd.getColumnName(2); // will fetch you "Password"
So my excel file is relatively small in size. It contains 8 sheets. Each sheet has "records" of data which i need to read. Each sheet also has the first row reserved for headers which i skip; so my data will begin from the 2nd row (1st index) of each sheet and end on the last record.
So, below is my code to iterate through the sheets and read each row however it fails to read each sheet. And i can't seem to figure out why. Please have look and any suggestions will be appreciated.
Thanks!
FileInputStream fis = new FileInputStream(new File(filePath));
XSSFWorkbook wb = new XSSFWorkbook(fis);
DataFormatter formatter = new DataFormatter();
//iterate over sheets
for (int i=0; i<NUM_OF_SHEETS; i++) {
sheet = wb.getSheetAt(i);
sheetName = sheet.getSheetName();
//iterate over rows
for (int j=1; j<=lastRow; j++) { //1st row or 0-index of each sheet is reserved for the headings which i do not need.
row = sheet.getRow(j);
if (row!=null) {
data[j-1][0] = sheetName; //1st column or 0th-index of each record in my 2d array is reserved for the sheet's name.
//iterate over cells
for (int k=0; k<NUM_OF_COLUMNS; k++) {
cell = row.getCell(k, XSSFRow.RETURN_BLANK_AS_NULL);
cellValue = formatter.formatCellValue(cell); //convert cell to type String
data[j-1][k+1] = cellValue;
}//end of cell iteration
}
}//end of row iteration
}//end of sheet iteration
wb.close();
fis.close();
At least there is one big logical error. Since you are putting the data of all sheets in one array, this array must be dimensioned like:
String[][] data = new String[lastRow*NUM_OF_SHEETS][NUM_OF_COLUMNS+1];
And then the allocations must be like:
...
data[(j-1)+(i*lastRow)][0] = sheetName; //1st column or 0th-index of each record in my 2d array is reserved for the sheet's name.
...
and
...
data[(j-1)+(i*lastRow)][k+1] = cellValue;
...
With your code, the allocations from second sheet will overwrite the ones from the first sheet, since j starts with 1 for every sheet.
i am new for upload Excel file POI API so i need to validate duplicate cell in particular column . example like
String dupcolumn = myRow.getcell(0);
Iterator iter = new Iterator();
while(iter.hesnext())
{
myRow = (Row) iter.next();
dupcolumn.contains(iter.toString());
}
Above code cannot support to read in particular column , but always continuous reading column .
This code may helpful to you.
InputStream xlsStream = excelFileUpload.getInputstream();
XSSFWorkbook wb = new XSSFWorkbook(xlsStream);
XSSFSheet sheet = wb.getSheetAt(0);
Iterator<Row> rows=sheet.iterator();
//need to keep retrieved values in a collection to check duplicates.
Set<String> values = new HashSet<String>();
//check all rows in excel sheet
while(rows.hasNext()){
//get next row
XSSFRow row =(XSSFRow)rows.next();
//pass '0' means first cell (column) in current row. if you need to get other cell value, you can pass relevant cell number instead of '0'.
XSSFCell cell=row.getCell(0);
if(values.contains(cell.getStringCellValue())){
//duplicated value
}else{
values.add(cell.getStringCellValue());
}
}