Develop Reference

Get each cells data from Excel converted to XML to stored to DB

After looking for some resources, I could load an Excel file which contain 1.000.000 rows data. But, I did not know how to get each data. Here is my code so far...
public void create(MultipartFile file) throws Exception {
try {
InputStream fileStream = new BufferedInputStream(file.getInputStream());
OPCPackage opc = OPCPackage.open(fileStream);
XSSFReader xssf = new XSSFReader(opc);
SharedStringsTable sst = xssf.getSharedStringsTable();
XSSFReader.SheetIterator itr = (XSSFReader.SheetIterator)xssf.getSheetData();
// I just realize, if I running below for-loop,
// this only print strings and in random order, not in the same order as the excel file.
// 20 is just an example
for (int i = 0; i < 20; i++) {
System.out.println(sst.getEntryAt(i).getT().toString());
}
while (itr.hasNext()) {
InputStream is = itr.next();
if (itr.getSheetName().equals("MY_SHEET_NAME")) {
while ("data is avaiable, this is just example, I'll use something like hasNext() for the row in the sheet, but I dont know how to do it" != null) {
// Want to process and get all data in each cells, then store to DB
// What I did not know, is how to get data in each cells
}
} else {
throw new Exception("Sheet not found");
}
}
} catch (Exception e) {
throw new Exception("Error is: " + e.getMessage());
} finally {
if (is != null) {
is.close();
}
if (opc != null){
opc.close();
}
if (fileStream != null) {
fileStream.close();
}
}
}
I've tried to look at here to process the sheet, but I did not get how to grab the data in each cells. Any help will really help me..
Update
if I read the doc of apache POI, here, from the link, code part that will process my excel is here:
public void processOneSheet(String filename) throws Exception {
OPCPackage pkg = OPCPackage.open(filename);
XSSFReader r = new XSSFReader( pkg );
SharedStringsTable sst = r.getSharedStringsTable();
XMLReader parser = fetchSheetParser(sst);
// To look up the Sheet Name / Sheet Order / rID,
// you need to process the core Workbook stream.
// Normally it's of the form rId# or rSheet#
InputStream sheet2 = r.getSheet("rId2");
InputSource sheetSource = new InputSource(sheet2);
parser.parse(sheetSource);
sheet2.close();
}
But, after call parser.parse(sheetSource), how I could get each data from each row and column? Because I wanna do validation on each data on each cells, then store it to database.
Update 2
I've try using this answer, https://stackoverflow.com/a/51818500/10454516. I could get the data, I've try to insert myObjectRepo.save(result) or myObjectRepo.save(myObject), both I placed the code inside void endRow method and I also try to place it right after switch but inside the if(lineNumber > 0), but its always return NullPointerException. But if I did not call save method, I try to print the result in the console, the result is printed.

One of the way you could grab the excel data is :
try {
InputStream excelFile = new FileInputStream(mFileName);
XSSFWorkbook wb = new XSSFWorkbook(excelFile);
XSSFSheet sheet = wb.getSheetAt(0);
XSSFRow row;
XSSFCell cell;
Iterator<Row> rows = sheet.rowIterator();
int col = 0, colPR = 1;
int pageRank = 0;
String url = null;
while (rows.hasNext()) {
row = (XSSFRow) rows.next();
url = row.getCell(col).getStringCellValue();
System.out.println("--------------------------");
}
FileOutputStream out = new FileOutputStream(mFileName);
wb.write(out);
out.flush();
out.close();
}
catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

Well, I'd like to answer my own question. After several experiments/trials and also based on this answer, I've made it to upload excel file which has at least 1.000.000 rows data and stored into db (postgresql). For upload, read and insert 1.000.000 rows data needs 5 minutes. Here is the link to the project. Hope this could help anybody who need it.

Looping through excel rows keeps given me an ArrayIndex...Exception

Okay so I am stuck and I am sure it is a simple solution. Basically I am beginning my initial conversion, or at least constructing a demo, to a keyword driven framework for selenium testing. Each row will contain data that will be used to drive the test. After the row is complete, the next row will contain the next test and so on. So I just started and I am having a little bit of trouble. Here is my code:
public List<String> getRowData(String row){
Sheet sheet = null;
List<String> getContents = new ArrayList<>();
try{
sheet = getWorkBook().getSheet("test");
for(int i=0; i<sheet.getRow(i).length; i++){
getRowData = sheet.getRow(i);
for(Cell rowData : getRowData){
System.out.print(String.format("Row info: %s\n", rowData.getContents()));
getContents.add(rowData.getContents());
}
}
}catch (BiffException e) {
e.printStackTrace();
} catch (IOException e) {
System.out.print("This is an exception!");
} catch (NullPointerException np){
System.out.print("File does not exist!");
}return getContents;
}
The strings for the exceptions are just placeholders, not going to be final. But I am getting an ArrayIndex....Exception. I do understand a little why I am getting the error, but I am trying to obviously break out of the loop once the row or row content is empty. I implemented a condition such that
if(sheet.getRow(i) == null){
break;
}
But it still evaluated and provides the same exception. I am stuck and would like your help in trying to break out of the loop when there are no contents. Thank you

change for(int i=0; i<sheet.getRow(i).length; i++)
to:
for(int i = 0; i < sheet.getRows(); i++){
if(sheet.getRow(i) == null) break;
//rest of the code
}
But are you sure you want to break the loop on an empty row and not continue to the next row?

reading variables buried in java exception handling

I am writing a function to take a text file and count how many lines it has while outputting the lines to an array of strings. Doing this I have several exceptions I need to look out for. The class function has several variables that should have a scope throughout the function but when I write a value to the function inside of an exception, the return statement cannot find it. I've moved the declaration around and nothing helps
The value returned "h5Files" "Might not have been initialized" Since I don't know how long the array will be I cannot initialize it to a certain length. I do this within the code and I need a way to tell the return statement that I now have a values
Here is the code
public String[] ReadScanlist(String fileIn){
int i;
String directory ="c:\\data\\"; // "\" is an illegal character
System.out.println(directory);
int linereader = 0;
String h5Files[];
File fileToRead = new File(directory + fileIn);
System.out.println(fileToRead);
try {
FileInputStream fin = new FileInputStream(fileToRead); // open this file
}
catch(FileNotFoundException exc) {
System.out.println("File Not Found");
}
try{
//read bytes until EOF is detected
do {
FileReader fr = new FileReader(fileToRead);// Need to convert to reader
LineNumberReader lineToRead = new LineNumberReader(fr); // Use line number reader class
//
while (lineToRead.readLine() != null){
linereader++;
}
linereader = 0;
lineToRead.setLineNumber(0); //reset line number
h5Files = new String[linereader];
while (lineToRead.readLine() != null){
h5Files[linereader] = lineToRead.readLine(); // deposit string into array
linereader++;
}
return h5Files;
}
while(i !=-1); // When i = -1 the end of the file has been reached
}
catch(IOException exc) {
System.out.println("Error reading file.");
}
try{
FileInputStream fin = new FileInputStream(fileToRead);
fin.close(); // close the file
}
catch(IOException exc) {
System.out.println("Error Closing File");
}
return h5Files;
}

Your code is very very odd. For example these two blocks make no sense:
try {
FileInputStream fin = new FileInputStream(fileToRead); // open this file
}
catch(FileNotFoundException exc) {
System.out.println("File Not Found");
}
try{
FileInputStream fin = new FileInputStream(fileToRead);
fin.close(); // close the file
}
catch(IOException exc) {
System.out.println("Error Closing File");
}
I don't know what you think they do, but besides the first one leaking memory, they do nothing at all. The comments are more worrying, they suggest that you need to do more reading on IO in Java.
Deleting those blocks and tidying the code a (moving declarations, formatting) gives this:
public String[] ReadScanlist(String fileIn) {
String directory = "c:\\data\\";
String h5Files[];
File fileToRead = new File(directory + fileIn);
try {
int i = 0;
do {
FileReader fr = new FileReader(fileToRead);
LineNumberReader lineToRead = new LineNumberReader(fr);
int linereader = 0;
while (lineToRead.readLine() != null) {
linereader++;
}
linereader = 0;
lineToRead.setLineNumber(0);
h5Files = new String[linereader];
while (lineToRead.readLine() != null) {
h5Files[linereader] = lineToRead.readLine();
linereader++;
}
return h5Files;
} while (i != -1);
} catch (IOException exc) {
System.out.println("Error reading file.");
}
return h5Files;
}
My first bone of contention is the File related code. First, File abstracts from the underlying OS, so using / is absolutely fine. Second, there is a reason File has a File, String constructor, this code should read:
File directory = new File("c:/data");
File fileToRead = new File(directory, fileIn);
But it should really be using the new Path API anyway (see below).
So, you declare h5Files[]. You then proceed to read the whole file to count the lines. You then assign h5Files[] to an array of the correct size. Finally you fill the array.
If you have an error anywhere before you assign h5Files[] you have not initialised it and therefore cannot return it. This is what the compiler is telling you.
I don't know what i does in this code, it is assigned to 0 at the top and then never reassigned. This is an infinite loop.
So, you need to rethink your logic. I would recommend throwing an IOException if you cannot read the file. Never return null - this is an anti-pattern and leads to all those thousands of null checks in your code. If you never return null you will never have to check for it.
May I suggest the following alternative code:
If you are on Java 7:
public String[] ReadScanlist(String fileIn) throws IOException {
final Path root = Paths.get("C:/data");
final List<String> lines = Files.readAllLines(root.resolve(fileIn), StandardCharsets.UTF_8);
return lines.toArray(new String[lines.size()]);
}
Or, if you have Java 8:
public String[] ReadScanlist(String fileIn) throws IOException {
final Path root = Paths.get("C:/data");
try (final Stream<String> lines = Files.lines(root.resolve(fileIn), StandardCharsets.UTF_8)) {
return lines.toArray(String[]::new);
}
}

Since I don't know how long the array will be I cannot initialize it
to a certain length.
I don't think an array is the correct solution for you then - not to say it can't be done, but you would be re-inventing the wheel.
I would suggest you use a LinkedList instead, something like:
LinkedList<String> h5Files = new LinkedList<>();
h5Files.add(lineToRead.readLine());
Alternatively you could re-invent the wheel by setting the array to an arbritary value, say 10, and then re-size it whenever it gets full, something like this:
h5Files = new String[10];
if (linereader = h5Files.size())
{
String[] temp = h5Files;
h5Files = new String[2 * linereader];
for (int i = 0; i < linereader; i++)
{
h5Files[i] = temp[i];
}
}
Either one of these solutions would allow you to initialize the array (or array alternative) in a safe constructor, prior to your try block, such that you can access it if any exceptions are thrown

Here is your problem. Please take a look on digested version of your code with my comments.
String h5Files[]; // here you define the variable. It still is not initialized.
try{
..................
do {
h5Files = new String[linereader]; // here you initialize the variable
} while(i !=-1); // When i = -1 the end of the file has been reached
..................
catch(IOException exc) {
// if you are here the variable is still not initialized
System.out.println("Error reading file.");
}
// you continue reading file even if exception was thrown while opening the file
I think that now the problem is clearer. You try to open the file and count lines. If you succeed you create array. If not (i.e. when exception is thrown) you catch the exception but still continue reading the file. But in this case you array is not initialized.
Now how to fix this?
Actually if you failed to read the file first time you cannot continue. This may happen for example if file does not exist. So, you should either return when first exception is thrown or just do not catch it at all. Indeed there is nothing to do with the file if exception was thrown at any phase. Exception is not return code. This is the reason that exceptions exist.
So, just do not catch exceptions at all. Declare your method as throws IOException and remove all try/catch blocks.

How do I add data from a file to a JTable?

Here is my method so far:
public void readfile(JTable table) {
try{
BufferedReader in = new BufferedReader( new FileReader("out.txt"));
for(int i = 0; i<10; i++) {
for(int j = 0; j<5; j++) {
table.setValueAt(in.readLine(), i, j);
}
}
in.close();
}catch (Exception e) {
System.err.println("error: " + e.getMessage());
}
}
Here are the contents of out.txt:
test1
test2
test3
test4
test5
Where I run the program and attempt to load the file to the table, nothing happens. I also get an output that says the following:
error: 0 >= 0
Help me please?

I would narrow your problem down to a smaller problem, solve this smaller problem, and then widen it until you have want you want.
Think of the contents of a File as a big blob of text.
Think of the table as a Vector of Vectors.
Smaller problem: How do I convert a big blob of text into a Vector of Vectors? You need to be able to sove this problem first before tackling File I/O or DefaultTableModels.

How to return a 2D String array in java?

I have a method wherein i have to return a 2D String array.
The part of code for that method is as follow:-
public String[][] retrieveData(){
try{
int noOfRows = 0;
pstmt = con.prepareStatement("select * from category");
ResultSet rs = pstmt.executeQuery();
while(rs.next())
noOfRows++;
rs.first();
categoryContent = new String[noOfRows][noOfCols];
for(int i = 0 ; i < noOfRows ; i++){
for(int j = 0 ; j < noOfCols ; j++){
if(j == 0){
Integer categoryNo = new Integer(rs.getInt(1));
categoryContent[i][j] = categoryNo.toString();
}
else{
categoryContent[i][j] = rs.getString(j+1);
}
}
rs.next();
}
return categoryContent ;
}
catch(Exception e){
e.printStackTrace();
}
}
The error which i am getting at compile time is as follows:-
I:\DynamicWebpageCreator\WEB-INF\classes>javac *.java
Category.java:134: missing return statement
public String[][] retrieveData(){
^**
1 error
Please help me soon. I am stuck with it.
All the answers are highly appreciated!

If an exception is thrown, you'll print the stack trace, but never return anything. That's what the compiler is complaining about.
In general, "handling" exceptions like this is a really bad idea:
Log any exception information somewhere more appropriate than just the console
Don't catch bare Exception - catch specific exceptions
If you can't actually handle an exception, let it propagate up the call stack
In this case I'd suggest you should probably just change your method to declare which exceptions might be thrown, and remove the try/catch block.
If you genuinely want to catch exceptions (specific ones, mind) then you'll need to work out what you want to return in that case.

You have indicated the return command in side the try statement try this way it will work.
public int x(){
try{
}catch (Exception e){}
return x;
}