Develop Reference

Importing a .csv or .xlsx file into mysql DB using Java?

YES, it sounds like a duplicate.
I'm practicing abit of Java on Intellij and tried writing a program to import a .xls excel file into a mysql database. Duplicate question, yes, but trawling the internet didnt yield much.
My code below currently does the job of importing any xls file perfectly. Unfortunately, it doesnt do anything for a .csv file nor an xlsx file.
When i try with a .csv file, the following error is thrown:
Invalid header signature; read 0x6972702C74786574, expected 0xE11AB1A1E011CFD0 - Your file appears not to be a valid OLE2 document
When a xlsx file is used, the following is instead thrown as an error:
Exception in thread "main" org.apache.poi.poifs.filesystem.OfficeXmlFileException: The supplied data appears to be in the Office 2007+ XML. You are calling the part of POI that deals with OLE2 Office Documents. You need to call a different part of POI to process this data (eg XSSF instead of HSSF)
at org.apache.poi.poifs.storage.HeaderBlock.<init>(HeaderBlock.java:152)
at org.apache.poi.poifs.storage.HeaderBlock.<init>(HeaderBlock.java:140)
at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.java:302)
at org.apache.poi.poifs.filesystem.POIFSFileSystem.<init>(POIFSFileSystem.java:85)
at FileExport.main(FileExport.java:21)
My code:
import java.io.FileInputStream;
import java.io.IOException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import org.apache.poi.poifs.filesystem.POIFSFileSystem;
import org.apache.poi.ss.usermodel.*;
public class FileExport {
public static void main(String[] args) throws Exception {
try {
Class forName = Class.forName("com.mysql.jdbc.Driver");
Connection con = null;
con = DriverManager.getConnection("jdbc:mysql://localhost:3306/test?useSSL=false", "root", "root");
con.setAutoCommit(false);
PreparedStatement pstm = null;
FileInputStream input = new FileInputStream("/Users/User/Desktop/Email/Test.xls");
POIFSFileSystem fs = new POIFSFileSystem(input);
Workbook workbook;
workbook = WorkbookFactory.create(fs);
Sheet sheet = workbook.getSheetAt(0);
Row row;
for (int i = 1; i <= sheet.getLastRowNum(); i++) {
row = (Row) sheet.getRow(i);
String text = row.getCell(0).getStringCellValue();
int price = (int) row.getCell(1).getNumericCellValue();
String sql = "INSERT INTO testtable (text, price) VALUES('" + text + "','" + price + "')";
pstm = (PreparedStatement) con.prepareStatement(sql);
pstm.setString(1, text);
pstm.setInt(2, price);
pstm.execute();
System.out.println("Import rows " + i);
}
con.commit();
pstm.close();
con.close();
input.close();
System.out.println("Success import excel to mysql table");
} catch (IOException e) {
}
}
}
Any suggestions on how to tweak this code to import .csv or xlsx files are greatly appreciated.

You should use PreparedStatement as it was intended for:
String sql = "INSERT INTO testtable (text, price) VALUES(?, ?)";
pstm = (PreparedStatement) con.prepareStatement(sql);
pstm.setString(1, text);
pstm.setInteger(2, text)
pstm.execute();
I guess it doesn't work because there is some punctuation in your text.
Also it is prone to SQL-injection.
Also you should close your closeable objects with try-with-resources or finally (if you stuck to java version prior version 7), as it done here:
https://stackoverflow.com/a/26516076/3323777
EDIT: ah yes, the empty catch block as Alex said. Don't do that, either.
EDIT2:
Apache POI was never designed to call on CSV files.
https://stackoverflow.com/a/1087984/3323777
There is another project for CSV at Apache:
http://commons.apache.org/proper/commons-csv/

Do you see the "Import rows" messages which prove that you actually have some rows to import?
Also the reason for not seeing any errors might be a "catch" block which does nothing. Add any output inside and observe new results, i.e.
System.out.println(e.getMessage());

Tweaked for read from .csv
public class FileExport {
public static void main(String[] args) throws Exception {
try {
Class forName = Class.forName("com.mysql.jdbc.Driver");
Connection con = null;
con = DriverManager.getConnection("jdbc:mysql://localhost:3306/test?useSSL=false", "root", "root");
con.setAutoCommit(false);
PreparedStatement pstm = null;
FileInputStream input = new FileInputStream("/Users/User/Desktop/Email/Test.csv");
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
String row = reader.readLine();
String text = row.split(";")[0];
String price = row.split(";")[1];
reader.close();
String sql = "INSERT INTO testtable (text, price) VALUES('" + text + "','" + price + "')";
pstm = (PreparedStatement) con.prepareStatement(sql);
pstm.execute();
System.out.println("Import rows " + i);
}
con.commit();
pstm.close();
con.close();
input.close();
System.out.println("Success import excel to mysql table");
} catch (IOException e) {
}
}

Try using 'poi-ooxml' for creating a Workbook object, its gradle dependency signature is provided below:
compile group: 'org.apache.poi', name: 'poi-ooxml', version: '3.14'
Below code may help you
InputStream inputStream = new FileInputStream("/Users/User/Desktop/Email/Test.xls");
XSSFWorkbook wb = new XSSFWorkbook(inputStream);
XSSFSheet sheet = wb.getSheetAt(0);

How to write fetched values from database to Excel file

I need your help in storing the fetched values from a SQL statement to an excel file. I wrote the below code, but I am facing difficulties on how to write the fetched values under the appropriate columns. For example at the beginning, I created 3 headers (ID - Name - Salary). Now, I need to write the fetched values from the SQL statement under each appropriate header, but I do not know how to write them. So kindly assist. The code is:
public void GenerateExcel() {
Connection conn = null;
Statement stmt = null;
FileOutputStream fileOut = new FileOutputStream("C:\\Desktop\\poi-test.xls");
HSSFWorkbook workbook = new HSSFWorkbook();
HSSFSheet worksheet = workbook.createSheet("Employee Details");
HSSFRow row1 = worksheet.createRow((short) 0);
HSSFCell cellA1 = row1.createCell((short) 0);
cellA1.setCellValue("ID");
HSSFCell cellB1 = row1.createCell((short) 1);
cellB1.setCellValue("Name");
HSSFCell cellC1 = row1.createCell((short) 1);
cellC1.setCellValue("Salary");
try{
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection(DB_URL, USER, PASS);
stmt = conn.createStatement();
String sql = "SELECT id, name, amount FROM Employee";
ResultSet rs = stmt.executeQuery(sql);
while(rs.next()){
int id = rs.getInt("id");
int age = rs.getString("name");
String first = rs.getInt("amount");
}
rs.close();
workbook.write(fileOut);
fileOut.flush();
fileOut.close();
}catch(SQLException se){
se.printStackTrace();
}catch(Exception e){
e.printStackTrace();
}finally{
try{
if(stmt!=null)
conn.close();
}catch(SQLException se){
}
try{
if(conn!=null)
conn.close();
}catch(SQLException se){
se.printStackTrace();
}
}
}
}

Actually I can't see where you add data to the workbook ?
I can only see you creating the workbook and adding the first row.
while(rs.next()){
int id = rs.getInt("id");
int age = rs.getString("name");
String first = rs.getInt("amount");
// Adding data here
Row newRow = worksheet.createRow(worksheet.getLastRowNum() + 1);
newRow.createCell(0).setCellValue(id);
newRow.createCell(1).setCellValue(age);
newRow.createCell(2).setCellValue(first);
}

Finally stop wasting time writing apache poi boiler plate. Look at MemPOI's solution:
You can try using MemPOI. Take a look:
File file = new File("C:\\Desktop\\poi-test.xls")
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection(DB_URL, USER, PASS);
PreparedStatement prepStmt = this.connection.prepareStatement("SELECT id, name, amount FROM Employee");
new MempoiBuilder()
.setFile(file)
.addMempoiSheet(new MempoiSheet(prepStmt, "Employee Details"))
.build()
.prepareMempoiReportToFile()
.get();
You can easily add a template or a subfoter. Take a look at the doc

Getting blob image in mysql using restful web service

I've done listing the strings like the name etc. but I've got problem in retrieving the image. Here's my code of retrieving it. It is in the arraylist. I don't know if i'm getting the right output. Please help me. Here's my code
public ArrayList<Objects> getTaxiDetails(Connection con, String taxi_plate_no) throws SQLException{
ArrayList<Objects> taxiDetailsList = new ArrayList<Objects>();
PreparedStatement stmt = con.prepareStatement("SELECT * from taxi_driver where taxi_plate_no = '" +taxi_plate_no+ "'");
//PreparedStatement stmt = con.prepareStatement("SELECT * from taxi_driver");
ResultSet rs = stmt.executeQuery();
try{
byte[] bytes = null;
while(rs.next()){
Objects taxiObjects = new Objects();
taxiObjects.setDriver_contact_no(rs.getString("driver_contact_no"));
taxiObjects.setDriver_name(rs.getString("driver_name"));
taxiObjects.setTaxi_plate_no(rs.getString("taxi_plate_no"));
taxiObjects.setDriver_operator(rs.getString("driver_operator"));
taxiObjects.setDriver_operator_address(rs.getString("driver_operator_address"));
taxiObjects.setImage(rs.getBytes("image"));
Blob image = rs.getBlob(1);
byte[] allBytesInBlob = image.getBytes(1, (int) image.length());
taxiDetailsList.add(taxiObjects);
}
} catch (SQLException e){
e.printStackTrace();
} return taxiDetailsList;
}
Here's the output

performance decreses after 45k record, when rewriting excel (.xlsm) file from java resultset

This code works perfectly for records below 35k and write data on Sheet but if record exist 45k performance slow down and dead without any exception or error,
and doesn't create any file.
how can i overcome this issue?
public void dbConnect(String driver_connect_string, String db_connect_string, String db_userid, String db_password){
try{
//database connectivity
Statement statement = conn.createStatement();
String queryString = propq.getProperty("Query");
ResultSet rs = statement.executeQuery(queryString);
OPCPackage pkg = OPCPackage.open(new File(("sourceFile")));
XSSFWorkbook wb_template;
wb_template = new XSSFWorkbook(pkg);
System.out.println("package loaded");
Sheet sheet = wb_template.getSheetAt(0);
Row row = null;
int index = 1;
while (rs.next()) {
try{
row = sheet.createRow(index);
row.createCell(0).setCellValue(rs.getString(1));
row.createCell(1).setCellValue(rs.getString(2));
row.createCell(2).setCellValue(rs.getString(3));
.
.
row.createCell(23).setCellValue(rs.getString(25));
System.out.println(index);
}
catch(Exception e){
System.out.println(e);
}
index++;
}
FileOutputStream out = new FileOutputStream(new File(("destFile")));
wb_template.write(out);
out.flush();
out.close();
}catch(Exception e){
e.printStackTrace();
}
}

Java servlet trying to output a xlsx file, but keep getting octet-stream

I'm trying to create a xlsx file in a servlet and send it to the browser where the user can either save or download the file. Currently it keeps sending application/octet-stream rather than an excel file.
What am i doing wrong?
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
SXSSFWorkbook wb = new SXSSFWorkbook();
Sheet sheet = wb.createSheet("Results");
List<List<TableData>> results = (List<List<TableData>>) request.getSession().getAttribute("results");
String tableName = (String) request.getSession().getAttribute("tableName");
DateFormat df = new SimpleDateFormat("yyyy-MM-dd");
Date date = new Date();
String filename = tableName+"-"+df.format(date)+".xlsx";
int rowCounter = 0;
// Create the rows
for(List<TableData> l : results) {
int counter = 0;
Row row = sheet.createRow(rowCounter);
for(TableData td : l) {
Cell cell = row.createCell(counter);
// if we're on the first row, get the column description, else get the data
if(rowCounter == 0) {
cell.setCellValue(td.getColumnDescription());
} else {
cell.setCellValue(td.getData());
}
counter++;
}
rowCounter++;
}
try {
OutputStream out = response.getOutputStream();
wb.write(out);
response.setContentType("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
response.setHeader("Expires:", "0"); // eliminates browser caching
response.setHeader("Content-Disposition", "attachment; filename="+filename);
out.flush();
out.close();
} catch (Exception e) {
LOG.debug(e.getStackTrace().toString());
}
}

Set the content type and the headers before sending the body. They're called headers for a good reason :-)