Correct way to use copy Postgres jdbc - java

Unable to use copy command with jdbc Postgres. Whats wrong with the below code snippet sample.
public boolean loadReportToDB(String date) {
// TODO Auto-generated method stub
Connection connection = DBUtil.getConnection("POSTGRESS");
String fileName = "C:/_0STUFF/NSE_DATA/nseoi_" + date + ".csv";
String sql = "\\copy fno_oi FROM 'C:\\_0STUFF\\NSE_DATA\\nseoi_27102017.csv' DELIMITER ',' CSV header";
try {
PreparedStatement ps = connection.prepareStatement(sql);
System.out.println("query"+ps.toString());
int rowsaffected = ps.executeUpdate();
System.out.println("INT+" + rowsaffected);
return true;
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return false;
}
org.postgresql.util.PSQLException: ERROR: syntax error at or near "\"
Position: 1
at org.
if we use
String sql = "copy fno_oi FROM 'C:\\_0STUFF\\NSE_DATA\\nseoi_27102017.csv' DELIMITER ',' CSV header";
then no rows are updated
postgres version postgresql-10.0-1-windows-x64

This works for me:
try (Connection conn = DriverManager.getConnection(connUrl, myUid, myPwd)) {
long rowsInserted = new CopyManager((BaseConnection) conn)
.copyIn(
"COPY table1 FROM STDIN (FORMAT csv, HEADER)",
new BufferedReader(new FileReader("C:/Users/gord/Desktop/testdata.csv"))
);
System.out.printf("%d row(s) inserted%n", rowsInserted);
}
Using copyIn(String sql, Reader from) has the advantage of avoiding issues where the PostgreSQL server process is unable to read the file directly, either because it lacks permissions (like reading files on my Desktop) or because the file is not local to the machine where the PostgreSQL server is running.

As your input file is stored locally on the computer running your Java program you need to use the equivalent of copy ... from stdin in JDBC because copy can only access files on the server (where Postgres is running).
To do that use the CopyManager API provided by the JDBC driver.
Something along the lines:
Connection connection = DBUtil.getConnection("POSTGRES");
String fileName = "C:/_0STUFF/NSE_DATA/nseoi_" + date + ".csv";
String sql = "copy fno_oi FROM stdin DELIMITER ',' CSV header";
BaseConnection pgcon = (BaseConnection)conection;
CopyManager mgr = new CopyManager(pgcon);
try {
Reader in = new BufferedReader(new FileReader(new File(fileName)));
long rowsaffected = mgr.copyIn(sql, in);
System.out.println("Rows copied: " + rowsaffected);
} catch (SQLException e) {
e.printStackTrace();
}

Related

Backslash not working for postgres query from jdbc [duplicate]

Unable to use copy command with jdbc Postgres. Whats wrong with the below code snippet sample.
public boolean loadReportToDB(String date) {
// TODO Auto-generated method stub
Connection connection = DBUtil.getConnection("POSTGRESS");
String fileName = "C:/_0STUFF/NSE_DATA/nseoi_" + date + ".csv";
String sql = "\\copy fno_oi FROM 'C:\\_0STUFF\\NSE_DATA\\nseoi_27102017.csv' DELIMITER ',' CSV header";
try {
PreparedStatement ps = connection.prepareStatement(sql);
System.out.println("query"+ps.toString());
int rowsaffected = ps.executeUpdate();
System.out.println("INT+" + rowsaffected);
return true;
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return false;
}
org.postgresql.util.PSQLException: ERROR: syntax error at or near "\"
Position: 1
at org.
if we use
String sql = "copy fno_oi FROM 'C:\\_0STUFF\\NSE_DATA\\nseoi_27102017.csv' DELIMITER ',' CSV header";
then no rows are updated
postgres version postgresql-10.0-1-windows-x64
This works for me:
try (Connection conn = DriverManager.getConnection(connUrl, myUid, myPwd)) {
long rowsInserted = new CopyManager((BaseConnection) conn)
.copyIn(
"COPY table1 FROM STDIN (FORMAT csv, HEADER)",
new BufferedReader(new FileReader("C:/Users/gord/Desktop/testdata.csv"))
);
System.out.printf("%d row(s) inserted%n", rowsInserted);
}
Using copyIn(String sql, Reader from) has the advantage of avoiding issues where the PostgreSQL server process is unable to read the file directly, either because it lacks permissions (like reading files on my Desktop) or because the file is not local to the machine where the PostgreSQL server is running.
As your input file is stored locally on the computer running your Java program you need to use the equivalent of copy ... from stdin in JDBC because copy can only access files on the server (where Postgres is running).
To do that use the CopyManager API provided by the JDBC driver.
Something along the lines:
Connection connection = DBUtil.getConnection("POSTGRES");
String fileName = "C:/_0STUFF/NSE_DATA/nseoi_" + date + ".csv";
String sql = "copy fno_oi FROM stdin DELIMITER ',' CSV header";
BaseConnection pgcon = (BaseConnection)conection;
CopyManager mgr = new CopyManager(pgcon);
try {
Reader in = new BufferedReader(new FileReader(new File(fileName)));
long rowsaffected = mgr.copyIn(sql, in);
System.out.println("Rows copied: " + rowsaffected);
} catch (SQLException e) {
e.printStackTrace();
}

Append row in Sql without replacing previous row java

So i have database with value like this...
i'm trying to append the value by using insert into without replacing it,the data from this txt file...
but when i reload/refresh the database there is no new data being appended into the database...,
here is my code....
public static void importDatabase(String fileData){
try{
File database = new File(fileData);
FileReader fileInput = new FileReader(database);
BufferedReader in = new BufferedReader(fileInput);
String line = in.readLine();
line = in.readLine();
String[] data;
while (line != null){
data = line.split(",");
int ID = Integer.parseInt(data[0]);
String Nama = data[1];
int Gaji = Integer.parseInt(data[2]);
int Absensi = Integer.parseInt(data[3]);
int cuti = Integer.parseInt(data[4]);
String Status = data[5];
String query = "insert into list_karyawan values(?,?,?,?,?,?)";
ps = getConn().prepareStatement(query);
ps.setInt(1,ID);
ps.setString(2,Nama);
ps.setInt(3,Gaji);
ps.setInt(4,Absensi);
ps.setInt(5,cuti);
ps.setString(6,Status);
line = in.readLine();
}
ps.executeUpdate();
ps.close();
con.close();
System.out.println("Database Updated");
in.close();
}catch (Exception e){
System.out.println(e);
}
}
When i run it, it shows no error but the data never get into database, where did i go wrong?.,...
Auto-commit mode is enabled by default.
The JDBC driver throws a SQLException when a commit or rollback operation is performed on a connection that has auto-commit set to true.
Symptoms of the problem can be unexpected application behavior
update the JVM configuration for the ActiveMatrix BPM node to use the following Oracle connection property:
autoCommitSpecCompliant=false Try once
Note:I am not able to put as comment so i posted as a answer

How do I read a CSV file into my local Oracle Database table?

My goal is to read from a CSV file, loading it into a sample database table (Oracle) that I have .
So far , I have the following Java code :
public class ParsingCSV {
public static void main(String[] args) {
ParsingCSV obj = new ParsingCSV();
obj.run();
}
#SuppressWarnings("oracle.jdeveloper.java.nested-assignment")
public void run() {
String csvFile = "C:\\Users\\IBM_ADMIN\\Desktop\\Work\\ConnectOne_Bancorp\\Database_Work\\Book1.csv";
BufferedReader br = null;
String line = "";
String cvsSplitBy = ",";
try {
br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
// use comma as separator
String[] country = line.split(cvsSplitBy);
System.out.println("Country [code= " + country[4]
+ " , name=" + country[5] + "]");
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
/* boilerplate */
any tips/pointers appreciated , thanks !
The tips could be:
Download the Oracle Database driver Oracle driver
Set the .jar to your classpath. [java setting class path]
Register the driver and connect to the database [java connect to oracle]
Execute sql "CREATE TABLE...." [sql create table example]
For every line that you read do sql = "INSERT INTO..." [sql insert into example]
For every step if you need more example and advice, I suggest google use the [] tags to search

JTDS not connecting to correct database

I am using JTDS to connect to a MS SQL Server. Connection to the database is no problem, but when I try to execute a statement, I get a Database 'java' does not exist exception.
ConnectionString:
conn = DriverManager.getConnection("jdbc:jtds:sqlserver://localhost;DatabaseName=MyDatabase;user=testuser;password=testpassword");
Trying to execute the script:
private void runStatement(String scriptLocation) {
if(scriptLocation == null) {
return;
}
try {
InputStream is = getClass().getClassLoader().getResourceAsStream(scriptLocation);
String query = is.toString();
is.close();
Statement stmt = conn.createStatement();
stmt.executeQuery(query);
} catch(IOException | SQLException ex) {
log.warning(ex.getMessage());
}
}
Stacktrace:
WARNING: Database 'java' does not exist. Make sure that the name is entered correctly.
java.sql.SQLException: Database 'java' does not exist. Make sure that the name is entered correctly.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:372)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2988)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2421)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:671)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQLQuery(JtdsStatement.java:505)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeQuery(JtdsStatement.java:1427)
at com.exampe.MyJTDSConnection.runStatement(MyJTDSConnection.java:238)
at com.exampe.MyJTDSConnection.loadPageTitle(MyJTDSConnection.java:208)
at com.exampe.MyJTDSConnection.runTesting(MyJTDSConnection.java:69)
at com.exampe.SeleniumTesting.runTest(SeleniumTesting.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
As mentioned in the comment to your question, applying the .toString() method to the InputStream object does not read the InputStream. Instead it just returns a String representation of the object itself, not what the object contains.
For example, my Java project has a resource file named "script.sql" that contains:
SELECT ##VERSION
The following code compares the result of simply doing a .toString() on the object vs. using Apache Commons IO to actually read the InputStream into a String:
package resourceTest;
import java.io.InputStream;
import org.apache.commons.io.IOUtils;
public class ResourceTestMain {
public static void main(String[] args) {
try (InputStream is = ResourceTestMain.class.getClassLoader().getResourceAsStream("resources/script.sql")) {
String toStringValue = is.toString();
String contents = IOUtils.toString(is, "UTF-8");
is.close();
System.out.println("is.toString() returned:");
System.out.println(" " + toStringValue);
System.out.println();
System.out.println("IOUtils.toString(is, \"UTF-8\") returned:");
System.out.println(" " + contents);
} catch (Exception e) {
e.printStackTrace(System.out);
}
}
}
The results are:
is.toString() returned:
java.io.BufferedInputStream#804a77
IOUtils.toString(is, "UTF-8") returned:
SELECT ##VERSION

MySQL DATA Truncation Error occuring from Java application but not directly on MySQL

I am using the JDBC driver to connect to a mysql database and using the "LOAD DATA INFILE" command in my java application to load(insert) a text file into the database. I am getting the following error: Data truncation: Data too long for column xxx at row 1.
However if I load the same text file manually by logging into the database and entering the SQL manually, the data loads fine.
Can someone pelase tell me what the error might be?
I am running this on Red Hat Enterprise Linux Server release 5.8 and the jdk version is 1.5.0_16 if that helps
This is the function used to load the data
public static void loaddata(Connection conn, String filename, String tablename)
{
try{
Statement stmt = null;
stmt = conn.createStatement();
File file = new File(filename);
file.getAbsolutePath().replace("\\", "\\\\");
String cmd = "LOAD DATA INFILE '"
+ file.getAbsolutePath().replace("\\", "\\\\")
+ "' INTO TABLE " + tablename + " FIELDS TERMINATED BY \'^\'";
stmt.executeUpdate(cmd);
System.out.println("cmd :" + cmd);
}
catch(SQLException sqle){
sqle.printStackTrace();
}
}
This is the function to create the JDBC connection:
public static Connection createConnection()
{
Connection conn=null;
try {
Class.forName("com.mysql.jdbc.Driver").newInstance();
}
catch(Exception e) {
e.printStackTrace();
}
try {
String url = ""; //URL mentioned in the actual code
conn = DriverManager.getConnection(url, user, pass);
} catch (SQLException sqe1) {
sqe1.printStackTrace();
}
return conn;
}

Categories

Resources