I am trying to find regular expression and if there are some duplicates, keep the unique ones and put the rest in a trash table.
but I get this Erro which I do not know what it is!
Here is my code:
public class RegexRemoverMain {
public static void main(String[] args) throws SQLException, ClassNotFoundException{
//Connection Parameters and Connect to Postgres Database
String data = "jdbc:postgresql://localhost:5432/postgres";
Class.forName("org.postgresql.Driver");
Connection conn = null;
//Connect to DB
conn = DriverManager.getConnection(
data, "username", "password");
//statements to get distinct owners
Statement ownerSt = conn.createStatement(
ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_UPDATABLE);
//statement to get Image Ids of a user
Statement ownersImagesIdsSt = conn.createStatement(
ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_UPDATABLE);
String insertSQL;
//an arraylist to store unique titles+tags reported by user
ArrayList<List<String>> result = new ArrayList<List<String>>();
//list for storing those Ids of a users which are filtered
List<String> filteredIds = new ArrayList<String>();
//list for storing those Ids of a users which are kept
List<String> ids = new ArrayList<String>();
//get the list of all the users
ResultSet distinctOwner = ownerSt.executeQuery("select distinct owner from \"flickrData_bulkUploadedFree\"");
distinctOwner.last();
distinctOwner.beforeFirst();
int count=0;
//RegularExpression Pattern
String theRegex= "((DSC)?(dsc)?(img)?(IMG)?(\\s?)(\\_?)((\\-?))[0-9]{1,9})";
Pattern checkRegex = Pattern.compile(theRegex);
//loop is going through all user's Images and check whether their the titles is one of the patterns if yes, check their title+description which are unique or not
//if yes, we keep them; if not; we throw them away or store in another place
while(distinctOwner.next()){
count = count++;
Statement insertSt = conn.createStatement(
ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_UPDATABLE);
//store filtered images
String insertString = "INSERT INTO regexIamges"
+ "( id , owner, descriptio, title, tags) VALUES"
+ "(?,?,?,?,?)";
PreparedStatement preparedStatement = conn.prepareStatement(insertString);
//for each user exist in "flickrData_bulkUploadedFree"
String owner = distinctOwner.getString("owner");
ResultSet ownersImages;
ownersImages = ownersImagesIdsSt.executeQuery("select id, title, tags, descriptio from \"flickrData_bulkUploadedFree\" where owner = '" + owner +"';");
ownersImages.last();
ownersImages.beforeFirst();
//an list of images of a user's with the information about id, title, tags and descriptions in order to find unique Images
ArrayList<List<String>> bulkUploadList = new ArrayList<List<String>>();
while(ownersImages.next()){
String id = ownersImages.getString("id");
String title = ownersImages.getString("title");
String tags = ownersImages.getString("tags");
String description = ownersImages.getString("descriptio");
Matcher regexMatcher = checkRegex.matcher(title);
if (regexMatcher.find()){
if(regexMatcher.group().length() != 0){
List<String> rowsList = new ArrayList<String>();
rowsList.add(id);
rowsList.add(title);
rowsList.add(tags);
rowsList.add(description);
bulkUploadList.add(rowsList);
bulkUploadList.add(rowsList);
}
}
else{
insertSQL = "INSERT INTO \"regBulkfreeFlickrData\" SELECT * FROM \"flickrData_bulkUploadedFree\" where id ='"+id+"';";
insertSt.addBatch(insertSQL);
}
}
HashSet<String> hashSet = new HashSet<String>();
for(List<String> item : bulkUploadList) {
String title, tags, id, desc, uniqueString;
title = item.get(1);
tags = item.get(2);
id = item.get(0);
desc = item.get(3);
uniqueString = (tags + "#" + desc).trim().toUpperCase();
System.out.println(item);
if(!hashSet.contains(uniqueString)) {
result.add(item);
hashSet.add(uniqueString);
insertSQL = "INSERT INTO \"regBulkfreeFlickrData\" SELECT * FROM \"flickrData_bulkUploadedFree\" where id ='"+id+"';";
insertSt.addBatch(insertSQL);
} else {
// System.out.println("Filtered element " + uniqueString + "id " + id);
filteredIds.add(id);
preparedStatement.setString(1, id);
preparedStatement.setString(2, owner);
preparedStatement.setString(3, desc);
preparedStatement.setString(4, title);
preparedStatement.setString(5, tags);
preparedStatement.addBatch();
}
}
preparedStatement.executeBatch();
preparedStatement.close();
insertSt.executeBatch();
insertSt.close();
}
}
and the Error is this:
Exception in thread "main" java.sql.BatchUpdateException: Batch entry 0 INSERT INTO regexIamges( id , owner, descriptio, title, tags) VALUES('4292220054.0000000000000','23352125#N07','NoValue','IMG_2720','NoValue') was aborted. Call getNextException to see the cause.
at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2743)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1928)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:405)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2892)
at uzh.textmining.RegexRemoverMain.main(RegexRemoverMain.java:116)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
and the table is:
CREATE TABLE "RegexImages"
(id numeric,
owner character varying(254),
descriptio character varying(254),
title character varying(254),
tags character varying(254),
PRIMARY KEY (id)
)
thanks hasnae;
I used try catch and I got that the tableName in my code does not match the table Name in database.
another problem was the name of the table: I change all the letters to lower case to solve all the errors.
A problem that looks similar: https://stackoverflow.com/a/39227828/755804
(spoiler:
The total number of values, that is, the number of columns multiplied by the number of rows must not exceed 32767 for a single INSERT statement.
You can divide 32767 by the number of columns to get the maximal number of rows per one SQL INSERT statement.)
Related
I am using a mysql table, and now I need to compare a columns all values with a given String.
I want to check if all values of the result set matches with encryptedString.
Need to understand what result set does and how it works.
Here I have a method, Some variables, and 2 mysql queries.
final String secretKey = "!!!!";
String name = jText.getText();
String pass = jTextPass.getText();
String originalString = pass;
String encryptedString = AES.encrypt(originalString, secretKey) ;
String decryptedString = AES.decrypt(encryptedString, secretKey) ;
PreparedStatement PS;
ResultSet result;
String query1 = "SELECT `pass` FROM `Remember_Pass` WHERE `name` =?";
PreparedStatement ps;
String query;
query = "UPDATE `tutor profile` SET `pass`=? WHERE `name`=?";
try {
PS = MyConnection.getConnection().prepareStatement(query1);
PS.setString(1, name);
PS.setString(2, encryptedString);
rs = PS.executeQuery();
//while(result.next() ){
//I am not understanding what to do here.
ps = MyConnection.getConnection().prepareStatement(query);
ps.setString(1, encryptedString);
ps.setString(2, name);
ps.executeUpdate();
PassSuccess success = new PassSuccess();
success.setVisible(true);
success.pack();
success.setLocationRelativeTo(null);
this.dispose();
//}
} catch (SQLException ex) {
Logger.getLogger(ForgetPassT.class.getName()).log(Level.SEVERE, null, ex);
}
First tip: using try-with-resources closes statement and result set even on exception or return. This also reduces the number of variable names for them because of the smaller scopes. This return from the innermost block I utilized. For unique names one can use if-next instead of while-next. A fail-fast by not just logging the exception is indeed also better; you can exchange the checked exception with a runtime exception as below, so it easier on coding.
String query1 = "SELECT `pass` FROM `Remember_Pass` WHERE `name` = ?";
String query = "UPDATE `tutor profile` SET `pass`=? WHERE `name`= ?";
try (PreparedStatement selectPS = MyConnection.getConnection().prepareStatement(query1)) {}
selectPS.setString(1, name);
//selectPS.setString(2, encryptedString);
try (ResultSet rs = selectPS.executeQuery()) {}
if (result.next()){ // Assuming `name` is unique.
String pass = rs.getString(1);
try (PreparedStatement ps = MyConnection.getConnection().prepareStatement(query)) {
ps.setString(1, encryptedString);
ps.setString(2, name);
int updateCount = ps.executeUpdate();
if (updateCount == 1) {
PassSuccess success = new PassSuccess();
success.setVisible(true);
success.pack();
success.setLocationRelativeTo(null);
return success;
}
}
}
}
} catch (SQLException ex) {
Logger.getLogger(ForgetPassT.class.getName()).log(Level.SEVERE, null, ex);
throw new IllegalStateException(e);
} finally {
dispose();
}
the ResultSet object contains all the information about the query that you perform, it will contain all columns. In your code the result variable will return anything since there is no part in your code where is executed, to do this you have to...
Statement statement = MyConnection.getConnection().createStatement();
ResultSet result = statement.executeQuery("YOUR SELECT STATEMENT HERE");
while(result.next()){
String column1 = result.getString("columnName");
}
The result.next() method is a boolean method that says if the ResultSet object still have values of the table inside and it will continue until it reaches the last row that your SELECT statement retrives. Now if you want to match the value of some column with other variables you can do it inside the while(result.next()).
result.getString("columnName") will extract the value from columnName as a String.
If you want to save things in an ArrayList to save the data and then use this list as you want the code can be like...:
Statement statement = MyConnection.getConnection().createStatement();
ResultSet result = statement.executeQuery("YOUR SELECT STATEMENT HERE");
List<Object> data = new ArrayList();
while(result.next()){
data.add(result.getString("columnName"));
}
return data;
Obviously you have to change the Object with the type of things that you want to store in the List.
Or if you want to store the data in an array. As I said in my comment this won't be dinamic, but...:
Statement statement = MyConnection.getConnection().createStatement();
ResultSet result = statement.executeQuery("YOUR SELECT STATEMENT HERE");
String[] data = new String[NUMBER_OF_COLUMNS_IN_RESULTSET];
while(result.next()){
data[0] = result.getString("columnName1");
data[1] = result.getString("columnName2");
data[2] = result.getString("columnName3");
//And so on...
}
return data;
The other way is that if you are returning an entity you can set the values of the ResultSet directly in the POJO:
Statement statement = MyConnection.getConnection().createStatement();
ResultSet result = statement.executeQuery("YOUR SELECT STATEMENT HERE");
Entity entity = new Entity();
while(result.next()){
entity.setColumnName1(result.getString("columnName1"));
entity.setColumnName2(result.getString("columnName2"));
entity.setColumnName3(result.getString("columnName3"));
//And so on...
}
return entity;
There are so many ways to store the data, just ask yourself how do you want to receive the data in the other parts of you code.
Regards.
I have used this method without using the join in the query and it was working as expected. But I added a inner join and now it can't update the "used" column
public HashMap<String, Comparable> getPhoneNumberAndMarkAsUsed() {
String[] colNames = { "phone_number.id", "phone_number.phone_number",
"phone_number.account_id", "phone_number.used AS used",
"(now() AT TIME ZONE account.timezone)::time AS local_time" };
String query = "select " + Stream.of(colNames).collect(Collectors.joining(", "))
+ " from account INNER JOIN phone_number ON account.id = phone_number.account_id where phone_number.used = false order by id DESC limit 1 for update";
HashMap<String, Comparable> account = new HashMap<String, Comparable>();
try (Connection conn = DriverManager.getConnection(url, props); // Make sure conn.setAutoCommit(false);
Statement stmt = conn.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE, ResultSet.CONCUR_UPDATABLE);
ResultSet rs = stmt.executeQuery(query)) {
conn.setAutoCommit(false);
ResultSetMetaData rsmd = rs.getMetaData();
int columnsNumber = rsmd.getColumnCount();
while (rs.next()) {
for (int i = 1; i <= columnsNumber; i++) {
if (i > 1)
System.out.print(", ");
String columnValue = rs.getString(i);
System.out.print(columnValue + " " + rsmd.getColumnName(i));
}
// Get the current values, if you need them.
account.put("phone_number", rs.getString("phone_number"));
account.put("account_id", rs.getLong("account_id"));
rs.updateBoolean("used", true);
rs.updateRow();
}
conn.commit();
} catch (SQLException e) {
e.printStackTrace();
}
return account;
}
the loop prints the following
7223 id, 10001234567 phone_number, 1093629 account_id, f used, 23:32:42.502472 local_time
accourding to the output above, then I am use that column "used" is part of the ResultSet. But I get the following Exception
org.postgresql.util.PSQLException: ERROR: column "used" of relation "account" does not exist
This is the query when printed
select phone_number.id, phone_number.phone_number, phone_number.account_id, phone_number.used AS used, (now() AT TIME ZONE account.timezone)::time AS local_time from account INNER JOIN phone_number ON account.id = phone_number.account_id where phone_number.used = false order by id DESC limit 1 for update
used belongs to the phone_number table not the account table. How can this be resolved?
here is the problem in your code:
rs.updateBoolean("used", true);
this statement will try to update the data of table through resultset but to do that you cannot user join and also there is one problem.
As you are updating via resultset it will try to update account table and if we find used column is account table then error occurs.
so your code is trying to find column "used" in account table but it is not there.
try this one:
String query = "select " + Stream.of(colNames).collect(Collectors.joining(", "))
+ " from phone_number INNER JOIN account phone_number ON account.id = phone_number.account_id where phone_number.used = false order by id DESC limit 1 for update";
I am trying to insert values into my database via java but I keep getting a "doesn't have a default value" error message. I have looked online to find some help but they haven't really solved my problem. This is the error that I am getting "java.sql.SQLException: Field 'firstName' doesn't have a default value".
import java.sql.*;
class MysqlCon {
public static void main(String[] args) {
String personalID = null;
String firstname = null;
String lastname = null;
String addressone = null;
String addresstwo = null;
String city = null;
String state = null;
String zipcode = null;
String phone = null;
String email = null;
try
{
Class.forName("com.mysql.jdbc.Driver");
Connection con = DriverManager.getConnection("jdbc:mysql://localhost:3306/Address_Book","root","");
Statement stmt = con.createStatement();
personalID ="3";
firstname = "John";
lastname = "Doe";
addressone = "4000 s.c.r. 222";
addresstwo = "5000 n.c.r. 333";
city = "place";
state = "Texas";
zipcode = "43523";
phone = "43523524";
email = "weqtatw#aadf.com";
stmt.executeUpdate("insert into Names(personID) values("+personalID+")");
stmt.executeUpdate("insert into Names(firstname) values("+firstname+")");
stmt.executeUpdate("insert into Names(lastname) values("+lastname+")");
stmt.executeUpdate("insert into addresses(address1) values("+addressone+")");
stmt.executeUpdate("insert into addresses(address2) values("+addresstwo+")");
stmt.executeUpdate("insert into addresses(city) values("+city+")");
stmt.executeUpdate("insert into addresses(state) values("+state+")");
stmt.executeUpdate("insert into addresses(zipcode) values("+zipcode+")");
stmt.executeUpdate("insert into phoneNumbers(phoneNumber) values("+phone+")");
stmt.executeUpdate("insert into emailAddresses(emailAddress) values("+email+")");
con.close();
}catch(Exception e){System.out.println(e);}
}
}
You're currently running three separate insert statements into one table, such as:
INSERT INTO Names (personID) values (3);
INSERT INTO Names (firstname) values ("John");
INSERT INTO Names (lastname) values ("Doe");
You will end up with 3 separate rows, instead of a single row. If any of the columns require a value (i.e. if you have a primary key without an auto_increment), the insert will fail since it will require a value for the column (edit: in this case you have a required value for the firstName field).
You'll also have a very useless table that will contain entries such as a row with nothing but the name "John", a row with nothing but the last name "Doe", and a row with personID of 3.
Instead, you should be running a single INSERT statement, such as:
stmt.executeUpdate("insert into Names(personID, firstname, lastname) values("+personalID+", "+firstname+", "+lastname+")");
Which should also populate all required cells in a row.
Please try to run one of insert into commands in database shell first - you will see that command like insert into Names(personID) values(3) is not correct because DB does not know what to insert into other fields. However, it would know if there is a default value for missing fields - that's why DB is complaining.
If you do not have default values for that table (they are set up while table creation), you must give all values at once like this:
INSERT INTO table_name (column1, column2, column3, ...) VALUES (value1, value2, value3, ...);
I want to add values in a table that has dynamic columns.
I managed to create a table with dynamic columns but I cannot figure out how to insert data.
//Create Table
sql = "CREATE TABLE MyDB.myTable" +
"(level INTEGER(255) )";
int columnNumber = 5; //Number of columns
//Add columns
for (i=0;i<columnNumber;i++){
String columnName = "Level_" +i:
String sql = "ALTER TABLE MyDB.myTable ADD " + columnName + " INTEGER(30)";
}
//Insert Data
//How to insert data dynamically, without knowing the number of columns?
You can also use database metadata to get the column names. This has the advantage that you even don't need to know the column names, rather they are retrieved dynamically in your code.
public static List<String> getColumns(String tableName, String schemaName) throws SQLException{
ResultSet rs=null;
ResultSetMetaData rsmd=null;
PreparedStatement stmt=null;
List<String> columnNames =null;
String qualifiedName = (schemaName!=null&&!schemaName.isEmpty())?(schemaName+"."+tableName):tableName;
try{
stmt=conn.prepareStatement("select * from "+qualifiedName+" where 0=1");
rs=stmt.executeQuery();//you'll get an empty ResultSet but you'll still get the metadata
rsmd=rs.getMetaData();
columnNames = new ArrayList<String>();
for(int i=1;i<=rsmd.getColumnCount();i++)
columnNames.add(rsmd.getColumnLabel(i));
}catch(SQLException e){
throw e;//or log it
}
finally{
if(rs!=null)
try {
rs.close();
} catch (SQLException e) {
// TODO Auto-generated catch block
throw e
}
if(stmt!=null)
try {
stmt.close();
} catch (SQLException e) {
// TODO Auto-generated catch block
throw e
}
}
return columnNames;
}
Once you have the column names, you can use it as you normally would (List.size() would of course give the number of columns).
UPDATE:
//I will assume that your values (data to be inserted) is a List of Object types and that it is already populated
List<Object> data = new ArrayList<>();
//you will populate this list
//getting the column names
List<String> columnNames = getColumns("MyTable", "MyDB");
String insertColumns = "";
String insertValues = "";
if(columnNames != null && columnNames.size() > 0){
insertColumns += columnNames.get(0);
insertValues += "?";
}
for(int i = 1; i < columnNames.size();i++){
insertColumns += ", " + columnNames.get(i) ;
insertValues += "?";
}
String insertSql = "INSERT INTO MyDB.MyTable (" + insertColumns + ") values(" + insertValues + ")";
try{
PrepareStatement ps = conn.prepareStatement(insertSql);
for(Object o : data){
ps.setObject(o); //you must pass objects of correct type
}
ps.execute(); //this inserts your data
}catch(SQLException sqle){
//do something with it
}
This code assume that you pass objects of correct types to PreparedStatement.setObject(Object o) method. It's also possible to retrieve column types using metadatabase information and then use that info to enforce type checking but that would make your code much more complicated
If you know the number of columns you want to insert, you can make your insert query same way you made your CREATE TABLE. Explicitly name the columns you want to add your data into, and make sure the columns you leave empty are allowed to be empty (NULL or default=0)
INSERT INTO MyDB.myTable (Level_1, Level_2, ...) VALUES (Val_1, Val_2, ...);
The alternative approach would be to store each inserted value in a separate row. In that way you don't vhave to change the number of columns in your table.
You need a table where you have a ID for every group of values:
- ID
- Level
- Value
You could have a separate table where you can register each ID:
- ID (bigInt, auto increment, primary key)
- info field
- timestamp
Now, for every set of data you want to insert, first insert need a Unique ID. If you use the second table, inserting a new row would give you a new ID:
INSERT INTO register_table (ID, info, timestamp) VALUES (NULL, 'some info', NOW());
This will give you a new ID (last_inserted_id).
With this ID now insert all values in the other table:
for(i=0i<columnNumber;i++){
"INSERT INTO _table (ID, Level, _Value) VALUES ("+ the_ID +", "+ i +", "+ the_VALUE +");";
}
If you want to fetch the data:
"SELECT Level, _Value FROM _table WHERE ID="+ the_ID +" ORDER BY Level;";
I'm trying to write a Java code snippet to return the keywords contained in the description of a specific entity & SELECT only those records that contain both keywords.
The following code works, however it returns entities whose body contains only one or the other of the words. I need it to return only those records that contain both keywords, not just one of them.
try {
String search = txt_search.getText();
search = search.trim();
search = search.replace (' ','|');
String sql = "SELECT * FROM catagory WHERE keywords REGEXP '" + search + "'";
pst = conn.prepareStatement(sql);
rs = pst.executeQuery();
if (rs.next()){
String add1 = rs.getString("CatagoryId");
txt_catagoryid.setText(add1);
String add2 = rs.getString("CatagoryName");
txt_catagoryName.setText(add2);
String add3 = rs.getString("keywords");
txt_keywords.setText(add3);
rs = stmt.executeQuery(sql);
catagoryTable.setModel(DbUtils.resultSetToTableModel(rs));
} catch(Exception e){
...
}