I'm writing a Java application that uses SQLite via JDBC for persistence.
What I need to do when the application starts is:
If the DB doesn't exists, create it with the right schema
If the DB exists, check if the DB has the right schema
In order to create the DB I thought of externalizing the SQL commands needed to create the structure (e.g. CREATE TABLE ...) in an external file, load it at runtime and execute it.
I thought of using BufferedReader with .readLine() and then feed each SQL command to a Statement.executeUpdate(). Is there some smarter way to do this?
Concerning the startup check of the database schema, while I think that this should be considered "good design", I don't know if in practice it could be overkill or useless.
One solution I came up with but has the disadvantage of being "vendor dependent" is this:
Create the DB by using a known prefix for all the structures' names (e.g. nwe_)
Query the sqlite_master table with WHERE name LIKE 'nwe_%'
Compare the content of the SQL column with the SQL command file I used for creating the DB
Again, is there some smarter way to do this? (maybe not "vendor dependent", but this at the time is not much of an issue for me)
Thanks.
I'm not a professional programmer and this is my first java application, but I decided to do
database verification too.
Fortunately, sqlite databases has a system table with sql statements for each table in database.
So there is a fast and dirty way to check database structure:
public class Model {
private Map<String, String> tableSql;
Model(){
// declare right database's tables syntax
tableSql = new HashMap<String, String>();
tableSql.put("settings", "CREATE TABLE [settings] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [item] ... )");
tableSql.put("scheta", "CREATE TABLE [scheta] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [sname] ...)");
tableSql.put("nomera", "CREATE TABLE [nomera] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [nvalue] ...)");
tableSql.put("corr", "CREATE TABLE [corr] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [cname] ...)");
tableSql.put("category", "CREATE TABLE [category] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [cname] ...)");
tableSql.put("price", "CREATE TABLE [price] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [cdate] ...)");
}
public void Connect( String path){
File DBfile = new File (path);
boolean DBexists = DBfile.exists();
Statement stmt = null;
ResultSet rs;
try {
Class.forName("org.sqlite.JDBC");
c = DriverManager.getConnection("jdbc:sqlite:" + path);
c.setAutoCommit(true);
stmt = c.createStatement();
if( DBexists ){
// check database structure
for (String tableName : tableSql.keySet()) {
rs = stmt.executeQuery( "SELECT sql FROM sqlite_master WHERE type = 'table' AND name = '" + tableName + "'");
if(rs.isBeforeFirst()){
rs.next();
// table and field names may be inside square brackets or inside quotes...
String table_schema_there = rs.getString(1).replaceAll("\\s+"," ").replaceAll("[\\[\\]'`]", "\"");
String table_schema_here = tableSql.get(tableName).replaceAll("\\s+"," ").replaceAll("[\\[\\]'`]", "\"");;
if( ! table_schema_there.equals(table_schema_here) ){
notifyListeners( new ModelResponse( false, "Structure error. Wrong structure of table " + tableName));
System.exit(0);
}
}
else{
notifyListeners( new ModelResponse( false, "Structure error. The table is missing: " + tableName ));
System.exit(0);
}
}
}
else{
// empty DB file created during connection so we need to create schema
for (String tableName : tableSql.keySet()) {
stmt.executeUpdate(tableSql.get(tableName));
}
}
}
catch ( Exception e ) {
notifyListeners( new ModelResponse( false, e.getMessage()));
System.exit(0);
}
}
}
You can write up your own verification of the database by looking in the metdata tables. This is not impossible, but it is a lot of code to maintain. You can also write up a lot of DDL statements to construct the database structures, again not impossible, but a lot of code to maintain.
If you follow such a path, I recommend a higher-level logic of
if (!checkDatabase()) {
try {
if (wantsToInstall()) {
installDatabase();
}
} catch (Exception e) {
exit();
}
}
or a specific installer (to reduce the chance of installing over existing items). Extensions that provide deleting data items can be done, but there are differing opinions as to what to do if encountering an unexpected pre-existing object during install / upgrade.
Now there are a few libraries that will do this for you; but by the time they add this feature, such a library probably takes care of much more than simple database structure maintenance. Look to JDO and a few others, I know that DataNucleus's environment will gladly create database structures tweaked to work well in just over 20 different databases (remember they all have subtle differences in how they do things).
Related
I am having code something like this.
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
stmt.setString(1, addressName);
Calculation of fullTableName is something like:
public String getFullTableName(final String table) {
if (this.schemaDB != null) {
return this.schemaDB + "." + table;
}
return table;
}
Here schemaDB is the name of the environment(which can be changed over time) and table is the table name(which will be fixed).
Value for schemaDB is coming from an XML file which makes the query vulnerable to SQL injection.
Query: I am not sure how the table name can be used as a prepared statement(like the name used in this example), which is the 100% security measure against SQL injection.
Could anyone please suggest me, what could be the possible approach to deal with this?
Note: We can be migrated to DB2 in future so the solution should compatible with both Oracle and DB2(and if possible database independent).
JDBC, sort of unfortunately, does not allow you to make the table name a bound variable inside statements. (It has its reasons for this).
So you can not write, or achieve this kind of functionnality :
connection.prepareStatement("SELECT * FROM ? where id=?", "TUSERS", 123);
And have TUSER be bound to the table name of the statement.
Therefore, your only safe way forward is to validate the user input. The safest way, though, is not to validate it and allow user-input go through the DB, because from a security point of view, you can always count on a user being smarter than your validation.
Never trust a dynamic, user generated String, concatenated inside your statement.
So what is a safe validation pattern ?
Pattern 1 : prebuild safe queries
1) Create all your valid statements once and for all, in code.
Map<String, String> statementByTableName = new HashMap<>();
statementByTableName.put("table_1", "DELETE FROM table_1 where name= ?");
statementByTableName.put("table_2", "DELETE FROM table_2 where name= ?");
If need be, this creation itself can be made dynamic, with a select * from ALL_TABLES; statement. ALL_TABLES will return all the tables your SQL user has access to, and you can also get the table name, and schema name from this.
2) Select the statement inside the map
String unsafeUserContent = ...
String safeStatement = statementByTableName.get(usafeUserContent);
conn.prepareStatement(safeStatement, name);
See how the unsafeUserContent variable never reaches the DB.
3) Make some kind of policy, or unit test, that checks that all you statementByTableName are valid against your schemas for future evolutions of it, and that no table is missing.
Pattern 2 : double check
You can 1) validate that the user input is indeed a table name, using an injection free query (I'm typing pseudo sql code here, you'd have to adapt it to make it work cause I have no Oracle instance to actually check it works) :
select * FROM
(select schema_name || '.' || table_name as fullName FROM all_tables)
WHERE fullName = ?
And bind your fullName as a prepared statement variable here. If you have a result, then it is a valid table name. Then you can use this result to build a safe query.
Pattern 3
It's sort of a mix between 1 and 2.
You create a table that is named, e.g., "TABLES_ALLOWED_FOR_DELETION", and you statically populate it with all tables that are fit for deletion.
Then you make your validation step be
conn.prepareStatement(SELECT safe_table_name FROM TABLES_ALLOWED_FOR_DELETION WHERE table_name = ?", unsafeDynamicString);
If this has a result, then you execute the safe_table_name. For extra safety, this table should not be writable by the standard application user.
I somehow feel the first pattern is better.
You can avoid attack by checking your table name using regular expression:
if (fullTableName.matches("[_a-zA-Z0-9\\.]+")) {
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
stmt.setString(1, addressName);
}
It's impossible to inject SQL using such a restricted set of characters.
Also, we can escape any quotes from table name, and safely add it to our query:
fullTableName = StringEscapeUtils.escapeSql(fullTableName);
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
stmt.setString(1, addressName);
StringEscapeUtils comes with Apache's commons-lang library.
I think that the best approach is to create a set of possible table names and check for existance in this set before creating query.
Set<String> validTables=.... // prepare this set yourself
if(validTables.contains(fullTableName))
{
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
//and so on
}else{
// ooooh you nasty haker!
}
create table MYTAB(n number);
insert into MYTAB values(10);
commit;
select * from mytab;
N
10
create table TABS2DEL(tname varchar2(32));
insert into TABS2DEL values('MYTAB');
commit;
select * from TABS2DEL;
TNAME
MYTAB
create or replace procedure deltab(v in varchar2)
is
LvSQL varchar2(32767);
LvChk number;
begin
LvChk := 0;
begin
select count(1)
into LvChk
from TABS2DEL
where tname = v;
if LvChk = 0 then
raise_application_error(-20001, 'Input table name '||v||' is not a valid table name');
end if;
exception when others
then raise;
end;
LvSQL := 'delete from '||v||' where n = 10';
execute immediate LvSQL;
commit;
end deltab;
begin
deltab('MYTAB');
end;
select * from mytab;
no rows found
begin
deltab('InvalidTableName');
end;
ORA-20001: Input table name InvalidTableName is not a valid table name ORA-06512: at "SQL_PHOYNSAMOMWLFRCCFWUMTBQWC.DELTAB", line 21
ORA-06512: at "SQL_PHOYNSAMOMWLFRCCFWUMTBQWC.DELTAB", line 16
ORA-06512: at line 2
ORA-06512: at "SYS.DBMS_SQL", line 1721
Go easy on me, middle school teacher taking a CS class. I've got a Java program that asks for user name, height, weight, does some calculations and gives results to the user. I now need to store this data in a database. I can get the data to store until I start using primary and foreign keys.
Here is the error I can't figure out:
Error: java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate key value in a unique or primary key constraint or unique index identified by 'SQL180429151131780' defined on 'USERPROFILE'.
Here is my table:
drop table stayfitapp.userdata;
drop table stayfitapp.userprofile;
drop schema stayfitapp restrict;
create schema stayfitapp;
create table stayfitapp.userprofile
(
profileName varchar(255) not null primary key,
profileGender varchar(255) not null
);
create table stayfitapp.userdata
(
profileAge double not null,
profileWeight double not null,
profileHeight double not null,
profileWaistCircumference double not null,
profileHipCircumference double not null,
profileName varchar(255),
foreign key (profileName) references stayfitapp.userprofile(profileName)
);
Here is the section of the "app" that writes to the table...
public void save(){
try {
String query = "insert into stayfitapp.userprofile" + "(profileName, profileGender)" + "values" + "(?,?)";
String query2 = "insert into stayfitapp.userdata" + "(profileAge, profileWeight, profileHeight, profileWaistCircumference, profileHipCircumference)" + "values" + "(?,?,?,?,?)";
Connection myConnection = DriverManager.getConnection("jdbc:derby://localhost:1527/stayfitDB2", "username", "password");
Statement myStatement = myConnection.createStatement();
//Statement myStatement2 = myConnection.createStatement();
PreparedStatement prepared = myConnection.prepareStatement(query);
prepared.setString(1, profileName);
prepared.setString(2, profileGender);
PreparedStatement prepared2 = myConnection.prepareStatement(query2);
prepared2.setDouble(1, profileAge);
prepared2.setDouble(2, profileWeight);
prepared2.setDouble(3, profileHeight);
prepared2.setDouble(4, profileWaistCircumference);
prepared2.setDouble(5, profileHipCircumference);
int rowsAffected = prepared.executeUpdate();
int rowsAffected2 = prepared2.executeUpdate();
if(rowsAffected==0)
{
System.out.println("Warning: User data did not save!");
}
else
{
System.out.println("User info saved!");
}
}
catch(SQLException e)
{
System.out.println("Error: "+e.toString());
}
Your save() method will attempt to add the user to the stayfitapp.userprofile table. This table has a field called profileName. profileName is the "primary key" so no duplicate values are allowed.
The error that you are getting is saying that you cannot add(insert) the record to the table because the table already has a record with the same name.
Does your program work okay if you use a different name each time?
You will need to add some logic to your program to deal with the scenario where the profileName already exists in the table. This will probably involve deleting or updating the existing record.
This is the problem.
insert into stayfitapp.userprofile"
+ "(profileName, profileGender)" + "values" , etc
You have nothing to check to see if a record already exists. Something like this would work better.
insert into stayfitapp.userprofile
profileName, profileGender
select distinct ?, ?
from someSmallTable
where not exists (
select 1
from stayfitapp.userprofile
where profileName = ?
)
The someSmallTable bit depends on your database engine, which you didn't specify.
I ended up writing a method to check if the username was already in the profile table. If the username was a duplicate I only wrote to the data table. If the username was new I wrote to both tables.
Thank you for your help! I'm sure there was a more efficient method (figuratively and literally) but I'm on to my final project and nearly surviving an actual CS class.
I need to use the database Firebird and for this I use the Jaybird 2.2.9.
When I used the MySQL driver, to converter of ResultSet to Object this way:
empresa.setBairro(rs.getString("empresa.bairro")); // (Table.Column)
empresa.setCep(rs.getString("empresa.cep")); // (Table.Column)
empresa.setCidade(rs.getString("empresa.cidade")); // (Table.Column)
But with Jaybird the resultSet don't return rs.getString("Table.Column")
I need this way when I have inner join in SQL.
Anyone help me?
This is my full code
public ContaLivros converterContaLivros(ResultSet rs, Integer linha) throws Exception {
if (rs.first()) {
rs.absolute(linha);
ContaLivros obj = new ContaLivros();
obj.setId(rs.getLong("cad_conta.auto_id"));
obj.setNome(rs.getString("cad_conta.nome"));
if (contain("cad_banco.auto_id", rs)) {
obj.setBancoLivros(converterBancoLivros(rs, linha));
} else {
obj.setBancoLivros(new BancoLivros(rs.getLong("cad_conta.banco"), null, null, null));
}
obj.setAgencia(rs.getInt("cad_conta.agencia"));
obj.setAgenciaDigito(rs.getInt("cad_conta.agencia_digito"));
obj.setConta(rs.getInt("cad_conta.conta"));
obj.setContaDigito(rs.getInt("cad_conta.conta_digito"));
obj.setLimite(rs.getDouble("cad_conta.limite"));
obj.setAtivo(rs.getString("cad_conta.ativo"));
return obj;
} else {
return null;
}
}
You can't. Jaybird retrieves the columns by its label as specified in JDBC 4.2, section 15.2.3. In Firebird the column label is either the original column name, or the AS alias, the table name isn't part of this. The extension of MySQL that you can prefix the table name for disambiguation is non-standard.
Your options are to specify aliases in the query and retrieve by this aliasname, or to process the result set metadata to find the right indexes for each column and retrieve by index instead.
However note that in certain queries (for example UNION), the ResultSetMetaData.getTableName cannot return the table name, as Firebird doesn't "know" it (as you could be applying a UNION to selects from different tables).
The name in jdbc will not have the table in it.
You can either
work with positional parameters ( getString (1); and so on )
Or
define column name alias in your select (select a.name namefroma from tableone a )
Or
simply do rs.getString ("column"); without the table prefix if name is unambigous
I have a table with unique index to eliminate duplicates (simplified example)
CREATE TABLE `domain` (
`id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`subdomain` VARCHAR(255) NOT NULL,
`domain` VARCHAR(63) NOT NULL,
`zone` VARCHAR(63) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE INDEX `UNIQUE` (`subdomain` ASC, `domain` ASC, `zone` ASC),
ENGINE = InnoDB;
I insert a lot of rows and i need to get primary keys returned (for other one-to-many inserts).
My problem is, that I insert a lot of duplicates and I need those keys returned too.
This is my solution which works, but isn't there more simple solution? With this I cannot use batch inserts and I want this to be most efficient.
PreparedStatement selectDomain = connection.prepareStatement("SELECT id FROM domain WHERE subdomain = ? AND domain = ? AND zone = ?");
PreparedStatement insertDomain = connection.prepareStatement("INSERT INTO domain(subdomain, domain, zone) VALUES (?,?,?)", Statement.RETURN_GENERATED_KEYS);
public int insertDomain(String subdomain, String domain, String zone) throws SQLException {
int domainId = 0;
selectDomain.setString(1, subdomain);
selectDomain.setString(2, domain);
selectDomain.setString(3, zone);
ResultSet resultSet = selectDomain.executeQuery();
if (resultSet.next()) {
domainId = resultSet.getInt(1);
} else {
insertDomain.setString(1, subdomain);
insertDomain.setString(2, subdomain);
insertDomain.setString(3, subdomain);
insertDomain.executeUpdate();
resultSet = insertDomain.getGeneratedKeys();
if (resultSet.next()) {
domainId = resultSet.getInt(1);
}
}
selectDomain.clearParameters();
insertDomain.clearParameters();
}
As I understand its not so easy approach for using batch execution. Your apporach is the best way to get the auto generated keys. There are few limitations of JDBC driver and it varies version to version, where getGeneratedKeys() works for single entry.
Please look into below links, it may help you :-
How to get generated keys from JDBC batch insert in Oracle?
http://docs.oracle.com/database/121/JJDBC/jdbcvers.htm#JJDBC28099
You could modify your INSERT to be something like this:
INSERT INTO domain (subdomain, domain, zone)
SELECT $subdomain, $domain, $zone
FROM domain
WHERE NOT EXISTS(
SELECT subdomain, domain, zone
FROM domain d
WHERE d.subdomain= $subdomain and d.domain=$domain and d.zone=$zone
)
LIMIT 1
Where $subdomain, $domain, $zone are the tag (properly quoted or as a placeholder of course) that you want to add if it isn't already there. This approach won't even trigger an INSERT (and the subsequent autoincrement wastage) if the tag is already there. You could probably come up with nicer SQL than that but the above should do the trick.
If your table is properly indexed then the extra SELECT for the existence check will be fast and the database is going to have to perform that check anyway.
Mysql tables
mysqltables
This is my query to insert data to the database.
public void voegSpelerToe(Speler speler, String spelNaam)
{
PreparedStatement invoerSpeler;
Speler huidigeSpeler = null;
try
{
Connection connection = PersistentieController.getInstance().getConnection();
invoerSpeler = connection.prepareStatement("INSERT INTO Speler " + "(naam, kleur, sector, aantalZilverstukken, Spel_naam) " + "VALUES ( ?, ?,?, ?, ?)");
invoerSpeler.setString(1, speler.getNaam());
invoerSpeler.setString(2, speler.getKleur());
invoerSpeler.setInt(3, speler.getSector().getCode());
invoerSpeler.setInt(4,speler.getKrediet());
invoerSpeler.setString(5, spelNaam);
invoerSpeler.executeUpdate();
} catch (SQLException sqlException)
{
sqlException.printStackTrace();
} finally
{
// PersistentieController.getInstance().closeConnection();
}
}
Everything has a value so I don't have nullexeptions.
But when I want to save the data I get this error:
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (`projecteng62`.`speler`, CONSTRAINT `fk_Speler_Spel1` FOREIGN KEY (`Spel_naam`) REFERENCES `spel` (`naam`) ON DELETE CASCADE ON UPDATE CASCADE)
How can I insert data in the foreign key.
First It save data to the table Spel
and then i need to save data in the table speler but I get a problem wit the foreign Key.
Like table spel:
naam: Game12
aantalTeSPelenRondes: 2
table Speler:
naam : player1
kleur : green
sector : 2
aantalZilverStukken : 10
Spel_Naam: game12
Spel_naam must be the same as naam in table Spel
It is telling you that it is expecting the contents of Spel_Naam to exist in some row in spels naam attribute, but it does not exist.
To fully figure out the issue I would also need to see where you are inserting into spel.
But in the example input you provided
naam: Game12
Spel_Naam: game12
there is an issue because one is capitalized and the other is not. If this is actually how the data is set up, then that is likely your problem. But it seems you have the right idea, you need to insert into naam first, then into Spel_Naam.