Does anyone know of a standard way to retrieve value defined with Defaults in a database when you insert?
These are not primary keys but other columns, the getGeneratedKeys method only returns for auto-increment, but I have other defaults like LastUpdate (date) or CreatedOn (date).
I realize that some databases like MSSQL have an output option or Oracle Return option, but I'm looking for a common way to do it.
Use the generated key so you can then follow up with a SELECT allTheFieldsYouCareAbout FROM tableYouJustAddedSomethingTo WHERE unid = generatedKeyYouJustGot.
Yeah, that's annoying and somewhat dubious from a performance perspective (the primary key is doubtlessly indexed, so not too pricey, but it's still another back-and-forth over TCP or whatever pipe you're using to talk to your database).
It's also the only way that reliably works on all major JDBC drivers.
Related
Similar to here I'm annotating my class with
#Table(indexes = {#Index(columnList = "name")})
which attempts to create a non-unique index with the maximum length of the varchar column. Unfortunately that's not possible because it's a varchar(255) column of type utf8mb4. phpMyAdmin added KEY '...' (name(191)) by clicking on the respective buttons in the UI, so at least my software runs efficient queries now.
Now I was wondering if it's possible to have my Java class auto-generate the index with limited length upon creating the database schema? The code builds on spring-boot-starter-data-jpa:1.4.2.RELEASE.
There are other answers than trying to get the 3rd party software to do something it may or may not allow for.
Live with 191 limitation on the column size. Or, do you really have a max between 191 and 255.
Change to utf8 (from utf8mb4). And lose the ability to store Emoji and some Chinese characters.
There is a clumsy process in 5.6 to raise the 767 limit you are bumping into.
Upgrade to 5.7, which makes virtually eliminates the problem.
You should only use the JPA generated table scripts as a starting point, and you should never use JPA to create you tables in production.
If you have "create table" privileges, so you don't need a DBA to create and modify the database, then I recommend that you use Flyway to manage database creation and migration. If you need to be database agnostic, and like long XML files, you can also use LiquideBase.
With flyway, you would add a new script every time you add one or more entities. I typically let JPA create the script, and then copy what I need, and maybe do some modifications - for instance varchar(255) means 255 bytes on some databases, so you may want modify that if you are storing something other than Latin-1.
Flyway is very simple to use, and it is fully integrated into Spring boot, so you just add the unique index the way you want it in the first (or later) flyway script src/main/resources/db/migration/V1__initial_script.sql.
sorry, if the question title is misleading or not accurate enough, but i didn't see how to ask it in one sentence.
Let's say we have a table where the PK is a String (numbers from '100,000' to '999,999', comma is for readability only).
Let's also say, the PK is not sequentially used.
Now i want to insert a new row into the table using java.sql and show the PK of the inserted row to the User. Since the PK is not generated by default (e.g. insert values without the PK didn't work, something like generated_keys is not available in the given environment) i've seen two different approaches:
in two different statements, first find a possible next key, then try to insert (and expect that another transaction used the same key in the time between the two statements) - is it valid to retry until success or could any sql trick with transaction-settings/locks help here? how can i realize that in java.sql?
for me, that's a disappointing solution, because of the non-deterministic behaviour (perhaps you could convince me of the contrary), so i searched for another one:
insert with a nested select statement that looks up the next possible PK. looking up other answers on generating the PK myself I came close to a working solution with that statement (left out the casts from string to int):
INSERT INTO mytable (pk,othercolumns)
VALUES(
(SELECT MIN(empty_numbers.empty_number)
FROM (SELECT t1.pk + 1 as empty_number
FROM mytable t1
LEFT OUTER JOIN mytable t2
ON t1.pk + 1 = t2.pk
WHERE t2.pk IS NULL
AND t1.pk > 100000)
as empty_numbers),
othervalues);
that works like a charm and has (afaik) a more predictable and stable solution than my first approach, but: how can i possibly retrieve the generated PK from that statement? I've read that there is no way to return the inserted row (or any columns) directly and most of the google results i've found, point to returning generated keys - even though my key is generated, it's not generated by the DBMS directly, but by my statement.
Note, that the DBMS used in development is MSSQL 2008 and the productive system is currently a DB2 on AS/400 (don't know which version) so i have to stick close to SQL standards. i can't change the db-structure in any way (e.g. use generated keys, i'm not sure about stored procedures).
DB2 for i allows generated keys, stored procedures, user defined functions - pretty much all of the things SQL Server can do. The exact implementation is different, but that's what manuals are for :-) Ask your admin what version of IBM i they're running, then hit up the Infocenter for specifics.
The constraining factor is that you can't alter the database design; you are stuck with apparently multiple processes trying to INSERT while backfilling 'holes' in the existing keyspace. That's a very tough nut to crack. Because you can't change the DB design, there's nothing to be done except to allow for and handle PK collisions. There's no SQL trick that'll help - the SQL way is to have the DB generate the PK, not the application.
There are several alternatives to suggest, in the event that some change is allowed. All have issues needing a workaround, but that is unavoidable at this point due to the application design.
Create a UDF that all INSERT clients use to retrieve the next available PK. Use a table of 'available numbers' and delete them as they are issued.
Pre-INSERT all the available numbers. Force clients to do an UPDATE. Make them FETCH...FOR UPDATE where (rest of data = not populated). This will lock the row, avoiding collisions as well as make the PK immediately available.
Leave the DB and the other application programs using this table as-is, but have your INSERT process draw from a block of keys that's been set aside for your use. Keep the next available number in an SQL SEQUENCE or an IBM i data area. This only works if there's a very large hole in the keyspace that's not yet used.
I have tagged this problem with both Oracle and Java because both Oracle and Java solutions would be accepted for this problem.
I am new to Oracle security and have been presented with the below problem to solve. I have done some research on the internet but I have had no luck so far. At first, I thought Oracle TDE might be helpful for my problem but here: Can Oracle TDE protect data from the DBA? it seems TDE doesn't protect data against DBA and this is an issue which is not to be tolerated.
Here is the problem:
I have a table containing millions of records. I have a Java application which queries this table using equality or range criteria against a column in the table which is the primary key column of the table. The primary key column contains sensitive data and thus has been encrypted already. As the result, querying data using normal (i.e. decrypted) values from the application cannot use the primary key's unique index access path. I need to improve the queries' performance without any changes on the application code (application config can be modified if necessary but not the code). It would be OK to do any changes that are necessary on the database side as long as that column remains encrypted.
Oracle people please: What solution(s) do you suggest to this problem? How can I create an index on decrypted column values and somehow force Oracle to utilize this index? How can I use partitioning such as hash-partitioning? How about views? Any, Any solution?
Java people please: I myself have this very vague idea which is to create a separate application in between (i.e between the database and the application) which acts as a proxy that receives the queries from the application and replaces the decrypted values with encrypted values and sends it for the database, it then receives the response and return the results back to the application. The proxy should behave like a database so that it should be possible for the application to connect to it by changing the connection string in the configuration file only. Would this work? How?
Thanks for all your help in advance!
which queries this table using equality or range criteria against a column in the table which is the primary key column of the table
To find a specific value it's simple enough - you can store the data encrypted any way you like - even as a hash and still retrieve a specific value using an index. But as per my comment elsewhere, you can't do range queries without either:
decrypting each and every row in the table
or
using an algorithm that can be cracked in a few seconds.
Using a linked list (or a related table) to define order instead of an algorithm with intrinsic ordering would force a brute force check on a much larger set of values - but it's nowhere near as secure as a properly encrypted value.
It doesn't matter if you use Oracle, Java or pencil and paper. Might be possible using quantum computing - but if you can't afford to ensure the security of your application / pay for good advice from an expert cryptographer, then you certainly won't be able to afford that.
How can I create an index on decrypted column values and somehow force Oracle to utilize this index?
Maybe you could create a function based index in which you index the decrypted value.
create index ix1 on tablename (decryptfunction(pk1));
Hey guys, I have this pre-existing SQLite database that I want to use with my Android application. I have created a sample database from scratch for testing purposes where each primary key is named _id and also adding the table android_metadata. This works great.
So now when I've tried to rename the primary keys of the database I already have, and upload it to the application, it doesn't work.
Can anyone tell me what exactly I have to do to my existing database to get it to work with the Android OS? Like what exactly has to be changed in the database for it to work?
And yes, I have looked at most tutorials, but most of them don't go into detail about what you have to change in the pre-existing database.
Here is the database I am using:
http://www.mediafire.com/file/bpbpm19y6kbpjot/database.db
Thanks.
Again, I have found this document to be very useful: http://www.reigndesign.com/blog/using-your-own-sqlite-database-in-android-applications/
I usually set the flag NO_LOCALIZED_COLLATORS when calling SQLiteDatabase.openDatabase(). Then you don't need the android_metadata table. As far as I know the _id column also must be of the type INTEGER PRIMARY KEY AUTOINCREMENT.
You don't actually need the primary ID column to be named _id -- you can just use something like SELECT my_id as _id, another_field ... in your select statement.
And you can either do as Omokoii said above and set the NO_LOCALIZED_COLLATORS flag, or you can create the android_metadata table and insert the value en-US into it.
As for using an existing DB, perhaps this blog post might help: http://www.reigndesign.com/blog/using-your-own-sqlite-database-in-android-applications/
Make sure that your existing SQLite database declares integer primary keys using only "INTEGER" (verbatim)--not "int" or "int16" or any of the other possibilities for declaring an integer that SQLite will recognize.
I ran into a related problem when importing a SQLite database in Adobe AIR (which has a common codebase with Goodle and Mozilla and other consortium members, IIRC). My PK had been defined as "int" (verbatim). SQLite treats "INTEGER" primary keys differently than it treats "int" or "INT" or "int16" etc primary keys!
Documented here: http://www.sqlite.org/datatypes.html
An INTEGER primary key is treated by SQLite as a synonym for the RowId. Any other int type is just like a standard column, and with a standard column RowId will not necessarily equal the value in the PK column.
However, Adobe and the other related subgroup of SQLite consortium members did not implement this (documented) behavior--for them any/every integer type used as the PK column is treated as a synonym for the row id-- and their failing to implement this distinction can result in erroneous joins when a pre-existing SQLite database is imported into their implementation(s), if the pre-existing database used anything other than "INTEGER" when declaring its integer-type primary keys.
P.S. I brought this to Adobe's attention and discussed it ad nauseam on the SQLite mailing list and on the Adobe AIR forum. Adobe wrote me that they would document their departure from "standard" SQLite behavior but leave it as is, so I believe Android will also differ from SQLite documented behavior in this regard.
P.P.S. It seems this subgroup of consortium members either did not envision the possibility that a database would be imported (i.e. they assumed the database would always be created anew via their interface) or they simply overlooked this (admittedly wonky) exceptional behavior in SQLite.
P.P.P.S. This table, for example, from the database the OP is using would return spurious results when involved in joins on the [stop_id] column if attached by an implementation of SQLite that did not implement the "standard" INTEGER/int (et al) exceptional behavior but treated any/every int-type when used with the PK as a synonym for the rowid:
CREATE TABLE mt_stop (
stop_id int NOT NULL PRIMARY KEY ASC,
stop_lat real NOT NULL CHECK (stop_lat >= -90 AND stop_lat <= 90),
stop_lon real NOT NULL CHECK (stop_lon >= -180 AND stop_lon <= 180),
stop_name varchar (120) DEFAULT 'Unknown'
)
I'm trying to make a piece of code using plain JDBC that fetches me the name of both target table and column of a foreign key of a specific column in specific table but going through the core interfaces I can't seem to find a direct way to do this.
Is there a way to get such information about foreign keys through JDBC directly or do I have to resort to metadata queries to specific database, in this case HSQLDB.
If I have to use the database specific metadata queries, which HSQLDB internal metadata tables hold that information?
Your best bet is Connection#getMetaData() which returns DatabaseMetaData with all methods to obtain information about all tables, columns, primary keys, foreign keys, etcetera. You're however dependent on the JDBC implementation (read: the JDBC driver make/version) whether this is fully supported.
JDBC does have support for this. Check out DatabaseMetaData.getCrossReference class. Other methods on DatabaseMetdata support for querying schema, catalog, tables, columns, etc. Bear in mind some databases require extra parameters on your URL to turn on Metadata (i.e. Oracle) to optimize the calls. Don't know if HQLSB requires this.