I have the following problem: I'm loading a "dif"-table with only deletes and inserts. A "change" is determined by a delete followed by an insert.
This table has no primary key and no field to order by. I want to use Hibernate to load this table. (SELECT obj FROM MyDifTable obj WHERE obj.group = groupId)
Right now, I'm using a key over all values and the above query and I'm not getting the rows in the same order as in the source table.
My questions are:
How do I let Hibernate always return the same order as in the table?
How to specify a primary key in this case?
I know this is such a badly designed thing and I've tried to argument with the design team but without success.
Best Regards,
Kai
Relational databases are based upon a branch of Set theory. Information is either in the database (in the Set) or absent. Order is not a concept in sets, only presence is a concept in sets. This means that you must model ordering explicitly if you need it.
If Order must be maintained, you must add an "order by" clause to sort the resulting members of the set. Sometimes that sort is external (so you can delay decision of the ordering until query submission time). On very rare occasions it is important to have that order internalized in the "data set".
On the odd case you need an internalized ordering, you must do so by storing the order along with the data in the set; but, to enforce the presence of the "order by" clause (query writers might omit it), you might also have to put a few stored procedures / views of the data to ensure that the query always returns with the correct ordering.
Examples of internalized ordering are common when storing a list to a database. One typically stores the list item along with it's index in the list in the same row. That way when it is retrieved, you can restore the items and the ordering.
Related
I am trying to have a table with an "order" column to allow rearranging the order of data. Is this possible using jpa? Maybe something similar to #OrderColumn but on the table itself.
Basically I want to add a new column called "order" that saves the order the records. If a record is added, it would automatically get a "order" value. If a record was deleted, the "order" of the remaining would be automatically updated. Additionally if possible, to rearrange the orders by moving one record to an lower "order" and it would push the others
There is no way to do this out of the box, but you can implement this yourself if you want. Just query for the count of objects right before persisting and set the count + 1 as value for that order column. Make sure that the order column is declared as being unique i.e. with a unique constraint.
Note that your requirement is pretty exotic and will likely require some kind of table lock or retry mechanism if you have high concurrency.
IMO you should ask whoever gave you this requirement what the goal is that should be achieved. I bet that you will find out you don't need this after all.
Consider this:
I have a database with 10 rows.
Each row has a unique id (int) paired with some value e.g. name (varchar).
These ids are incremented from 1 to 10.
I delete 2 of the records - 2 and 8.
I add 2 more records 11 and 12.
Questions:
Is there a good way to redistribute unique ids in this database so it would go from 1 to 10 again ?
Would this be considered bad practice ?
I ask this question, because after some use of this database: adding and deleting values the ids would differ significantly.
One way to approach this would be to just generate the row numbers you want at the time you actually query, something like this:
SET #rn = 0;
SELECT
(#rn:=#rn + 1) AS rn, name
FROM yourTable;
ORDER BY id;
Generally speaking, you should not be worrying about the auto increment values which MySQL is assigning. MySQL will make sure that the values are unique without your intervention.
If you set the ID column to be primary key and an auto-increment as well, "resetting" is not really necessary because it will keep assigning unique IDs anyways.
If the thing that bothers you are the "gaps" among the existing values, then you might resort to "sort deletion", by employing the is_deleted column with bit/boolean values. Default value would be 0 (or b0), of course. In fact, soft-deleting is advised if there are some really important data that might be useful later on, especially if it involves possibility for payment-related entries where user can delete one of such entries either by omission or deliberately.
There is no simple way to employ the deletion where you simply remove one value and re-arrange the remaining IDs to retain the sequence. A workaround might be to do the following steps:
DELETE entry first. i.e. delete from <table> where ID = _value
INSERT INTO SELECT (without id column). please note that the table need to be identical in terms of columns and types in order for this query to work properly, so to speak... and you can also utilize temporary as the backup_table. i.e. insert into <backup_table> select <coluum1, column2, ...> from <table>
TRUNCATE your table, i.e. truncate table <table>
copy the values from the temp table back into the existing table. You can utilize the INSERT INTO SELECT once again, but make sure to drop the temp table in the end
Please note that I would NOT advise you to do this, mainly because most people utilize some sort of caching in their applications and they also utilize the specific ways to evaluate whether a specific object is the same.
I.e. in Java, the equals() and hashCode() methods for POJOs are overriden and programmers generally rely on IDs to be permanent way of identifying a specific object. By utilizing the above method, you essentially break the whole concept and I would not advise you to change the object's autoincrement ID value for this reason, before anything else.
Essentially, what you want to do is simply an anti-pattern and will generally make common patterns and practices employed by experienced programmers into solutions that are prone to unexpected issues and/or failures... and this especially applies if/when advanced features are involved, such as employing this such anti-pattern into an application that utilizes galera cluster and/or application caching.
I need to persist a queue of tasks in MySQL. When reading them from DB I have to make sure the order is exactly the same as they have been persisted.
In general I prefer to have the solution DB agnostic (i.e. pure JPA) but adding some flavor of Hibernate and/or MySQL is acceptable as well.
My (probably naive) first version looks like:
em.createNamedQuery("MyQuery", MyTask.class).setFirstResult(0).setMaxResults(count).getResultList();
Where MyQuery doesn't have any "order by" clause i.e. it looks like:
SELECT t FROM MyTasks
Would such approach guarantee that the incoming results/entities are ordered in the way they have been persisted? What if I enable caching as well?
I was also thinking of adding an extra field to the task entity which is a timestamp in milliseconds (UTC from 1970-01-01) and then order by it in the query but then I might be in a situation where two tasks get generated immediately one after the other and they have the same timestamp.
Any solutions/ideas are welcome!
EDIT:
I just realised that auto increment (at least in MySQL) would throw an exception once it reaches its max value and no more inserts would be possible. This means I shouldn't worry about having the counter reset by the DB and I could explicitly order by an "auto increment" column in my query. Of course I would have another problem to deal with i.e. what to do in case the volume is so high that the largest possible unsigned integer type in MySQL is not big enough but this problem is not nesessarily coupled with the problem I am dealing right now.
Focusing in a pure JPA solution, cause the entity MyTasks must have a primary key I suggest you to use Sequence Generator for its primary key and sort the result of your query using order by clause on the key.
For example:
#Entity
class MyTask {
#Id #GeneratedValue(strategy=GenerationType.SEQUENCE)
private Long id;
You can also tight it a little bit with your database using #SequenceGenerator to specify a generator defined in the database.
Edit: Did you take a look at the #PrePersist option for setting the timestamp? Maybe you can combine the timestamp field and the id sequenced generation and order by in this order, so timestamp conflicts are resolved by id comparation (which are unique).
Most RDBMS's will store in the order of insertion and given no other instruction will order results that way too. If you don't want to leave it to chance, you have a couple of options.
1) You can generate a reasonably unique ID by using a timestamp and a incrementing fixed-length number,
OR
2) You can just define your table with an autonumbered primary key (which is probably easier).
If the table has a primary key to order by, then by default, most RDBMS's will return things in ascending primary key order... or you can enforce it explicitly in your query.
JPA (with or without cache) and RDBMS not guarantee of persisting or uploading sequence when you do not use order instruction. To solve task you should add integral primary key to the entity and use it when gather data as it mentioned other answereres.
Even I've searched on google I want to be very sure about a thing.
Does ScrollMode.FORWARD_ONLY means that I will receive the results in the order they are in DB?
I have something like:
Scroller<Integer> foundRecs = new Scroller<Integer>(query.scroll(ScrollMode.FORWARD_ONLY));
Maybe is a stupid question...
That specific API is Hibernate, which I don't know too much about, but I guess it maps down to TYPE_FORWARD_ONLY in the end (and its documentation agrees by mentioning that constant).
If that's the case, then no: this will not influence the order in which items are returned.
It only means that you can only traverse the result once and can only navigate forward (not backwards).
Databases don't really have an "order" in which they store data, because their tables are sets (not lists). If you need some order for your results, then you need to add an ORDER BY to your SQL (or the equivalent in whichever query system you use).
You cannot rely on the physical order of data in the database. This might work if you query only a single table, but will fail as soon as you are using joins.
If you want your data to appear in a specific order, you need an ORDER BY clause.
I need to generate encoding String for each item I inserted into the database. for example:
x00001 for the first item
x00002 for the sencond item
x00003 for the third item
The way I chose to do this is counting the rows. Before I insert the third item, I count against the database, I know there're already 2 rows, so the next encoding is ended with 3.
But there is a problem. If I delete the second item, the forth item will not be the x00004,but x00003.
I can add additional columns to table, to store the next encoding, I don't know if there's other better solutions ?
Most databases support some sort of auto incrementing identity field. This field is normally also setup to be unique, so duplicate ids do not occur.
Consult your database documentation to see how it is done in your database and use that - don't reinvent the wheel when you have a good mechanism in place already.
What you want is SELECT MAX(id) or SELECT MAX(some_function(id)) inside the transaction.
As suggested in Oded's answer a lot of databases have their own methods of providing sequences which are more efficient and depending on the DBMS might support non numeric ids.
Also you could have id broken down into Y and 00001 as separate columns and having both columns make up primary key; then most databases would be able to provide the sequence.
However this leads to the question if your primary key should have a meaning or not; Y suggest that there is some meaning in the part of the key (otherwise you would be content with a plain integer id).