I use the postgresql database.
I have a serial field in one table, the serial field is not primary key, and it is used to allocate port number to avoid duplication, the serial starts from 30001 for example.
I use hibernate to insert table, but the field is always 0.
My sequence description:
last_value | bigint | 30001
start_value | bigint | 30001
increment_by | bigint | 1
max_value | bigint | 9223372036854775807
min_value | bigint | 30001
and my entity java code is:
#Basic
#Column(name = "XXX")
#SequenceGenerator(name="XXX_seq",sequenceName="XXX_seq", allocationSize=1, initialValue = 30001)
#GeneratedValue(strategy=GenerationType.SEQUENCE,generator="XXX_seq")
Because the field is not primary key, so I don't use the #Id.
How should I do?
As #GeneratedValue can only be used with #Id (answered earlier by tom)
Provides for the specification of generation strategies for the values
of primary keys. The GeneratedValue annotation may be applied to a
primary key property or field of an entity or mapped superclass in
conjunction with the Id annotation.
DOC
Solution
You can get max value by query like: it returns you the max used port
Query query = em.createQuery("select max(u.portId) from XXXX u");
List<Long> list = query.getResultList();
return (list.isEmpty() || list.get(0) == null ? maxId : list.get(0));
#GeneratedValue
only works on identifiers (#Id)
You could use:
an Insert Trigger
a separate Table where you store the last used number
Database specific solution (Postgres can have a Sequence on a Column if i'm not mistaken)
Not really JPA but I don't think that can be done automagically by JPA/Hibernate
Related
Environment:
mariadb-java-client-2.7.0
DB : MariaDB 10.5.7
ojdbc8 - Oracle 11.2.0.3.0 JDBC 4.0
DB : Oracle Database 11g
Hibernate 4.3.8
Code :
Session session = sessionFactory.openSession();
Criteria fetchCriteria = session.createCriteria("Student");
Disjunction disjunction = Restrictions.disjunction();
for (int i = 1; i <= 10000; i++) {
Conjunction conjunction = Restrictions.conjunction();
conjunction.add(Restrictions.eq("RollNumber", i+""));
disjunction.add(conjunction);
}
fetchCriteria.add(disjunction);
long start1 = System.currentTimeMillis();
List resultList = fetchCriteria.setFirstResult(0).setResultTransformer(Criteria.ALIAS_TO_ENTITY_MAP).list();
long end1 = System.currentTimeMillis();
System.out.println("Time took :"+(end1-start1) +"ms");
Issue
If i run above code with Hibernate 4.3.8 + Oracle 8 it taking less than 5000 milliseconds.
If i run above code with Hibernate 4.3.8 +mariadb-java-client-2.7.0 it taking more than 40,000 milliseconds.
Extra Configuration :
I have set hibernate.jdbc.fetch_size to 100 in hibernate.cfg.xml
along with jdbc URL ,username and password.
Findings:
The query generated in both cases are same and if i execute those
query with SQL Client it takes 10-11 seconds for ORACLE and 41-42 seconds for MariaDB.
The query which is generated by both database if i invoke using JDBC
program (both for ORACLE and MariaDB) it is taking approx 600 milliseconds
Note: Both tables (Oracle and MariaDB) have 15,000 records.
Can anyone help me why MariaDB is taking time?
or some extra settings are required to improve the MariaDB performance.
I have tried defaultFetchSize which is mentioned in https://mariadb.com/kb/en/about-mariadb-connector-j/ but no luck.
SQL Query Generated by the databases:
select this_.rollNo as RollNo1_0_0_, this_.VersionID as Version2_0_0_,
this_.Name as Name3_0_0_, this_.dept as dept4_0_0_,
this_.favSubj as favSubj5_0_0_,
this_.ID as ID33_0_0_
from Student this_
where ((this_.ID='1')
or (this_.ID='2')
or (this_.ID='3')
or ....
or (this_.ID='10000')
MariaDB DDL
CREATE TABLE `student` (
`RollNo` bigint(20) NOT NULL ,
`VersionID` bigint(20) NOT NULL,
`Name` varchar(100) COLLATE ucs2_bin DEFAULT NULL,
`dept` varchar(100) COLLATE ucs2_bin DEFAULT NULL,
`favSubj` varchar(100) COLLATE ucs2_bin DEFAULT NULL,
`ID` varchar(100) COLLATE ucs2_bin DEFAULT NULL,
PRIMARY KEY (`RollNo`),
UNIQUE KEY `UK_student` (`ID`)
) ENGINE=InnoDB AUTO_INCREMENT=20258138 DEFAULT CHARSET=ucs2 COLLATE=ucs2_bin
Oracle DDL
CREATE TABLE student (
RollNo NUMBER(19,0),
VersionID NUMBER(19,0) NOT NULL ENABLE,
Name VARCHAR2(100),
dept VARCHAR2(100),
favSubj VARCHAR2(100),
ID VARCHAR2(100),
PRIMARY KEY ("RollNo"),
CONSTRAINT "UK_student" UNIQUE ("ID")
)
MariaDB explain select query output
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
this_
range
UK_Student
UK_Student
203
NULL
10000
Using index condition
An OR with 10K items takes a long time to parse. Faster would be an IN:
where this_.ID IN ('1', '2', ..., '10000')
However, even that is likely to take a long time to run.
In the case of MariaDB, I think the Optimizer will say
Oh, that's too many items for me to look up each one, so
I will, instead, simply scan the table, checking each row for an ID in that list (using some kind of efficient lookup in the 10K-long list).
However, if there are 20M rows in the table, that will take a long time.
Can you provide the query plan (EXPLAIN) so we can confirm what I am hypothecating?
This seems logical and faster, but will not work correctly:
where this_.ID BETWEEN '1' AND '10000'
because it is a VARCHAR!!
Performance -- Make id an INT, not a VARCHAR!
This question already has an answer here:
JPA, Reuse of deleted ID
(1 answer)
Closed 2 years ago.
Assuming I have a simple hibernate entity like so
#Entity
public class SomeEntity {
#Id
private long id;
// Other fields...
}
I want to generate sequence for the id column with consideration of any deleted entities. Meaning I don't want the sequence to be incremented for every new entity saved to the database, but to reuse values assigned to entities that were deleted.
For example in the db I have these entities saved
|---------------------|
| ID |
|---------------------|
| 1 |
|---------------------|
| 2 |
|---------------------|
So the next entity id should get the value 3. But in case the entity with id '1' was deleted, the next time I save an entity to the database I want its id to be '1'
How can this behaviour can be implemented using Hibernate?
In the end, a DB sequence doesn't work that way. You will have to implement a DB query to get the minimum available ID and set the entity ID manually.
I run the next SQL (JDBC)
CREATE TABLE IF NOT EXISTS PUBLIC.MY_DATA(
ID BIGINT IDENTITY PRIMARY KEY NOT NULL,
...,
LAST_MODIFIED BIGINT DEFAULT UNIX_MILLIS() NOT NULL)
and get SQLSyntaxErrorException: unexpected token: UNIX_MILLIS
But according to the documentation
UNIX_MILLIS ( [ ] )
This function returns a BIGINT value. With no parameter, it returns the number of milliseconds since 1970-01-01. With a DATE or TIMESTAMP parameter, it converts the argument into number of milliseconds since 1970-01-01. (HyperSQL)
Any help is appreciated
Looking at the definition of the DEFAULT clause (in table creation), and the fact that what is allowed in there is limited to
<default option> ::= <literal> | <datetime value function> | USER
| CURRENT_USER | CURRENT_ROLE | SESSION_USER | SYSTEM_USER |
CURRENT_CATALOG | CURRENT_SCHEMA | CURRENT_PATH | NULL
And <datetime value function> is defined as
datetime value function
<datetime value function> ::= ...
Specify a function that returns a datetime value. The supported
datetime value functions are listed and described in the Built In
Functions chapter.
As UNIX_MILLIS does not return a datetime value, but a BIGINT, it is entirely possible that UNIX_MILLIS is not considered a <datetime value function>, and therefor not available in the DEFAULT clause.
This seems to be supported by looking at the parser of the default clause, which filters allowed expressions based on their result type.
As a tip, I haven't tested it, but it is possible that enabling PostgreSQL compatibility mode might allow you to use DEFAULT UNIX_TIME() or maybe DEFAULT (UNIX_TIME()).
I tried running this statement in Postgres:
insert into field (id, name) values (DEFAULT, 'Me')
and I got this error:
ERROR: null value in column "id" violates not-null constraint
I ended up having to manually set the id. The problem with that is when my app inserts a record I get a duplicate key error. I am building a java app using Play framework and ebean ORM. So the entire schema is generated automatically by ebean. In this case, what is the best practice for inserting a record manually into my db?
Edit:
Here is how I'm creating my Field class
#Entity
public class Field {
#id
public Long id;
public String name;
}
Edit:
I checked the field_seq sequence and it looks like this:
CREATE SEQUENCE public.field_seq INCREMENT BY 1 MINVALUE 1 MAXVALUE 9223372036854775807 START 1 CACHE 1;
Edit:
Here is the generated SQL in pgAdmin III:
CREATE TABLE field
(
id bigint NOT NULL,
created timestamp without time zone,
modified timestamp without time zone,
name character varying(255),
enabled boolean,
auto_set boolean,
section character varying(17),
input_type character varying(8),
user_id bigint,
CONSTRAINT pk_field PRIMARY KEY (id),
CONSTRAINT fk_field_user_3 FOREIGN KEY (user_id)
REFERENCES account (id) MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION,
CONSTRAINT ck_field_input_type CHECK (input_type::text = ANY (ARRAY['TEXT'::character varying, 'TEXTAREA'::character varying]::text[])),
CONSTRAINT ck_field_section CHECK (section::text = ANY (ARRAY['MAIN_CONTACT_INFO'::character varying, 'PARTICIPANT_INFO'::character varying]::text[]))
);
CREATE INDEX ix_field_user_3
ON field
USING btree
(user_id);
There is no column default defined for field.id. Since the sequence public.field_seq seems to exist already (but is not attached to field.id) you can fix it with:
ALTER SEQUENCE field_seq OWNED BY field.id;
ALTER TABLE field
ALTER COLUMN id SET DEFAULT (nextval('field_seq'::regclass));
Make sure the sequence isn't in use for something else, though.
It would be much simpler to create your table like this to begin with:
CREATE TABLE field
(
id bigserial PRIMARY KEY,
...
);
Details on serial or bigserial in the manual.
Not sure how the the Play framework implements this.
This works.
insert into field (id, name) values (nextval('field_seq'), "Me");
I'm having an Entity which has a primary key / id field like the following:
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
This works well. I'm using EclipseLink to create the DDL-Schema, and the column is correctly created like so:
`id` bigint(20) NOT NULL AUTO_INCREMENT
However, I've got several entities for which I do want to specify the PK myself (it's a little application that transfers data from an old database to the new one we're building). If I specify the ID for the POJO (using setId(Long id)) and persist it, EclipseLink does not save it (i.e. the record is saved, but the id is auto generated by eclipseLink).
Is there a way to manually specify the value of a column which has a #GeneratedValue ?
Here some thoughts on the issue:
I tried to work around the problem by not using #GeneratedValue at all, but simply manually define the column to be AUTO_INCREMENTed. However this forces me to manually provide an IDs always, since EclipseLink validates the primary key (so it may not be null, zero, or a negative number). The exception message reads that I should specify eclipselink.id_validation, however this does not seem to make any difference (I annotated #PrimaryKey(validation = IdValidation.NONE) but still got the same message).
To clarify: I'm using EclipseLink (2.4.0) as persistence provider and I can't switch away from it (large portions of the project depend on eclipselink specific query hints, annotations, and classes).
EDIT (In Response to the answers):
Custom Sequencing: I tried to implement my own sequencing. I tried subclassing DefaultSequence, but EclipseLink will tell me Internal Exception: org.eclipse.persistence.platform.database.MySQLPlatform could not be found. But I've checked: The class is on the classpath.
So I subclassed another class, NativeSequence:
public class MyNativeSequence extends NativeSequence {
public MyNativeSequence() {
super();
}
public MyNativeSequence(final String name) {
super(name);
}
#Override
public boolean shouldAlwaysOverrideExistingValue() {
return false;
}
#Override
public boolean shouldAlwaysOverrideExistingValue(final String seqName) {
return false;
}
}
However, what I get is the following:
javax.persistence.RollbackException: Exception [EclipseLink-7197] (Eclipse Persistence Services - 2.4.0.v20120608-r11652): org.eclipse.persistence.exceptions.ValidationException
Exception Description: Null or zero primary key encountered in unit of work clone [de.dfv.datenbank.domain.Mitarbeiter[ id=null ]], primary key [null]. Set descriptors IdValidation or the "eclipselink.id-validation" property.
at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.commitInternal(EntityTransactionImpl.java:102)
...
Caused by: Exception [EclipseLink-7197] (Eclipse Persistence Services - 2.4.0.v20120608-r11652): org.eclipse.persistence.exceptions.ValidationException
Exception Description: Null or zero primary key encountered in unit of work clone [de.dfv.datenbank.domain.Mitarbeiter[ id=null ]], primary key [null]. Set descriptors IdValidation or the "eclipselink.id-validation" property.
at org.eclipse.persistence.exceptions.ValidationException.nullPrimaryKeyInUnitOfWorkClone(ValidationException.java:1451)
...
(stack trace shortened for clarity). This is the same message which I got before. Shouldn't I subclass NativeSequence? If so, I don't know what to implement for the abstract methods in Sequence or StandardSequence.
It may also be worth noting, that simply subclassing (without overriding any methods) the class works as expected. However, returing false in shouldAlwaysOverrideExistingValue(...) will not generate a single value at all (I stepped through the program and getGeneratedValue() is not called once).
Also, when I insert like 8 entities of a certain kind within a transaction it resulted in 11 records in the database (what the hell?!).
EDIT (2012-09-01): I still do not have a Solution for the problem, Implementing my own sequence did not solve it. What I need is a way to be able to not set an Id explicitly (so it will be auto generated) and to be able to set an Id explicitly (so it will be used for the creation of the record in the database).
I tried to define the column as auto_increment myself and ommit #GeneratedValue, however Validation will kick in and not allow me to save such an entity. If I specify a value != 0 and != zero, mysql will complain for a duplicate primary key.
I'm running out of ideas and options to try. Any? (starting a bounty)
This works with eclipselink. It will create a seperate table for the sequence, but that shouldn't pose a problem.
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
#Column(name="id", insertable=true, updatable=true, unique=true, nullable=false)
private Long id;
GenerationType.AUTO will choose the ideal generation strategy. Since the field is specified as insertable and updateable, a TABLE generation strategy will be used. This means eclipselink will generate another table holding the current sequence value and generate the sequence itself instead of delegating it to the database. Since the column is declared insertable, if id is null when persisting, eclipselink will generate the id. Otherwise the existing id will be used.
If you use TABLE sequencing, then EclipseLink will allow you to override the value (or SEQUENCE if your database supports this, MySQL does not).
For IDENTITY, I'm not even sure that MySQL will allow you to supply your own Id, you might want to verify this. In general I would never recommend using IDENTITY as it does not support preallocation.
There are a few issues with allowing IDENTITY to provide the id or not. One is that two different insert SQL will need to be generated depending on the id value, as for IDENTITY the id cannot be in the insert at all. You may wish to log a bug to have IDENTITY support user provided ids.
You should still be able to get it working with your own Sequence subclass, or possibly MySQLPlatform subclass. You would set your MySQLPlatform subclass using the "eclipselink.target-database" persistence unit property.
Database-centric solution to your problem:
Create an auxiliary, nullable column in your table. It will hold your manually assigned ids:
CREATE TABLE `test_table`
(
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`manual_id` bigint(20) NULL,
`some_other_field` varchar(200) NOT NULL,
PRIMARY KEY(id)
);
Map this column to a normal field in your Entity:
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name="manual_id")
private Integer manualId;
Create a trigger that sets the table id to the manual assigned id if it is not null:
DELIMITER //
CREATE TRIGGER `test_table_bi` BEFORE INSERT ON `test_table`
FOR EACH ROW
BEGIN
IF NEW.`manual_id` IS NOT NULL THEN
SET NEW.`id` = NEW.`manual_id`;
END IF;
END;//
DELIMITER;
Always use the manualId when you need to assign a custom id. The trigger will do the magic for you:
testEntiy.setManualId(300);
entityManager.persist(testEntity);
After the database import phase, simple remove the trigger, the auxiliary column and it's mapping.
DROP TRIGGER `test_table_bi`;
ALTER TABLE `test_table` DROP COLUMN `manual_id`;
Warning
If you manually specify an id greater than the current AUTO_INCREMENT value, the next generated id will jump to the value of the manually assigned id plus 1, e.g.:
INSERT INTO `test_table` (manual_id, some_other_field) VALUES (50, 'Something');
INSERT INTO `test_table` (manual_id, some_other_field) VALUES (NULL, 'Something else');
INSERT INTO `test_table` (manual_id, some_other_field) VALUES (90, 'Something 2');
INSERT INTO `test_table` (manual_id, some_other_field) VALUES (NULL, 'Something else 2');
INSERT INTO `test_table` (manual_id, some_other_field) VALUES (40, 'Something 3');
INSERT INTO `test_table` (manual_id, some_other_field) VALUES (NULL, 'Something else 3');
Will wield the results:
+----+-----------+------------------+
| id | manual_id | some_other_field |
+----+-----------+------------------+
| 50 | 50 | Something |
| 51 | NULL | Something else |
| 90 | 90 | Something 2 |
| 91 | NULL | Something else 2 |
| 40 | 40 | Something 3 |
| 92 | NULL | Something else 3 |
+----+-----------+------------------+
To avoid problems it is highly recommended to set the AUTO_INCREMENT column to start with a number greater than all of the existing ids in your previous database, e.g.:
ALTER TABLE `test_table` AUTO_INCREMENT = 100000;
I might be missing something obvious, but why not just define another Entity with the same #Table(name=".....") annotation, with the same fields, but make the id not generated? Then you can use that Entity for the code that copies data from the old DB to the new, and the one with the generated Id can be used for normal creates that require id generation.
I can't tell you if it works with EclipseLink, but we're using Hibernate here and it doesn't seem to mind it.
Using GenerationType.SEQUENCE with PostgreSQL and EclipseLink worked for me.
1) Change
#GeneratedValue(strategy = GenerationType.IDENTITY)
by
#GeneratedValue(strategy=GenerationType.SEQUENCE, generator="step_id_seq")
#SequenceGenerator(name="step_id_seq", sequenceName="step_id_seq")
Now, you can call sequence using NativeQuery:
return ((Vector<Integer>) em.createNativeQuery("select nextval('step_id_seq')::int").getSingleResult()).get(0);
and set the returned Id to your Entity before call EntityManager.persist() method.
Hope it's not too late!
Look for Custom Id Generator
http://blog.anorakgirl.co.uk/2009/01/custom-hibernate-sequence-generator-for-id-field/
maybe this could help.
My way (MySql) is deactivate GeneratedValue:
#Id
//#GeneratedValue
#Column(unique = true, nullable = false, columnDefinition ="BINARY(16)")
private UUID id;
And add in Entity:
#PrePersist
protected void onCreation() {
if (id == null) setId(UUID.randomUUID());
}
Now in my code I can do (on service for example):
String clientID = env.getParam("id");
Entity entity = entityRepository.findFirstById(UUID.fromString(clientID));
//is new?
if (entity==null){
entity = new Entity();
entity.setId(UUID.fromString(clientID));//set cumstom ID here
}
Entity entityNew = entityRepository.save(entity); //insert
if (entityNew.getId().equals(entity.getId()) ){
Log.i("OK!")
}
👌