I've mapped my class as follow (omitted other fields as only ID matters):
#Entity
#Table(name = "MODEL_GROUP")
#Cache(usage = CacheConcurrencyStrategy.TRANSACTIONAL)
public class SettlementModelGroup implements Serializable
{
#Id
#GeneratedValue(generator = "MODEL_GROUP_SEQ", strategy = GenerationType.SEQUENCE)
#GenericGenerator(name = "MODEL_GROUP_SEQ",
strategy = "sequence",
parameters = #Parameter(name = "sequence", value = "SEQ_MODEL_GROUP_MODEL_GROUP_ID"))
#Column(name = "MODEL_GROUP_ID", nullable = false)
private Integer modelId;
}
when I'm saving new object:
Integer modelGroupId = sessionFactory.getCurrentSession().save( modelGroup );
System.out.println( modelGroupId );
ID is set as for example 23, but when I look at the database it is actually 24. This is leading to many problems, as I'm using this ID later on. Any idea why it is making this gap?
SQL logs show that everything is fine (I thinks so):
Hibernate:
select
SEQ_MODEL_GROUP_MODEL_GROUP_ID.nextval
from
dual
Hibernate:
insert
into
MODEL_GROUP
(DOMAIN_ID, DESCRIPTION, NAME, PERIOD_TYPE_ID, MODEL_GROUP_TYPE_ID, STATUS_ID, OWNER_ID, MODEL_GROUP_ID)
values
(?, ?, ?, ?, ?, ?, ?, ?)
Trigger and Sequence:
CREATE SEQUENCE "SEQ_MODEL_GROUP_MODEL_GROUP_ID"
INCREMENT BY 1
START WITH 1
NOMAXVALUE
MINVALUE 1
NOCYCLE
NOCACHE
NOORDER
;
CREATE OR REPLACE TRIGGER "TRG_MODEL_GROUP_MODEL_GROUP_ID"
BEFORE INSERT
ON "MODEL_GROUP"
FOR EACH ROW
WHEN (NEW."MODEL_GROUP_ID" is NULL)
BEGIN
SELECT "SEQ_MODEL_GROUP_MODEL_GROUP_ID".NEXTVAL
INTO :NEW."MODEL_GROUP_ID"
FROM DUAL;
END;
Apparently, when Hibernate ask your database for nextValue of ID, it fires also Trigger. So when I ask for ID, I've got number 23 but when actually saving to database by commiting transaction, it is increased again so I've got 24. Solution is described here:
HIbernate issue with Oracle Trigger for generating id from a sequence
To make it work correctly, I changed Trigger:
CREATE OR REPLACE TRIGGER "TRG_MODEL_GROUP_MODEL_GROUP_ID"
BEFORE INSERT
ON "MODEL_GROUP"
FOR EACH ROW
WHEN (NEW."MODEL_GROUP_ID" is NULL)
BEGIN
SELECT "SEQ_MODEL_GROUP_MODEL_GROUP_ID".NEXTVAL
INTO :NEW."MODEL_GROUP_ID"
FROM DUAL;
END;
Related
I get this error when I try to run start my application:
org.springframework.dao.DataIntegrityViolationException: could not execute query; SQL [SELECT * FROM testquestions ORDER by id DESC LIMIT 1]; nested exception is org.hibernate.exception.DataException: could not execute query
As seen in previous problems on StackOverflow, I tried to adjust the length of my data input in my sql file and I've set the length of my #Column to the same amount of characters. this didn't help.
this is my #Table class:
#Entity
#Getter
#Setter
#Table(name = "testquestions")
public class TestQuestion {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private int id;
#Column(name = "questiontitle", length = 2000)
private String questionTitle;
#Column(name = "info", length = 4096)
private String Info;
#Column(name = "solvetime")
private int solveTime;
#Column(name = "difficultylevel")
private DifficultyLevel difficultyLevel;
#Column(name = "questionimage")
private Image questionImage;
public TestQuestion(){
}
public TestQuestion(int id, String questionTitle, String info, DifficultyLevel difficultyLevel) {
this.id = id;
this.questionTitle = questionTitle;
Info = info;
this.difficultyLevel = difficultyLevel;
}
public String getInfo() {
return Info;
}
}
This is my # Query in my QuestionRepository:
#Query(value = "SELECT * FROM testquestions ORDER by id DESC LIMIT 1", nativeQuery = true)
TestQuestion fetchLastQuestion();
This is my database.sql file, it writes to a PostgreSQL data base:
TRUNCATE TABLE users CASCADE;
TRUNCATE TABLE testquestions CASCADE;
DROP TABLE users;
DROP TABLE testquestions;
CREATE TABLE users(
id int,
username varchar(255),
password varchar(255),
role varchar(255)
);
CREATE TABLE testquestions(
id int primary key ,
questiontitle varchar(2000),
info varchar(4096),
solvetime int,
difficultylevel varchar(255),
questionimage bytea
);
INSERT INTO users(id, username, password, role)
VALUES (0, 'user', 'u', 'user'),
(1, 'user','u','user');
INSERT INTO testquestions(id,questiontitle, info, solvetime, difficultylevel, questionimage)
VALUES (0, 'Multiple Databases', 'A company wants to use Spring Boot in a web application which should use JPA as a database abstraction. The unit tests should be run on an H2 database while the production should run on a MySQL database.
Select all the things that need to be done or that will be done automatically by Spring Boot.', 3, 'Easy',
''),
(1, 'Screen Orientation', 'Which of these methods are called when the screen changes orientation from portrait to landscape in Android?',
3, 'Easy',''),
(2, 'Merge Names', 'Implement the uniqueNames method. When passed two arrays of names, it will return an array containing the names that appear in either or both arrays. The returned array should have no duplicates.
For example, calling MergeNames.uniqueNames(new String[]{''Ava'', ''Emma'', ''Olivia''}, new String[]{''Olivia'', ''Sophia'', ''Emma''}) should return an array containing Ava, Emma, Olivia, and Sophia in any order.',
10, 'Easy',''),
(3, 'Date', 'Write a function that converts user entered date formatted as M/D/YYYY to a format required by an API (YYYYMMDD). The parameter "userDate" and the return value are strings.
For example, it should convert user entered date "12/31/2014" to "20141231" suitable for the API.', 10, 'Easy', ''),
(4, 'Inspector', 'Fix the bugs in the following HTML code.', 10, 'Easy',''),
(5, 'Train Composition', 'A TrainComposition is built by attaching and detaching wagons from the left and the right sides, efficiently with respect to time used.
For example, if we start by attaching wagon 7 from the left followed by attaching wagon 13, again from the left, we get a composition of two wagons (13 and 7 from left to right). Now the first wagon that can be detached from the right is 7 and the first that can be detached from the left is 13.
Implement a TrainComposition that models this problem.', 20, 'Hard', '');
Has anyone got an idea how to fix this error?
Thanks!
Tom
I have two tables, AccountData and Relations.
They have one to many relation, to one account can belong more relations.
I try to write a test for its repository to test the saving functionality and I get:
Caused by: org.h2.jdbc.JdbcSQLException: Unique index or primary key violation: "PRIMARY KEY ON PUBLIC.ACC_DATA(ID)"; SQL statement:
insert into ACC_DATA (ACC_CURRENCY, STATUS, CREATED_AT, CODE, ACC_NBR, OWNER_ID, EP_ID, SUBTYPE_ID, TYPE_ID, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) [23505-197]
I define the primary key as follows:
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "SEQ_ACC_DATA")
#SequenceGenerator(name = "SEQ_ACC_DATA", sequenceName = "SEQ_ACC_DATA", allocationSize = 50)
private Long id;
Any idea?
Thank yoou in advance!
Code seems to be correct.
Compare the value of the SEQ_ACC_DATA Sequence in the DB and the Max value for the ID column of the ACC_DATA table.
If the ID column has value greater than the sequence, you'll need to increment the value of the Sequence so that new inserts can take place through code.
I had an existed entity (TableEntity.java), the table existed in db, and also the data
Here is how the column id already delcared in TableEntity.java
#Id
#SequenceGenerator(name = "table_name_id_seq", sequenceName = "table_name_id_seq", allocationSize = 1)
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "table_name_id_seq")
#Column(name = "id", nullable = false, updateable = false)
private int id;
In the database, I look at the properties of the table, for id column, the data type is int and set as Not NULL
The previous code, for inserting data to db, is only doing "repository.save()"
But now, I want to use PreparedStatement to insert data to db per batch
Here is how I create the query
String query = "INSERT INTO table_name (column1, column2) values (?, ?)";
the problem is, when the query executed, its violates null constraint for the id column. How to make the database can handle my column id? Since, when I just use repository.save() is work fine, but when I use that (could I say it a native query?) query, that exception appear. How to solve this? or any reference to solve this?
Set a default value for the column:
alter table table_name
alter column id set default nextval('table_name_id_seq');
Now if an INSERT statement does not supply a value for the id column (by not mentioning it), the next sequence value is used to populate the column.
I'm currently facing the well-known and common Hibernate insert batch problem.
I need to save batches 5 millions of rows long. I'm first trying with a much lighter payload. Since I have to insert entities of only 2 types (first all records of type A, then all records of type B, all pointing to common type C ManyToOne parent), I would like to take the most advantage from JDBC batch insert.
I have already read lots of documentation, but none that I have tried worked.
I know that in order to use batch inserts I must not use an entity generator. So I removed the AUTO_INCREMENT ID and I'm setting the ID with a trick: SELECT MAX(ID) FROM ENTITIES and increment every time.
I know that I must flush the session regularly. I'll post code ahead, but anyway I perform a transaction every 500 elements.
I know that I have to set hibernate.jdbc.batch_size consistent with my application's bulk size, so I set it in the LocalSessionFactoryBean (Spring ORM integration)
I know I have to enable rewriting batched statements in connection URL.
Here are my entities
Common parent entity. This gets inserted first in a single transaction. I don't care about auto increment column here. Only one record per batch job
#Entity
#Table(...)
#SequenceGenerator(...)
public class Deal
{
#Id
#Column(
name = "DEAL_ID",
nullable = false)
#GeneratedValue(
strategy = GenerationType.AUTO)
protected Long id;
................
}
One of the children (let's say 2.5M records per batch)
#Entity
#Table(
name = "TA_LOANS")
public class Loan
{
#Id
#Column(
name = "LOAN_ID",
nullable = false)
protected Long id;
#ManyToOne(
optional = false,
targetEntity = Deal.class,
fetch = FetchType.LAZY)
#JoinColumn(
name = "DEAL_ID",
nullable = false)
protected Deal deal;
.............
}
The other children type. Let's say the other 2.5M records
#Entity
#Table(
name = "TA_BONDS")
public class Bond
{
#Id
#Column(
name = "BOND_ID")
#ManyToOne(
fetch = FetchType.LAZY,
optional = false,
targetEntity = Deal.class)
#JoinColumn(
name = "DEAL_ID",
nullable = false,
updatable = false)
protected Deal deal;
}
Simplified code that inserts records
long loanIdCounter = loanDao.getMaxId(), bondIdCounter = bondDao.getMaxId(); //Perform SELECT MAX(ID)
Deal deal = null;
List<Bond> bondList = new ArrayList<Bond>(COMMIT_BATCH_SIZE); //500 constant value
List<Loan> loanList = new ArrayList<Loan>(COMMIT_BATCH_SIZE);
for (String msg: inputStreamReader)
{
log.debug(msg.toString());
if (this is a deal)
{
Deal deal = parseDeal(msg.getMessage());
deal = dealManager.persist(holder.deal); //Called in a separate transaction using Spring annotation #Transaction(REQUIRES_NEW)
}
else if (this is a loan)
{
Loan loan = parseLoan(msg.getMessage());
loan.setId(++loanIdCounter);
loan.setDeal(deal);
loanList.add(loan);
if (loanList.size() == COMMIT_BATCH_SIZE)
{
loanManager.bulkInsert(loanList); //Perform a bulk insert in a single transaction, not annotated but handled manually this time
loanList.clear();
}
}
else if (this is a bond)
{
Bond bond = parseBond(msg.getMessage());
bond.setId(++bondIdCounter);
bond.setDeal(deal);
bondList.add(bond);
if (bondList.size() == COMMIT_BATCH_SIZE) //As above
{
bondManager.bulkInsert(bondList);
bondList.clear();
}
}
}
if (!bondList.isEmpty())
bondManager.bulkInsert(bondList);
if (!loanList.isEmpty())
loanManager.bulkInsert(loanList);
//Flush remaining items, not important
Implementation of bulkInsert:
#Override
public void bulkInsert(Collection<Bond> bonds)
{
// StatelessSession session = sessionFactory.openStatelessSession();
Session session = sessionFactory.openSession();
try
{
Transaction t = session.beginTransaction();
try
{
for (Bond bond : bonds)
// session.persist(bond);
// session.insert(bond);
session.save(bond);
}
catch (RuntimeException ex)
{
t.rollback();
}
finally
{
t.commit();
}
}
finally
{
session.close();
}
}
As you can see from comments, I have tried several combinations of stateful/stateless session. None worked.
My dataSource is a ComboPooledDataSource with following URL
<b:property name="jdbcUrl" value="jdbc:mysql://server:3306/db?autoReconnect=true&rewriteBatchedStatements=true" />
My SessionFactory
<b:bean id="sessionFactory" class="class.that.extends.org.springframework.orm.hibernate3.LocalSessionFactoryBean" lazy-init="false" depends-on="dataSource">
<b:property name="dataSource" ref="phoenixDataSource" />
<b:property name="hibernateProperties">
<b:props>
<b:prop key="hibernate.dialect">${hibernate.dialect}</b:prop> <!-- MySQL5InnoDb-->
<b:prop key="hibernate.show_sql">${hibernate.showSQL}</b:prop>
<b:prop key="hibernate.jdbc.batch_size">500</b:prop>
<b:prop key="hibernate.jdbc.use_scrollable_resultset">false</b:prop>
<b:prop key="hibernate.cache.use_second_level_cache">false</b:prop>
<b:prop key="hibernate.cache.provider_class">org.hibernate.cache.EhCacheProvider</b:prop>
<b:prop key="hibernate.cache.use_query_cache">false</b:prop>
<b:prop key="hibernate.validator.apply_to_ddl">false</b:prop>
<b:prop key="hibernate.validator.autoregister_listeners">false</b:prop>
<b:prop key="hibernate.order_inserts">true</b:prop>
<b:prop key="hibernate.order_updates">true</b:prop>
</b:props>
</b:property>
</b:bean>
Even if my project-wide class extends LocalSessionFactoryBean, it does not override its methods (only adds few project-wide methods)
I'm getting mad since a few days. I read a few articles and none helped me enable batch inserts. I run all of my code from JUnit tests instrumented with Spring context (so I can #Autowire my classes). All of my attempts only produce a lots of separate INSERT statements
https://stackoverflow.com/questions/12011343/how-do-you-enable-batch-inserts-in-hibernate
https://stackoverflow.com/questions/3469364/faster-way-to-batch-saves-with-hibernate
https://forum.hibernate.org/viewtopic.php?p=2374413
https://stackoverflow.com/questions/3026968/high-performance-hibernate-insert
What am I missing?
It's likely your queries are being rewritten but you wouldn't know if by looking at the Hibernate SQL logs. Hibernate does not rewrite the insert statements - the MySQL driver rewrites them. In other words, Hibernate will send multiple insert statements to the driver, and then the driver will rewrite them. So the Hibernate logs only show you what SQL Hibernate sent to the driver, not what SQL the driver sent to the database.
You can verify this by enabling MySQL's profileSQL parameter in connection url:
<b:property name="jdbcUrl" value="jdbc:mysql://server:3306/db?autoReconnect=true&rewriteBatchedStatements=true&profileSQL=true" />
Using an example similar to yours, this is what my output looks like:
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
Wed Feb 05 13:29:52 MST 2014 INFO: Profiler Event: [QUERY] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) duration: 1 ms, connection-id: 81, statement-id: 33, resultset-id: 0, message: insert into Person (firstName, lastName, id) values ('person1', 'Name', 1),('person2', 'Name', 2),('person3', 'Name', 3),('person4', 'Name', 4),('person5', 'Name', 5),('person6', 'Name', 6),('person7', 'Name', 7),('person8', 'Name', 8),('person9', 'Name', 9),('person10', 'Name', 10)
The first 10 lines are being logged by Hibernate though this not what is actually being sent to MySQL database. The last line is coming from MySQL driver and it clearly shows a single batch insert with multiple values and that is what is actually being sent to the MySQL database.
I have written an application that parses an xml document using jaxb and then inserts into a database using JPA.
I have three JPA entities.
1.ItemEntity
2.PromotionEntity
3.SellPriceEntity
The item entity has a one to one relationship with the PromotionEntity and a one to one relationship with the SellPrice Entitiy.
When try to insert into the db using only the ItemEntity my application works and the item record is inserted into the db. However when I try to insert into the db using the ItemEntity, PromotionEntity and the SellPriceEntity I start to get errors.
org.apache.openjpa.persistence.PersistenceException: Incorrect integer value: '\xAC\xED\x00\x05sr\x00,org.apache.camel.example.etl.PromotionEntity$\x0C\xF5\xF1\x08\x0B\xA2\x81\x02\x00\x05L\x00\x02idt\x00\x10' for column 'PROMOTION_ID' at row 1 {prepstmnt 1554452939 INSERT INTO item (id, ATTRIBUTE_1, ATTRIBUTE_3, ATTRIBUTE_2, BRAND_LOGO_FILE_NAME, BRAND_NAME, CLASS_NO, DEFAULT_MARGIN, DESCRIPTION, EXTENDED_DESCRIPTION, EXTENDED_DESCRIPTION_2, GST_CODE, IMAGE_FILE_NAME, ITEM_NO, OUT_OF_STOCK_IND, PACK_QTY, PROMOTION_ID, SELL_PRICE_ID, SELLING_UNIT, SIZE_APPLICABLE, STOCK_AVAILABLE, SPPLR_NO, VOLUME, WEB_AGE_GROUP, WEB_COLOR_DESCRIPTION, WEB_DESCRIPTION, WEB_SIZE_DESCRIPTION, WEIGHT) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)} [code=1366, state=HY000]
FailedObject: org.apache.camel.example.etl.ItemEntity#333f8b4c
Here is how I have set my entity beans up...
#Entity(name = "item")
public class ItemEntity implements java.io.Serializable {
private static final long serialVersionUID = -9063279672222036437L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#Column(name = "ITEM_NO")
private String itemNo;
#OneToOne(cascade = CascadeType.ALL)
#JoinColumn(name="PROMOTION_ID")
private PromotionEntity promotion;
#OneToOne(cascade = CascadeType.ALL)
#JoinColumn(name="SELL_PRICE_ID")
private SellPriceEntity sellPrice;
gets and sets...
Promotion Entity
#Entity(name = "promotion")
public class PromotionEntity implements java.io.Serializable {
private static final long serialVersionUID = 2597721500656837249L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#Column(name = "PROMOTION_ID")
private String promotionId;
#Column(name = "PROMOTION_PRICE")
private String promotionPrice;
gets and sets...
Sell Price Entity
#Entity(name = "sell_price")
public class SellPriceEntity implements java.io.Serializable {
private static final long serialVersionUID = -205334787672950850L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#Column(name = "SELL_PRICE_EFFECTIVE_DATE_1")
private String sellPriceEffectiveDateOne;
#Column(name = "SELL_PRICE_1")
private String sellPriceOne;
gets and sets...
I believe that I have defined the relationship fields properly so not sure what is going wrong. On trying to debug through the JDBCStoreManager class I can see that the sql being executed is....
com.mysql.jdbc.JDBC4PreparedStatement#6d66cc49:
INSERT INTO item (
id,
ATTRIBUTE_1,
ATTRIBUTE_3,
ATTRIBUTE_2,
BRAND_LOGO_FILE_NAME,
BRAND_NAME,
CLASS_NO,
DEFAULT_MARGIN,
DESCRIPTION,
EXTENDED_DESCRIPTION,
EXTENDED_DESCRIPTION_2,
GST_CODE, I
MAGE_FILE_NAME,
ITEM_NO,
OUT_OF_STOCK_IND,
PACK_QTY,
PROMOTION_ID,
SELL_PRICE_ID,
SELLING_UNIT,
SIZE_APPLICABLE,
STOCK_AVAILABLE,
SPPLR_NO,
VOLUME,
WEB_AGE_GROUP,
WEB_COLOR_DESCRIPTION,
WEB_DESCRIPTION,
WEB_SIZE_DESCRIPTION,
WEIGHT)
VALUES (701, '', '', '', '', '', '350', '.00', 'KHOMBU APFOOTA KOKO HIGH', '', '', '1', '', '93501250080', 'Y', '0', ** STREAM DATA **, ** STREAM DATA **, 'Each', 'Y', '0', 'KHOMBU', '.0000', '', 'Black', '', '8', '.00')
Where I should be inserting promotion_id and sell_price_id it is telling me ** STREAM DATA **. Is that right? Maybe this is why I'm getting a data type error.
thanks
It is not clear what you are after from the description, which also conflicts with the model you posted. A joincolumn is used to specify the foreign key field name as well as the primary key it references in the target entity. But JPA requires this to be the primary key in the referenced entity, so your mappings end up using "ITEM"."PROMOTION_ID" to reference "PROMOTION"."ID". You get the exception because on of the "PROMOTION_ID" fields doesn't exist on one of the tables - either the item or promotion tables.
Decide which table will have the foreign key to the parents table and go from there. I'm not sure it makes sense to have a single auto generated pk value reused for the 3 entities involved in the model shown, as it means the other two entitities can never exist with out the one who gets the id set. For example, there would be no way to create a new promotion and keep the old one around. That said, it would be possible in JPA 2.0 to set a relationship as the #ID. So you could remove the id from item like this:
#Column(name = "ITEM_NO")
private String itemNo;
#Id
#OneToOne(cascade = CascadeType.ALL)
#JoinColumn(name="PROMOTION_ID")
private PromotionEntity promotion;
#OneToOne(cascade = CascadeType.ALL)
#JoinColumn(name="SELL_PRICE_ID")
private SellPriceEntity sellPrice;
This would have item use the generated long id value from promotion as its id as well, and store it in the "PROMOTION_Id" field. You could do something similar to SellPriceEntity so it has a relationship to item or promotion that it could use to get its id set.