It seems that JOOQ is completely ignoring the default values of database columns. Neither gets the ActiveRecord object updated nor does it skip this column on INSERT. Instead it tries to set it to NULL which fails on NOT NULL columns.
Example:
CREATE TABLE bug (
foo int,
bar int not null default 42
);
BugRecord b = jooq.newRecord(BUG);
b.setFoo(3);
b.store();
assertNotNull(b.getBar()); // fails
Record r = jooq.select().from(BUG).fetchOne();
assertEquals(new Integer(-1), r.getValue(BUG.BAR)); // fails
// DataMapper pattern
Bug b = new Bug();
b.setFoo(3);
bugDao.insert(b); // Fails because it tries to set "bar" to NULL
The behaviour I would expect is that either the newRecord() initializes all default variables with the korrekt values (although I understand that this could be difficult if the result is the outcome of a custom function :-)).or that the INSERT INTO does not insert all unmodified columns with default values and then that the INSERT INTO is followed by a SELECT that fetches the now existing values from the database (similar to a RETURNING).
Is this really a bug/limitation or am I missing some config option etc which makes it
possible to use "not null default" columns?
You've spotted a couple of things here (all relevant to jOOQ 3.1 and previous versions):
Returning default values from inserts:
BugRecord b = jooq.newRecord(BUG);
b.setFoo(3);
b.store();
assertNotNull(b.getBar()); // fails
That would be a nice-to-have feature, indeed. Currently, jOOQ only fetches IDENTITY column values. You can use the INSERT .. RETURNING syntax or the UPDATE .. RETURNING syntax to explicitly chose which columns ought to be returned after an insert or update. But being able to do so in regular CRUD operations would be much better.
This had also been mentioned in this thread. The relevant feature request for this is #1859.
You can work around this issue by calling
b.refresh(); // Refresh all columns
b.refresh(BUG.BAR, ...); // Refresh only some columns
Inserting NULL vs. inserting DEFAULTs through UpdatableRecord:
Record r = jooq.select().from(BUG).fetchOne();
assertEquals(new Integer(-1), r.getValue(BUG.BAR)); // fails
This is a bug, in my opinion. jOOQ's CRUD operations should be DEFAULT value safe. Only those values that have been set explicitly prior to a store() / insert() / update() operation ought to be rendered in the generated SQL. I have registered #2698 for this.
Inserting NULL vs. inserting DEFAULTs through DAO:
// DataMapper pattern
Bug b = new Bug();
b.setFoo(3);
bugDao.insert(b); // Fails because it tries to set "bar" to NULL
Nice catch. This is non-trivial to solve / enhance, as a POJO does not ship with an internal "changed" / "dirty" flag per column. It is thus not possible to know the meaning of a null reference in a POJO.
On the other hand, jOOQ already knows whether a column is nullable. If jOOQ also maintained metadata about the presence of a DEFAULT clause on a column, it could deduce that the combination NOT NULL DEFAULT would have to lead to:
INSERT INTO bug(foo, bar)
VALUES(3, DEFAULT)
And to
UPDATE bug SET bar = DEFAULT WHERE foo = 3
I have registered
#2699: Adding some metadata information to generated code
#2700: Leveraging the above metadata in SQL from DAOs
Related
It appears that Spring Boot is not capable of interpreting SELECT query results when id is not unique.
For example, let us say that there is a table t in the database which has two columns c and c2. Table does not have a primary key, so both columns may be duplicated, but it has an unique constraint on combination of c and c2 columns so that rows can not be duplicated.
Here are sample data for this table:
c | c2
______
a | a1
a | a2
a | a3
b | a1
What I need to achieve in this table is to have one and only one b in the c column and zero to many as.
I defined a Spring's repository for this table as follows:
package P;
public interface tr extends org.springframework.data.repository.PagingAndSortingRepository<P.t, java.lang.String>
{
public java.util.List<P.t> findAllByc(java.lang.String c);
}
Here I specified that Spring should use a string for an table id. Although I do not use any of repository's "*ById" methods. This is just something what is required to be specified here whether or not I need it.
And the table is represented in the Java Persistence Application programming interface class t like this:
package P;
#javax.persistence.Entity()
#javax.persistence.Table(schema = "s")
public class t implements java.io.Serializable
{
#javax.persistence.Id()
public java.lang.String c;
public java.lang.String c2;
}
Here I marked column c as an id, because it is Spring's requirement, although, as I said, I do not use "*byId" methods, so, it is not important to me.
Now, in the repository I defined (or, to be precise, declared) a custom method findAllByc which, as the name suggests, should be able to find zero to many rows of the table t where c column matches the given string.
Unfortunately, this only works when I try to search for row with unique value in the c column, for example, findAllByc("b");. When the c column is not unique, this method behaves very strangely. It works, it finds correct number of rows, but, strange thing which it does is that it finds only the first row in the table that matches the given c value, and then it simply copies that row the number of times of the count of the rows that match given c value.
So, for example, for this sample table, if I execute findAllByc("a"); the result will be:
a|a1
a|a1
a|a1
So, as you can see from the sample table which I provided above, a1 value is written only once in the table where c column is equal to a.
So what Spring Boot does is that it simply copies the first row that matches into all other rows.
Why, on the Earh, Spring would do that??? This is very strange and unexpected behaviour.
This only occurs on the column which is marked as table's id. So, for example, if I declare findAllByc2 method, then everything works as expected, that is, if I call that method and give it, for example, a2 as parameter, it will find only one row, and if I give it a1 as a parameter, it will find both rows, and will not duplicate them, for example:
a|a1
b|a1
So, I know that the cause of the problem is that c is marked as id, and id is supposed to be unique, but that should be true only for "*ById" methods which construct the queries based on the id. I do not use these methods for this table. The method which I use is "Byc" method, which means that it will use c when constructing queries, and not id. In this case, c is the same as id, I know. But the difference between findById and findAllByc method should be in that that findById method can assume that id is unique and perform some optimisations, simplifications or whatever it does based on that information, but findAllByc can not since when searching by some column in the table, you can not assume that that column is unique. Even one more reason why Spring should not assume that that column is unique is because I added "All" to the method name, which indicates to the Spring Boot that I expect this method to return a list which may contain more than one item. So how could Spring assume that c is unique inside this method when it returnes a list of rows. So if row is unique, then how could there be multiple rows with the same id? It simply does not make any sense to assume this.
Since I did not implement this method, but merely declared it, and let the Spring implement it, this, I would say, is a bug in the Spring Boot.
In any case, I went to investigate how Spring Boot parses this method name and converts it into query, and it turned out that it does it correctly! Just like one would expect it to parse this method name.
I went on the server log and looked up what query is sent to the server by my program when findAllByc("a"); is called and this is the result written in the server log at the time of program execution:
select t0_.c as c1_0_, t0_.c2 as c2_0_ from s.t t0_ where t0_.c=$1
parameters: $1 = 'a'
As you can see, Spring Boot constructed the correct query from the method name, but still managed to mess up the results.
Of course, if I directly execute this query using plain Java Data Base Connectivity Application Programming Interface (so by bypassing Spring and Hibernate), the returned result is as expected:
a|a1
a|a2
a|a3
So why does Spring Boot messes up results despite that correct query is used, and correct table snippet really is returned from the server? Why does it duplicate rows?
Sure, quick fix is to mark both c and c2 columns with Id annotation in my t class, to explicitely tell Spring that c can not be assumed to be unique, since it is only part of the id, but, first of all, why would I have to do that. It is clear from the method name what I want, and correct Structured Query Language query is the proof that Spring Boot understood me perfectly well.
And secondly, marking both columns as id would work just fine if both as and bs can occur zero to many times in the c column. But, in this case as can occur zero to many times, but a b must occur once and only once in the column. So, if I would to mark both c and c2 as id, I would get that Spring Boot treats a b in the c column as being just one of multiple bs. It would allow multiple bs to be inserted into the table.
For example, if I mark both columns as being ids, and then try to update c2 value of a b in c column by using the save method, Spring would simply think of this entity as being another entity and would simply add another b into the table, which should not happen.
So, I want entities with a in the c column to be inserted into the table when save method is called and the one with b in the c column to be updated when this method is called. Since save is not splited into insert and update methods, I can not tell Spring explicitely whether I want it to perform an update or insert operation. So that is why I can not set both columns as id. I mean save does not work now as it is because it only updates the first row with a in the c column instead of inserting it, but that is another problem. I will override the save method in tr repository, so this is not important for this question. Focus in this question is in findAllByc method.
Sure, the best solution is to add id column in the t table and give it some random number to make Spring happy and then move the Id annotation to that field in t class, sure, but why would I be required to do so when findAllByc is recognized correctly and correct query is produced. Spring simply messes up when extracting the data returned from the server. Beside, this only generates useless data in the database table where I simply do not need ids. If I were to add id column, I would never use it. Is it really worh wasting data storage just to make Spring happy?
Is this a bug?
Can I fix it on some other way without adding unique id column to the table?
I have the below class structure:
class A{
int id;
List<B> blist;
List<C> clist;
List<D> dlist;
}
I get a json as an input which is mapped to object A by a mapper. Now, i have object A which has the list of B,C and D objects. I want to use batching to save the insert time taken. I went through the documentation which describes the solution if I want to save multiple parent objects. How would I use the batching capability in my case which has nested list of objects of multiple type.
I have enabled batch inserts using
<property name="hibernate.jdbc.batch_size">50</property>
This by itself doesnt give me any batching unless I clear and flush the session. Any suggestions on how do I go about with this?
The problem is that you're using IDENTITY strategy.
Whenever you save a new entity, Hibernate will place it into the Session's 1LC; however, in order to do that the identifier must be known. The problem with IDENTITY strategy is that Hibernate must actually perform the insert to determine the identifier value.
In the end, batch insert capabilities are disabled.
You should either try to load your data using business key values that are known up front or worse case use SEQUENCE generation type with a sequence optimizer to minimize the database hit. This will allow batch inserts to work.
UPDATE
For situations where you have no business key that defines the uniqueness for a row and your database doesn't have SEQUENCE support, you could manage the identifiers yourself. You can either elect to do this using a custom identifier generator or just doing this in your loop as code.
The caveat here is that this solution is not thread-safe. You should guarantee that at no point would you ever be running this logic in two threads simultaneously, which is typically not something one does anyway with bulk data loads.
Define a variable to store your identifier in. We will need to initialize this variable based on the existing max value of the identifier in the database. If no rows in the database exist, we likely will want to initialize it as 1.
Long value = ... // createQuery ( "SELECT MAX(id) FROM YourEntity" )
value = ( value == null ? 1L : value + 1);
The next step is to change the #Id annotated field. It should not be marked as #GeneratedValue since we're going to allow the application to provide the value.
For each row you're going to insert, simply call your #setId( value ) method with the value variable generated from step 1.
Increment your value variable by 1.
I'm executing stored procedure with Hibernate native method, this stored procedure creates column names depending on another table Ids. So its columns look like something like this:
| id | ... some other columns ... | name | c_1 | c_2 | c_4 | c_.. |
If I call Query.getResultList() it returns List<Object[]>, and I don't know column names. I have to know column names (and corresponding column index) to continue my further business logic. I also cannot use EntityManager.createNativeQuery(String s, Class aClass) since it is not one POJO class.
Currently I'm getting List<Object[]> without problem, but I need, for example, Map<String,Object[]> column name as a key and column values as an array of Objects.
How can I get all column names with their values?
Finally, found the solution, and I'm going to share with you(maybe helps someone). Here is what I did:
I added Spring JDBC Support to my application, then,
I used Spring SimpleJdbcTemplate Querying, there is a method: SimpleJdbcDaoSupport.getSimpleJdbcTemplate().queryForList(nativeQuery) which returns List<Map<String,Object>>
(This is just an idea, I haven't tested it. And it needs Criteria API.)
If you manage to get the result as List<javax.persistence.Tuple>, you can take one Tuple and call getElements() on it - you'll get a list of TupleElement<?>s on which you can call getAlias(). Presumably this will be the column name. But I'm not 100% sure and it won't work if there are no tuples (results) returned, but I think it's worth a try. See 9.2. Tuple criteria queries.
For this approach, this might help:
Calling stored procedure from Java / JPA
How can i call stored procedure using spring with jpa
Some of the non-nullable fields in table have default value. When inserting new rows into table via JPA, I do not want to pass any value for these fields so that they get the default values. However, when inserting new row via Spring JPA repository classes, I get an error that null values cannot be inserted. I noticed that the insert statement JPA sends to the database have all fields listed:
insert into table (field1, field2, field3) values ('abc',null,null);
Since field2 and field3 have null specified, the default values are not assigned and database throws error that null values cannot be inserted. Is there a workaround?
You can try to configure insertable property for #Column, which shouldn't be persisted & to exclude it from the insert statement.
From Documentation - insertable : (Optional) Whether the column is included in SQL INSERT statements generated by the persistence provider.
If you want to assign default value to database. you shouldn't insert it as NULL rather you have to leave it out, and don't insert it, to do that you can use, #Column(insertable = false)
annotation.
As the matter of fact I think it's not good job to assign default value in database when you work to gather with ORM. choose an other way such as JPA Events to initiate all values in JAVA.
I have a search.jsp page where I search & retrieve a record from Database. I have to duplicate this record and make modifications and save both records.
Ex:
method1(){
Response response=new Response();
//To perform DB search
response=delegate.requestSearch(//some request criteria);
Types obj1=(Types) response.getBaseEntity();
Types obj2=obj1;
obj2.setPromaryKeyObject(//some unique value);
}
In my DAO, save method:
session.save(obj1);
session.save(obj2);
transaction.commit(); // transaction
When the debug point commits transaction, I get one update query only. But I expect two insert queries.
But this works fine. This was an attempt but this is not what I want to do.
Types tempObj1 = new Types();
tempObj1.setValue1(obj1.getValue1();
// all values of the object should be set
session.save(tempObj1);
Types tempObj1 = new Types();
tempObj2.setValue1(obj2.getValue1();
// all values of the object should be set
session.save(tempObj2);
transaction.commit(); // transaction
In this case I trigger two insert queries.
I'm pretty sure that the mistake is in my hibernate method as I'm very new to Hibernate. This same code works fine when I was using JDBC and MySql queries.
It's all about ids. If object with given id exist in database it is updated. If no id is set then it is generated according to generation strategy and new record is inserted. Sometimes (depends on configuration) you must set the id explicitly.
Since you use MySQL i guess you use autoincrement column as id and IDENTITY generation strategy. What you probably need to do is to leave id as is for updated record and set id to null for the one you want to insert as new.
This is due the obj2 == obj1. Try to obj2 = obj1.clone();
Hibernate internally keeps track of objects you have save()'d. You did not really create a new instance with this code
Types obj1=(Types) response.getBaseEntity();
Types obj2=obj1;
so hibernate thinks you're saving the same object again with no data changed, so naturally it did nothing.
There are two ways to solve this:
Copy the data to a new instance and call save again. This is a bit
like the second code fragment you had.
Use the merge() method which does not associate the argument with the session, so that each time you call merge(something), an insert is generated. (This assumed you've cleared the id of something and disassociated it with the session first.)