I have the below class structure:
class A{
int id;
List<B> blist;
List<C> clist;
List<D> dlist;
}
I get a json as an input which is mapped to object A by a mapper. Now, i have object A which has the list of B,C and D objects. I want to use batching to save the insert time taken. I went through the documentation which describes the solution if I want to save multiple parent objects. How would I use the batching capability in my case which has nested list of objects of multiple type.
I have enabled batch inserts using
<property name="hibernate.jdbc.batch_size">50</property>
This by itself doesnt give me any batching unless I clear and flush the session. Any suggestions on how do I go about with this?
The problem is that you're using IDENTITY strategy.
Whenever you save a new entity, Hibernate will place it into the Session's 1LC; however, in order to do that the identifier must be known. The problem with IDENTITY strategy is that Hibernate must actually perform the insert to determine the identifier value.
In the end, batch insert capabilities are disabled.
You should either try to load your data using business key values that are known up front or worse case use SEQUENCE generation type with a sequence optimizer to minimize the database hit. This will allow batch inserts to work.
UPDATE
For situations where you have no business key that defines the uniqueness for a row and your database doesn't have SEQUENCE support, you could manage the identifiers yourself. You can either elect to do this using a custom identifier generator or just doing this in your loop as code.
The caveat here is that this solution is not thread-safe. You should guarantee that at no point would you ever be running this logic in two threads simultaneously, which is typically not something one does anyway with bulk data loads.
Define a variable to store your identifier in. We will need to initialize this variable based on the existing max value of the identifier in the database. If no rows in the database exist, we likely will want to initialize it as 1.
Long value = ... // createQuery ( "SELECT MAX(id) FROM YourEntity" )
value = ( value == null ? 1L : value + 1);
The next step is to change the #Id annotated field. It should not be marked as #GeneratedValue since we're going to allow the application to provide the value.
For each row you're going to insert, simply call your #setId( value ) method with the value variable generated from step 1.
Increment your value variable by 1.
Related
I found that if the entity mapping without a hibernate generated primary key value, the SelectBeforeUpdate(false) will not working.
#Entity
#SelectBeforeUpdate(false)
class X {
protected X(){}
#Id
#GeneratedValue
UUID id;
int x;
}
If comment out #GeneratedValue, assign id value in X ctor, the #SelectBeforeUpdate(false) will lose its functionality.
Due to hibernate will check a detached object is transient or not by ForeignKeys.isTransient in org\hibernate\engine\internal\ForeignKeys.java which finally call IdentifierValue.isUnsaved in org\hibernate\engine\spi\IdentifierValue.java. Without #GeneratedValue, hibernate use a static UNDEFINED instance of class IdentifierValue which isUnsaved method always return null. That will cause ForeignKeys.isTransient return null so that hibernate will build the snapshot at last which will trigger a selection SQL.
However, the problem is that #GeneratedValue split the object construction to 2 phase: first new it then pass it into session.save. I don't like this, is there a workaround to make #SelectBeforeUpdate(false) works without let hibernate generated the #Id value? I use UUID to be the pk so it is unnecessary let hibernate to do this.
I use hibernate 5
Finally I figure out this mechanism. For hibernate, the #Id field are processed in an uniformed procedure whatever the type is: if you want hibernate generate the id value for you, you must call session.save, even the id value can be determined at the stage of object construction.
This is problematic: if user use UUID to be the primary key, that means user want to determine the value ahead of communicating with database. For some complex logic, after the object A create, the object A need to do some additional things, after that, the object A need along with another object B in an transaction to commit to database. If the object A's id is null till database commit, it is very inconvenient due to object is not a complete object yet.
Hibernate use the uniform id generation procedure as an assumption which used to determine the object is transient or not. If #GeneratedValue is add, hibernate consider object is transient only if the id value is null(This is quite simple, if the id value is not null, then session.save must be called). If without a generator, hibernate can not know the fact, the only way to find out is select it out from database. So it have to omit the #SelectBeforeUpdate(false)
The only way to alter the procedure is self define a Interceptor which override the Interceptor.isTransient, however doing this require user implement the transient checking. If always return false in it, the save and update must be called precisely.
I have a primary key defined as a string value. So two tuples with primary keys as "Hello" and "hello" cant exist because the primary key is case insensitive.
What will be the sql statement to create a table whose primary key is case insensitive?
You basically need a check constraint that uses a sub query. But since sub-queries are not permitted in check constraints you can write a function that runs the query and returns a
result: (Assuming your PK is called ID)
create or replace
FUNCTION check_id(ID_INPUT IN VARCHAR2)
RETURN NUMBER
IS count_id NUMBER;
Begin
SELECT count(*) INTO count_id FROM table WHERE UPPER(ID_INPUT) = UPPER(ID);
Return(Count_Id);
END;
Then use this function in your check constraint for your primary key
CHECK check_id(ID) = 0
Try this and let me know. Have not verified it for syntax.
Having a user defined function called by a constraint is one way to go. The drawbacks are that a complete table scan must be performed every time a row is inserted -- or whenever the key field is updated, I suppose, but that shouldn't be happening too often. Also, developing a udf that can be referenced by a constraint is not always easy. Even when it's allowed, the function has to meet strict requirements about being deterministic and not altering the data state. Some DBMSs make this relatively easy, others...do not.
Another option would be to define a "shadow" field. This field, populated by insert and update trigger(s), would be the contents of the ID field converted to all one case, upper or lower. A unique constraint on the shadow field would then serve the purpose you need. This could also be implemented as a computed column, where they exist, but triggers would be better. In case of duplicates, the triggers could then intercept the exception and throw their own, with the ID field instead of the shadow named as the culprit.
The drawback to this method is that you've doubled the space needed to contain your key data. That may or may not be significant.
Interestingly, Oracle 12c has introduced the "invisible" column. It acts just like any other column except it must be referenced explicitly, it doesn't show up in select * for example. This would be a perfect use for such an invisible column.
I've been trying to do a simple one to many object binding in DataNucleus JDO. It's just two classes (i stripped a simple fields):
#PersistenceCapable(table="ORDER",schema="mgr")
public class Order {
#PrimaryKey(column="id")
#Persistent(valueStrategy=IdGeneratorStrategy.NATIVE,column="id")
private Long id;
#Persistent(defaultFetchGroup="false",column="customer_id")
#Element(column="customer_id")
private Customer customer;
}
And a class Customer having a list of orders
#PersistenceCapable(table="customer",schema="mgr",identityType=IdentityType.DATASTORE)
#DatastoreIdentity(strategy=IdGeneratorStrategy.NATIVE)
public class Customer {
#PrimaryKey
#Persistent(valueStrategy=IdGeneratorStrategy.NATIVE,column="id")
private Long id;
#Persistent(mappedBy="customer")
private List<Order> orders;
}
The database table setup is extremely simple(a table for customer and a table for orders with a foreign key (customer_id) referencing customer). Yet, when i try to insert some orders for customer i receive an error
javax.jdo.JDODataStoreException: Insert of object
"test.Order#17dd585" using statement "INSERT INTO
ORDER
(USER_COMMENT,ORDER_DATE,STATUS,CUSTOMER_ID,ORDERS_INTEGER_IDX)
VALUES (?,?,?,?,?)" failed : Unknown column 'ORDERS_INTEGER_IDX' in
'field list'
Somehow DataNucleus is assuming, there is a column ORDERS_INTEGER_IDX (such column does not exist in the database). The only idea, that came to my mind is http://www.datanucleus.org/products/datanucleus/jdo/metadata_xml.html
In some situations DataNucleus will add a special datastore column to
a join table so that collections can allow the storage of duplicate
elements. This extension allows the specification of the column name
to be used. This should be specified within the field at the
collection end of the relationship. JDO2 doesnt allow a standard place
for such a specification and so is an extension tag.
So cool! 'in some situations'. I have no idea how to make my situation not to be a subset of 'some situations' but I have no idea, how to get this working. Perhaps someone has allready met the "INTEGER_IDX" problem? Or (it is also highly possible) - im not binding the data correctly :/
So you create the schema yourself. Your schema is inconsistent with metadata. You run persistence without validating your metadata against schema, and an exception results. DataNucleus provides you with SchemaTool to create or validate the schema against your metadata, so that would mean that you can detect the problem.
You're using an indexed list, so it needs an index for each element (or how else is it to know what position an element is in?). How can it assume there is an index? well it's a thing called the JDO spec (publically available), which defines indexed lists. If you don't want positions of elements storing then don't use a List (the Java util class for retaining the position of elements) ... so I'd suggest using a Set since that doesn't need position info (hence no index).
You also have a class marked as datastore identity, and then have a primary-key. That is a contradiction ... you have one or the other. The docs define all of that, as well as how to have a 1-N List relation ("JDO API" -> "Mapping" -> "Fields/Properties" -> "1-N Relations" -> "Lists" or "Sets")
I have a search.jsp page where I search & retrieve a record from Database. I have to duplicate this record and make modifications and save both records.
Ex:
method1(){
Response response=new Response();
//To perform DB search
response=delegate.requestSearch(//some request criteria);
Types obj1=(Types) response.getBaseEntity();
Types obj2=obj1;
obj2.setPromaryKeyObject(//some unique value);
}
In my DAO, save method:
session.save(obj1);
session.save(obj2);
transaction.commit(); // transaction
When the debug point commits transaction, I get one update query only. But I expect two insert queries.
But this works fine. This was an attempt but this is not what I want to do.
Types tempObj1 = new Types();
tempObj1.setValue1(obj1.getValue1();
// all values of the object should be set
session.save(tempObj1);
Types tempObj1 = new Types();
tempObj2.setValue1(obj2.getValue1();
// all values of the object should be set
session.save(tempObj2);
transaction.commit(); // transaction
In this case I trigger two insert queries.
I'm pretty sure that the mistake is in my hibernate method as I'm very new to Hibernate. This same code works fine when I was using JDBC and MySql queries.
It's all about ids. If object with given id exist in database it is updated. If no id is set then it is generated according to generation strategy and new record is inserted. Sometimes (depends on configuration) you must set the id explicitly.
Since you use MySQL i guess you use autoincrement column as id and IDENTITY generation strategy. What you probably need to do is to leave id as is for updated record and set id to null for the one you want to insert as new.
This is due the obj2 == obj1. Try to obj2 = obj1.clone();
Hibernate internally keeps track of objects you have save()'d. You did not really create a new instance with this code
Types obj1=(Types) response.getBaseEntity();
Types obj2=obj1;
so hibernate thinks you're saving the same object again with no data changed, so naturally it did nothing.
There are two ways to solve this:
Copy the data to a new instance and call save again. This is a bit
like the second code fragment you had.
Use the merge() method which does not associate the argument with the session, so that each time you call merge(something), an insert is generated. (This assumed you've cleared the id of something and disassociated it with the session first.)
In legacy database tables we have numbered columns like C1, C2, C3, C100 or M1, M2, M3, M100.
This columns represent BLOB data.
It is not possible to change anything it this database.
By using JPA Embeddable we map all of the columns to single fields. And then during embedding we override names by using 100 override annotations.
Recently we have switched to Hibernate and I've found things like UserCollectionType and CompositeUserType. But I hadn't found any use cases that are close to mine.
Is it possible to implement some user type by using Hibernate to be able to map a bundle of columns to a collection without additional querying?
Edit:
As you probably noticed the names of columns can differ from table to table. I want to create one type like "LegacyArray" with no need to specify all of the #Columns each time I use this type.
But instead I'd use
#Type(type = "LegacyArrayUserType",
parameters =
{
#Parameter(name = "prefix", value = "A"),
#Parameter(name = "size", value = "128")
})
List<Integer> legacyA;
#Type(type = "LegacyArrayUserType",
parameters =
{
#Parameter(name = "prefix", value = "B"),
#Parameter(name = "size", value = "64")
})
List<Integer> legacyB;
I can think of a couple of ways that I would do this.
1. Create views for the collection information that simulates a normalized table structure, and map it to Hibernate as a collection:
Assuming your existing table is called primaryentity, I would create a view that's similar to the following:
-- untested SQL...
create view childentity as
(select primaryentity_id, c1 from primaryentity union
select primaryentity_id, c2 from primaryentity union
select primaryentity_id, c3 from primaryentity union
--...
select primaryentity_id, c100 from primaryentity)
Now from Hibernate's perspective, childentity is just a normalized table that has a foreign key to primarykey. Mapping this should be pretty straight forward, and is covered here:
http://docs.jboss.org/hibernate/stable/core/reference/en/html/collections.html
The benefits of this approach:
From Hibernate's point of view, the tables are normalized, it's a fairly simple mapping
No updates to your existing tables
The drawbacks:
Data is read-only, I don't think your view can be defined in an updatable manner (I could be wrong)
Requires change to the database, you may need to create lots of views
Alternately, if your DBA won't even let you add a view to the database, or if you need to perform updates:
2. Use Hibernate's dynamic model mapping facility to map your C1, C2, C3 properties to a Map, and have some code you your DAO layer do the appropriate conversation between the Map and the Collection property:
I have never done this myself, but I believe Hibernate does allow you to map tables to HashMaps. I'm not sure how dynamically Hibernate allows you to do this (i.e., Can you get away with simply specifying the table name, and having Hibernate automatically map all the columns?), but it's another way I can think of doing this.
If going with this approach though, be sure to use the data access object pattern, and ensure that the internal implementation (use of HashMaps) is hidden from the client code. Also be sure to check before writing to the database that the size of your collection does not exceed the number of available columns.
The benefits of this approach:
No change to the database at all
Data is updatable
O/R Mapping is relatively simple
The drawbacks:
Lots of plumbing in the DAO layer to map the appropriate types
Uses experimental Hibernate features that may change in the future
Personally, I think that design sounds like it breaks first normal form for relational databases. What happens if you need C101 or M101? Change your schema again? I think it's very intrusive.
If you add Hibernate to the mix it's even worse. Adding C101 or M101 means having to alter your Java objects, your Hibernate mappings, everything.
If you have 1:m relationships with C and M tables, you'd be able handle the cases I just cited by adding additional rows. Your Java objects contain Collection<C> or Collection<M>. Your Hibernate mappings are one-to-many that don't change.
Maybe the reason that you don't see any Hibernate examples to match your case because it's a design that's not recommended.
If you must, maybe you should look at Hibernate Component Mapping.
UPDATE: The fact that this is legacy is duly noted. My point in bringing up first normal form is as much for others who might find this question in the future as it is for the person who posted the question. I would not want to answer the question in such a way that it silently asserted this design as "good".
Pointing out Hibernate component mapping is pertinent because knowing the name of what you're looking for can be the key when you're searching. Hibernate allows an object model to be finer grained than the relational model it maps. You are free to model a denormalized schema (e.g., Name and Address objects as part of a larger Person object). That's just the name they give such a technique. It might help find other examples as well.
Sorry if I'm misunderstanding your problem here, I don't know much about Hibernate. But couldn't you just concatenate during selection from database to get something like what you want?
Like:
SELECT whatever
, C1||C2||C3||C4||...||C100 AS CDATA
, M1||M2||M3||M4||...||M100 AS MDATA
FROM ...
WHERE ...
(Of course, the concatenation operator differs between RDBMSs.)
[EDIT] I suggest to use a CompositeUserType. Here is an example. There is also a good example on page 228f in the book "Java Persistence With Hibernate".
That allows you to handle the many columns as a single object in Java.
The mapping looks like this:
#org.hibernate.annotations.Columns(columns = {
#Column(name="C1"),
#Column(name="C2"),
#Column(name="C3"),
...
})
private List<Integer> c;
Hibernate will load all columns at once during the normal query.
In your case, you must copy the int values from the list into a fixed number of columns in nullSafeSet. Pseudocode:
for (int i=1; i<numColumns; i++)
if (i < list.size())
resultSet.setInt(index+i, list.get(i));
else
resultSet.setNull(index+i, Hibernate.INTEGER.sqlType());
In nullSafeGet you must create a list and stop adding elements when a column is NULL. For additional safety, I suggest to create your own list implementation which doesn't allow to grow beyond the number of columns (inherit from ArrayList and override ensureCapacity()).
[EDIT2] If you don't want to type all the #Column annotations, use a code generator for them. That can be as simple as script which you give a name and a number and it prints #Column(...) to System.out. After the script ran, just cut&paste the data into the source.
The only other solution would be to access the internal Hibernate API to build that information at runtime but that API is internal, so a lot of stuff is private. You can use Java reflection and setAccessible(true) but that code probably won't survive the next update of Hibernate.
You can use UserTypes to map a given number of columns to any type you wish. This could be a collection if (for example) for collections are always bounded in size by a known number of items.
It's been a while (> 3 years) since I used Hibernate so I'm pretty rusty but I recall it being very easy to do; your BespokeUserType class gets passed the ResultSet to hydrate your object from it.
I too have never used Hibernate.
I suggest writing a small program in an interpreted language (such as Python) in which you can execute a string as if it were a command. You could construct a statement which takes the tedious work out of doing what you want to do manually.