Consider these beans:
class Country {
String name;
String code;
...
List<City> cities;
}
class City {
String name;
String zip;
...
List<Street> streets;
}
class Street {
String name;
}
I have to get that nested beans from 3 tables in a database.
I can solve in two ways:
Query loop (query on countries, looping results querying their citiy, looping results querying their streets....)
Full flat datasource (a single wide select joining all 3 tables with all rows at maximum details ordered by outer to inner fields) and after that split that.
The first question: is it the second solution the best choice considering that the nesting level can be deeper than 3 levels?
Let's say yes, I suppose to use the second option:
select *
from countries c
join cities t ...
join streets s ...
order by c.name, c.code, ..., t.name, t.zip, ...
The second question: how can I store that ResultSet in beans with JdbcTemplate?
Is there something for this purpose that split rows to nested beans? I cannot use a custom RowMapper because I do not have a single outer bean for each row.
Performance-wise, the first solution is really bad. This means that you'll run a lot of queries against your database, and the number of these queries really depends on number of rows.
I believe the database just won't be able to handle that eventually.
The second approach is much better, you can handle it in one query and since you use jdbc template, creating the corresponding objects must be not hard.
If you still have a lot of data, you might want to consider retrieving with paging (bring 100 records, than bring yet another 100 records).
Most of the DB drivers already do this in one way or another, you might want to add an "application level" handling for this if you don't want to keep in memory all the data at once, but this really depends on use cases.
If the tables a really big though joins can also be costly, but in this case the first approach will fail in any case (just too many queries because the data size is big).
Related
We have a list of ENUMS for different purposes.
Sample Enum:
public enum EJobTrackingType implements Lookup<EJobTrackingType> {
//I want to save all these enums into database
NOTIFY_SERVICE(230000001, 1,"Notify service", "desc"),
COMPLETED_SERVICE(230000002, 2,"Completed service", "desc"),
..... //more than 100 enums like this
}
we use .sql file to save this enums,
INSERT INTO FTTHAPP.TBL_LOOKUP (id, parent_lookup_id, group_id, group_name, code, name, desc) values (230000001, NULL, 23, 'JOB_TRACKING_TYPE', 1, 'NOTIFY_SERVICE', "desc")
Problem:
We need to write values two times (One in enum and another one .sql statement)
It seems not efficient way, Is any other way to save ENUMS to avoid this repetition?
From the above mentioned details what I understood is that we have to insert all the enums present in the java file to database.
The possible answer I think would be to iterate over all the enums and insert each of them to database.
EnumSet.allOf(EJobTrackingType.class)
.forEach(eJob -> ejobRepository.saveAnFlush(eJob));
This is not the actual implementation though but it gives idea how we can proceed.
We can have a class mapping to the values that has to be stored marked as #Entity.
Populate that class with values in Enums with the help of some adapter.
And the entity class can be saved to database.
We can have some attribute to specify an insert or update has to be made. So that we can have some filter during the forEach of Enums.
This job can be ran on the start up of application where a new class which will handle this job can be annotated with #Configuration. If there are too many rows to be updated/inserted there might be a possibility of running this job in a separate thread.
I think this helps.. These are few things I thought of..
I would like to avoid having column names as strings in the code. Is there any other way to accomplish this?:
String query = "SELECT c.foo1.columnA, c.foo1.foo2.columnB FROM Table c";
session.createQuery(query).list();
I'm able to iterate over a column as string like c.foo1.foo2.columnB by splitting and getting the ClassMetadata, the property Type and other Hibernate functions until I reach the last element. However, I can't think a way to get a column string from Java beans, iterating through properties too.
Not sure what is the intention. Couple of thoughts
If you are worried about possibility of property names being wrong, current day IDEs does a good job by validating the property names in JPA queries
Object reflection can give you the property names. But not necessarily all properties are mapped to columns. You can look at this and use it along with bean property names via reflection.
Hope that helps.
There is no way to achieve what you are looking for. But, if your concern is correctness of these queries and worry that the problem will not be known until the execution hits this, you could use NamedQuery
#Entity
#NamedQuery(
name="findAllEmployeesByFirstName",
queryString="SELECT OBJECT(emp) FROM Employee emp WHERE emp.firstName = 'John'"
)
public class Employee implements Serializable {
...
}
Usage
List employees = em.createNamedQuery("findAllEmployeesByFirstName").getResultList();
The benefit is that queries defined in NamedQuery annotations are compiled to actual SQL at start up time. So incorrect field references(typo etc) will cause a start up error and the application will not start.
Another option will be as mentioned in the other answer to trust in a good IDE to refactor all occurrences properly when you rename fields (Idea does a great job at this, so would any other IDE)
EDIT: I do not think there is any performance degradation with named queries. Rather it may appear to be faster as compiled queries are cached(very subjective)
Finally, its better to use the actual query as-is as mentioned in comments. It is far more readable and debug in its context. If you are concerned about correctness, unit-test the heck out of it and be confident.
We are examining 2 different methods to do our entities updates:
"Standard" updates - Fetch from DB, set Fields, persist.
We have a Query for each entity that looks like this:
String sqlQuery = "update Entity set entity.field1 = entity.field1, entity.field2 = entity.field2, entity.field3 = entity.field3, .... entity.fieldn = entity.fieldn"
We receive the fields that changed (and their new values) and we replace the string fields (only those required) with the new values. i.e. something like :
for each field : fields {
sqlQuery.replace(field.fieldName, getNewValue(field));
}
executeQuery(sqlQuery);
Any ideas if these options differ in performance to a large extent? Is the 2nd option even viable? what are the pros / cons?
Please ignore the SQL Injection vulnerability, the example above is only an example, we do use PreparedStatements for the query building.
And build a mix solution?
First, create a hasmap .
Second, create a new query for a PreparedStament using before hasmap (for avoid SQL injection)
Third, set all parameters.
The O(x)=n, where n is the number of parameters.
The first solution is much more flexible You can rely on Hibernate dirty checking mechanism for generating the right updates. That's one good reason why an ORM tool is great when writing data.
The second approach is no way way better because it might generate different update plans, hence you can't reuse the PreparedStatement statement cache across various column combinations. Instead of using string based templates (vulnerable to SQL injections) you could use JOOQ instead. JOOQ allows you to reference your table columns in Java, so you can build the UPDATE query in a type-safe fashion.
I'm writing a J2EE/JPA/Spring 3 application, trying to stay pure JPA 2.0. I want to get a count of child objects without having to load them, as it's obviously an expensive operation. For example here's a simplified example
Organisation
- OrgID
- OrgName
Employee
- EmployeeID
- OrgID (key to Organisation table)
- EmployeeName
On a jsp page I want to show the list of all organisations and a count of the number of employees without loading the employees themselves. If it can be a single database hit that loads all the Organisation objects and somehow loads a count of the Employee objects that would be great. I'd rather avoid one query to list the organisations then one for each organisation to count the employees. I guess I could add a transient property to hold the count, I'm not sure how to best do that.
Just to give an idea of scale, there will be around 50 organisations, and each organisation will have between 0 and 500 employees. I would rather avoid any implementation specific extensions, as I've changed JPA providers once and may change again.
In SQL I'd just do a join, group, and count, but I don't have a clue how to do it in JPA. Any help appreciated!
You can select directly into an object that you define to be the result that holds the organisation and the count. Then you just write the query. The only trick is that you have to manually group by every field on the Organisation. 'GROUP BY ORGANISATION' is not legal.
public class OrgWithEmpCount {
private Organisation org;
private Long empCount;
public OrgWithEmpCount(Organisation org, Long empCount) {
this.org = org;
this.empCount = empCount;
}
}
Select new full.package.OrgWithEmpCount(o, count(e.employeeId))
from Organisation o, IN(o.employees) e
group by o.orgId, o.orgName, o.whateverElse
Going off of the the accepted answer from Affe - this worked except for the case of you wanting to have the count for companies even with no employees. The IN query will end up just excluding those companies. Hard to imagine a company with zero employees, but for the sake of the Query example, you could do it like this:
select new full.package.OrgWithEmpCount(o, count(e.employeeId))
FROM Organisation o LEFT JOIN e.employees AS e
group by o.orgId, o.orgName, o.whateverElse
You get the idea... just instead of doing an IN, do a LEFT JOIN.
In legacy database tables we have numbered columns like C1, C2, C3, C100 or M1, M2, M3, M100.
This columns represent BLOB data.
It is not possible to change anything it this database.
By using JPA Embeddable we map all of the columns to single fields. And then during embedding we override names by using 100 override annotations.
Recently we have switched to Hibernate and I've found things like UserCollectionType and CompositeUserType. But I hadn't found any use cases that are close to mine.
Is it possible to implement some user type by using Hibernate to be able to map a bundle of columns to a collection without additional querying?
Edit:
As you probably noticed the names of columns can differ from table to table. I want to create one type like "LegacyArray" with no need to specify all of the #Columns each time I use this type.
But instead I'd use
#Type(type = "LegacyArrayUserType",
parameters =
{
#Parameter(name = "prefix", value = "A"),
#Parameter(name = "size", value = "128")
})
List<Integer> legacyA;
#Type(type = "LegacyArrayUserType",
parameters =
{
#Parameter(name = "prefix", value = "B"),
#Parameter(name = "size", value = "64")
})
List<Integer> legacyB;
I can think of a couple of ways that I would do this.
1. Create views for the collection information that simulates a normalized table structure, and map it to Hibernate as a collection:
Assuming your existing table is called primaryentity, I would create a view that's similar to the following:
-- untested SQL...
create view childentity as
(select primaryentity_id, c1 from primaryentity union
select primaryentity_id, c2 from primaryentity union
select primaryentity_id, c3 from primaryentity union
--...
select primaryentity_id, c100 from primaryentity)
Now from Hibernate's perspective, childentity is just a normalized table that has a foreign key to primarykey. Mapping this should be pretty straight forward, and is covered here:
http://docs.jboss.org/hibernate/stable/core/reference/en/html/collections.html
The benefits of this approach:
From Hibernate's point of view, the tables are normalized, it's a fairly simple mapping
No updates to your existing tables
The drawbacks:
Data is read-only, I don't think your view can be defined in an updatable manner (I could be wrong)
Requires change to the database, you may need to create lots of views
Alternately, if your DBA won't even let you add a view to the database, or if you need to perform updates:
2. Use Hibernate's dynamic model mapping facility to map your C1, C2, C3 properties to a Map, and have some code you your DAO layer do the appropriate conversation between the Map and the Collection property:
I have never done this myself, but I believe Hibernate does allow you to map tables to HashMaps. I'm not sure how dynamically Hibernate allows you to do this (i.e., Can you get away with simply specifying the table name, and having Hibernate automatically map all the columns?), but it's another way I can think of doing this.
If going with this approach though, be sure to use the data access object pattern, and ensure that the internal implementation (use of HashMaps) is hidden from the client code. Also be sure to check before writing to the database that the size of your collection does not exceed the number of available columns.
The benefits of this approach:
No change to the database at all
Data is updatable
O/R Mapping is relatively simple
The drawbacks:
Lots of plumbing in the DAO layer to map the appropriate types
Uses experimental Hibernate features that may change in the future
Personally, I think that design sounds like it breaks first normal form for relational databases. What happens if you need C101 or M101? Change your schema again? I think it's very intrusive.
If you add Hibernate to the mix it's even worse. Adding C101 or M101 means having to alter your Java objects, your Hibernate mappings, everything.
If you have 1:m relationships with C and M tables, you'd be able handle the cases I just cited by adding additional rows. Your Java objects contain Collection<C> or Collection<M>. Your Hibernate mappings are one-to-many that don't change.
Maybe the reason that you don't see any Hibernate examples to match your case because it's a design that's not recommended.
If you must, maybe you should look at Hibernate Component Mapping.
UPDATE: The fact that this is legacy is duly noted. My point in bringing up first normal form is as much for others who might find this question in the future as it is for the person who posted the question. I would not want to answer the question in such a way that it silently asserted this design as "good".
Pointing out Hibernate component mapping is pertinent because knowing the name of what you're looking for can be the key when you're searching. Hibernate allows an object model to be finer grained than the relational model it maps. You are free to model a denormalized schema (e.g., Name and Address objects as part of a larger Person object). That's just the name they give such a technique. It might help find other examples as well.
Sorry if I'm misunderstanding your problem here, I don't know much about Hibernate. But couldn't you just concatenate during selection from database to get something like what you want?
Like:
SELECT whatever
, C1||C2||C3||C4||...||C100 AS CDATA
, M1||M2||M3||M4||...||M100 AS MDATA
FROM ...
WHERE ...
(Of course, the concatenation operator differs between RDBMSs.)
[EDIT] I suggest to use a CompositeUserType. Here is an example. There is also a good example on page 228f in the book "Java Persistence With Hibernate".
That allows you to handle the many columns as a single object in Java.
The mapping looks like this:
#org.hibernate.annotations.Columns(columns = {
#Column(name="C1"),
#Column(name="C2"),
#Column(name="C3"),
...
})
private List<Integer> c;
Hibernate will load all columns at once during the normal query.
In your case, you must copy the int values from the list into a fixed number of columns in nullSafeSet. Pseudocode:
for (int i=1; i<numColumns; i++)
if (i < list.size())
resultSet.setInt(index+i, list.get(i));
else
resultSet.setNull(index+i, Hibernate.INTEGER.sqlType());
In nullSafeGet you must create a list and stop adding elements when a column is NULL. For additional safety, I suggest to create your own list implementation which doesn't allow to grow beyond the number of columns (inherit from ArrayList and override ensureCapacity()).
[EDIT2] If you don't want to type all the #Column annotations, use a code generator for them. That can be as simple as script which you give a name and a number and it prints #Column(...) to System.out. After the script ran, just cut&paste the data into the source.
The only other solution would be to access the internal Hibernate API to build that information at runtime but that API is internal, so a lot of stuff is private. You can use Java reflection and setAccessible(true) but that code probably won't survive the next update of Hibernate.
You can use UserTypes to map a given number of columns to any type you wish. This could be a collection if (for example) for collections are always bounded in size by a known number of items.
It's been a while (> 3 years) since I used Hibernate so I'm pretty rusty but I recall it being very easy to do; your BespokeUserType class gets passed the ResultSet to hydrate your object from it.
I too have never used Hibernate.
I suggest writing a small program in an interpreted language (such as Python) in which you can execute a string as if it were a command. You could construct a statement which takes the tedious work out of doing what you want to do manually.