I'm working with Java Apache Cayenne, under a MySQL DB.
I have quite a large table with a single bigint PK and some fields.
I'd like to retrieve just only the PK values and not all the object that maps this entity, as it would be too resource-consuming.
Is there a snippet that I can use, instead of this one that retrieves all the objects?
ObjectContext context = ...
SelectQuery select = new SelectQuery(MyClass.class);
List<MyClass> result = context.performQuery(select);
You should try using SQLTemplate instead of SelectQuery.
Here's a quick example:
ObjectContext context = ...
SQLTemplate select = new SQLTemplate(MyClass.class, "SELECT #result('PK_COLUMN' 'long') FROM MY_TABLE");
List result = context.performQuery(select);
You can find more information here
+1 for Josemando's answer. And here is another way that may work in case you are planning to only work with a subset of fetched objects:
ObjectContext context = ...
SelectQuery select = new SelectQuery(MyClass.class);
select.setPageSize(100);
List<MyClass> result = context.performQuery(select);
'setPageSize' ensures that 'result' only contains ids, until you attempt to read an object from the list. And when you do, it will resolve it page-by-page (100 objects at a time in the example above). This may fit a number of scenarios. Of course if you iterate through the entire list, eventually all objects will be fully resolved, and there will be no memory benefit.
Related
I need to copy the records of one table to another table based on some condition.
String query = "insert into public.ticket_booking_archive select * from public.ticket_booking where ticketId in (:ticketIds)";
So here the :ticketIds are dynamic, where i need to pass ticketIds to make sure whether it satisfies the condition. So it may be the matching and non matching ticket id's here at runtime.
The values of ticketIds are something like this
('f1fa3a42-5837-11ec-bf63-0242ac130002','516fd14d-3c9d-4b4b-91a0-b684d8592dfe','c9652f86-734c-4df5-8ef9-d407cb3eaf7a','df7f2812-b445-45b4-b731-da23c36d7738','f1fa3a42-5837-11ec-bf63-0242ac130002'). And this is just an example. And the list might goes on.
Since it is of type UUID, I'm storing it into a Set<UUID>
Set<UUID> tktIds = new HashSet<UUID>();
for(int i=0 ; i<ticketIds.size(); i++) {
String ticketId = ticketIds[i];
tktIds.add(UUID.fromString(ticketId));
}
Map<String, Object> params = new HashMap<>();
params.put("ticketIds", tktIds);
SqlParameterSource namedParameters =
new MapSqlParameterSource().addValue("ticketIds",params.get("ticketIds"));
Since I'm using NamedParameterJdbcTemplate, so I'm using like below
int res = writeNamedJdbcTemplate.update(query, namedParameters);
res = 3 when executed programmatically.
Here the problem is, as soon as it finds the first matching value in the IN clause it executes. And it is not considering the other matching values (ticketIds here)
But if I execute the same query in pgadmin it works fine
insert into public.ticket_booking_archive select * from public.ticket_booking where ticketId in ('f1fa3a42-5837-11ec-bf63-0242ac130002','516fd14d-3c9d-4b4b-91a0-b684d8592dfe','c9652f86-734c-4df5-8ef9-d407cb3eaf7a','df7f2812-b445-45b4-b731-da23c36d7738','f1fa3a42-5837-11ec-bf63-0242ac130002');
result is 6. Working as expected.
writeNamedJdbcTemplate.queryForObject(query, namedParameters, Integer.class); //. throws an error
Can anyone please assist? I'm really not sure where I'm making a mistake
I am not quite sure whether you are using the appropriate JDBC template for the named parameters, but you can do the following:
you can consult this article to use the right template and employ proper SQL query composition,
for string passing you can wrap the parameter mapping as shown here
after all your named parameter should work
I am using Spring Boot endpoints to return results from database queries. It works fine when using getResultList() on the TypedQuery. However I know I will have to managed very large data sets. I am looking into using ScrollableResults via hibernate but I cannot figure out how to actually reference the contents of each row.
StatelessSession session = ((Session) entityManager.getDelegate()).getSessionFactory().openStatelessSession();
criteriaQuery.multiselect(selections);
criteriaQuery.where(predicates.toArray(new Predicate[]{}));
Query<?> query = session.createQuery(criteriaQuery);
query.setMaxResults(5);
query.setFetchSize(1000);
query.setReadOnly(true);
ScrollableResults results = query.scroll(ScrollMode.FORWARD_ONLY);
while(results.next()){
Object row = results.get();
}
results.close();
session.close();
I have tried results.get(0), results.get(0)[0], results.getLong(0), Object[] row vs Object row, etc. With and without toString() on all of the options. Nothing I do gets more out of the row than the java object reference. I've tried casting as well and get a "cannot cast error". Sometimes I get an error, "query specifies a holder class". Not sure what that means because my criteria query is built by joining 1 or more entities where the entities and selected columns are not known before hand. So I am not actually specifying a class. They entities and selects are specified by user input. Any thoughts? Thanks!
UPDATE:
I can do System.out.println(scroll.getType(0)); and in this case observe a long.
But when I try to save that long (.getLong(0)) I get the error, "query specifies a holder class". Or again the cannot cast error.
Got it figured out. queryDetails is a CriteriaQuery<Tuple>
StatelessSession session = entityManagers.get("DatasourceName").unwrap(Session.class).getSessionFactory().openStatelessSession();
Stream<Tuple> resultStream = session.createQuery(queryDetails)
.setReadOnly(true)
.setMaxResults(100)
.setFetchSize(1000)
.setCacheable(false)
.getResultStream();
Iterator<Tuple> itr = resultStream.iterator();
while (itr.hasNext()){
//Get the next row:
Tuple row = itr.next();
}
A CriteriaQuery that uses multiselect produces an Object[] or javax.persistence.Tuple as result type. Maybe you should try debugging to see the actual object type and from there you can work further.
If you are processing and returning all rows anyway, there is no need to use the ScrollableResults API as you will have to create objects for all rows anyway. If your use case is to do some kind of aggregation, I would recommend you use an aggregate function instead and let the database do the aggregation.
I want to make my small spring project effectively. So I use IN clause instead of using loops in hql.
01) Question in setParameterList()
To use setParameterList(), we have to pass list object
List<Department> listDeptmntId = reportService.listDepartmentID(companyId); //list of objects
String hql = "select s.department.departmentName, g.dateTime from Gauge g inner join g.survey s where s.department in (:dpts)";
Query query = sessionFactory.getCurrentSession().createQuery(hql);
query.setParameterList("dpts",listDeptmntId);
The query works fine. But this return only one (first object in listDeptmntId list) resultset, does not return other results.
I tries to pass integer list like [1,2,3] using following method also not working.
List<Integer> dptIds=listDeptmntId.stream().map((Department::getDepartmentId()).collect(Collectors.toList());
02) Question in setParameter()
int cId=10;
String hql="...... companyId=:id"
Query query = sessionFactory.getCurrentSession().createQuery(hql);
query.setParameter("id",cId);
Sometime when I use parameter passing ("=:") , it does not work for Integers . But directly setting variable to query like following is working
int cId=10;
String hql="...... companyId="+cId
Query query = sessionFactory.getCurrentSession().createQuery(hql);
My code may be wrong because I'm going through ebooks and referring materials to do the project. Thank you in advance.
I generally do IN(?, ..., ?) using an java.sql.Array.
long[] deptIds = listDeptmntId.toArray(new long[listDeptmntId.size()];
java.sql.Array array = conn.createArrayOf("LONG", deptIds);
query.setPameter("depts", array);
q1) Check the join (can't see any other reasons to return just one object). And passing [1,2,3] won't work with that query because in query you're dealing with department object. If you want [1,2,3] to work change the query to check for dept-ids like this -> "select s.department.departmentName, g.dateTime from Gauge g inner join g.survey s where s.department.id in (:id-list)"
q2) You've already created the query object when you set the parameter, so setting parameter at that point might not affect the query object. (Your "+" approach works since the parameter is set to the query when creating the query object.)
I have a query that select a set from one database and would like to turn that into a temp table that I can transform with some aggregate functions. I would like to do this in memory using MyBatis, and I can't for the life of me seem to get it sorted out. In my mapper xml, I have:
<select id="selectExample" resultType="POJOclass">
SELECT DISTINCT anon_cookie,
search,
site
FROM
T1
WHERE
is_cookied_user = 1
and is_iab_robot = 0
and is_pattern_match_robot = 0
and has_site_search_phrase = 1
and day_dt >= current_date - 1
</select>
In my java class I am retrieving this as a list of the bean class,
List<POJO> list = selectList("selectExample", className);
I would like to find a way to query this result set, either as map, or casting it to the POJO class or perhaps creating a temp table. I can't seem to find a good way to do this using MyBatis... any suggestions?
So, I'm getting a number of instances of a particular entity by id:
for(Integer songId:songGroup.getSongIds()) {
session = HibernateUtil.getSession();
Song song = (Song) session.get(Song.class,id);
processSong(song);
}
This generates a SQL query for each id, so it occurred to me that I should do this in one, but I couldn't find a way to get multiple entities in one call except by running a query. So I wrote a query
return (List) session.createCriteria(Song.class)
.add(Restrictions.in("id",ids)).list();
But, if I enable 2nd level caching doesn't that mean that my old method would be able to return the objects from the 2nd level cache (if they had been requested before) but my query would always go to the database.
What the correct way to do this?
What you're asking to do here is for Hibernate to do special case handling for your Criteria, which is kind of a lot to ask.
You'll have to do it yourself, but it's not hard. Using SessionFactory.getCache(), you can get a reference to the actual storage for cached objects. Do something like the following:
for (Long id : allRequiredIds) {
if (!sessionFactory.getCache().containsEntity(Song.class, id)) {
idsToQueryDatabaseFor.add(id)
} else {
songs.add(session.get(Song.class, id));
}
}
List<Song> fetchedSongs = session.createCriteria(Song.class).add(Restrictions.in("id",idsToQueryDatabaseFor).list();
songs.addAll(fetchedSongs);
Then the Songs from the cache get retrieved from there, and the ones that are not get pulled with a single select.
If you know that the IDs exist, you can use load(..) to create a proxy without actually hitting the DB:
Return the persistent instance of the given entity class with the given identifier, obtaining the specified lock mode, assuming the instance exists.
List<Song> list = new ArrayList<>(ids.size());
for (Integer id : ids)
list.add(session.load(Song.class, id, LockOptions.NONE));
Once you access a non-identifier accessor, Hibernate will check the caches and fallback to DB if needed, using batch-fetching if configured.
If the ID doesn't exists, a ObjectNotFoundException will occur once the object is loaded. This might be somewhere in your code where you wouldn't really expect an exception - you're using a simple accessor in the end. So either be 100% sure the ID exists or at least force a ObjectNotFoundException early where you'd expect it, e.g. right after populating the list.
There is a difference between hibernate 2nd level cache to hibernate query cache.
The following link explains it really well: http://www.javalobby.org/java/forums/t48846.html
In a nutshell,
If you are using the same query many times with the same parameters then you can reduce database hits using a combination of both.
Another thing that you could do is to sort the list of ids, and identify subsequences of consecutive ids and then query each of those subsequences in a single query. For example, given List<Long> ids, do the following (assuming that you have a Pair class in Java):
List<Pair> pairs=new LinkedList<Pair>();
List<Object> results=new LinkedList<Object>();
Collections.sort(ids);
Iterator<Long> it=ids.iterator();
Long previous=-1L;
Long sequence_start=-1L;
while (it.hasNext()){
Long next=it.next();
if (next>previous+1) {
pairs.add(new Pair(sequence_start, previous));
sequence_start=next;
}
previous=next;
}
pairs.add(new Pair(sequence_start, previous));
for (Pair pair : pairs){
Query query=session.createQuery("from Person p where p.id>=:start_id and p.id<=:end_id");
query.setLong("start_id", pair.getStart());
query.setLong("end_id", pair.getEnd());
results.addAll((List<Object>)query.list());
}
Fetching each entity one by one in a loop can lead to N+1 query issues.
Therefore, it's much more efficient to fetch all entities at once and do the processing afterward.
Now, in your proposed solution, you were using the legacy Hibernate Criteria, but since it's been deprecated since Hibernate 4 and will probably be removed in Hibernate 6, so it's better to use one of the following alternatives.
JPQL
You can use a JPQL query like the following one:
List<Song> songs = entityManager
.createQuery(
"select s " +
"from Song s " +
"where s.id in (:ids)", Song.class)
.setParameter("ids", songGroup.getSongIds())
.getResultList();
Criteria API
If you want to build the query dynamically, then you can use a Criteria API query:
CriteriaBuilder builder = entityManager.getCriteriaBuilder();
CriteriaQuery<Song> query = builder.createQuery(Song.class);
ParameterExpression<List> ids = builder.parameter(List.class);
Root<Song> root = query
.from(Song.class);
query
.where(
root.get("id").in(
ids
)
);
List<Song> songs = entityManager
.createQuery(query)
.setParameter(ids, songGroup.getSongIds())
.getResultList();
Hibernate-specific multiLoad
List<Song> songs = entityManager
.unwrap(Session.class)
.byMultipleIds(Song.class)
.multiLoad(songGroup.getSongIds());
Now, the JPQL and Criteria API can benefit from the hibernate.query.in_clause_parameter_padding optimization as well, which allows you to increase the SQL statement caching mechanism.
For more details about loading multiple entities by their identifier, check out this article.