Develop Reference

How can I avoid Stream<X> return type method in JpaRepository loading all data into memory with EclipseLink JPA?

We are using Spring data 2.4.4 + EclipseLink 2.7.0 (JPA 2.2) in our project.
Recently, we are developing a function allowing user downloading data by xlsx, and when I test interface with large dataset, it ran out of memory (OutOfMemoryError) unsurprisingly. So we are considering using Stream tpye method in JpaRepository expecting EclipseLink will return a Stream implemented by CursoredStream or ScrollableCursor. However, it seems to behave just like getting a List.
To verify, I defined a method that would fetch all job orders from db:
#Query("select jo from JobOrder jo order by jo.oid")
Stream<JobOrder> streamAll();
And wrap it with a transaction:
#Repository
public class JobOrderTestDAO {
#Autowired
private JobOrderRepository repository;
#Transactional(readOnly = true)
public Stream<JobOrder> testGetAllByStream() {
return repository.streamAll();
}
}
Finally, in the test, I limit the stream size to 10 and print their oid in console. If Cursor is used as container, results should be returned immediately.
#Autowired
private JobOrderTestDAO testDAO;
#Test
void testGetAllByStream() {
Stream<JobOrder> joStream = testDAO.testGetAllByStream();
joStream.limit(10).forEach(System.out::println);
joStream.close();
}
However, no results returned, only to find memory exploding. We checked the source code, EclipseLink seems not providing real-streaming solution for getResultStream() that said to "provide additional capabilities".
default Stream<X> getResultStream() {
return getResultList().stream();
}
Now we're using a somehow tricky workaround by downgrading JPA to 2.1.x. Since StreamExecutor will explicitly calling Cursor based function.
protected Object doExecute(final AbstractJpaQuery query, JpaParametersParameterAccessor accessor) {
if (!SurroundingTransactionDetectorMethodInterceptor.INSTANCE.isSurroundingTransactionActive()) {
throw new InvalidDataAccessApiUsageException(NO_SURROUNDING_TRANSACTION);
}
Query jpaQuery = query.createQuery(accessor);
// JPA 2.2 on the classpath
if (streamMethod != null) {
return ReflectionUtils.invokeMethod(streamMethod, jpaQuery);
}
// Fall back to legacy stream execution
PersistenceProvider persistenceProvider = PersistenceProvider.fromEntityManager(query.getEntityManager());
//Implementation here is using Cursor
CloseableIterator<Object> iter = persistenceProvider.executeQueryWithResultStream(jpaQuery);
return StreamUtils.createStreamFromIterator(iter);
}
It may not be a good practice to exclude a jar that match the versions and reinclude a jar that has been out of date. Thus, we are seeking for a solution, that might possibly keep JpaRepository and JpaSpecificationExecutor instead of coding directly with ExpressionBuilder and with stream underlying Stream.

Have the same issue, what I found is that spring-data-jpa starting from 1.11.8 version changed implementation of JpaQueryExecution.doExecute. So instead of running persistenceProvider.executeQueryWithResultStream it calls Query.getResultStream method. The default implementation for getResultStream method is getResultList().stream(). That means that instead of real streaming and using scrollable cursors it tries to put all the data in memory. Eclipselink do not overrides default behavior for getResultStream method till current version 3.0.
A few options could be used here:
Instead of spring-data use directly JDBC like spring-data did on versions before 1.11.8.
Instead of eclipselink use hibernate or any other persistence provider who has full support of JPA 2.2 features.

Calling stored procedure in spring boot with Jpa returns empty objects

I'm trying to fetch data from the store procedures in my spring boot rest api. The procedure call is successful but it return empty objects instead of returning any data. When I run those stored procedures in SQL Server those return the actual data.
Here are my stored procedure calls from the repository:
Get all stats without any param
#Repository
public interface GirdawariInfoAllDivisionsRepo extends
JpaRepository<GirdawariInfoOfAllDivisions, Long> {
#Query(value = "GetGirdawariInfoOfAllDivisions", nativeQuery = true)
List<GirdawariInfoOfAllDivisions> getAllDivisionsStatsList();
}
Get Division's data based on division id
#Repository
public interface GirdawariInfoOfDistrictsByDivisionRepo extends
JpaRepository<GirdawariInfoOfDistrictsByDivision, Long> {
#Query(value = "{CALL GetGirdawariInfoOfDistrictsByDivision(:Division_Id)};", nativeQuery =
true)
List<GirdawariInfoOfDistrictsByDivision> getStatsBylDivisionsId(#Param("Division_Id") Long
Division_Id);
}
The 1st one return 9 records in SQL Server but return 9 empty objects in Jpa like
[{},{},{},{},{},{},{},{},{}]
The 2nd returns 5 records in SQL Server but return 5 empty objects in Jpa like
[{},{},{},{},{}]
The number of objects spring boot is returning is the same as expected but the problem is those are empty.
Please let me know if I have to add any other information.
Any help will be appreciated. Thanks

There are two different approaches to call stored procedures in spring-boot application.
Approach 1 – #NamedStoredProcedureQuery Annotation
Approach-2 #Procedure Annotation
Refer here. It has code example for both the approaches.

how to write not in clause using springboot findAll() method

I am try to get data from the database using spring boot JPA and CRUD Repository findAll() method with using not in clause but not find any where. There is any solution making dynamic query using findAll() method.
Example:
hql="select * from urlMaster urlmast where urlmast.urlcd not in (select distinct acturl.acuCd from ActionUrl acturl) order by urlmast.urlcd";
Currently I am just getting data from the urlMaster data. But i want acturl.acuCd not in ActionUrl table.
Repository class:
public interface ActionUrlRepository extends CrudRepository<urlMaster, Long>,JpaRepository<urlMaster, Long> {
}
Service implementation:
actionurlrepository.findAll();
If there is any provision? Please guide me. Thanks!

You could put your query string in an #Query annotation, as described in https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.query-methods.at-query
So something like :
public interface ActionUrlRepository extends JpaRepository<urlMaster, Long> {
#Query("select * from urlMaster urlmast where urlmast.urlcd not in (select distinct acturl.acuCd from ActionUrl acturl) order by urlmast.urlcd")
List<urlMaster> findByNotIn();
}
By the way, you don't need CrudRepository on the declaration, since JpaRepository already extends from it (via PagingAndSortingRepository).

How to make serverStatus() query via spring data in mongoDb

I got database with several collections in micorservice, so there's controllers which works fine. But I need to collect statistics from servers, including database. There's a good query serverStats() which gives all the information about the server, but how can I pass it through app layers?
I made interface repository extended ReactiveCrudRepository using SomeModel class as a place holder and wrote one method like String getStatistics() with #Query annotation, but it doesn't work
public interface MongoMonitoring extends ReactiveCrudRepository<SomeModel, String> {
#Query("{ serverStatus: 1 }")
String getStatus();

Use MongoTemplate. The code like:
private MongoTemplate mongoTemplate;
Document result = this.mongoTemplate.executeCommand("{ serverStatus: 1 }");
The document is from package org.bson.Document which is like JSON.

How to findOneAndDelete in mongodb using MongoRepository in Spring

I am using a Spring web service and MongoDb to hold my data. Currently my Repository class extends MongoRepository and an obj of its interface gets injected into my Controller.
There is no method for finding and deleting a specific entity in the MongoRepository interface. How can I do this without having to provide a concrete implementation? I need to do the operation at the same time.
Here's my code on github if its useful: https://github.com/RyanNewsom/DentistAppointmentSchedulerService

I ended up figuring this one out. I made a custom class and used MongoTemplate. You can then submit a query using mongoTemplate. It contains a lot more mongo specific implementation.
#Repository
public class AppointmentCustomRepository {
#Autowired
MongoTemplate mongoTemplate;
public Appointment getAppointmentAndDelete(String id) {
return mongoTemplate.findAndRemove(Query.query(Criteria.where("id").is(id)), Appointment.class);
}
}

Sample snippet to find and delete one document in Mongo DB using Java API's
MongoCollection<Document> collection = database.getCollection("PasstheCollectionName");
Document document = collection.find.first();
Object value = document.get("_id");
Bson filter = Filter.and(Filter.eq("_id",value));
collection.findOneAndDelete(filter);