Develop Reference

LastModifiedFileListFilter for Sftp inbound adapter

I am trying to implement LastModifiedFileListFilter as it looks like there is no similar filter for spring-integration-sftp yet for 5.3.2 release, I tried to copy the LastModifiedFileListFilter from spring-integration-file but the discard callback isn't working. Here is my implementation:
#Slf4j
#Data
public class LastModifiedLsEntryFileListFilter implements DiscardAwareFileListFilter<ChannelSftp.LsEntry> {
private static final long ONE_SECOND = 1000;
private static final long DEFAULT_AGE = 30;
private volatile long age = DEFAULT_AGE;
#Nullable
private Consumer<ChannelSftp.LsEntry> discardCallback;
public LastModifiedLsEntryFileListFilter(final long age) {
this.age = age;
}
#Override
public List<ChannelSftp.LsEntry> filterFiles(final ChannelSftp.LsEntry[] files) {
final List<ChannelSftp.LsEntry> list = new ArrayList<>();
final long now = System.currentTimeMillis() / ONE_SECOND;
for (final ChannelSftp.LsEntry file : files) {
if (this.fileIsAged(file, now)) {
log.info("File [{}] is aged...", file.getFilename());
list.add(file);
} else if (this.discardCallback != null) {
log.info("File [{}] is still being uploaded...", file.getFilename());
this.discardCallback.accept(file);
}
}
return list;
}
#Override
public boolean accept(final ChannelSftp.LsEntry file) {
if (this.fileIsAged(file, System.currentTimeMillis() / ONE_SECOND)) {
return true;
}
else if (this.discardCallback != null) {
this.discardCallback.accept(file);
}
return false;
}
private boolean fileIsAged(final ChannelSftp.LsEntry file, final long now) {
return file.getAttrs().getMTime() + this.age <= now;
}
#Override
public void addDiscardCallback(#Nullable final Consumer<ChannelSftp.LsEntry> discardCallbackToSet) {
this.discardCallback = discardCallbackToSet;
}
}
The filter is able to correctly identify the age of file and discards it but that file is not retried which I believe is part of discard callback.
I guess my question is how to set discard callback to keep retrying the discarded file until files ages. Thanks

not retried which I believe is part of discard callback.
I wonder what makes you think that way...
The fact that FileReadingMessageSource with its WatchService option has the logic like this:
if (filter instanceof DiscardAwareFileListFilter) {
((DiscardAwareFileListFilter<File>) filter).addDiscardCallback(this.filesToPoll::add);
}
doesn't mean that SFTP implementation is similar.
The retry is there anyway: on the next poll not accepted file will be checked again.
You probably don't show other filters you use, and your file is filtered out before this LastModifiedLsEntryFileListFilter, e.g. with the SftpPersistentAcceptOnceFileListFilter. You need to consider to have your "last-modified" as a first one in the chain.
If you are not going to support discard callback from the outside, you probably don't need to implement that DiscardAwareFileListFilter at all.

How to fetch data from different sources (device cache, web server and sqlite database) synchronously?

and happy new year!
I am Learning android development using java, and I started by creating my first application using architecture components following this tutorial:
Guide to app architecture
I have a good understanding about android architecture design and the other basic stuff. But, I still have a problem in handling background tasks because there are many classes that provide multi-threading like : Executor, IntentService and asynctask..
My code :
Repository class:
public class MovieRpository {
private IWebService m_webService;
private Movie _movie;
private AppExecuters m_executers;
private MovieRpository(AppExecuters executers, TMDB webService){
this.m_executers = executers;
this.m_webService = webService;
}
private static MovieRpository m_singleInstance;
public static MovieRpository getInstance(){
if(m_singleInstance == null){
m_singleInstance = new
MovieRpository(AppExecuters.getInstance(), new TMDB(new JsonParser<Movie>()));
}
return m_singleInstance;
}
public LiveData<Movie> getMovie( String movieId){
final MutableLiveData<Movie> _liveDataMovie = new MutableLiveData<Movie>();
m_executers.networkIO().execute(()->{
_movie =(Movie)m_webService.getObject(movieId);});
_liveDataMovie.setValue(_movie);
return _liveDataMovie;
}
}
My problem :
In Repository class when i use debugger inside the second thread which is created by m_executers.networkIO() _movie has a value inside the network thread. but, it is always null outside execute method.
The rest of my code:
-IWebService interface
public interface IWebService<T> {
public T getObject(String url);
}
TMDB:
public class TMDB implements IWebService<Movie> {
private JsonParser<Movie> m_jsonParser;
private Movie _movie;
public TMDB(JsonParser<Movie> jsonParser){
m_jsonParser = jsonParser;
}
#Override
public Movie getObject(String url) {
try {
HttpURLConnection _connection = (HttpURLConnection) (new URL(url)).openConnection();
_connection.setRequestMethod(Settings.CONNECTION_GET_METHOD);
_connection.setConnectTimeout(Settings.CONNECTION_TIMEOUT);
_connection.setReadTimeout(Settings.DATA_READ_TIMEOUT);
_connection.connect();
InputStreamReader _streamReader = new InputStreamReader(_connection.getInputStream());
BufferedReader _bufferedReader = new BufferedReader(_streamReader);
StringBuilder _data = new StringBuilder();
String _line;
while((_line = _bufferedReader.readLine()) != null ){
_data.append(_line);
}
_bufferedReader.close();
_streamReader.close();
Gson _gson = new Gson();
_movie = _gson.fromJson(_data.toString(), Movie.class);
}
catch(MalformedURLException e) {
e.printStackTrace();
}
catch(IOException e){
e.printStackTrace();
}
return _movie;
}
}
AppExecuters singleton class for managing threads:
public class AppExecuters {
private static AppExecuters m_singltonInstance;
private Executor m_NetworkIO;
private Executor m_diskIO;
private Executor m_mainThread;
private static final Object LOCK = new Object();
private AppExecuters(Executor diskThread, Executor networkThread, Executor mainThread){
this.m_diskIO = diskThread;
this.m_mainThread = mainThread;
this.m_NetworkIO = networkThread;
}
public static AppExecuters getInstance(){
if(m_singltonInstance == null){
// In-order to make the singleton instance synchronized between all threads.
synchronized (LOCK){
m_singltonInstance = new AppExecuters(Executors.newSingleThreadExecutor(), Executors.newFixedThreadPool(3), new MainThread());
}
}
return m_singltonInstance;
}
public Executor networkIO(){
return m_NetworkIO;
}
public Executor diskIO(){
return m_diskIO;
}
public Executor mainThread(){
return m_mainThread;
}
private static class MainThread implements Executor{
private Handler mainThreadHandler = new Handler(Looper.getMainLooper());
#Override
public void execute(Runnable command) {
mainThreadHandler.post(command);
}
}
}
I also need to know the best practice for managing
threads in such case.

Get rowId after inserted Room Persistence Library

All I have to do is get the return value from the insert as a long. I got that but it isn't working, I am getting back 0 from the returned value. I am using a DAO, Repository and ViewModel as stated in the Google CodeLabs. I have followed this post Rowid after Insert in Room.
Player Class
#Entity(tableName = "player_table")
public class Player {
#PrimaryKey(autoGenerate = true)
private long id;
#NonNull
#ColumnInfo(name = "username")
private String username;
}
DAO
#Insert
long insert(Player player);
Repository
public long insert(Player player) {
new insertAsyncTask(mPlayerDao).execute(player);
rowId = player.getId();
return rowId;
}
ViewModel
public long insert(Player player){
rowId = mRepository.insert(player);
return rowId;
}
Activity
String playerString = editTextUsername.getText().toString();
Player player = new Player(playerString);
long rowId = mDreamViewModel.insert(player);

The problem is that you return player.getId() before the AsyncTask finishes its background work of insertion; you must wait until it delivers you the correct result of insertion, and here is a suggested solution using the thread-safe CountDownLatch which pauses the the execution of the subsequent code using .await() method until count of the CountDownLatch reaches 0; where it decrements by 1 each time .countDown() is invoked.
public class Repository {
private long rowId = -1;
private CountDownLatch mLatch;
public long insert(Player player) {
mLatch = new CountDownLatch(1);
new insertAsyncTask(mPlayerDao).execute(player);
try {
mLatch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
Log.i("LOG_TAG", String.valueOf(rowId));
return rowId;
}
class insertAsyncTask extends AsyncTask<Player, Void, Void> {
#Override
protected Void doInBackground(Player... players) {
rowId = mDatabase.getContactDAO().addContact(players[0]);
mLatch.countDown();
return null;
}
}
}

The insert method in your Repository class can be changed to this:
public long insert(Player player) {
return new insertAsyncTask(mPlayerDao).execute(player).get();
}
I found this by searching GitHub for AsyncTasks with "Void, Long" as the last two types, along with some other Room-specific terms.
When debugging my own application I saw that execution of the return statement in the insert method was occurring before the doInBackground method was executed, which seemed obvious after I saw it happen.

executing a method in parallel from a call method

I have a library which is being used by customer and they are passing DataRequest object which has userid, timeout and some other fields in it. Now I use this DataRequest object to make a URL and then I make an HTTP call using RestTemplate and my service returns back a JSON response which I use it to make a DataResponse object and return this DataResponse object back to them.
Below is my DataClient class used by customer by passing DataRequest object to it. I am using timeout value passed by customer in DataRequest to timeout the request if it is taking too much time in getSyncData method.
public class DataClient implements Client {
private RestTemplate restTemplate = new RestTemplate();
// first executor
private ExecutorService service = Executors.newFixedThreadPool(15);
#Override
public DataResponse getSyncData(DataRequest key) {
DataResponse response = null;
Future<DataResponse> responseFuture = null;
try {
responseFuture = getAsyncData(key);
response = responseFuture.get(key.getTimeout(), key.getTimeoutUnit());
} catch (TimeoutException ex) {
response = new DataResponse(DataErrorEnum.CLIENT_TIMEOUT, DataStatusEnum.ERROR);
responseFuture.cancel(true);
// logging exception here
}
return response;
}
#Override
public Future<DataResponse> getAsyncData(DataRequest key) {
DataFetcherTask task = new DataFetcherTask(key, restTemplate);
Future<DataResponse> future = service.submit(task);
return future;
}
}
DataFetcherTask class:
public class DataFetcherTask implements Callable<DataResponse> {
private DataRequest key;
private RestTemplate restTemplate;
public DataFetcherTask(DataRequest key, RestTemplate restTemplate) {
this.key = key;
this.restTemplate = restTemplate;
}
#Override
public DataResponse call() throws Exception {
// In a nutshell below is what I am doing here.
// 1. Make an url using DataRequest key.
// 2. And then execute the url RestTemplate.
// 3. Make a DataResponse object and return it.
// I am calling this whole logic in call method as LogicA
}
}
As of now my DataFetcherTask class is responsible for one DataRequest key as shown above..
Problem Statement:-
Now I have a small design change. Customer will pass DataRequest (for example keyA) object to my library and then I will make a new http call to another service (which I am not doing in my current design) by using user id present in DataRequest (keyA) object which will give me back list of user id's so I will use those user id's and make few other DataRequest (keyB, keyC, keyD) objects one for each user id returned in the response. And then I will have List<DataRequest> object which will have keyB, keyC and keyD DataRequest object. Max element in the List<DataRequest> will be three, that's all.
Now for each of those DataRequest object in List<DataRequest> I want to execute above DataFetcherTask.call method in parallel and then make List<DataResponse> by adding each DataResponse for each key. So I will have three parallel calls to DataFetcherTask.call. Idea behind this parallel call is to get the data for all those max three keys in the same global timeout value.
So my proposal is - DataFetcherTask class will return back List<DataResponse> object instead of DataResponse and then signature of getSyncData and getAsyncData method will change as well. So here is the algorithm:
Use DataRequest object passed by customer to make List<DataRequest> by calling another HTTP service.
Make a parallel call for each DataRequest in List<DataRequest> to DataFetcherTask.call method and return List<DataResponse> object to customer instead of DataResponse.
With this way, I can apply same global timeout on step 1 along with step 2 as well. If either of above step is taking time, we will just timeout in getSyncData method.
DataFetcherTask class after design change:
public class DataFetcherTask implements Callable<List<DataResponse>> {
private DataRequest key;
private RestTemplate restTemplate;
// second executor here
private ExecutorService executorService = Executors.newFixedThreadPool(10);
public DataFetcherTask(DataRequest key, RestTemplate restTemplate) {
this.key = key;
this.restTemplate = restTemplate;
}
#Override
public List<DataResponse> call() throws Exception {
List<DataRequest> keys = generateKeys();
CompletionService<DataResponse> comp = new ExecutorCompletionService<>(executorService);
int count = 0;
for (final DataRequest key : keys) {
comp.submit(new Callable<DataResponse>() {
#Override
public DataResponse call() throws Exception {
return performDataRequest(key);
}
});
}
List<DataResponse> responseList = new ArrayList<DataResponse>();
while (count-- > 0) {
Future<DataResponse> future = comp.take();
responseList.add(future.get());
}
return responseList;
}
// In this method I am making a HTTP call to another service
// and then I will make List<DataRequest> accordingly.
private List<DataRequest> generateKeys() {
List<DataRequest> keys = new ArrayList<>();
// use key object which is passed in contructor to make HTTP call to another service
// and then make List of DataRequest object and return keys.
return keys;
}
private DataResponse performDataRequest(DataRequest key) {
// This will have all LogicA code here which is shown in my original design.
// everything as it is same..
}
}
Now my question is -
Does it have to be like this? What is the right design to solve this problem? I mean having call method in another call method looks weird?
Do we need to have two executors like I have in my code? Is there any better way to solve this problem or any kind of simplification/design change we can do here?
I have simplified the code so that idea gets clear what I am trying to do..

As already mentioned in the comments of your question, you can use Java's ForkJoin framework. This will save you the extra thread pool within your DataFetcherTask.
You simply need to use a ForkJoinPool in your DataClient and convert your DataFetcherTask into a RecursiveTask (one of ForkJoinTask's subtypes). This allows you to easily execute other subtasks in parallel.
So, after these modifications your code will look something like this:
DataFetcherTask
The DataFetcherTask is now a RecursiveTask which first generates the keys and invokes subtasks for each generated key. These subtasks are executed in the same ForkJoinPool as the parent task.
public class DataFetcherTask extends RecursiveTask<List<DataResponse>> {
private final DataRequest key;
private final RestTemplate restTemplate;
public DataFetcherTask(DataRequest key, RestTemplate restTemplate) {
this.key = key;
this.restTemplate = restTemplate;
}
#Override
protected List<DataResponse> compute() {
// Create subtasks for the key and invoke them
List<DataRequestTask> requestTasks = requestTasks(generateKeys());
invokeAll(requestTasks);
// All tasks are finished if invokeAll() returns.
List<DataResponse> responseList = new ArrayList<>(requestTasks.size());
for (DataRequestTask task : requestTasks) {
try {
responseList.add(task.get());
} catch (InterruptedException | ExecutionException e) {
// TODO - Handle exception properly
Thread.currentThread().interrupt();
return Collections.emptyList();
}
}
return responseList;
}
private List<DataRequestTask> requestTasks(List<DataRequest> keys) {
List<DataRequestTask> tasks = new ArrayList<>(keys.size());
for (DataRequest key : keys) {
tasks.add(new DataRequestTask(key));
}
return tasks;
}
// In this method I am making a HTTP call to another service
// and then I will make List<DataRequest> accordingly.
private List<DataRequest> generateKeys() {
List<DataRequest> keys = new ArrayList<>();
// use key object which is passed in contructor to make HTTP call to another service
// and then make List of DataRequest object and return keys.
return keys;
}
/** Inner class for the subtasks. */
private static class DataRequestTask extends RecursiveTask<DataResponse> {
private final DataRequest request;
public DataRequestTask(DataRequest request) {
this.request = request;
}
#Override
protected DataResponse compute() {
return performDataRequest(this.request);
}
private DataResponse performDataRequest(DataRequest key) {
// This will have all LogicA code here which is shown in my original design.
// everything as it is same..
return new DataResponse(DataErrorEnum.OK, DataStatusEnum.OK);
}
}
}
DataClient
The DataClient will not change much except for the new thread pool:
public class DataClient implements Client {
private final RestTemplate restTemplate = new RestTemplate();
// Replace the ExecutorService with a ForkJoinPool
private final ForkJoinPool service = new ForkJoinPool(15);
#Override
public List<DataResponse> getSyncData(DataRequest key) {
List<DataResponse> responsList = null;
Future<List<DataResponse>> responseFuture = null;
try {
responseFuture = getAsyncData(key);
responsList = responseFuture.get(key.getTimeout(), key.getTimeoutUnit());
} catch (TimeoutException | ExecutionException | InterruptedException ex) {
responsList = Collections.singletonList(new DataResponse(DataErrorEnum.CLIENT_TIMEOUT, DataStatusEnum.ERROR));
responseFuture.cancel(true);
// logging exception here
}
return responsList;
}
#Override
public Future<List<DataResponse>> getAsyncData(DataRequest key) {
DataFetcherTask task = new DataFetcherTask(key, this.restTemplate);
return this.service.submit(task);
}
}
Once you are on Java8 you may consider changing the implementation to CompletableFutures. Then it would look something like this:
DataClientCF
public class DataClientCF {
private final RestTemplate restTemplate = new RestTemplate();
private final ExecutorService executor = Executors.newFixedThreadPool(15);
public List<DataResponse> getData(DataRequest initialKey) {
return CompletableFuture.supplyAsync(() -> generateKeys(initialKey), this.executor)
.thenApply(requests -> requests.stream().map(this::supplyRequestAsync).collect(Collectors.toList()))
.thenApply(responseFutures -> responseFutures.stream().map(future -> future.join()).collect(Collectors.toList()))
.exceptionally(t -> { throw new RuntimeException(t); })
.join();
}
private List<DataRequest> generateKeys(DataRequest key) {
return new ArrayList<>();
}
private CompletableFuture<DataResponse> supplyRequestAsync(DataRequest key) {
return CompletableFuture.supplyAsync(() -> new DataResponse(DataErrorEnum.OK, DataStatusEnum.OK), this.executor);
}
}
As mentioned in the comments, Guava's ListenableFutures would provide similar functionality for Java7 but without Lambdas they tend to get clumsy.

As I know, RestTemplate is blocking, it is said in ForkJoinPool JavaDoc in ForkJoinTask:
Computations should avoid synchronized methods or blocks, and should minimize other blocking synchronization apart from joining other tasks or using synchronizers such as Phasers that are advertised to cooperate with fork/join scheduling. ...
Tasks should also not perform blocking IO,...
Call in call is redundant.
And you don't need two executors. Also you can return partial result in getSyncData(DataRequest key). This can be done like this
DataClient.java
public class DataClient implements Client {
private RestTemplate restTemplate = new RestTemplate();
// first executor
private ExecutorService service = Executors.newFixedThreadPool(15);
#Override
public List<DataResponse> getSyncData(DataRequest key) {
List<DataResponse> responseList = null;
DataFetcherResult response = null;
try {
response = getAsyncData(key);
responseList = response.get(key.getTimeout(), key.getTimeoutUnit());
} catch (TimeoutException ex) {
response.cancel(true);
responseList = response.getPartialResult();
}
return responseList;
}
#Override
public DataFetcherResult getAsyncData(DataRequest key) {
List<DataRequest> keys = generateKeys(key);
final List<Future<DataResponse>> responseList = new ArrayList<>();
final CountDownLatch latch = new CountDownLatch(keys.size());//assume keys is not null
for (final DataRequest _key : keys) {
responseList.add(service.submit(new Callable<DataResponse>() {
#Override
public DataResponse call() throws Exception {
DataResponse response = null;
try {
response = performDataRequest(_key);
} finally {
latch.countDown();
return response;
}
}
}));
}
return new DataFetcherResult(responseList, latch);
}
// In this method I am making a HTTP call to another service
// and then I will make List<DataRequest> accordingly.
private List<DataRequest> generateKeys(DataRequest key) {
List<DataRequest> keys = new ArrayList<>();
// use key object which is passed in contructor to make HTTP call to another service
// and then make List of DataRequest object and return keys.
return keys;
}
private DataResponse performDataRequest(DataRequest key) {
// This will have all LogicA code here which is shown in my original design.
// everything as it is same..
return null;
}
}
DataFetcherResult.java
public class DataFetcherResult implements Future<List<DataResponse>> {
final List<Future<DataResponse>> futures;
final CountDownLatch latch;
public DataFetcherResult(List<Future<DataResponse>> futures, CountDownLatch latch) {
this.futures = futures;
this.latch = latch;
}
//non-blocking
public List<DataResponse> getPartialResult() {
List<DataResponse> result = new ArrayList<>(futures.size());
for (Future<DataResponse> future : futures) {
try {
result.add(future.isDone() ? future.get() : null);
//instead of null you can return new DataResponse(DataErrorEnum.NOT_READY, DataStatusEnum.ERROR);
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
//ExecutionException or CancellationException could be thrown, especially if DataFetcherResult was cancelled
//you can handle them here and return DataResponse with corresponding DataErrorEnum and DataStatusEnum
}
}
return result;
}
#Override
public List<DataResponse> get() throws ExecutionException, InterruptedException {
List<DataResponse> result = new ArrayList<>(futures.size());
for (Future<DataResponse> future : futures) {
result.add(future.get());
}
return result;
}
#Override
public List<DataResponse> get(long timeout, TimeUnit timeUnit)
throws ExecutionException, InterruptedException, TimeoutException {
if (latch.await(timeout, timeUnit)) {
return get();
}
throw new TimeoutException();//or getPartialResult()
}
#Override
public boolean cancel(boolean mayInterruptIfRunning) {
boolean cancelled = true;
for (Future<DataResponse> future : futures) {
cancelled &= future.cancel(mayInterruptIfRunning);
}
return cancelled;
}
#Override
public boolean isCancelled() {
boolean cancelled = true;
for (Future<DataResponse> future : futures) {
cancelled &= future.isCancelled();
}
return cancelled;
}
#Override
public boolean isDone() {
boolean done = true;
for (Future<DataResponse> future : futures) {
done &= future.isDone();
}
return done;
}
//and etc.
}
I wrote it with CountDownLatch and it looks great, but note there is a nuance.
You can get stuck for a little while in DataFetcherResult.get(long timeout, TimeUnit timeUnit) because CountDownLatch is not synchronized with future's state. And it could happen that latch.getCount() == 0 but not all futures would return future.isDone() == true at the same time. Because they have already passed latch.countDown(); inside finally {} Callable's block but didn't change internal state which is still equals to NEW.
And so calling get() inside get(long timeout, TimeUnit timeUnit) can cause a small delay.
Similar case was described here.
Get with timeout DataFetcherResult.get(...) could be rewritten using futures future.get(long timeout, TimeUnit timeUnit) and you can remove CountDownLatch from a class.
public List<DataResponse> get(long timeout, TimeUnit timeUnit)
throws ExecutionException, InterruptedException{
List<DataResponse> result = new ArrayList<>(futures.size());
long timeoutMs = timeUnit.toMillis(timeout);
boolean timeout = false;
for (Future<DataResponse> future : futures) {
long beforeGet = System.currentTimeMillis();
try {
if (!timeout && timeoutMs > 0) {
result.add(future.get(timeoutMs, TimeUnit.MILLISECONDS));
timeoutMs -= System.currentTimeMillis() - beforeGet;
} else {
if (future.isDone()) {
result.add(future.get());
} else {
//result.add(new DataResponse(DataErrorEnum.NOT_READY, DataStatusEnum.ERROR)); ?
}
}
} catch (TimeoutException e) {
result.add(new DataResponse(DataErrorEnum.TIMEOUT, DataStatusEnum.ERROR));
timeout = true;
}
//you can also handle ExecutionException or CancellationException here
}
return result;
}
This code was given as an example and it should be tested before using in production, but seems legit :)

spring data mongodb Cannot perform cascade save on child object without id set

I am using #CascadeSave to save child object in separate collection.
My Document classes are :
public class FbUserProfile{
#Id
private long id;
#DBRef(lazy=true)
#CascadeSave()
private Set<FacebookFriend> friends;
#DBRef(lazy=true)
#CascadeSave()
private Set<FacebookFriendList> customFriendList;
}
public class FacebookFriend{
#Id
private long id;
private String name;
}
public class FacebookFriendList{
#Id
private long id;
private String name;
private String list_type;
}
I add some object in both friends,customFriendList.
and try to update fbUserProfile object using:
mongoTemplate.save(fbUserProfile);
note: fbUserProfile already exists in db. Now I am updating this
Error Message: Cannot perform cascade save on child object without id set
If I remove #CascadeSave. It works fine for me. How I can Cascade set objects.
I am also using #CascadeSave with other objects. Its working fine but they are not set object.

I found the same tutorials somewhere else: Baeldung's and JavaCodeGeeks (this is the one i've followed)
I've had that same problem, and I could solve it.
It happens when you try to persist a collection. It doesn't matter that the collection's items have the #Id, because the collection itself won't have it. I edited the code in the EventListener's onBeforeConvert to check if the field you're trying to CascadeSave is a collection (in my case a List). If it's a list, you just cycle through it checking each individual item for #Id and saving them.
If it's not a collection you still have to persist them the same way you did before
#Override
public void onBeforeConvert(Object source) {
ReflectionUtils.doWithFields(source.getClass(), new ReflectionUtils.FieldCallback() {
#Override
public void doWith(Field field)
throws IllegalArgumentException, IllegalAccessException {
ReflectionUtils.makeAccessible(field);
if (field.isAnnotationPresent(DBRef.class) && field.isAnnotationPresent(CascadeSave.class)){
final Object fieldValue = field.get(source);
if(fieldValue instanceof List<?>){
for (Object item : (List<?>)fieldValue){
checkNSave(item);
}
}else{
checkNSave(fieldValue);
}
}
}
});
}
private void checkNSave(Object fieldValue){
DbRefFieldCallback callback = new DbRefFieldCallback();
ReflectionUtils.doWithFields(fieldValue.getClass(), callback);
if (!callback.isIdFound()){
throw new MappingException("Oops, something went wrong. Child doesn't have #Id?");
}
mongoOperations.save(fieldValue);
}

The best way to set an ID on the dependent child object is to write a listener class by extending AbstractMongoEventListener class and override the onConvert() method.
public class CustomMongoEventListener extends
AbstractMongoEventListener<Object> {
#Autowired
private MongoOperations mongoOperations;
#Override
public void onBeforeConvert(final Object entity) {
if (entity.id == null || entity.id.isEmpty()) {
entity.id = generateGuid(); //generate random sequence ID
}
public static String generateGuid() {
SecureRandom randomGen = new SecureRandom();
byte[] byteArray = new byte[16];
randomGen.nextBytes(byteArray);
return new Base32().encodeToString(byteArray).substring(0,26);
}
}
Finally register your custom listener in `your configuration file. For annotation approach use the following code to register :
#Bean
public CustomMongoEventListener cascadingMongoEventListener() {
return new CustomMongoEventListener();
}

The above solution works fine incase if you have a list. But we can avoid firing a save query for each element from the list, as it reduces the performance. Here is the solution I have found out of the above code.
#Override
public void onBeforeConvert(BeforeConvertEvent<Object> event) {
Object source = event.getSource();
ReflectionUtils.doWithFields(source.getClass(), new ReflectionUtils.FieldCallback() {
#Override
public void doWith(Field field)
throws IllegalArgumentException, IllegalAccessException {
ReflectionUtils.makeAccessible(field);
if (field.isAnnotationPresent(DBRef.class) && field.isAnnotationPresent(CascadeSave.class)){
final Object fieldValue = field.get(source);
if(fieldValue instanceof List<?>){
for (Object item : (List<?>)fieldValue){
checkNAdd(item);
}
}else{
checkNAdd(fieldValue);
}
mongoOperations.insertAll(documents);
}
}
});
}
private void checkNAdd(Object fieldValue){
DbRefFieldCallback callback = new DbRefFieldCallback();
ReflectionUtils.doWithFields(fieldValue.getClass(), callback);
if (!callback.isIdFound()){
throw new MappingException("Oops, something went wrong. Child doesn't have #Id?");
}
documents.add(fieldValue);
}

Okey I extend the class and it will check if the document is exist if it exist it will update the document else it insert the document:
#Component
class GenericCascadeMongo(
private val mongoTemplate: MongoTemplate
) : AbstractMongoEventListener<Any>() {
override fun onBeforeConvert(event: BeforeConvertEvent<Any?>) {
val source = event.source
?: return
ReflectionUtils.doWithFields(source.javaClass) { field ->
ReflectionUtils.makeAccessible(field)
if (field.isAnnotationPresent(DBRef::class.java) && field.isAnnotationPresent(CascadeSave::class.java)) {
val fieldValue = field[source]
?: return#doWithFields
if (fieldValue is List<*>) {
fieldValue.filterNotNull().forEach {
checkAndSave(it)
}
} else {
checkAndSave(fieldValue)
}
}
}
}
private fun checkAndSave(fieldValue: Any) {
try {
val callback = DbRefFieldCallback(fieldValue)
ReflectionUtils.doWithFields(fieldValue.javaClass, callback)
if (!callback.isIdFound && callback.id == null) {
mongoTemplate.insert(fieldValue)
}
if (callback.id != null) {
val findById = mongoTemplate.exists(Query(Criteria.where(MConst.MONGO_ID).`is`(callback.id)), fieldValue.javaClass)
if (findById) {
mongoTemplate.save(fieldValue)
} else {
mongoTemplate.insert(fieldValue)
}
}
} catch (e: Exception) {
e.printStackTrace()
}
}
private class DbRefFieldCallback(val fieldValue: Any?) : FieldCallback {
var isIdFound = false
private set
var id: String? = null
private set
#Throws(IllegalArgumentException::class, IllegalAccessException::class)
override fun doWith(field: Field) {
ReflectionUtils.makeAccessible(field)
if (field.isAnnotationPresent(Id::class.java)) {
isIdFound = true
id = ReflectionUtils.getField(field, fieldValue)?.toString()
}
}
}
}