Merging several Flux into one from Mongo Collection - java

I am a bit new to spring reactive programming. I am currently developing an app using spring-boot-starter-webflux:2.0.0.M5.
I have a Foodtruck mongodb model which contains menus :
#Document(collection = "Foodtruck")
#lombok.Getter
#lombok.Setter
#lombok.NoArgsConstructor
#lombok.AllArgsConstructor
public class Foodtruck {
#Id
private String id;
#URL
private String siteUrl;
#NotBlank
private String name;
private String description;
#NotBlank
private String address;
private List<Menu> menus;
#JsonIgnore
private GridFS image;
}
And here is the model for Menu :
#lombok.Getter
#lombok.Setter
#lombok.NoArgsConstructor
#lombok.AllArgsConstructor
#Document(collection = "menus")
public class Menu {
#Id
private String id;
private List<DayOfWeek> days;
#NotBlank
private String label;
private String description;
private Double price;
private List<Dish> dishes;
#JsonIgnore
private GridFS image;
}
To get all my menus, I first need to get all my restaurants and then merge all the Flux obtained, and return them through my rest controller.
Here is the service I call from my rest resource :
#Service
public class MenuServiceImpl implements MenuService {
FoodtruckService foodtruckService;
public MenuServiceImpl(FoodtruckService foodtruckService) {
this.foodtruckService = foodtruckService;
}
#Override
public Flux<Menu> getAllMenus() {
Flux<Menu> allMenuFlux = Flux.empty();
Flux<Foodtruck> foodtruckFlux = foodtruckService.getAllFoodtrucks();
foodtruckFlux.toStream().forEach(foodtruck -> {
Flux<Menu> currentMenuFlux = Flux.fromIterable(foodtruck.getMenus());
allMenuFlux.mergeWith(currentMenuFlux);
});
return allMenuFlux;
}
}
The mergeWith does not seems to add anything to allMenuFlux. I think I have an understanding problem here.
I have been through the documentation, tested other methods like concat or zip, but it does not interleave flux events as I wish. I can't manage to merge these Flux correctly.
I also think there is a better way to get mongo embedded documents through a menu repository, since my method seems to be a bit overkill, and can cause performance problems in the long run. I have tried and it does not work.
EDIT:
After trying the following code (Being sure that my list is not empty), the resulting fluxtest3 variable is merged correctly :
Flux<Menu> allMenuFlux = Flux.empty();
Flux<Foodtruck> foodtruckFlux = foodtruckService.getAllFoodtrucks();
List<Foodtruck> test = foodtruckFlux.collectList().block();
Flux<Menu> fluxtest1 = Flux.fromIterable(test.get(0).getMenus());
Flux<Menu> fluxtest2 = Flux.fromIterable(test.get(1).getMenus());
Flux<Menu> fluxtest3 = fluxtest1.mergeWith(fluxtest2);
But that's not exactly what I want. Why isn't it working with an empty flux as parent flux.
What am I missing here ?
Thanks in advance for your help.

I think there are a couple of misunderstandings at work here.
If you ever convert a Flux to a Collection, Stream or similar and then something obtained from that back to a Flux you are most certainly doing something wrong. These classes force your pipeline to collect multiple elements, then process them and then convert them back to a Flux. In almost every case this should be possible just with operations offered by Flux.
The methods on Flux don't change the Flux but create new instances with additional behavior. So if you have code like this:
Flux<Menu> allMenuFlux = Flux.empty();
Flux<Foodtruck> foodtruckFlux = foodtruckService.getAllFoodtrucks();
foodtruckFlux.toStream().forEach(foodtruck -> {
Flux<Menu> currentMenuFlux = Flux.fromIterable(foodtruck.getMenus());
allMenuFlux.mergeWith(currentMenuFlux);
});
return allMenuFlux;
Everytime you call allMenuFlux.mergeWith(currentMenuFlux); you create a new Flux just so the Garbage Collector can take care of it. allMenuFlux is still the empty `Flux you started with.
What you really seem to want is:
return foodtruckService.getAllFoodtrucks()
.flatMap(Foodtruck::getMenus);
See the documentation for flatMap. The difference between flatMap and mergeWith is that mergeWith keeps the original elements. Which is superfluous if there are none as in your use-cases.
Bonus: Your additional question
Flux<Menu> fluxtest1 = Flux.fromIterable(test.get(0).getMenus());
Flux<Menu> fluxtest2 = Flux.fromIterable(test.get(1).getMenus());
Flux<Menu> fluxtest3 = fluxtest1.mergeWith(fluxtest2);
Here you are not returning the original fluxtest1 but the newly generated fluxtest2. Therefore it does work.

Related

Lombok's `#Builder` annotation stubbornly stays package - private

I have the following #Builder - annotated class:
#Data
#Builder(access = AccessLevel.PUBLIC)
#Entity
public class LiteProduct
{
// Minimal information required by our application.
#Id
private Long id; // Unique
private String name;
private Long costInCents;
private String type;
// Model the product types as a Hash Set in case we end up with several
// and need fast retrieval.
public final static Set<String> PRODUCT_TYPES = new HashSet<>
(Arrays.asList("flower", "topical", "vaporizer", "edible", "pet"));
// Have to allow creation of products without args for Entities.
public LiteProduct()
{
}
public LiteProduct(#NonNull final Long id, #NonNull final String name,
#NonNull final String type, #NonNull final Long costInCents)
{
if(!PRODUCT_TYPES.contains(type))
{
throw new InvalidProductTypeException(type);
}
this.name = name;
this.id = id;
this.costInCents = costInCents;
}
Whenever I want to use the builder class that Lombok is purported to give me, despite the fact that the IDE seems to detect it just fine:
I get a compile-time error about its visibility:
I have looked at some workarounds such as this or this, and they all seem to imply that my problem ought to already be solved automatically and that Lombok by default produces public Builder classes. This does not seem to be implied from my output, and does not happen even after I put the parameter access=AccessLevel.PUBLIC in my #Builder annotation in LiteProduct. Any ideas? Is there something wrong with the fact that this class is also an #Entity? Something else I'm not detecting?
// Edit: I determined that when I move the class in the same package as the one I am calling the builder pattern from, it works just fine. This is not an #Entity issue, but a package visibility issue which based on what I'm reading should not be there.
The problem was that I was using the following line of code to create an instance of LiteProduct:
return new LiteProduct.builder().build();
instead of:
return LiteProduct.builder().build();
which is what the #Builder annotation allows you to do. Clearly builder() is like a factory method for Builders that already calls new for you.

How to Map a Java Entity to Multiple MongoDB Collections in Spring Boot and use Different Indexing for both Collections?

Say I have a pojo class:
public class event {
String eventId;
String date;
String state;
}
And I want to save my event class in 2 separate collections in MongoDB.
I can use mongoTemplate for this:
mongoTemplate.save(event, "collection_1");
mongoTemplate.save(event, "collection_2");
But I run into problems because the collections need to have a different way of handeling the documents.
The first collection should store every event as a new
document. But the documents should expire after x seconds.
The second collection should only store the latest entry for a given
eventId.
Using annotations I can achieve 1. and 2. separately in the following way:
Criteria 1.
#Document
public class event {
String eventId;
#Indexed(expireAfterSeconds = x)
String date;
String state;
}
Criteria 2.
#Document
public class event {
#Id
String eventId;
String date;
String state;
}
I can't find a good way of achieving both criteria at the same time.
I know I could for example create 2 more classes that have the same fields as the event class but with different annotations and have a constructor that takes an event as an argument. But this really does not feel like a good solution with all the duplicate code.
Is there a more elegant solution for this problem?
So I finally found an awnser to my question. I'll post it here incase anyone else encounters the same problem.
The trick is to set the indexing through a configuration file instead of using annotations.
#Configuration
#DependsOn("mongoTemplate")
public class MongoCollectionConfiguration {
private final MongoTemplate mongoTemplate;
#Autowired
public MongoCollectionConfiguration(MongoTemplate mongoTemplate) {
this.mongoTemplate = mongoTemplate;
}
#PostConstruct
public void initIndexes() {
mongoTemplate.indexOps("collection_1")
.ensureIndex(
new Index().on("location.timestamp", Sort.Direction.DESC).expire(604800)
);
mongoTemplate.indexOps("collection_2")
.ensureIndex(
new Index().on("location.timestamp", Sort.Direction.DESC).unique()
);
}
I hope this helps someone in the future, and ofcourse I am open to any improvement!

Spring data mongodb query add boolean field if value exists in array

I'm trying to query some data from mongoDb which contains likes array. Each like object holds user.id of liker.
I need an extra boolean field that says if document is liked by user or not. So I need isLiked to be true if user has liked the document.
Here is what I have done till now:
I used ConditionalOperators.Cond to check if likes.userId is equal to userId of visitor.
#Override
public List<PostExtra> findPostsNearBy(double[] point, Distance distance, String thisUserId) {
mongoTemplate.indexOps(CheckInEntity.class).ensureIndex(new GeospatialIndex("position"));
ConditionalOperators.Cond conditionalOperators = new ConditionalOperators.ConditionalOperatorFactory(Criteria.where("likes.userId").is(thisUserId)).then(true).otherwise(false);
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.geoNear(
locationBasedOperationHelper.findNear(point,distance)
.query(new Query(privacyConsideredOperationHelper.privacyConsideredQuery(userRelationsEntity)))
,"distance"
),
//Aggregation.unwind("likes"),
Aggregation.project("user", "description").and(conditionalOperators).as("isLiked")
);
final AggregationResults<PostExtra> results =
mongoTemplate.aggregate(aggregation, PostEntity.class, PostExtra.class);
return results.getMappedResults();
}
if I remove the comment on Aggregation.unwind("likes") I can only get posts that this user has liked not those he hasn't.
I have seen the same matter here but I dont know whats the MontoTemplate code related to that?
Also I have seen approaches with setIsSubset, still I dont know java implementation.
I'm using spring boot 2.0.4.RELEASE and spring-boot-starter-data-mongodb.
#Document(collection = EntityCollectionNames.POST_COLLECTION_NAME)
public class PostEntity{
#Id
private String id;
#Field
#DBRef
#Indexed
private UserEntity user;
#GeoSpatialIndexed(type = GeoSpatialIndexType.GEO_2DSPHERE)
#Field(value = "position")
private Point position;
#Field(value = "description")
private String description;
#Field
private int likesCount;
#Field
private List<LikeEntity> likes;
}
Post Extra:
public class PostExtra extends PostEntity {
private double distance;
private boolean isLiked;
}
Like:
public class LikeEntity {
#Field
private String userId;
}
After some searching I found out to do it without using unwind I need to implement something like this to project isLiked field:
$cond:[
{$gt:[
{$size:
{$setIntersection:["$likes.userId",userIdsToCheckIn]}
},0]
},true,false]
I tried to do it in spring like this:
ConditionalOperators.Cond conditionalOperators = new ConditionalOperators.ConditionalOperatorFactory(
ComparisonOperators.Gt.valueOf(
ArrayOperators.Size.lengthOfArray(
SetOperators.SetIntersection.arrayAsSet("$likes.userId").intersects(????????)
)
).greaterThanValue(0)
).then(true).otherwise(false);
But I didnt know what to pass to intersects() method and apparently Spring does not let you pass List or Array as that input. Then I found out I can implement my own AggregationExpressions and override toDocument() method.
So I came up with this and it worked:
ConditionalOperators.Cond conditionalOperators = new ConditionalOperators.ConditionalOperatorFactory(
new AggregationExpression() {
#Override
public Document toDocument(AggregationOperationContext aggregationOperationContext) {
return Document.parse("{$gt:[{$size:{$setIntersection:[\"$likes.userId\",[\""+userId+"\"]]}},0]}");
}
}
).then(true).otherwise(false);
Ofcourse I could add $cond to the implementation directly too.
I would still like to know if there are any spring solutions that does not contain mongodb queries directly for this.

How to use db references with reactive Spring Data MongoDB?

I'm new to MongoDB and Reactor and I'm trying to retrieve a User with its Profiles associated
Here's the POJO :
public class User {
private #Id String id;
private String login;
private String hashPassword;
#Field("profiles") private List<String> profileObjectIds;
#Transient private List<Profile> profiles; }
public class Profile {
private #Id String id;
private #Indexed(unique = true) String name;
private List<String> roles; }
The problem is, how do I inject the profiles in the User POJO ?
I'm aware I can put a #DBRef and solve the problem but in it's documentation, MongoDB specify manual Ref should be preferred over DB ref.
I'm seeing two solutions :
Fill the pojo when I get it :
public Mono<User> getUser(String login) {
return userRepository.findByLogin(login)
.flatMap(user -> ??? );
}
I should do something with profileRepository.findAllById() but I don't know or to concatene both Publishers given that profiles result depends on user result.
Declare an AbstractMongoEventListener and override onAfterConvert method :
But here I am mistaken since the method end before the result is Published
public void onAfterConvert(AfterConvertEvent<User> event) {
final User source = event.getSource();
source.setProfiles(new ArrayList<>());
profileRepository.findAllById(source.getProfileObjectIds())
.doOnNext(e -> source.getProfiles().add(e))
subscribe();
}
TL;DR
There's no DBRef support in reactive Spring Data MongoDB and I'm not sure there will be.
Explanation
Spring Data projects are organized into Template API, Converter and Mapping Metadata components. The imperative (blocking) implementation of the Template API uses an imperative approach to fetch Documents and convert these into domain objects. MappingMongoConverter in particular handles all the conversion and DBRef resolution. This API works in a synchronous/imperative API and is used for both Template API implementations (imperative and the reactive one).
Reuse of MappingMongoConverter was the logical decision while adding reactive support as we don't have a need to duplicate code. The only limitation is DBRef resolution that does not fit the reactive execution model.
To support reactive DBRefs, the converter needs to be split up into several bits and the whole association handling requires an overhaul.
Reference : https://jira.spring.io/browse/DATAMONGO-2146
Recommendation
Keep references as keys/Id's in your domain model and look up these as needed. zipWith and flatMap are the appropriate operators, depending on what you want to archive (enhance model with references, lookup references only).
On a related note: Reactive Spring Data MongoDB comes partially with a reduced feature set. Contextual SpEL extension is a feature that is not supported as these components assume an imperative programming model and thus synchronous execution.
For the first point, I finally achieve doing what I wanted :
public Mono<User> getUser(String login) {
return userRepository.findByLogin(login)
.flatMap( user ->
Mono.just(user)
.zipWith(profileRepository.findAllById(user.getProfileObjectIds())
.collectionList(),
(u, p) -> {
u.setProfiles(p);
return u;
})
);
}
In my case, I have managed this problem using the follow approuch:
My Entity is:
#Getter
#Setter
#NoArgsConstructor
#AllArgsConstructor
#Document(collection = "post")
public class Post implements Serializable {
private static final long serialVersionUID = -6281811500337260230L;
#EqualsAndHashCode.Include
#Id
private String id;
private Date date;
private String title;
private String body;
private AuthorDto author;
private Comment comment;
private List<Comment> listComments = new ArrayList<>();
private List<String> idComments = new ArrayList<>();
}
My controller is:
#GetMapping(FIND_POST_BY_ID_SHOW_COMMENTS)
#ResponseStatus(OK)
public Mono<Post> findPostByIdShowComments(#PathVariable String id) {
return postService.findPostByIdShowComments(id);
}
Last, but not, least, my Service (here is the solution):
public Mono<Post> findPostByIdShowComments(String id) {
return postRepo
.findById(id)
.switchIfEmpty(postNotFoundException())
.flatMap(postFound -> commentService
.findCommentsByPostId(postFound.getId())
.collectList()
.flatMap(comments -> {
postFound.setListComments(comments);
return Mono.just(postFound);
})
);
}
public Flux<Comment> findCommentsByPostId(String id) {
return postRepo
.findById(id)
.switchIfEmpty(postNotFoundException())
.thenMany(commentRepo.findAll())
.filter(comment1 -> comment1.getIdPost()
.equals(id));
}
Thanks, this helped a lot.
Here is my solution:
public MappingMongoConverter mappingMongoConverter(MongoMappingContext mongoMappingContext) {
MappingMongoConverter converter = new MappingMongoConverter(NoOpDbRefResolver.INSTANCE, mongoMappingContext);
converter.setTypeMapper(new DefaultMongoTypeMapper(null));
converter.setCustomConversions(mongoCustomConversions());
return converter;
}
The trick was to use the NoOpDbRefResolver.INSTANCE

spring data elasticsearch dynamic multi tenant index mismatch?

I am experimenting with spring data elasticsearch by implementing a cluster which will host multi-tenant indexes, one index per tenant.
I am able to create and set settings dynamically for each needed index, like
public class SpringDataES {
#Autowired
private ElasticsearchTemplate es;
#Autowired
private TenantIndexNamingService tenantIndexNamingService;
private void createIndex(String indexName) {
Settings indexSettings = Settings.builder()
.put("number_of_shards", 1)
.build();
CreateIndexRequest indexRequest = new CreateIndexRequest(indexName, indexSettings);
es.getClient().admin().indices().create(indexRequest).actionGet();
es.refresh(indexName);
}
private void preapareIndex(String indexName){
if (!es.indexExists(indexName)) {
createIndex(indexName);
}
updateMappings(indexName);
}
The model is created like this
#Document(indexName = "#{tenantIndexNamingService.getIndexName()}", type = "movies")
public class Movie {
#Id
#JsonIgnore
private String id;
private String movieTitle;
#CompletionField(maxInputLength = 100)
private Completion movieTitleSuggest;
private String director;
private Date releaseDate;
where the index name is passed dynamically via the SpEl
#{tenantIndexNamingService.getIndexName()}
that is served by
#Service
public class TenantIndexNamingService {
private static final String INDEX_PREFIX = "test_index_";
private String indexName = INDEX_PREFIX;
public TenantIndexNamingService() {
}
public String getIndexName() {
return indexName;
}
public void setIndexName(int tenantId) {
this.indexName = INDEX_PREFIX + tenantId;
}
public void setIndexName(String indexName) {
this.indexName = indexName;
}
}
So, whenever I have to execute a CRUD action, first I am pointing to the right index and then to execute the desired action
tenantIndexNamingService.setIndexName(tenantId);
movieService.save(new Movie("Dead Poets Society", getCompletion("Dead Poets Society"), "Peter Weir", new Date()));
My assumption is that the following dynamically index assignment, will not work correctly in a multi-threaded web application:
#Document(indexName = "#{tenantIndexNamingService.getIndexName()}"
This is because TenantIndexNamingService is singleton.
So my question is how achieve the right behavior in a thread save manner?
I would probably go with an approach similar to the following one proposed for Cassandra:
https://dzone.com/articles/multi-tenant-cassandra-cluster-with-spring-data-ca
You can have a look at the related GitHub repository here:
https://github.com/gitaroktato/spring-boot-cassandra-multitenant-example
Now, since Elastic has differences in how you define a Document, you should mainly focus in defining a request-scoped bean that will encapsulate your tenant-id and bind it to your incoming requests.
Here is my solution. I create a RequestScope bean to hold the indexes per HttpRequest
how does singleton bean handle dynamic index

Categories

Resources