Develop Reference

Unable to insert data into db via an Observable

I have the following Observable where I am expecting some DB insertions to occur upon subscribing to it.
But nothing happens, no DB inserts and same time no errors either.
But If I directly subscribe to the method that does the DB calls, the DB insert occurs as expected.
How can I fix this such that the subscription to the Observable call below will perform the DB insert?
Please advice. Thanks.
This is the Observable where no DB insert occurs and no errors. I want to change this such that the DB insertion occurs when I subscribe to this Observable.
public Observable<KafkaConsumerRecord<String, RequestObj>> apply(KafkaConsumerRecords<String, RequestObj> records) {
Observable.from(records.getDelegate().records().records("TOPIC_NAME"))
.buffer(2)
.map(this::convertToEventRequest)
.doOnNext(this::handleEventInsertions)
.doOnSubscribe(() -> System.out.println("Subscribed!"))
.subscribe(); // purposely subscribing here itself to test
return null; // even if I return this observable and subscribe at the caller, same outcome.
}
Just to test if the query works, if I were to directly subscribe to the method that does the insertion, it works as expected as follows.
Doing this in debug mode.
client.rxQueryWithParams(query, new JsonArray(params)).subscribe() // works
The following are references to see whats happening inside the convertToEventRequest and handleEventInsertions methods
private Map<String, List<?>> convertToEventRequest(Object records) {
List<ConsumerRecord<String, RequestObj>> consumerRecords = (List<ConsumerRecord<String, RequestObj>>) records;
List<AddEventRequest> addEventRequests = new ArrayList<>();
List<UpdateEventRequest> updateEventRequests = new ArrayList<>();
consumerRecords.forEach(record -> {
String eventType = new String(record.headers().headers("type").iterator().next().value(), StandardCharsets.UTF_8);
if("add".equals(eventType)) {
AddEventRequest request = AddEventRequest.builder()
.count(Integer.parseInt(new String(record.headers().headers("count").iterator().next().value(), StandardCharsets.UTF_8)))
.data(record.value())
.build();
addEventRequests.add(request);
} else {
UpdateEventRequest request = UpdateEventRequest.builder()
.id(new String(record.headers().headers("id").iterator().next().value(), StandardCharsets.UTF_8))
.status(Integer.parseInt(new String(record.headers().headers("status").iterator().next().value(), StandardCharsets.UTF_8)))
.build();
updateEventRequests.add(request);
}
});
return new HashMap<String, List<?>>() {{
put("add", addEventRequests);
put("update", updateEventRequests);
}};
}
private void handleEventInsertions(Object eventObject) {
Map<String, List<?>> eventMap = (Map<String, List<?>>) eventObject;
List<AddEventRequest> addEventRequests = (List<AddEventRequest>) eventMap.get("add");
List<UpdateEventRequest> updateEventRequests = (List<UpdateEventRequest>) eventMap.get("update");
if(addEventRequests != null && !addEventRequests.isEmpty()) {
insertAddEvents(addEventRequests);
}
if(updateEventRequests != null && !updateEventRequests.isEmpty()) {
insertUpdateEvents(updateEventRequests);
}
}
private Single<ResultSet> insertAddEvents(List<AddEventRequest> requests) {
AddEventRequest request = requests.get(0);
List<Object> params = Arrays.asList(request.getCount(), request.getData());
String query = "INSERT INTO mytable(count, data, creat_ts) " +
"VALUES (?, ?, current_timestamp)";
return client.rxQueryWithParams(query, new JsonArray(params));
}
private Single<ResultSet> insertUpdateEvents(List<UpdateEventRequest> requests) {
UpdateEventRequest request = requests.get(0);
return client.rxQueryWithParams(
"UPDATE mytable SET status=?, creat_ts=current_timestamp WHERE id=?",
new JsonArray(Arrays.asList(request.getStatus(), request.getId())));
}

Can you try to wrap it into Observable.defer?
Observable.defer(() -> Observable.from(records.getDelegate().records().records("TOPIC_NAME"))...

Combine CompletionStage in play framework action in Java

I try to combine CompletionStages in play framework and then return a Result like ok(). This is my setup:
AccountDao which has two methods:
public CompletionStage<Account> getUserByEmail(String email) {
return supplyAsync(() -> ebeanServer.find(Account.class).setUseCache(true).where().eq(EMAIL, email).findOne(), executionContext).thenApply(account -> {
return account;
});
}
public CompletionStage<Void> updateAccount(Account account) throws OptimisticLockException{
return runAsync(() -> {
ebeanServer.update(account);
}, executionContext);
}
And then i have my controller with the action:
public CompletionStage<Result> editAccount() {
Map<String, String[]> form_values = request().body().asFormUrlEncoded();
return CompletableFuture.completedFuture(ok());
}
So now in the action i want first to execute getUserByEmail and then i want to set some values and Update this with updateAccount method. How can i combine this two stages without blocking play context? I tried different setups with thenCompose and combine but i dont get it ...
Here one of my tries:
public CompletionStage<Result> editAccount() {
Map<String, String[]> form_values = request().body().asFormUrlEncoded();
accountDao.getUserByEmail(session().get("accountEmail")).thenCompose(x -> accountDao.updateAccount(x).thenApplyAsync(account -> {
return ok("Going to save account edits");
}, httpExecutionContext.current()));
return CompletableFuture.completedFuture(ok("Fehler am Ende"));
}
The problem here is, that i cannot access the account (x) from before because i cannot set this as function ... like this:
public CompletionStage<Result> editAccount() {
Map<String, String[]> form_values = request().body().asFormUrlEncoded();
accountDao.getUserByEmail(session().get("accountEmail")).thenCompose(x -> {
//Update vars in x and then save to database
accountDao.updateAccount(x);
}.thenApplyAsync(account -> {
return ok("Going to save account edits");
}, httpExecutionContext.current()));
return CompletableFuture.completedFuture(ok("Fehler am Ende"));
}
Here i get the error: The target type of this expression must be a functional interface and plays says that i have to include the return statement at the end of the function!
I just dont get it ... Thanks for your help!
#Marimuthu Madasamy Thats no exactly what i want. In your awnser i would update the account twice. On etime in accountDao.updateAccount(account) and in accountDao.saveAccount(account); I want something like this:
return accountDao.getUserByEmail("mail").thenCompose(account -> {
account.setName("NewName");
accountDao.save(account);
} .thenApplyAsync(voidInput -> {
return ok("Account saved");
}, httpExecutionContext.current()));
In this case in only update the account once and only return the result on the httpExecutionContext

If I understand your question correctly, you want to access (to save?) account after updateAccount(account) method call.
Since updateAccount method returns CompletionStage<Void>, when you call thenApplyAsync on this stage, the input type would only be Void which is not Account. But with the following code, you would still have access to the account returned from getUserByEmail assuming updateAccount mutates the account by your text "update vars in x":
public CompletionStage<Result> editAccount() {
return accountDao
.getUserByEmail(email)
.thenCompose(account -> accountDao.updateAccount(account)
.thenApplyAsync(voidInput -> {
// here you still have access to the `account` from `getUserByEmail` method
accountDao.saveAccount(account);
return ok("Account saved");
}, httpExecutionContext.current());
}

Ok i found my own awnser here with the support of Marimuthu Madasamy! Thanks. I trie to explain it. First here is the code:
public CompletionStage<Result> editAccount() {
Map<String, String[]> form_values = request().body().asFormUrlEncoded();
return accountDao.getUserByEmail(session().get("accountEmail")).thenApply(account -> {
System.out.println("Async get Account / "+Thread.currentThread());
account.setCompany(form_values.get("company")[0]);
return accountDao.updateAccount(account);
}).thenApplyAsync(account -> {
System.out.println("Async resutl / "+Thread.currentThread());
return ok("Account saved normal");
}, httpExecutionContext.current()).exceptionally(e ->{
System.out.println("Async exception / "+Thread.currentThread());
System.out.println(e.getLocalizedMessage());
return ok(e.getLocalizedMessage());
});
}
Ok at first i execute accountDao.getUserByEmail() as you can see at top in my awnser this returns CompletionStage and is executed in my database execution context. At next with thenApply i get the result and i execute the next Async mehtod. I use thenApply instand of thenApplyAsync so the next call is also executed with the database execution context without setting it explicitly. After the accountDao.updateAccount() i execute the next stage on the httpExecutionContext to replay a result or to quit exceptionally! I really hope it is clear and helps some one!

Convert Observable of List to List of Observables and merge in RxJava

I'm learning Java with Android by creating Hacker News reader app.
What I'm trying to do is:
Send a request to /topstories, return Observable<List<int>>, emit when
request finishes.
Map each storyId to Observable<Story>
Merge Observables into one entity, which emits List<Story>, when all requests finishes.
And to the code:
private Observable<Story> getStoryById(int articleId) {
BehaviorSubject<Story> subject = BehaviorSubject.create();
// calls subject.onNext on success
JsonObjectRequest request = createStoryRequest(articleId, subject);
requestQueue.add(request);
return subject;
}
public Observable<ArrayList<Story>> getTopStories(int amount) {
Observable<ArrayList<Integer>> topStoryIds = (storyIdCache == null)
? fetchTopIds()
: Observable.just(storyIdCache);
return topStoryIds
.flatMap(id -> getStoryById(id))
// some magic here
}
Then we would use this like:
getTopStories(20)
.subscribe(stories -> ...)

You can try something like that
Observable<List<Integers>> ids = getIdsObservable();
Single<List<Story>> listSingle =
ids.flatMapIterable(ids -> ids)
.flatMap(id -> getStoryById(id)).toList();
Then you can subscribe to that Single to get the List<Story>

Please have a look at my solution. I changed your interface to return a Single for getStoryById(), because it should only return one value. After that, I created a for each Story a Single request and subscribed to all of them with Single.zip. Zip will execute given lambda, when all Singles are finished. On drawback is, that all requestes will be fired at once. If you do not want this, I will update my post. Please take into considerations that #elmorabea solution will also subscribe to the first 128 elements (BUFFER_SIZE = Math.max(1, Integer.getInteger("rx2.buffer-size", 128));), and to the next element when one finishes.
#Test
void name() {
Api api = mock(Api.class);
when(api.getTopStories()).thenReturn(Flowable.just(Arrays.asList(new Story(1), new Story(2))));
when(api.getStoryById(eq(1))).thenReturn(Single.just(new Story(888)));
when(api.getStoryById(eq(2))).thenReturn(Single.just(new Story(888)));
Flowable<List<Story>> listFlowable =
api.getTopStories()
.flatMapSingle(
stories -> {
List<Single<Story>> collect =
stories
.stream()
.map(story -> api.getStoryById(story.id))
.collect(Collectors.toList());
// possibly not the best idea to subscribe to all singles at the same time
Single<List<Story>> zip =
Single.zip(
collect,
objects -> {
return Arrays.stream(objects)
.map(o -> (Story) o)
.collect(Collectors.toList());
});
return zip;
});
TestSubscriber<List<Story>> listTestSubscriber =
listFlowable.test().assertComplete().assertValueCount(1).assertNoErrors();
List<List<Story>> values = listTestSubscriber.values();
List<Story> stories = values.get(0);
assertThat(stories.size()).isEqualTo(2);
assertThat(stories.get(0).id).isEqualTo(888);
assertThat(stories.get(1).id).isEqualTo(888);
}
interface Api {
Flowable<List<Story>> getTopStories();
Single<Story> getStoryById(int id);
}
static class Story {
private final int id;
Story(int id) {
this.id = id;
}
}

Prevent race condition with RxJava asynchronous thread implementation

I am using spring framework StringRedisTemplate to update an entry which happen with multiple threads.
public void processSubmission(final String key, final Map<String, String> submissionDTO) {
final String hashKey = String.valueOf(Hashing.MURMUR_HASH.hash(key));
this.stringRedisTemplate.expire(key, 60, TimeUnit.MINUTES);
final HashOperations<String, String, String> ops = this.stringRedisTemplate.opsForHash();
Map<String, String> data = findByKey(key);
String json;
if (data != null) {
data.putAll(submissionDTO);
json = convertSubmission(data);
} else {
json = convertSubmission(submissionDTO);
}
ops.put(key, hashKey, json);
}
In this json entry looks below,
key (assignmentId) -> value (submissionId, status)
As seen in code, before update the cache entry, I fetch current entry and add the new entry and put them all. But since this operation can be do in multiple threads, there can be situation of race condition leads to data lost. I could synchronize above method, but then it will be a bottle neck for the parallel processing power of RxJava implementation where processSubmission method is call via RxJava on two asynchronous threads.
class ProcessSubmission{
#Override
public Observable<Boolean> processSubmissionSet1(List<Submission> submissionList, HttpHeaders requestHeaders) {
return Observable.create(observer -> {
for (final Submission submission : submissionList) {
//Cache entry insert method invoke via this call
final Boolean status = processSubmissionExecutor.processSubmission(submission, requestHeaders);
observer.onNext(status);
}
observer.onCompleted();
});
}
#Override
public Observable<Boolean> processSubmissionSet2(List<Submission> submissionList, HttpHeaders requestHeaders) {
return Observable.create(observer -> {
for (final Submission submission : submissionList) {
//Cache entry insert method invoke via this call
final Boolean status = processSubmissionExecutor.processSubmission(submission, requestHeaders);
observer.onNext(status);
}
observer.onCompleted();
});
}
}
Above will call from below service API.
class MyService{
public void handleSubmissions(){
final Observable<Boolean> statusObser1 = processSubmission.processSubmissionSet1(subListDtos.get(0), requestHeaders)
.subscribeOn(Schedulers.newThread());
final Observable<Boolean> statusObser2 = processSubmission.processSubmissionSet2(subListDtos.get(1), requestHeaders)
.subscribeOn(Schedulers.newThread());
statusObser1.subscribe();
statusObser2.subscribe();
}
}
So handleSubmissions is calling with multiple threads per assignment id. But then per main thread is create and call two reactive java threads and process the submission list associate with each assignment.
What would be the best approach I could prevent redis entry race condition, while keep the performance of the RxJava implementation? Is there a way I could do this redis operation more efficient way?

It looks like you're only using the ops variable to do a put operation at the end, and you could isolate that point which is where you need to synchronize.
In the short research that I did, I couldn't find if HashOperations is not already thread-safe).
But an example of how you could just isolate the part you're concerned about is to do something like:
public void processSubmission(final String key, final Map<String, String> submissionDTO) {
final String hashKey = String.valueOf(Hashing.MURMUR_HASH.hash(key));
this.stringRedisTemplate.expire(key, 60, TimeUnit.MINUTES);
Map<String, String> data = findByKey(key);
String json;
if (data != null) {
data.putAll(submissionDTO);
json = convertSubmission(data);
} else {
json = convertSubmission(submissionDTO);
}
putThreadSafeValue(key, hashKey, json);
}
And have a method that is synchronized just for the put operation:
private synchronized void putThreadSafeValue(key, hashKey, json) {
final HashOperations<String, String, String> ops = this.stringRedisTemplate.opsForHash();
ops.put(key, hashKey, json);
}
There are a number of ways to do this, but it looks like you could restrict the thread contention down to that put operation.

How to query the Alfresco Audit Service in Java

I can easily query the Alfresco audit log in REST using this query:
http://localhost:8080/alfresco/service/api/audit/query/audit-custom?verbose=true
But how to perform the same request in Java within Alfresco module?
It must be synchronous.
A lazy solution would be to call the REST URL in Java, but it would probably be inefficient, and more importantly it would require me to store an admin's password somewhere.
I noticed AuditService has a auditQuery method so I am trying to call it. Unfortunately it seems to be for asynchronous operations? I don't need callbacks, as I need to wait until the queried data is ready before going on to the next step.

Here is my implementation, mostly copied from the source code of the REST API:
int maxResults = 10000;
if (!auditService.isAuditEnabled(AUDIT_APPLICATION, ("/" + AUDIT_APPLICATION))) {
throw new WebScriptException(
"Auditing for " + AUDIT_APPLICATION + " is disabled!");
}
final List<Map<String, Object>> entries =
new ArrayList<Map<String,Object>>(limit);
AuditQueryCallback callback = new AuditQueryCallback() {
#Override
public boolean valuesRequired() {
return true; // true = verbose
}
#Override
public boolean handleAuditEntryError(
Long entryId, String errorMsg, Throwable error) {
return true;
}
#Override
public boolean handleAuditEntry(
Long entryId,
String applicationName,
String user,
long time,
Map<String, Serializable> values) {
// Convert values to Strings
Map<String, String> valueStrings =
new HashMap<String, String>(values.size() * 2);
for (Map.Entry<String, Serializable> mapEntry : values.entrySet()) {
String key = mapEntry.getKey();
Serializable value = mapEntry.getValue();
try {
String valueString = DefaultTypeConverter.INSTANCE.convert(
String.class, value);
valueStrings.put(key, valueString);
}
catch (TypeConversionException e) {
// Use the toString()
valueStrings.put(key, value.toString());
}
}
entry.put(JSON_KEY_ENTRY_VALUES, valueStrings);
}
entries.add(entry);
return true;
}
};
AuditQueryParameters params = new AuditQueryParameters();
params.setApplicationName(AUDIT_APPLICATION);
params.setForward(true);
auditService.auditQuery(callback, params, maxResults);
Though the callback might it look asynchronous, it is not.