I am trying to use source.queue with cachedHostConnectionPool in java but not able to get the correct output. Below is my attempt at it. I am always getting success response, which i understand why but not sure what is the right way to do it.
Reason why i am getting success response is because i have defined Try as HttpResponse.create() , which is always giving me success response but my doubt is why it is not getting overwritten when i am doing queue.offer(Pair.create(httprequest,promise)).
Can someone please help me to understand what am i doing wrong here.
// Method to return queue
public SourceQueueWithComplete<Object> httpGetRequestWithPool
(Materializer materializer)
{
final Http http = Http.get(system);
final Flow<Pair<HttpRequest, Object>, Pair<Try<HttpResponse>, Object>, HostConnectionPool> flow;
flow = http.cachedHostConnectionPool(ConnectHttp.toHost("host:port"));
final SourceQueueWithComplete<Object> queue= Source.queue(3,OverflowStrategy.dropNew())
.map(d -> Pair.create(HttpRequest.create("contextpath"),d))
.via(flow)
.toMat(Sink.foreach(p -> p.first()), Keep.left())
.run(materializer);
return queue;
}
// Here i am trying to use.
SourceQueueWithComplete<Object> queue = util.httpGetRequestWithPool(materializer);
Source<Object, NotUsed> source = Source.from(ListOfObject);
source.mapAsyncUnordered(3,x-> {
String url = util.getServiceUrl("servicename").append(x.getId).toString();
HttpRequest httpRequest = HttpRequest.create(url);
Try<HttpResponse> promise = Try.apply(()->HttpResponse.create());
queue.offer(Pair.create(httpRequest,promise))
.thenCompose(result -> {
if(result instanceof QueueOfferResult.Enqueued$){
return CompletableFuture.completedFuture(promise)
.thenApply(res ->{
if(res.get().status().intValue()==200){
System.out.println("success");
}
return res;
});
}
else{
return CompletableFuture.completedFuture(HttpResponse.create().withStatus(StatusCodes.SERVICE_UNAVAILABLE));
}
});
return null;
}).run(materializer);
```
Related
I am using Vert.x in my project, I used future() to get the results from a MongoDB query. However when I do future().result it returns "null". I want the result to be saved in the future and I will use it for other APIs. Is there any guide for me, I will be very grateful and appreciate if someone give me a solution. Thanks
router.class
rivate void getClazzById(RoutingContext rc) {
Future<JsonObject> future = Future.future();
String clazzId = rc.request().getParam("clazzId");
classService.getClazzById(clazzId, res -> {
System.out.println(res.result());
if (res.succeeded()) {
JsonObject result = res.result();
if (result != null) {
future.complete(res.result());
rc.response().setStatusCode(200).putHeader("Content-Type", "application/json; charset=utf-8")
.end(result.encodePrettily());
} else {
rc.response().setStatusCode(400).putHeader("Content-Type", "application/json; charset=utf-8")
.end(new JsonObject().put("error", "Class not found!").encodePrettily());
}
} else
rc.fail(res.cause());
});
future.setHandler(s -> {
if (s.succeeded()) {
System.out.println("sss: " +s.result()); // print: {"_id":"123", "name":"abc"}
}
else {
System.out.println("fail");
}
});
System.out.println("hhhhhh: " + future.result()); // I want to print {"_id":"123", "name":"abc"}
}
service.class
public void getClazzById(String clazzId, Handler<AsyncResult<JsonObject>> resultHandler) {
JsonObject query = new JsonObject().put("_id", clazzId);
client.findOne(Collection.CLAZZ, query, null, ar -> {
if (ar.succeeded()) {
if (ar.succeeded()) {
JsonObject result = ar.result();
if (result == null) {
resultHandler.handle(Future.failedFuture("Error"));
} else {
resultHandler.handle(Future.succeededFuture(result));
}
} else {
ar.cause();
}
}
});
}
When writing asynchronous code, you are carried to use the framework / runtime semantics and tools for communication.
You are already leveraging one of Vert.x's way of async communication, the Future - but in the wrong manner trying to retrieve its result inline.
Instead of having the Future result accessed within your method, you need to return it as your mean of communication to a caller, which would be able to set a completion handler (Handler) to it:
private Future<JsonObject> getClazzById(RoutingContext rc) {
Future<JsonObject> future = Future.future();
String clazzId = rc.request().getParam("clazzId");
classService.getClazzById(clazzId, res -> {
if (res.succeeded()) {
JsonObject result = res.result();
if (result != null) {
future.complete(res.result()); // set the retrieved result
rc.response().setStatusCode(200).putHeader("Content-Type", "application/json; charset=utf-8")
.end(result.encodePrettily());
} else {
future.complete(null); // you need to provide 'null' result to avoid caller blocking
rc.response().setStatusCode(400).putHeader("Content-Type", "application/json; charset=utf-8")
.end(new JsonObject().put("error", "Class not found!").encodePrettily());
}
} else
rc.fail(res.cause());
});
return future; // return the future to caller
}
An interested caller would be able to set a handler for Future completion as needed:
getClassById(rc).setHandler(s -> {
if (s.succeeded()) {
System.out.println("sss: " +s.result()); // print: {"_id":"123", "name":"abc"}
}
else {
System.out.println("fail");
}
});
As a side note: you are not setting the API boundaries properly in your business logic as you are trying to resolve the HTTP Response result which generally denotes the request processing end while still returning the query result to be handled in some other manner.
I need to call certain API with multiple query params simultaneously, in order to do that I wanted to use reactive approach. I ended up with reactive client that is able to call endpoint based on passed SearchQuery, handle pagination of that response and call for remaining pages and returns Flux<Item>. So far it works fine, however what I need to do now is to:
Collect data for all search queries and save them as initial state
Once the initial data is collected, I need to start repeating those calls in small time intervals and validate each item against initial data. Basically, I need to find new items from here.
But I'm running out of options how to solve that, I came up with probably the dirties solution ever, but I bet there are much better ways to do that.
So first of all, this is relevant code of my client
public Flux<Item> collectData(final SearchQuery query) {
final var iteration = new int[]{0};
return invoke(query, 0).expand(res ->
this.handleResponse(res, query, iteration))
.flatMap(response -> Flux.fromIterable(response.collectItems()));
}
private Mono<ApiResponse> handleResponse(final ApiResponse response, final SearchQuery searchQuery, final int[] iteration) {
return hasNextPage(response) ? invoke(searchQuery, calculateOffset(++iteration[0])) : Mono.empty();
}
private Mono<ApiResponse> invoke(final SearchQuery query, final int offset) {
final var url = offset == 0 ? query.toUrlParams() : query.toUrlParamsWithOffset(offset);
return doInvoke(url).onErrorReturn(ApiResponse.emptyResponse());
}
private Mono<ApiResponse> doInvoke(final String endpoint) {
return webClient.get()
.uri(endpoint)
.retrieve()
.bodyToMono(ApiResponse.class);
}
And here is my service that is using this client
private final Map<String, Item> initialItems = new ConcurrentHashMap<>();
void work() {
final var executorService = Executors.newSingleThreadScheduledExecutor();
queryRepository.getSearchQueries().forEach(query -> {
reactiveClient.collectData(query).subscribe(item -> initialItems.put(item.getId(), item));
});
executorService.scheduleAtFixedRate(() -> {
if(isReady()) {
queryRepository.getSearchQueries().forEach(query -> {
reactiveClient.collectData(query).subscribe(this::process);
});
}
}, 0, 3, TimeUnit.SECONDS);
}
/**
* If after 2 second sleep size of initialItems remains the same,
* that most likely means that initial population phase is over,
* and we can proceed with further data processing
**/
private boolean isReady() {
try {
final var snapshotSize = initialItems.size();
Thread.sleep(2000);
return snapshotSize == initialItems.size();
} catch (Exception e) {
return false;
}
}
I think the code speaks for itself, I just want to finish first phase, which is initial data population and then start processing all incomming data.
I am trying to return a CompletableFuture that will return a response from Amazon. My code first checks to see if a response is cached locally. If so, it returns the response, otherwise it calls Amazon. [Note: The real version will also cache the response received from Amazon, but I haven't included that as the code is already pretty complicated.]
Is there a way to change the implementation of my callAmazon method (or otherwise reorganize the code) so that I don't have to "manually" copy the response state from amazonApi to the finalResponse?
I don't want to return cacheCheck directly because I don't want the caller to be able to complete() it.
public CompletableFuture<Response> fetchProductList() {
CompletableFuture<Response> finalResponse = new CompletableFuture<>();
CompletableFuture<Response> cacheCheck = //...
// First, see if we have a cached copy
cacheCheck.whenComplete((response, throwable) -> {
if (throwable == null) {
// Cache hit. Return the cached response
finalResponse.complete(response);
} else {
// Cache miss. Call Amazon
callAmazon(finalResponse);
}
});
return finalResponse;
}
private void callAmazon(CompletableFuture<Response> finalResponse) {
CompletableFuture<Response> amazonApi = //...
amazonApi.whenComplete((response, throwable) -> {
// Copy the state to the `finalResponse`
if (throwable == null) {
finalResponse.complete(response);
} else {
finalResponse.completeExceptionally(throwable);
}
});
}
What makes your requirement so complex is the fact that cacheCheck can throw an exception.
What I would do in your case is to refactor the cache to deliver either null if the value was not found in the cache, or the actual Response, if the value was in the cache.
Furthermore, I would modify callAmazon to return directly the CompletableFuture:
private CompletableFuture<Response> callAmazon() {
CompletableFuture<Response> amazonApi = //...
return amazonApi;
}
This way you can use thenCompose:
final CompletableFuture<Response> cacheCheck = //...
final CompletableFuture<Response> amazonResponse = callAmazon();
final CompletableFuture<Response> finalResult =
cachedResponse.thenCompose(cacheResult -> {
return cacheResult == null ? amazonResponse : CompletableFuture.completedFuture(cacheResult);
});
If you really need to throw an exception from the cache, you can use exceptionally to convert the exception to a null value, and then use thenCompose to decide if you use the cache value, or call Amazon:
final CompletableFuture<Response> finalResult = cachedResponse.exceptionally(e -> {
return null;
}).thenCompose(cacheResult -> {
return cacheResult == null ? amazonResponse : CompletableFuture.completedFuture(cacheResult);
});
I am using IBM's ILOG 7.1 via Java (aka JRules). I need to debug some pre-existing functionality, where there is a set of rules being executed. Only some of the rules that should be executed are actually in the output of my trace. So I'm left wondering:
Are the rules not actually getting executed?
Are the rules simply not getting outputted as part of the trace?
I am not sure.
The Java code that I have to execute and log the traced rules is as follows:
public RulesDTO executeRules(RulesDTO rulesDTO) {
try {
String sessionId = rulesDTO.getSessionId();
String clientIp = rulesDTO.getClientIp();
LOG.info("Starting to execute rules.");
String ruleSetName = rulesDTO.getRuleSetName();
IlrSessionRequest request = this.rulesSessionProvider
.createRequest(ruleSetName);
Map<String, Object> parNameToObjectMap = new HashMap<String, Object>();
parNameToObjectMap.put("params", rulesDTO);
parNameToObjectMap.put("timeOut",
getEngineExecutionTimeout());
request.setInputParameters(parNameToObjectMap);
boolean isTraceEnabled = true;
// Enable trace to retrieve info on executed rules
request.setTraceEnabled(isTraceEnabled);
// get all traces
request.getTraceFilter().setInfoAllFilters(isTraceEnabled);
IlrSessionResponse response = null;
try {
// calls ILOG engine, itself, to execute rules
response = executeRulesRequest(request, ruleSetName);
}
catch (TimeoutException te) {
LOG.error(String.format("Rules %s timed out [timeout=%d]",
ruleSetName, getEngineExecutionTimeout()));
throw te;
}
logRuleExecutions(rulesDTO.getID(), sessionId, response);
return rulesDTO;
}
catch (Throwable t) {
LOG.error("Rule set execution failed. ilogVersion=7.1", t);
throw new RulesException(
"Rule set execution failed. ilogVersion=7.1", t);
}
}
private void logRuleExecutions(
long searchId, String sessionId, IlrSessionResponse response) {
IlrExecutionTrace executionTrace = response.getRulesetExecutionTrace();
List<IlrExecutionEvent> executionEvents = executionTrace.getExecutionEvents();
// ExecutedRule is a custom class I have wrapping IlrExecutionEvent
List<ExecutedRule> executedRules = getRuleExecutions(executionEvents);
int numRulesExecuted = executedRules.size();
LOG.info("sessionId={}, SearchId={}, numRules={}",
new Object[] {sessionId, searchId, numRulesExecuted});
// loop over executedRules list and just log each one
logFilteredRules(executedRules, searchId, sessionId);
}
private List<ExecutedRule> getRuleExecutions(List<IlrExecutionEvent> executionEvents) {
List<ExecutedRule> executedRules = new LinkedList<ExecutedRule>();
for (IlrExecutionEvent event : executionEvents) {
if (event instanceof IlrRuleEvent) {
IlrRuleEvent ruleEvent = (IlrRuleEvent) event;
allRuleExecutions.add(convertToRuleExecution(ruleEvent));
}
else {
List<IlrExecutionEvent> subEvents = ((IlrTaskEvent) event).getSubExecutionEvents();
LOG.debug("Size of subEvents={}", subEvents.size());
List<ExecutedRule> executedSubRules = getRuleExecutions(subEvents);
executedRules.addAll(executedSubRules);
}
}
}
In addition to the fact only some of the rules appear to be getting invoked, it appears that only earlier rules are invoked, and not rules that ought to be executed after the initial rules. Any help would be greatly appreciated. Thanks. :-)
I'm attempting to perform a synchronous write/read in a demux-based client application with MINA 2.0 RC1, but it seems to get stuck. Here is my code:
public boolean login(final String username, final String password) {
// block inbound messages
session.getConfig().setUseReadOperation(true);
// send the login request
final LoginRequest loginRequest = new LoginRequest(username, password);
final WriteFuture writeFuture = session.write(loginRequest);
writeFuture.awaitUninterruptibly();
if (writeFuture.getException() != null) {
session.getConfig().setUseReadOperation(false);
return false;
}
// retrieve the login response
final ReadFuture readFuture = session.read();
readFuture.awaitUninterruptibly();
if (readFuture.getException() != null) {
session.getConfig().setUseReadOperation(false);
return false;
}
// stop blocking inbound messages
session.getConfig().setUseReadOperation(false);
// determine if the login info provided was valid
final LoginResponse loginResponse = (LoginResponse)readFuture.getMessage();
return loginResponse.getSuccess();
}
I can see on the server side that the LoginRequest object is retrieved, and a LoginResponse message is sent. On the client side, the DemuxingProtocolCodecFactory receives the response, but after throwing in some logging, I can see that the client gets stuck on the call to readFuture.awaitUninterruptibly().
I can't for the life of me figure out why it is stuck here based upon my own code. I properly set the read operation to true on the session config, meaning that messages should be blocked. However, it seems as if the message no longer exists by time I try to read response messages synchronously.
Any clues as to why this won't work for me?
The reason this wasn't working for me was because of an issue elsewhere in my code where I stupidly neglected to implement the message response encoder/decoder. Ugh. Anyway, the code in my question worked as soon as I fixed that.
I prefer this one (Christian Mueller : http://apache-mina.10907.n7.nabble.com/Mina-Client-which-sends-receives-messages-synchronous-td35672.html)
public class UCPClient {
private Map<Integer, BlockingQueue<UCPMessageResponse>> concurrentMap = new ConcurrentHashMap<Integer, BlockingQueue<UCPMessageResponse>>();
// some other code
public UCPMessageResponse send(UCPMessageRequest request) throws Throwable {
BlockingQueue<UCPMessageResponse> queue = new LinkedBlockingQueue<UCPMessageResponse>(1);
UCPMessageResponse res = null;
try {
if (sendSync) {
concurrentMap.put(Integer.valueOf(request.getTransactionReference()), queue);
}
WriteFuture writeFuture = session.write(request);
if (sendSync) {
boolean isSent = writeFuture.await(transactionTimeout, TimeUnit.MILLISECONDS);
if (!isSent) {
throw new TimeoutException("Could not sent the request in " + transactionTimeout + " milliseconds.");
}
if (writeFuture.getException() != null) {
throw writeFuture.getException();
}
res = queue.poll(transactionTimeout, TimeUnit.MILLISECONDS);
if (res == null) {
throw new TimeoutException("Could not receive the response in " + transactionTimeout + " milliseconds.");
}
}
} finally {
if (sendSync) {
concurrentMap.remove(Integer.valueOf(request.getTransactionReference()));
}
}
return res;
}
}
and the IoHandler:
public class InnerHandler implements IoHandler {
// some other code
public void messageReceived(IoSession session, Object message) throws Exception {
if (sendSync) {
UCPMessageResponse res = (UCPMessageResponse) message;
BlockingQueue<UCPMessageResponse> queue = concurrentMap.get(res.getTransactionReference());
queue.offer(res);
}
}
}
I had this exact problem. It turns out that it's because I was doing reads/writes in my IoHandler.sessionCreated() implementation. I moved the processing onto the thread that established the connection, instead of just waiting for the close future.
You must not use your login() function in IoHandler Thread :
If you call IoFuture.awaitUninterruptibly() in the override event function of IoHandler,
IoHandler don't work and get stuck.
You can call login() in other Thread and it will be work properly.