This code is returning only 100 results, even if there are tens of thousands of entries in the redis database:
RedisFuture<List<Object>> future = redis.dispatch(CommandType.SCAN, new ArrayOutput<String, String>(StringCodec.UTF8), new CommandArgs<String, String>(StringCodec.UTF8).add("aircraft"));
List<Object> result = Collections.emptyList();
try {
result = future.get();
} catch (InterruptedException | ExecutionException e) {
// TODO: Do something
}
if(result.size() > 1) {
#SuppressWarnings("unchecked")
List<List<String>> aircraft = (List<List<String>>) result.get(1);
Map<String, Aircraft> retval = aircraft.stream().collect(Collectors.toMap(it -> it.get(0), it -> {
try {
JsonNode positionJson = objectMapper.readTree(it.get(1)).get("coordinates");
return new Aircraft(positionJson.get(0).asDouble(), positionJson.get(1).asDouble());
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}));
return retval;
} else {
return Collections.emptyMap();
}
Related
I am using Mongo async driver 3.11.2 for the following code to get distinct values from a collection along with a filter query. The distinct method doesn't seem to return any values at all. I have tried after removing the filter but no luck. A similar code works with find.
public CompletableFuture<List<String>> findDistinctByCondition(JsonNode condition, String collectionName, String distinctField) {
MongoCollection<Document> collection = mongoDatabase.getCollection(collectionName);
List<String> tripNumberList = new ArrayList<>();
CompletableFuture<List<String>> finalResult = new CompletableFuture<>();
Block<String> printDocumentBlock = new Block<String>() {
#Override
public void apply(String tripNumber) {
tripNumberList.add(tripNumber);
}
};
SingleResultCallback<Void> callbackWhenFinished = new SingleResultCallback<Void>() {
#Override
public void onResult(final Void result, final Throwable t) {
logger.info("Operation Finished!");
finalResult.complete(tripNumberList);
}
};
DistinctIterable<String> distinct = collection.
distinct(distinctField, Document.parse(condition.toString()), String.class);
distinct.forEach(printDocumentBlock , callbackWhenFinished);
try {
logger.info("Printing distinct tripNumbers");
finalResult.get().forEach(s -> {
logger.info(s);
});
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
return finalResult;
}
What might be going wrong here and how to make it work?
I am currently facing the following issue with Java concurrency. I want to parse a friend list on a website, then search for the friends of the friends etc. recursively.
Here it is represented visually:
0
/|\
0 0 0
/|\
0 0 0
I came up with a solution but it does not perform as well as I expected, I assume my logic might be somewhat faulty.
private ArrayList<String> getUserFriendsProfileURLs(final String uri, final int indexOfDeep, int foldenesLevel)
throws IOException {
var usersURIs = getUsers(uri); //long network call
ArrayList<String> uris = new ArrayList<>();
uris.addAll(usersURIs);
if (indexOfDeep != foldenesLevel) {
List<CompletableFuture<ArrayList<String>>> futures = new ArrayList<>();
usersURIs.forEach(useruri -> {
CompletableFuture<ArrayList<String>> future = CompletableFuture.supplyAsync(new Supplier<>() {
public ArrayList<String> get() {
var friendsOfUser = new ArrayList<String>();
try {
friendsOfUser = getUserFriendsProfileURLs(useruri, indexOfDeep, 1 + foldenesLevel);
} catch (IOException e) {
throw new IllegalStateException(e);
}
return friendsOfUser;
}
});
futures.add(future);
});
CompletableFuture<Void> allFuturesResult = CompletableFuture
.allOf(futures.toArray(new CompletableFuture[futures.size()]));
var res = allFuturesResult.thenApply(v -> futures.stream().map(CompletableFuture::join).toList());
try {
uris = (ArrayList<String>) res.get().stream().flatMap(Collection::stream).collect(Collectors.toList());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
return uris;
}
I wrote 2nd version of function:
/**
* getUserFriendsProfileURLs returns list of friends of given steamid DFS
* algorithm is used
*
*/
private ArrayList<String> getUserFriendsProfileURLs(final int indexOfDeep, int foldenesLevel,
ArrayList<String> usersURIs) throws IOException {
ArrayList<String> uris = new ArrayList<>();
uris.addAll(usersURIs);
if (indexOfDeep != foldenesLevel) {
List<CompletableFuture<ArrayList<String>>> futures = new ArrayList<>();
usersURIs.forEach(useruri -> {
CompletableFuture<ArrayList<String>> future = CompletableFuture.supplyAsync(new Supplier<>() {
public ArrayList<String> get() {
ArrayList<String> ur = null;
try {
ur = getUsers(useruri); // long network call
} catch (IOException e1) {
e1.printStackTrace();
}
return ur;
}
});
futures.add(future);
});
CompletableFuture<Void> allFuturesResult = CompletableFuture
.allOf(futures.toArray(new CompletableFuture[futures.size()]));
var res = allFuturesResult.thenApply(v -> futures.stream().map(CompletableFuture::join).toList());
try {
uris = (ArrayList<String>) res.get().stream().flatMap(Collection::stream).collect(Collectors.toList());
uris = getUserFriendsProfileURLs(indexOfDeep, 1 + foldenesLevel, uris);
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
return uris;
}
I had two seperate taskLists and want to common / generic method to execute both using invokeAll() in ExecutorService . I am able to get the response and invokeAll() for each TaskList .
But unable to write a common to execute and get the result .
LocateUser Tasks :
List<LocateUser> taskList = new ArrayList<>();
taskList.add(new BSLocateUserClient(url, locateName, username, token));
List<Future<LocateUserResponse>> locateResponse = executor.invokeAll(taskList);
locateResponse.forEach(response -> {
LocateUserResponse user;
try {
user = response.get();
} catch (InterruptedException | ExecutionException e) {
//
} finally {
executor.shutdown();
}
});
LoginResponse Tasks :
List<LoginUser> taskList = new ArrayList<>();
for (String url : urls) {
taskList.add(new BSWebserviceClient(url, username, password, isOciLogin22, ociWebServiceTemplateFactory));
}
List<Future<LoginResponse>> response = executor.invokeAll(taskList);
List<LoginResponse> loginResponses = new ArrayList<>();
response.forEach(loginResponse -> {
try {
LoginResponse loginDetails = loginResponse.get();
} catch (InterruptedException | ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
executor.shutdown();
}
});
Can you help me to resolve this ?
I achieved this one - by adding one interface ResponseTask and implements this interface to both response class and create a generic / common method for invokeAll and getResponse .
public interface ResponseTask extends Serializable {
}
invokeAll :
public <T extends ResponseTask> List<T> invokeAll(Set<Callable<ResponseTask>> callables, int threadCount) {
ThreadFactory customThreadfactory = new CustomThreadFactoryBuilder().setNamePrefix("MultiThreadExecutor-Calls")
.setDaemon(false).build();
ExecutorService executor = Executors.newFixedThreadPool(threadCount, customThreadfactory);
try {
List<Future<ResponseTask>> threadResponse = executor.invokeAll(callables);
return getResponse(threadResponse);
} catch (InterruptedException e) {
// Restore interrupted state...
Thread.currentThread().interrupt();
} finally {
executor.shutdown();
}
return Collections.emptyList();
}
getResponse :
public <T extends ResponseTask> List<T> getResponse(List<Future<ResponseTask>> threadResponse) {
List<BSLocateUserResponse> locateResponse = new ArrayList<>();
List<LoginUser> userResponse = new ArrayList<>();
for (Future<ResponseTask> response : threadResponse) {
ResponseTask result = null;
try {
result = response.get();
} catch (ExecutionException e) {
// Todo Need to capture the specific exception to ignore here
LOG.info("Exception : {} occurred when calling multithread ", e.getMessage());
continue;
} catch (InterruptedException e) {
// Restore interrupted state...
Thread.currentThread().interrupt();
}
if (result instanceof BSLocateUserResponse) {
locateResponse.add((BSLocateUserResponse) result);
} else if (result instanceof LoginUser) {
userResponse.add((LoginUser) result);
}
}
if (locateResponse.isEmpty()) {
return (List<T>) userResponse;
}
return (List<T>) locateResponse;
}
I am using Playframework(scalaVersion 2.12.4) with Java 1.8 and Couchbase(Couchbase java-client 2.4.2). I am fetching UserDetails document and updating the same document in the database.
My code is working in normal scenarios, couldn't test the conflicting edge cases. Hence, not sure whether my code will work or not.
How to handle TemporaryLockFailureException? My current exception handling seems to be wrong.
#Override
public void submitUsersDetails(DashboardRequest request) {
logger.debug("inside submitDashboardDetails");
CompletableFuture.runAsync(() -> {
try {
bucket = cluster.openBucket(config.getString(PctConstants.COUCHBASE_BUCKET_NAME),
config.getString(PctConstants.COUCHBASE_BUCKET_PASSWORD));
String documentID = PctConstants.USER_PROFILE;
JsonDocument retrievedJsonDocument = bucket.getAndLock(documentID, 5);
if (retrievedJsonDocument != null) {
JsonNode json = Json.parse(retrievedJsonDocument.content().toString());
UserDetails userDetails = Json.fromJson(json, UserDetails.class);
// UPDATING userDetails
JsonObject jsonObject = JsonObject.fromJson(Json.toJson(userDetails).toString());
bucket.replace(JsonDocument.create(documentID, jsonObject, retrievedJsonDocument.cas()));
logger.info("Successfully entered record into couchbase");
} else {
logger.error("Fetching User_details unsuccessful");
throw new DataNotFoundException("User_details data not found");
}
} catch (TemporaryLockFailureException e) {
try {
logger.debug("Inside lock failure **************************");
Thread.sleep(5000);
String documentID = PctConstants.USER_PROFILE;
JsonDocument retrievedJsonDocument = bucket.getAndLock(documentID, 5);
if (retrievedJsonDocument != null) {
JsonNode json = Json.parse(retrievedJsonDocument.content().toString());
UserDetails userDetails = Json.fromJson(json, UserDetails.class);
// UPDATING userDetails
JsonObject jsonObject = JsonObject.fromJson(Json.toJson(userDetails).toString());
bucket.replace(JsonDocument.create(documentID, jsonObject, retrievedJsonDocument.cas()));
logger.info("Successfully entered record into couchbase");
submitDashboardDetails(request);
} else {
logger.error("Fetching User_details unsuccessful");
throw new DataNotFoundException("User_details data not found");
}
} catch (InterruptedException e1) {
e1.printStackTrace();
}
} catch (Exception e) {
e.printStackTrace();
try {
throw e;
} catch (Exception e1) {
e1.printStackTrace();
}
}
}, ec.current()).join();
}
I'm trying to create pull-based source of items. For some reason after pushing one item nothing arrives to the consumer. Whole pipeline is stuck. What am I missing ?
private static Flux<ByteBuffer> testRun(String path) {
return Flux.using(() -> {
FileInputStream in = new FileInputStream(path);
FileChannel channel = in.getChannel();
return Tuple.of(in, channel);
}, t -> s -> {
ByteBuffer bb = ByteBuffer.allocate(1000);
try {
if (t._2.read(bb) > 0)
s.onNext(bb.rewind());
else
s.onComplete();
} catch (IOException ex) {
s.onError(ex);
}
}, t -> {
try {
t._1.close();
t._2.close();
} catch (IOException ex) {
throw new RuntimeException(ex);
}
});
}
I got it. sourceSupplier of using is called only once.
private static Flux<ByteBuffer> testRun(String path) {
return Flux.using(() -> {
final FileInputStream in = new FileInputStream(path);
final FileChannel channel = in.getChannel();
return Tuple.of(in, channel);
}, t -> {
return Flux.generate(s -> {
ByteBuffer bb = ByteBuffer.allocate(1000);
try {
if (t._2.read(bb) > 0)
s.next(bb.rewind());
else
s.complete();
} catch (IOException ex) {
s.error(ex);
}
});
}, t -> {
try {
t._1.close();
t._2.close();
} catch (IOException ex) {
throw new RuntimeException(ex);
}
});
}