I have been working on some methods which perform parallel operations on an LDAP server. My problem is that the method code is basically duplicate except for one line in all methods and I have no idea on how to refactor the methods such that the code would remain relatively readable. There are multiple duplicate methods; I will show two as an example.
Example of the methods:
public static void addEntriesInParallel(LdapConnParams ldapConnParams, List<LDAPEntry> entries) {
ExecutorService executor = Executors.newCachedThreadPool();
CountDownLatch latch = new CountDownLatch(batchCount(entries.size()));
LdapConnParamsBuilder connParamsBuilder = new LdapConnParamsBuilder(ldapConnParams);
for (List<LDAPEntry> entryBatch : Lists.partition(entries, BATCH_SIZE)) {
LdapConnParams threadConnParams = connParamsBuilder.build();
executor.submit(() -> {
try {
LdapTransactions.addEntries(threadConnParams, entryBatch);
latch.countDown();
} catch (LDAPException ex) {
// Exception handling
}
});
}
try {
latch.await();
} catch (InterruptedException ex) {
// Exception handling
}
}
public static void deleteAttributeInParallel(LdapConnParams ldapConnParams, String attribute, List<LDAPEntry> entries) {
ExecutorService executor = Executors.newCachedThreadPool();
CountDownLatch latch = new CountDownLatch(batchCount(entries.size()));
LdapConnParamsBuilder connParamsBuilder = new LdapConnParamsBuilder(ldapConnParams);
for (List<LDAPEntry> entryBatch : Lists.partition(entries, BATCH_SIZE)) {
LdapConnParams threadConnParams = connParamsBuilder.build();
executor.submit(() -> {
try {
LdapTransactions.deleteAttribute(threadConnParams, attribute, entryBatch);
latch.countDown();
} catch (LDAPException ex) {
// Exception handling
}
});
}
try {
latch.await();
} catch (InterruptedException ex) {
// Exception handling
}
}
The only point of difference is the line (first method):
LdapTransactions.addEntries(threadConnParams, entryBatch);
Compared to (second method):
LdapTransactions.deleteAttribute(threadConnParams, attribute, entryBatch);
Now, the problem also is that the methods called within don't match in signatures.
Is there a way to rewrite the code in such a manner that I don't have to keep the same method multiple times with a single line difference?
Thanks for any tips!
You already did most of the work by figuring out what the methods have in common, and what differs.
Now, you only need to extract the common parts into a more general method for executing LDAP jobs, while trying to keep enough flexibility for differences to be supported.
Assuming all functions that you call inside the processing loop take a threadConnParams and entryBatch, and the rest is function-dependent, you could start with this interface:
public static void performInParallel(
LdapConnParams ldapConnParams,
List<LDAPEntry> entries,
LDapAction action)
LDapAction is a functional interface that is meant to be implemented by a lambda (or a concrete class if that's useful) and looks like this:
#FunctionalInterface
interface LDapAction {
void apply(LdapConnParams threadConnParams,
List<LDAPEntry> entryBatch) throws LDAPException;
}
The implementation changes slightly from what you have now, with the main call becoming:
executor.submit(() -> {
try {
action.accept(threadConnParams, entryBatch);
latch.countDown();
} catch (LDAPException ex) {
// Exception handling
}
});
It could look something like this from the API consumer point-of-view:
addEntries:
performInParallel(ldapConnParams, entries,
(threadConnParams, entryBatch) ->
LdapTransactions.addEntries(threadConnParams, entryBatch));
deleteAttribute:
performInParallel(ldapConnParams, entries,
(threadConnParams, entryBatch) ->
LdapTransactions.deleteAttribute(threadConnParams, attribute, entryBatch));
If the performInParallel function requires more input, you can add that to the action interface, but from the code you've provided so far that doesn't seem to be necessary.
Related
Im with java 11, Let's say I have multiple runnable methods all them are same structure except the number of parameters as in example:
private Runnable deleteIndex(String targetIndex) {
return () -> {
try {
iIndexerManagementService.deleteIndex(targetIndex);
} catch (Exception e) {
log.error("Cannot perform delete:", e);
}
};
}
private Runnable deleteDataBySource(String targetIndex, String source) {
return () -> {
try {
indexerDocumentService.deleteBySource(targetIndex, source);
} catch (Exception e) {
log.error("Cannot perform delete:", e);
}
};
}
private Runnable createDeleteTaskByScanID(String targetIndex, String scandId, String source) {
return () -> {
try {
indexerDocumentService.deleteByScanId(targetIndex, scandId, source);
} catch (Exception e) {
log.error("Cannot perform delete:", e);
}
};
} ....
and more ....
Later on I use it like here on each runnable method:
public void submitDeleteIndexTask(String targetIndex) {
Optional.ofNullable(syncControlRepository.findByName(CATALOG_ENTITY_TYPE_KEYWORD))
.ifPresent(strategy -> {
Future<?> future = uploadersExecutor.submit(deleteIndex(targetIndex));
if (featureFlagEnabled && config.isTestProfile()) {
try {
future.get(1, TimeUnit.MINUTES);
} catch (Exception e) {
log.error("Cannot wait on thread block:", e);
}
}
});
}
.... and more
I works perfect.
But I want to find nicer elegant way not to hold lots of runnable methods in class cause it might be more.
My purpose create 1 generic runnable wrapper to be able pass any kind of function so when I'll use it i will pass only the function itself and the parameters to invoke.
I thought about this way, bit i'm not sure it is correct
// generic runnable wrapper to pass any function
public Runnable runAsync(Function foo, Object o) {
return () -> {
try {
foo.apply(o);
} catch (Exception e) {
log.error("Cannot perform delete:", e);
}
};
}
and then it should be 1 general method to be able receive function to fit the runnables methods:
public void runGeneralAsync(... HERE PASS FUNCTION ..??. ) {
Optional.ofNullable(syncControlRepository.findByName(CATALOG_ENTITY_TYPE_KEYWORD))
.ifPresent(strategy -> {
Future<?> future = uploadersExecutor.submit(runAsync(....FUNCTOIN... ?? ));
if (featureFlagEnabled && config.isTestProfile()) {
try {
future.get(1, TimeUnit.MINUTES);
} catch (Exception e) {
log.error("Cannot wait on thread block:", e);
}
}
});
}
Please advice
What you're asking for is tantamount to 'stringly typed' / dynamic, and is fundamentally unjavalike. You may think you want this; you don't. The entire java ecosystem will be fighting you every step of the way if you try.
NB: Your runGeneralAsync is deplorably bad - you're really abusing Optional here. That code is a long-winded way of saying: Look up a thingie, if it does exist, do this, if it does not, silently do nothing. Aside from the long-windedness (there's nothing wrong with if!), "Do nothing" is a horrible fallback. Horrible in the sense of: A single bug in the name or setup means nothing happens, so, you'll be chasing that bug for hours. An NPE takes 1 minute. If you ever write code where you can't fathom null occuring, just write it. If null does happen you want that exception, so you know where to fix things.
At any rate:
Instead, Runnable, itself, is the answer. Wrap lambdas in lambdas.
You're thinking of something like:
// API:
public void runGeneralAsync(String methodName, Object... args) {
// Lots of reflective shenanigans to find it.
// Lots of issues with dealing with exceptions -
// what if the method name does not exist? What if the wrong amount
// of args were passed? Or the wrong type? Or there is an overload?
// .. and then of course code to stick it all into an executor.submit
}
// usage:
runGeneralAsync("deleteIndexTask", "my-index");
but what you want to do is something like this:
public void runGeneralAsync(Runnable task) {
uploadersExecutor.submit(runAsync(....));
}
// usage:
runGeneralAsync(() -> deleteIndex("my-index"));
Depending on needs, you may want to use Callable instead of Runnable here (you can return things, and throw things).
Don't really know if the title fits the problem that I'm trying to solve but here it goes (feel free to modify it for something that makes more sense).
Actually I'm appending String related operations within a Consumer, so that when requested the value is transformed within the chain. However, I see the same exception being thrown in multiple methods and I would like to compress that exact same try/catch clause in an upper level, so that methods only contain the new Consumer definition to be appended.
public static class Clazz {
private Consumer<String> modifications;
public Clazz() {
// ...
}
public void addOp() {
this.modifications = modifications.andThen(item -> {
try {
item.toUpperCase();
} catch (Exception e) {
// logging and exception handling - Merge in common lambda?
}
});
}
public void anotherOp() {
this.modifications = modifications.andThen(item -> {
try {
item.toLowerCase();
} catch (Exception e) {
// logging and exception handling -- Merge in common lambda?
}
});
}
}
Would there be any option to compress all those exception clauses in an upper class? Which is the approach when simplifying lambda try/catch clauses?
Consider we have a scheduled executor service:
ScheduledExecutorService threadPool = Executors.newScheduledThreadPool(...);
And for some logic we want to retry a task execution. The following approach seems to be smelling for me, but I can't understand why:
threadPool.submit(new Runnable() {
#Override
public void run() {
// ...
if (needToBeScheduled()) {
threadPool.schedule(this, delay, TimeUnit.MINUTES);
} else if (needToBeResubmitted()) {
threadPool.submit(this);
}
}
});
The one obvious problem I see is that this code is not possible to convert to lambda:
threadPool.submit(()-> {
// ...
if (needToBeScheduled()) {
threadPool.schedule(this, delay, TimeUnit.MINUTES);
} else if (needToBeResubmitted()) {
threadPool.submit(this);
}
});
^^ this won't compile, as we can not refer this from lambda. Though it can be solved by introducing a method which produces such an instance and provide it instead of this.
But this is only one disadvantage I see. Is anything else here which can cause any problems? Perhaps there is a more proper approach? Move this logic to ThreadPoolExecutor.afterExecute() (this causes type conversion though...)?
Assuming that object is stateless, i.e. there are no object variables in Runnable instance.
P.S. The logic of what to do (reschedule task or resubmit or do nothing) is based on some information retrieved from the database (or any external source). So Runnable is still stateless, but it calculates the outcome based on some results of its work.
Honestly, I don't like the approach where a task (a simple independent unit of work) decides whether it should put itself in the service or not and interacts with the ExecutorService directly. I believe // ... is the only part a task should execute.
I would convert a Runnable in a Callable<Boolean>:
Callable<Boolean> task = () -> {
// ...
return needToBeScheduled; // or sth more complex with several boolean fields
};
And I would definitely move that logic outside a task (for example, into a service method):
Future<Boolean> future = threadPool.submit(task);
try {
boolean needToBeScheduled = future.get();
if (needToBeScheduled) {
threadPool.schedule(task, delay, TimeUnit.MINUTES);
}
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
By something more complex I meant a class that comprises 2 boolean fields. It takes Supplier<Boolean>s to make things lazy.
final class TaskResult {
private final Supplier<Boolean> needToBeScheduled;
private final Supplier<Boolean> needToBeResubmitted;
private TaskResult(Supplier<Boolean> needToBeScheduled, Supplier<Boolean> needToBeResubmitted) {
this.needToBeScheduled = needToBeScheduled;
this.needToBeResubmitted = needToBeResubmitted;
}
public static TaskResult of(Supplier<Boolean> needToBeScheduled, Supplier<Boolean> needToBeResubmitted) {
return new TaskResult(needToBeScheduled, needToBeResubmitted);
}
public boolean needToBeScheduled() {
return needToBeScheduled != null && needToBeScheduled.get();
}
public boolean needToBeResubmitted() {
return needToBeResubmitted != null && needToBeResubmitted.get();
}
}
With a few changes to the above example, we have:
Callable<TaskResult> task = () -> {
// ...
return TaskResult.of(() -> needToBeScheduled(), () -> needToBeResubmitted());
};
final Future<TaskResult> future = threadPool.submit(task);
try {
final TaskResult result = future.get();
if (result.needToBeScheduled()) {
threadPool.schedule(task, delay, TimeUnit.MINUTES);
}
if (result.needToBeResubmitted()) {
threadPool.submit(task);
}
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
So I have 20 setters in a row all of which could potentially fail. Rather than skip them all if one fails or surround them each with a try catch is there a way to do this with some of Java 8s features?
For example, I was thinking something like this:
public void mapElement(Function function) {
try the function
catch if something goes wrong
}
Then I could use this like so:
mapElement(myObject.putA(a));
mapElement(myObject.putB(b));
mapElement(myObject.putC(c));
....
How about something like this:
public void tryTo(Runnable action) {
try {
action.run();
} catch (Exception e) {
// do something?
}
}
tryTo(() -> myObject.putA(a));
tryTo(() -> myObject.putB(b));
tryTo(() -> myObject.putC(c));
Note that myObject, a, b and c all need to be effectively final for this to work.
A spin on the above is to have a single method that takes an array of Runnables and executes them in a loop:
public void tryAll(Runnable... actions) {
for (Runnable action : actions) {
try {
action.run();
} catch (Exception e) {
// do something?
}
}
}
tryAll(() -> myObject.putA(a),
() -> myObject.putB(b),
() -> myObject.putC(c));
You could use the Runnable functional interface.
Its function descriptor is () -> void.
It suits perfectly for your need as the mapping operation returns no result and you don't need to specify any parameter as input of the function either.
Indeed here : myObject.putA(a), you don't want to pass the a parameter to the function.
You want rather pass the whole expression myObject.putA(a) as a lambda body :
() -> myObject.putA(a);
You could so write this mapElement() method :
public static void mapElement(Runnable mapProcessor) {
// Retry logic
try {
mapProcessor.run();
} catch (Exception e) {
// exception handling
}
}
Note that to catch any Java exception (both checked and runtime), you have to catch RuntimeException rather than Exception.
And you can use mapElement() in this way :
MyObject myObject = new MyObject();
...
mapElement(() -> myObject.putA(a));
mapElement(() -> myObject.putB(b));
mapElement(() -> myObject.putC(c));
Runnable may not convey the expected meaning as it was primarily
designed for thread execution.
If it makes sense you also may introduce your own functional interface.
With this way, you may also declare any specific checked exception in the function if it appears relevant.
Which is not possible with Runnable.
For example :
#FunctionalInterface
public interface MapProcessor{
void map() throws MappingException;
}
You could so use it in this way :
public static void mapElement(MapProcessor mapProcessor) {
// Retry logic
try {
mapProcessor.map();
}
catch (MappingException e) {
// Mapping exception handling
}
catch (Exception e) { // or RuntimeException if more relevant
// other exception handling
}
}
I'm writing a converter program using JavaFX, and am using the recommended javafx.concurrent.Task to do the heavy work away from the JavaFX Application Thread. I'm also using java.util.concurrent.Future and java.util.concurrent.ExecutorService within it to do extra work that can be done concurrently.
However a lot of the work involves methods that can potentially throw, and I need the process to stop in its tracks if that occurs. I'm currently spamming try-catch blocks everywhere and returning false rather than letting the exceptions bubble up.
But as call() in both Future and Task have the throws Exception declaration in case there are any uncaught ones, is it ok for me to just not catch the exceptions in the Futures and let them be handled by the Application Thread when the Task terminates?
As the exceptions will do what i want and terminate the thread, while providing extra information as to why the thread stopped to the application thread so i can show an appropriate alert.
The other reason I want to do this is within the Futures I need access to values from the Task thread, but i don't need to alter them so I'd like to make values final and create the Futures in lambdas. The try-catch blocks are complicating matters as i can't make as many values final (or effectively final) as I'd like. This is because I'm assigning the return values from methods that could potentially throw when i initialise them. As a result I have to surround the assignment in a try-catch, thus need to keep creating temp variables before copying them to a final variable, which makes the process look a mess and likely wastes memory.
Is not catching exceptions inside a Task or Future a good idea or not? Are there any major pitfalls or issues with not catching exceptions until the Application Thread?
Here's an example of what i currently have in my controller class to deal with the event that should trigger this process:
ExecutorService converterExecutor = Executors.newSingleThreadExecutor();
ConverterTask converterThread = new ConverterTask()
converterExecutor.execute(converterThread);
Boolean success = null;
try
{
success = converterThread.get();
}
catch (InterruptedException | ExecutionException e1)
{
e1.printStackTrace();
return false;
}
if (success == null)
{
return false;
}
And the ConverterTask class containing the long-running logic to me run in another thread
public class ConverterTask extends Task< Boolean >
{
public Boolean call() throws Exception
{
//do stuff
ExecutorService executor = Executors.newSingleThreadExecutor();
Future< String > skeletonThread = executor.submit(new Callable< String >()
{
//stuff that can throw exceptions
});
//do more stuff
String temp_sklData = null;
try
{
temp_sklData = skeletonThread.get();
}
catch (InterruptedException | ExecutionException e1)
{
e1.printStackTrace();
return false;
}
if (temp_sklData == null)
{
return false;
}
final String sklData = temp_sklData;
//do more stuff including use sklData in a Lambda Future
}
}
and what I'd like to do in converterTask instead if my propagating the exceptions is a good idea
public class converterTask extends Task< Boolean >
{
public Boolean call() throws Exception
{
//do stuff
ExecutorService executor = Executors.newSingleThreadExecutor();
Future< String > skeletonThread = executor.submit(new Callable< String >()
{
//stuff that can throw exceptions
});
//do more stuff
final String sklData = skeletonThread.get();
//do more stuff including use sklData in a Lambda Future
}
}
I don't really understand the structure of your code, for reasons pointed out by others in the comments. But the basic answer to your question is that you should use try-catch if you can handle the exceptions gracefully (this typically means you can still return a meaningful result), and let them propagate if not.
So a ConverterTask that converted one String into another, which may potentially throw exceptions which prevent the conversion from taking place, would look like
public class ConverterTask extends Task<String> {
private final String textToConvert ;
public ConverterTask(String textToConvert) {
this.textToConvert = textToConvert ;
}
#Override
public String call() throws Exception {
String result = doConversion(textToConvert); // may throw exception...
return result ;
}
}
and a typical usage would be
Executor conversionExec = ... ;
// ...
ConverterTask converterTask = new ConverterTask(someText);
converterTask.setOnSucceeded(e -> {
String result = converterTask.getValue();
// process result...
});
converterTask.setOnFailed(e -> {
// code to execute in case of unhandled exception...
// note you can get the exception with
Throwable thingThatWentWrong = converterTask.getException();
// ...
});
conversionExec.execute(converterTask);
Note that the code in either handler (onSucceeded or onFailed) is executed on the FX Application Thread for you, so you should not block in those methods, but you can access UI elements.
If you had exceptions from which you could recover you would do something along the lines of
#Override
public String call() {
try {
String result = doConversion(textToConvert);
return result ;
} catch (Exception e) {
// fallback computation that doesn't throw exception...
String result = doSafeConversion(textToConvert);
return result ;
}
}
and then you don't need the onFailed handler.
And obviously if you have some recoverable exceptions and some non-recoverable, you can combine the two techniques:
#Override
public String call() throws Exception {
String partialResult ;
// recoverable:
try {
partialResult = doSomeProcessing(textToConvert);
} catch (Exception e) {
partialResult = computeFallbackValue(textToConvert);
}
// non-recoverable: may throw exception which is propagated out
String result = completeComputation(partialResult);
return result ;
}
(and now you need the onFailed handler again).