Develop Reference

How to correctly handle a spark.sql.AnalysisException

I've been using Spark Dataset API to perform operations on a JSON to extract certain fields as needed. However, when the specification that I provide to let spark know what field to extract goes wrong, spark spits out an
org.apache.spark.sql.AnalysisException
How can unchecked runtime exceptions be handled in a distributed processing scenario like this ? I understand that throwing a try-catch would get things sorted but what is the recommended way to handle such a scenario
dataset = dataset.withColumn(current, functions.explode(dataset.col(parent + Constants.PUNCTUATION_PERIOD + child.substring(0, child.length() - 2))));

In scala, you should simply wrap the call in a Try and manage Failure. Something like:
val result = Try(executeSparkCode()) match {
case s: Success(_) => s;
case Failure(error: AnalysisException) => Failure(new MyException(error));
}
Note 1: If your question implies how to manage exception in scala, there are a lot of doc and post about this subject (i.e. don't throw). For example, you can check that answer (of mine)
Note 2: I don't have a scala dev env right here, so I didn't test this code)
In java there is a tricky situation however: the compiler doesn't expect an AnalysisException which is unchecked so you cannot catch this exception specifically. Probably some scala/java misunderstanding because scala doesn't track checked exceptions. What I did was:
try{
return executeSparkCode();
} catch (Exception ex) {
if(ex instanceOf AnalysisException){
throw new MyException(ex);
} else {
throw ex; // unmanaged exceptions
}
}
Note: In my case, I also tested the content of the error message for a specific exception that I must managed (i.e "path does not exist") in which case I return an empty dataset instead of throwing another exception. I was looking for a better solution and happened to get here...

Reactive patterns with argument preconditions

I'm working with RxJava and I don't know where the right place to check arguments would be. For example, say I have the following:
public Completable createDirectory(final String path) {
return Completable.create(emitter -> {
final File directory = new File(path);
final boolean createdSuccessfully = directory.mkDirs();
if (createdSuccessfully) {
emitter.onComplete();
} else {
emitter.onError(new IOException("Failed to create directory.");
}
}
}
Would it be better to check for a null path in the root of the method, or at the start of the completable? I'm leaning towards the former, but I'm interested in the pros and cons of both approaches.

I would say: it depends on what you are trying to achieve.
If you check for null-value on method-entry, before Completeable.create, you would throw the NPE/ IllegalArgumentException on calling-thread. This would follow the 'fail fast'-philosophy. Possible pros:
* fails fast on method-invocation
* Exception in callstack of calling-thread
If you check in Completable.create-lambda, it will be invoked on lambda-evaluation (subscription).
Example: Call createDirectory-Method on calling-thread, add subscribeOn and subscribe: NPE / IllegalArgumentException will be thrown on subscribeOn-Scheduler-Thread on subscription.
It will only be thrown, when you subscribe to returned Completable. Maybe you create an Completable but never subscribe to it. Then no exception will be thrown. This could be a pro for checking in lambda. If you would not check, a NPE would be thrown anyways on subscribeOn thread. Exception thrown on subscribeOn thread could be negative, because you only see the trace for subscribedOn-thread. Sometimes it is not easy to see the flow, when switching threads with only on callstack. You would not see that createDirectory-method was invoked. You would only see the lambda invocation plus some reactive-stack-polution.
I would say: use fail fast, if you subscribe to created Completable at some point. If it is possible, that no subscription to created Completable happens at any time, it would probably use some Precondition with a message, because it will only throw, when subscribed to (lazy). Anyways, if you check with Objects#requireNonNull() without any message in the lambda, you could just discard checking for null, because the NPE will be thrown anyway.

Null analysis failing to understand code

I'm currently having issues with null analysis on both Eclipse and IntelliJ where they don't understand a piece of code that handles null pointer exceptions. Basically I got the method below:
#Nullable
public static <T> T getValueOrNull(Supplier<T> resolver) {
try {
T result = resolver.get();
return Optional.ofNullable(result).orElse(null);
} catch (NullPointerException e) {
return null;
}
}
Normally I wouldn't catch those types of exceptions but in the project I'm working this method makes things much more readable.
It is called this way:
getValueOrNull(() -> obj.getObj2().getObj3()...getObjN())
This call returns the value or null if anything is null in the chain.
The problem is that null analysis will complain about possible null pointer exceptions while calling the method. While it is true that any of them may be null the method is handling that error. Is there a way to either skip null analysis for the parameters of a method or help the analyzer understand what is happening?
It seems that even a very simple test where null pointer exceptions are being caught still flags potential null issues inside the try block. That makes me think that try catch blocks are never even considered by the analyzer.
Also I do know that there are other ways of doing this such as the code below but if possible I'd like to be able to use what we are currently doing due to readability. Also even the code below has issues with the null analysis for the IDEs mentioned above.
Optional.ofNullable(objectA)
.map(a -> a.getObjectB())
.map(b -> b.getObjectC())
.map(c -> c.getObjectD())
.map(d -> d.getObjectE())
.map(e -> e.getName())
.orElse("");

Handling exceptions in Java (GWT)

I'm currently dealing with exceptions handling and I'm wondering where should I catch them.
Here is an stack from the GWT app :
A helper with a method which can throws NumerFormatExeption
(FormHelper.java)
A widget which uses this helper (CostWidget.java)
A presenter which calls this widget to retrieve data (BuildingPresenter.java)
FormHelper.java
public static Integer prepareIntegerForDb(String string) {
return Integer.parseInt(string);
}
CostWidget.java
public DetailCostProxy getCostDetail() {
...
costDetail.setQuantity(FormHelper.prepareDoubleForBd(qtTextBox.getText()));
...
return costDetail;
}
public List<DetailCostProxy> getCostList() {
...
costDetails .add(ligneCout.getCostDetail());
...
}
BuildingPresenter.java
public void saveBuilding(final BuildingProxy inter, final CollectRequestContext savecontext) {
savecontext.save(display.getCostWidget().getCoutList()).fire(new Receiver<BuildingProxy >() {....
}
I am thinking about :
1) adding "throws NumberFormatException" to prepareIntegerForDb() in the helper
2) adding "throws NumberFormatException" to getCostDetail() in the widget
3) adding "throws NumberFormatException" to getCostList() in the widget
4) caching the exception in the presenter (in saveBuilding)
The aim is :
to log the exception
to provide the user with a message saying that something went wrong
What do you think about this approach considering that this in an example and I will have to apply this pattern into the entire app (more than 20 presenters).
Is my way a good way to handle exceptions in GWT ? or should I log the error directly in the helper or elsewhere ?

prepareIntegerForDB() should throw the exception. This happens automatically when Integer.parse() fails, and you do not have to actually throw the Exception.
getCostDetail() should explicitly catch and throw the exception, and possibly expand upon why it was thrown. Something like "The cost was not in a readable format". That method is responsible for only that one line.
getCostList() should catch and handle the exceptions. That method is responsible for an entire collection. If you do not handle the bad data here, you will lose the good data. Here is one way to handle the bad data.
public List<DetailCostProxy> getCostList() {
...
try {
DetailCostProxy cost = lineCount.getCostDetail()
costDetails.add(cost);
catch (NumberFormatException e) {
costDetails.add(null);
}
...
}
Finally, the method that displays your data to the user should interpret the data passed to it before displaying it. If you used my example above, this would be as simple as checking for null values.

What do you think about this approach considering that this in an
example and I will have to apply this pattern into the entire app
(more than 20 presenters).
Adding throws NumberFormatException declarations won't help you to "provide the user with a message saying that something went wrong". NumberFormatException-s are RuntimeException-s so the throws declaration won't even force to try/catch in the code that uses these methods.
Is my way a good way to handle exceptions in GWT ? or should I log the
error directly in the helper or elsewhere ?
4) catching the exception in the presenter (in saveBuilding)
The aim is :
to log the exception
to provide the user with a message saying that something went wrong
This question is not specific to GWT.
To catch the Exception is a good idea if you know how to deal with it.
If you signal the error to the user, you need to be able to have the user decide how to handle the issue (for example a pop-up message proposing two actions to resume the application execution).

Java coding practice, runtime exceptions and this scenario

In the following scenario, I was trying to see how to handle this code and it how it relates to Runtimexception. I have read that is generally better to throw runtime exceptions as opposed to rely on static exceptions. And maybe even better to catch a static checked exception and throw an unchecked exception.
Are there any scenarios where it is OK to catch a static exception, possibly the catch-all Exception and just handle the exception. Possibly log an error message and continue on.
In the code below, in the execute1 method and execute2 method, let us say there is volatile code, do you catch the static exception and then rethrow? Or possibly if there are other errors:
if (null == someObj) { throw new RuntimeException(); }
Is this an approach you use?
Pseudo Code:
public class SomeWorkerObject {
private String field1 = "";
private String field2 = "";
public setField1() { }
public setField2() { }
// Do I throw runtime exception here?
public execute1() {
try {
// Do something with field 1
// Do something with field 2
} catch(SomeException) {
throw new RuntimeException();
}
}
// Do I throw runtime exception here?
public execute2() {
try {
// Do something with field 1
// Do something with field 2
} catch(SomeException) {
throw new RuntimeException();
}
}
}
public class TheWeb {
public void processWebRequest() {
SomeWorkerObject obj = new SomeWorkerObject();
obj.setField1("something");
obj.setField2("something");
obj.execute1();
obj.execute2();
// Possibility that runtime exception thrown?
doSomethingWith(obj);
}
}
I have a couple of problems with this code. There are times when I don't want a runtimeexception to be thrown because then execution stops in the calling method. It seems if I trap the errors in the method, maybe I can continue. But I will know if I can continue later on the program.
In the example above, what if obj.execute1() throws a Runtimeexception, then the code exits?
Edited: This guy seems to answer a lot of my questions, but I still want to hear your opinions.
http://misko.hevery.com/2009/09/16/checked-exceptions-i-love-you-but-you-have-to-go/
"Checked exceptions force me to write catch blocks which are meaningless: more code, harder to read, and higher chance that I will mess up the rethrow logic and eat the exception."

When catching an exception and throwing RuntimeException instead, it is important to set the original exception as a cause for the RuntimeException. i.e.
throw new RuntimeException(originalException).
Otherwise you will not know what was the problem in the first place.

Rethrowing checked exceptions as unchecked exceptions should only be done if you are sure that the checked exception is not to be expected.
Here's a typical example:
try {
hash = MessageDigest.getInstance("MD5").digest(string.getBytes("UTF-8"));
} catch (NoSuchAlgorithmException e) {
// Unexpected exception. "MD5" is just hardcoded and supported.
throw new RuntimeException("MD5 should be supported?", e);
} catch (UnsupportedEncodingException e) {
// Unexpected exception. "UTF-8" is just hardcoded and supported.
throw new RuntimeException("UTF-8 should be supported?", e);
}

There are times when I don't want a
runtimeexception to be thrown because
then execution stops in the calling
method. It seems if I trap the errors
in the method, maybe I can continue.
But I will know if I can continue
later on the program.
You have the right idea. The advice about throwing RuntimeException is that it doesn't require the caller to use a try-block or a 'throws' clause.
If your code can recover from an exception than it really should catch it and not throw anything.

One of the first rules about exceptions is to not abuse them to pass state in your application. They should be used for exceptional situations, not as alternative return values.
The second rule is to catch exceptions at the level you process them. Catch and rethrow does not add much. Any cleanup code in your method should be done in a finally block.
In my opinion catching checked exceptions and rethrowing them as runtime exceptions is abusing the system. It feels like working around the "limitations" of design by contract instead of using those "limitations" to get a more robust application.

Whether or not to handle an exception or simply rethrow it depends on your use case.
For example, if you're reading a file to load data into your application, and some IO error occurs, you're unlikely to recover from the error, so rethrowing the error to the top and consequently terminating the application isn't a bad course of action.
Conversely, if you're anticipating recoverable errors then you should absolutely catch and handle the errors. For example, you may have users entering data in a form. If they enter data incorrectly, your input processing code may throw an exception (e.g. NumberFormatException when parsing a malformed number string). Your code should catch these exceptions and return an error the user, prompting for correct input.
On an additional note, it's probably bad form to wrap all your exceptions with RuntimeException. If your code is going to be reused somewhere else, it is very helpful to have checked exceptions to signify that your code can fail in certain ways.
For example, assume your code is to parse configuration data from a file. Obviously, an IO error may occur, so you will have to catch an IOException somewhere in your code. You probably won't be able to do anything about the error, so you will have to rethrow it. However, someone calling into your code may well be able to handle such an error, for example by backing off to configuration defaults if the configuration can't be loaded from the file. By marking your API with checked exceptions, someone using your code can clearly see where an error may occur, and can thus write the error handling code at the appropriate place. If instead you simply throw a RuntimeException, the developer using your code won't be aware of possible errors until they creep up during testing.