I've been using Spark Dataset API to perform operations on a JSON to extract certain fields as needed. However, when the specification that I provide to let spark know what field to extract goes wrong, spark spits out an
org.apache.spark.sql.AnalysisException
How can unchecked runtime exceptions be handled in a distributed processing scenario like this ? I understand that throwing a try-catch would get things sorted but what is the recommended way to handle such a scenario
dataset = dataset.withColumn(current, functions.explode(dataset.col(parent + Constants.PUNCTUATION_PERIOD + child.substring(0, child.length() - 2))));
In scala, you should simply wrap the call in a Try and manage Failure. Something like:
val result = Try(executeSparkCode()) match {
case s: Success(_) => s;
case Failure(error: AnalysisException) => Failure(new MyException(error));
}
Note 1: If your question implies how to manage exception in scala, there are a lot of doc and post about this subject (i.e. don't throw). For example, you can check that answer (of mine)
Note 2: I don't have a scala dev env right here, so I didn't test this code)
In java there is a tricky situation however: the compiler doesn't expect an AnalysisException which is unchecked so you cannot catch this exception specifically. Probably some scala/java misunderstanding because scala doesn't track checked exceptions. What I did was:
try{
return executeSparkCode();
} catch (Exception ex) {
if(ex instanceOf AnalysisException){
throw new MyException(ex);
} else {
throw ex; // unmanaged exceptions
}
}
Note: In my case, I also tested the content of the error message for a specific exception that I must managed (i.e "path does not exist") in which case I return an empty dataset instead of throwing another exception. I was looking for a better solution and happened to get here...
Related
I am not able to understand why Java doesn't allow to change exception message of an exception of type Exception (or its superclass Throwable) once it has been created. It allows to change the stackTrace using setStackTrace but not the message.
The use case I have seems genuine to me and would appreciate some alternative.
Use case
I have a controller service X that calls let's say 10 other dependent services. To make debugging easy, if a dependent service throws some exception we want to surface some identifiers from service X to its upstream to identify the failed request easily. For this we have requestId which we create and set for each dependency.
Now to avoid duplication across all dependent services and simplify code, we can create a common interceptor that allows us to do some work before and after each call. Something like -
// do some work
requestId = getRequestId(); // create or somehow get requestId
dependentService.setRequestId(requestId);
try {
dependentService.call();
}
catch (Exception e) {
e.setMessage(e.getMessage() + ... + requestId);
throw e;
}
//do some work
But Java doesn't allow us to set message. At the same time, we want to preserve the exception and its type (which could be any of the custom types defined by dependent services), so I don't want to do something like throw new Exception(e.getMessage() + ...)
It's not really what it's meant for, but you could attach another exception with addSuppressed:
} catch (Exception e) {
e.addSuppressed(new ExtraInfoException(...));
throw e;
}
where ... contains the extra info you want to include.
The advantage of this over adding to the exception message is that you can define your ExtraInfoException so that it has the info you want in fields, rather than having to parse it back out of the exception message.
With that said, a more idiomatic way to attach more exception info it would be:
} catch (Exception e) {
throw new ExtraInfoException(e, ...);
}
which has exactly the same advantage of allowing you to return structured information, with the additional advantage that you can catch ExtraInfoException directly, rather than catching Exception and then hunting for the extra info reflectively.
Why doesn't Java have setMessage in Exception/Throwable classes?
The answer to your question is that they (the library designers) did not think that changing a message on an exception was a useful thing to do.
To a large degree1, the designers have taken the view that they shouldn't design the APIs to directly support all possible use-cases ... including the obscure ones that almost nobody will encounter. Like your one2.
And in your case, there are other ways to achieve what you are trying to do; see the other answers.
I did a quick search of the Java bugs database to see if someone else had put in an RFE to request a setMessage method for Throwable or Exception. I couldn't find anything. If your requirement was even slightly common, there would probably be an RFE with an explanation of why it was declined.
1 - Obviously, there are exceptions to this, but that is beside the point.
2 - Obviously you would disagree that your use-case is obscure, but that that is also beside the point. The question is why they haven't implemented this, not whether they were wrong. (Asking / debating whether they were wrong is off-topic, because it is a matter of opinion.)
Resetting a message it's some kind of rewriting the history. You have a catch block when you catch exception and handle them. If you need to throw an exception during the handling, it's a different problem and an exception should be different.
} catch (SomeException e) {
// here we have SomeException and we want to handle it.
// if we can't we throw a new one, because we have a problem with handling.
// if the handling problem cause is SomeException we put it at the cause.
throw new AnotherException("with some message", e);
}
And in the stacks trace we will see that we have AnotherException because of SomeException which gives us information about the root of problem.
Just simply throw new instance like this:
try {
...
} catch(Exception ex) {
throw new Exception ("new message", ex);
}
I'm currently dealing with exceptions handling and I'm wondering where should I catch them.
Here is an stack from the GWT app :
A helper with a method which can throws NumerFormatExeption
(FormHelper.java)
A widget which uses this helper (CostWidget.java)
A presenter which calls this widget to retrieve data (BuildingPresenter.java)
FormHelper.java
public static Integer prepareIntegerForDb(String string) {
return Integer.parseInt(string);
}
CostWidget.java
public DetailCostProxy getCostDetail() {
...
costDetail.setQuantity(FormHelper.prepareDoubleForBd(qtTextBox.getText()));
...
return costDetail;
}
public List<DetailCostProxy> getCostList() {
...
costDetails .add(ligneCout.getCostDetail());
...
}
BuildingPresenter.java
public void saveBuilding(final BuildingProxy inter, final CollectRequestContext savecontext) {
savecontext.save(display.getCostWidget().getCoutList()).fire(new Receiver<BuildingProxy >() {....
}
I am thinking about :
1) adding "throws NumberFormatException" to prepareIntegerForDb() in the helper
2) adding "throws NumberFormatException" to getCostDetail() in the widget
3) adding "throws NumberFormatException" to getCostList() in the widget
4) caching the exception in the presenter (in saveBuilding)
The aim is :
to log the exception
to provide the user with a message saying that something went wrong
What do you think about this approach considering that this in an example and I will have to apply this pattern into the entire app (more than 20 presenters).
Is my way a good way to handle exceptions in GWT ? or should I log the error directly in the helper or elsewhere ?
prepareIntegerForDB() should throw the exception. This happens automatically when Integer.parse() fails, and you do not have to actually throw the Exception.
getCostDetail() should explicitly catch and throw the exception, and possibly expand upon why it was thrown. Something like "The cost was not in a readable format". That method is responsible for only that one line.
getCostList() should catch and handle the exceptions. That method is responsible for an entire collection. If you do not handle the bad data here, you will lose the good data. Here is one way to handle the bad data.
public List<DetailCostProxy> getCostList() {
...
try {
DetailCostProxy cost = lineCount.getCostDetail()
costDetails.add(cost);
catch (NumberFormatException e) {
costDetails.add(null);
}
...
}
Finally, the method that displays your data to the user should interpret the data passed to it before displaying it. If you used my example above, this would be as simple as checking for null values.
What do you think about this approach considering that this in an
example and I will have to apply this pattern into the entire app
(more than 20 presenters).
Adding throws NumberFormatException declarations won't help you to "provide the user with a message saying that something went wrong". NumberFormatException-s are RuntimeException-s so the throws declaration won't even force to try/catch in the code that uses these methods.
Is my way a good way to handle exceptions in GWT ? or should I log the
error directly in the helper or elsewhere ?
4) catching the exception in the presenter (in saveBuilding)
The aim is :
to log the exception
to provide the user with a message saying that something went wrong
This question is not specific to GWT.
To catch the Exception is a good idea if you know how to deal with it.
If you signal the error to the user, you need to be able to have the user decide how to handle the issue (for example a pop-up message proposing two actions to resume the application execution).
In the following piece of code :
try{
....
}
catch(Exception e)
{
logger.error("Exception caught in method: \n");
logger.error(e.getMessage());
String fault = "<Fault><FaultCode>" + e.getClass().getName() + "</FaultCode><FaultDescription>" + e.getMessage() + "</FaultDescription></Fault>";
return XmlObject.Factory.parse(fault);
}
,
is there a way to convert the "fault" string to an XML object using a code that wouldn't throw any exception or require any exception handling?
Basically, we need to return an XML object in both success and fauilure scenarios, without the method using any throws clause. That's the reason, we've removed all the specific Exception cases, that could occur in the main flow, like ParserConfiguration, IOExceptoin, XmlException etc. from the catch and kept a generic one and hard coded a string that would gather the required fault info and pass back as an XML object. But, even with this minimal coding, there is a chance of an XmlException occuring in the parse() method. Any other way, like reading from an errorConfig file or creating a separate XML object would bring in more exceptions to be handled.
So, just want to know whether its even possible to develop such a code, i.e. an exception-free catch block returning XML? Or, if anyone could recommend any other approach would also be fine.
FYI : The main code basically transforms an XML or enriches it by appending more tags and returns it.
Hope I've made myself clear.
If the parser cannot parse the XML (and aborts with an exception), where are you going to get your XML object from?
If you think that in your flow of operation no XML exception can ever really happen (which the compiler of course cannot guarantee), then assume just that, catch the exception, log a "oops, that should never have happened", re-throw a RuntimeException and don't worry about it any longer (it won't ever happen, right?)
String fault = "<Fault><FaultCode>" + e.getClass().getName() +
"</FaultCode> <FaultDescription>" + e.getMessage()
+ "</FaultDescription></Fault>";
return XmlObject.Factory.parse(fault);
I think (if that error message does not need escaping), you can be sure that this will never fail. So that would fit the above pattern.
I don't know if that is "checked-exception-free", but you could forgo the XML string and re-parsing it and construct a DOM programmatically instead.
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
Document doc = dbf.newDocumentBuilder().newDocument();
Element rootElement = doc.createElement("fault");
// etc ...
I don't know if that is "checked-exception-free",
Looking at the Javadocs: Nope. All kinds of configuration exceptions. :-(
Let's say I am designing an API for storing passwords. According to Effective Java it is a good idea to use throw clauses to show checked exceptions, however by having a throw clause that throws a SQLException, which is a checked exception, then I am revealing the underlying implementation details of my API and thus I will be unable to change the implementation details of my API at a later stage. One of the pros to throwing a checked exception is that the programmer who uses the API should be able to handle the exception in a manner of their choosing. Which of these two methods should I choose to do, add a throw clause which reveals the implementation or hide it or use a different approach?
Your motivation is correct for not "leaking" SQLException to the users of your class.
The fact that you're using SQL could be considered an implementation detail. You may even swap SQL persistence for say, an in-memory one at a later time, and this change shouldn't impact the users of your class.
If you are inclined to use checked exceptions, I would define your own exception type (say, PasswordStoreException -- just an example name). You can use it to wrap the original exception that was thrown, e.g.:
try {
// do whatever
} catch (SQLException ex) {
throw new PasswordStoreException(ex);
}
It is today considered bad design for an API to declare checked exceptions. If you have ever used such an API, you should already know why.
In any case your API should never throw (let alone declare) exceptions belonging to other APIs. If you do that, you hang a completely unrelated dependency on your client's back. The only "exception" to this rule are JDK's standard exceptions like NPE, ISE and the like.
Catch the SQLException, and wrap it into your own exception:
try {
// JDBC code
}
catch (SQLException e) {
throw new MyException("Error persisting the secret", e); // choose a better name, of course
}
Whether this exception should be a checked exception or a runtime exception depends on what the caller can do about it. If it's recoverable (which, I think, is not the case here), it should be a checked exception. If it's not recoverable (the caller can just display an error message), then it should be a runtime exception.
If it's a checked exception, you have no choice; the exception MUST be declared in the throws clause of the method.
As is, it is always a good idea to throw your own exception checked/unchecked. But before that, try to fix the underlying exception if possible. I always prefer the below way,
try {
// JDBC code
}
catch (SQLException e) {
// try to solve the exception at API level
bollean solvable = trySolveException(e);
if (!solvable) {
// Alert the admin, or log the error statement for further debugging.
mailClient.sendMailToDBAdmin("Some issue storing password", e);
// or
LOG.severe("some issue in storing password " + e.toString);
throw MyException("A request/logstatement is logged on your behalf regarding the exception", e);
}
LOG.info("The exception is solved");
}
finally {
// don't forget to free your resources - to avoid garbage and memory leaks, incase you have solved the issue in trySolveException(e).
}
So,
1) You don't expose the SRQException directly, but you throw your own version of the exception.
2) You tried to solve the exception once and if not you alerted somehow - through a mail or a log statement.
3) Finally, you ve released all the resources if you succeed in solving the exception.
The finally clause can be avoided if you use the new Java7's try with resource close option.
For whether to throw checked or unchecked exception, I will give you an example
1) Say an exceptions like NPE - they are programmatic errors and the developer should be more responsible to have not created a NullPointer. You don't expect your code to account for such careless errors and put a try(NPE), catch(NPE). So throw a unchecked exceptions.
2) On the other hand the exceptions like SQL exceptions are at the rare cases, account for some external dependency. So, better throw a user defined checked exceptions. And the user can determine if he can connect to the backup SQL server if any.
3) There are another clause of exceptions, where the program cannot continue furhter. Say a Memory Out of Bounds. They should be thrown as Errors.
Try this..
fillInStackTrace() method is called to re-initialize the stack trace data in the newly created throwable. Will be helpful in masking the info about the exception when tries to access the API.
For my Java application, I am creating an instance of a user information object and populating it with a service that I don't control the source for.
The code looks like this:
// username given as parameter
UserInfo ui = new UserInfo();
try {
DirectoryUser du = LDAPService.findUser(username);
if (du!=null) {
ui.setUserInfo(du.getUserInfo());
}
} catch (Exception e) {
// Whatever
}
If LDAPService.findUser() can't locate a user, it will throw a NullPointerException and grind the rest of my application to a stop. It's okay if the user information isn't populated, so I want to be able to continue without causing everything else to start throwing exceptions.
Is there a way to do this?
I've upvoted Amir Afghani's answer, which seems to be the only one as of yet that actually answers the question.
But I would have written it like this instead:
UserInfo ui = new UserInfo();
DirectoryUser du = null;
try {
du = LDAPService.findUser(username);
} catch (NullPointerException npe) {
// It's fine if findUser throws a NPE
}
if (du != null) {
ui.setUserInfo(du.getUserInfo());
}
Of course, it depends on whether or not you want to catch NPEs from the ui.setUserInfo() and du.getUserInfo() calls.
You could catch the NullPointerException explicitly and ignore it - though its generally not recommended. You should not, however, ignore all exceptions as you're currently doing.
UserInfo ui = new UserInfo();
try {
DirectoryUser du = LDAPService.findUser(username);
if (du!=null) {
ui.setUserInfo(du.getUserInfo());
}
} catch (NullPointerException npe) {
// Lulz # your NPE
Logger.log("No user info for " +username+ ", will find some way to cope");
}
You are already doing it in your code. Run this example below. The catch will "handle" the exception, and you can move forward, assuming whatever you caught and handled did not break code down the road which you did not anticipate.
try{
throw new Exception();
}catch (Exception ex){
ex.printStackTrace();
}
System.out.println("Made it!");
However, you should always handle an exception properly. You can get yourself into some pretty messy situations and write difficult to maintain code by "ignoring" exceptions. You should only do this if you are actually handling whatever went wrong with the exception to the point that it really does not affect the rest of the program.
It's generally considered a bad idea to ignore exceptions. Usually, if it's appropriate, you want to either notify the user of the issue (if they would care) or at the very least, log the exception, or print the stack trace to the console.
However, if that's truly not necessary (you're the one making the decision) then no, there's no other way to ignore an exception that forces you to catch it. The only revision, in that case, that I would suggest is explicitly listing the the class of the Exceptions you're ignoring, and some comment as to why you're ignoring them, rather than simply ignoring any exception, as you've done in your example.
You are actually ignoring exception in your code. But I suggest you to reconsider.
Here is a quote from Coding Crimes: Ignoring Exceptions
For a start, the exception should be logged at the very least, not
just written out to the console. Also, in most cases, the exception
should be thrown back to the caller for them to deal with. If it
doesn't need to be thrown back to the caller, then the exception
should be handled. And some comments would be nice too.
The usual excuse for this type of code is "I didn't have time", but
there is a ripple effect when code is left in this state. Chances are
that most of this type of code will never get out in the final
production. Code reviews or static analysis tools should catch this
error pattern. But that's no excuse, all this does is add time to the
maintainance and debugging of the software.
Even if you are ignoring it I suggest you to use specific exception names instead of superclass name. ie., Use NullPointerException instead of Exception in your catch clause.
You can write a try - catch block around the line you want to have ignored.
Like in the example code of yours. If you just continue your code below the closing bracket of the catch block everythings fine.
LDAPService should contain method like LDAPService.isExists(String userName) use it to prevent NPE to be thrown. If is not - this could be a workaround, but use Logging to post some warning..
Printing the STACK trace, logging it or send message to the user, are very bad ways to process the exceptions. Does any one can describe solutions to fix the exception in proper steps then can trying the broken instruction again?