I use KafkaBolt in Storm to publish messages to various Kafka topics. I want to put logging and metrics around the publishing logic so I can create alerts around any exceptions that might be thrown when there's a publishing failure. Exposing those exceptions is done through a Callback function that's passed into KafkaProducer.send(), which is executed after publishing succeeds or fails.
The problem is that KafkaBolt completely encapsulates its KafkaProducer so there's no way to inject a custom Callback, so if I want to see any errors I have to look in the Storm UI. I've worked around this by creating a wrapper for KafkaBolt. This wrapper will, in turn, wrap the OutputCollector passed into KafkaBolt.prepare() in a custom OutputCollector that overrides the behavior of OutputCollector.reportError(). I can then add my own logging and metrics reporting code there and then have it call the original method.
This solution seems perfectly adequate for what I need, but it seems odd that KafkaBolt makes it so difficult to programmatically access those exceptions. I was wondering if maybe I was missing something obvious and if there was a better way to do this.
I don't think you're missing anything, you're likely just the first person with this need. Someone has to hit this issue and decide to fix it :)
If you'd like to make changes to the bolt to support custom error handling (e.g. by allowing the user to provide a callback as you suggest), you can raise an issue at https://issues.apache.org/jira/projects/STORM/issues and make a PR against https://github.com/apache/storm/pulls. You're of course also welcome to only raise the issue, someone else might see it and decide to fix it, but contributing the fix yourself would likely be faster.
edit: You can find the bolt code at https://github.com/apache/storm/blob/master/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/bolt/KafkaBolt.java
Related
We are running are a series JAX-WS webservices and I'm currently trying to improve the applications logging.
I am currently failing to find a way to capture Runtime exceptions so they can be logged with stacktrace etc. With the hope that we don't have to have to wrap each of our functions in a try/catch to facilitate the logging.
It appeared that the web service handlers are too late.
It feels like this should be well tread ground but my googling is not feeding me the results I need.
Has anyone got any ideas about how best to achieve this, feeling stuck.
Thanks,
Jon
You can try AOP here .
Create a #annotation and make that particular annotation as an pointcut in AOP. And write around function so that you can log whatever you want around the execution.
The only thing you have to do is to provide annotation to the function.
You can check this : https://www.baeldung.com/spring-aop-annotation
The OSGi ConfigurationAdmin specification mentions that implementations of ManagedService and ManagedServiceFactory may signal an invalid incoming configuration by throwing a ConfigurationException. Yet, apart from this statement, the spec is silent on how the various actors should handle the situation, and most importantly, what should be the state of the environment after such an exception.
For example, suppose that a ManagedServiceFactory currently has a service instance (lets say service.pid=example.12345) with a valid set of properties; that service instance is published by the factory into the service registry. Then, the factory is informed of a configuration update for that service instance; however, on verification, the update method determine that incoming properties are invalid. Based on the spec, the factory should therefore throw a ConfigurationException.
But then, if nothing else is done, the environment remains in an unstable state: there is now a published service in the registry based on a configuration that no longer exists; consequently, whenever the ManagedServiceFactory service gets restarted (for example because of a bundle update or a whole framework restart), it will not be possible to reinstantiate that service, its former valid configuration having been lost. This breaks the persistence paradigm of the Configuration Admin subsystem, and pose severe issues regarding the stability of some OSGi environment.
Unfortunately, there is no easy way for the initial configurator bundle to detect that its configuration change caused a ConfigurationException, making it hardly possible in general to restore a stable configuration from that place. It seems to me that it would be more appropriate, in such situation, for the ConfigurationAdmin to (persistently) restore the previously valid configuration, but there is unfortunately no mention of such behaviour in the spec, and I don't see any trace of such mechanism in Felix's implementation.
Given these facts, it seems that the only possibility to maintain the stability of the environment would be for a ManagedServiceFactory implementation to first unregister and destroy existing service instances for which it has received invalid configuration properties, and only after that, throw the mandated ConfigurationException. This would effectively result in the same environment state as what would be if the framework was relaunched at that point. Similarly, a ManagedService implementation should handle an invalid configuration by first entirely restoring its default configuration, and then throw a ConfigurationException.
So, how exactly should errors in ManagedService and ManagedServiceFactories configuration updates be handled? Is my understanding correct? From what I see out there in example/open source implementations of ManagedService and ManagedServiceFactory, this aspect seems to be totally ignored by most developers. Does the spec provides any clarification on the subject?
The general strategy is to log it as an error and pray it will be solved soon. The purpose of the Configuration Exception is to provide detailed information to the devops so it can be corrected quickly.
The strategies you describe are imho so hopelessly complex and open ended that they tend to create more problems then they ever can solve. Someone made a mistake to create wrong configuration, the only solution is to fix that configuration. I find that in general systems that handle these exceptional case to become very fragile. Once something is wrong, you're in an infinite space and software is extremely bad in reasoning about things you don't know about.
So unless you have some very specific use cases I do not think it can, nor should it, have a general solution.
In general there are three strategies to handle this:
Rejecting the invalid configuration but keeping the previous state
Rejecting the invalid configuration and destroy the current state as if there was no configuration before
Rejecting invalid values, apply valid values as much as possible
What to choose, an wheather you descide to throw an exception, print a warning to the log, send an e-mail or bring up a popup heavyly depends on your system and use cases.
For example if you have an UI and the user can change the config you can simply save the old config and if you detect an error you can ask the user to either correct or revert the configuration.
Even better, you can describe the Configuration requirements with the MetaTypeService so you can validate the config before apply it.
If you have a set of config files, you better make a backup before so you can revert :)
Is it possible to create something like an event in java which will be triggered if an Exception was thrown?
I want to make something like a exception tracking system which sends all Exception to a central server.
Is this possible or do I have to call a method on myself within every catch block?
The recommended way to handle this would be with logging (for example SLF4J and Logback). You can configure loggers to send the messages to a centralized logging server, but you still need to manually log the exception and if applicable, handle it.
If you were hoping for an automagical way to just forget about exceptions, that's not really possible. Exception handling is not as easy as it may seem, if you intend to make your applications robust and easily debuggable.
A good additional "trick" is to use Thread.setDefaultUncaughtExceptionHandler() to provide a final fail-safe. Its only job should be to log any exceptions that aren't handled elsewhere. However it should happen only in case there's a massive failure and the stacktrace might otherwise be lost. If it's being invoked just because some input was of the wrong format, your design and code is of poor quality.
There is a lot of code for this SSCCE, however I've added this on github to make it easier to interact with.
I'm attempting to wrap my head around this annotation, but I can't seem to figure out its use. I've created a sample application. I'm able to create the MBean, monitor it, and connect to my local MBean server to modify the application while running, but see no real use for #ManagedNotification and #ManagedNotifications.
I can remove the ManagedNotifications on my class, and still receive the notifications. The only difference is that I don't see the description of the Notificaiton, however so long as your messages are good, I don't see the need for this annotation. Is there anything extra I'm missing? The javadocs on it state it is to be used on a method, but the target for #ManagedNotifications and #ManagedNotification is limited to type only. Is there more to this annotation than I am seeing?
I am writing a wrapper around the DefaultHttpClient to handle some of the error-prone configuration options.
For example I will pre-configure everything to handle UTF-8 properly and to shutdown the connection cleanly.
When a non-200 is returned, I thought about the client registering a handler for a specific status code and then calling it.
I would provide some default handlers to take care of simple cases.
Is this a good pattern for a clean API? If I throw exceptions, the client has to handle cases which might not happen at all as I would have to throw an exception per possible HTTP status code (or most).
The thing I like about handlers is that I can provide a couple of 'default handlers' which might be overwritten...
I'd like to hear your input and maybe get some more creative ideas.
Cheers
Currently, accepted practice for Java APIs is to employ unchecked exceptions, so client won't need to change it's internal code just to accommodate API's exceptions into client's codebase.
Instead, if you use unchecked exceptions, you'll save your code unchanged except the places where you really need to handle exceptions.
Here're slides about Robert Martin's "Clean code" book which talk about error handling best practices: slides.
I wouldn't create a different exception for each http error code. At most create one or two general exceptions and store the exact error code as part of the exception. That way if the client code just wants to log or ignore them it can, or it can get more details based on the error code.