The title might be incorrect, but I will try to explain my issue. My project is a Spring Boot project. I have services which do calls to external REST endpoints.
I have a service method which contains several method calls to other services I have. Every individual method call can be successful or not. Every method call is done to a REST endpoint and there can be issues that for example the webservice is not available or that it throws an unknown exception in rare cases. What ever happens, I need to be able to track which method calls were successful and if any one of them fails, I want to rollback to the original state as if nothing happened, see it a bit as #Transactional annotation. All REST calls are different endpoints and need to be called separately and are from an external party which I don't have influence on. Example:
public MyServiceImpl implements MyService {
#Autowired
private Process1Service;
#Autowired
private Process2Service;
#Autowired
private Process3Service;
#Autowired
private Process4Service;
public void bundledProcess() {
process1Service.createFileRESTcall();
process2Service.addFilePermissionsRESTcall();
process3Service.addFileMetadataRESTcall(); <-- might fail for example
process4Service.addFileTimestampRESTcall();
}
}
If for example process3Service.addFileMetadataRESTcall fails I want to do something like undo (in reverse order) for every step before process3:
process2Service.removeFilePermissionsRESTcall();
process1Service.deleteFileRESTcall();
I read about the Command pattern, but that seems to be used for Undo actions inside an application as a sort of history of actions performed, not inside a Spring web application. Is this correct for my use case too or should I track per method/webservice call if it was successful? Is there a best practice for doing this?
I guess however I track it, I need to know which method call failed and from there on perform my 'undo' method REST calls. Although in theory even these calls might also fail of course.
My main goal is to not have files being created (in my example) which any further processes have not been performed on. It should either be all successful or nothing. A sort of transactional.
Update1: improved pseudo implementation based on comments:
public Process1ServiceImpl implements Process1Service {
public void createFileRESTcall() throws MyException {
// Call an external REST api, pseudo code:
if (REST-call fails) {
throw new MyException("External REST api failed");
}
}
}
public class BundledProcessEvent {
private boolean createFileSuccess;
private boolean addFilePermissionsSuccess;
private boolean addFileMetadataSuccess;
private boolean addFileTimestampSuccess;
// Getters and setters
}
public MyServiceImpl implements MyService {
#Autowired
private Process1Service;
#Autowired
private Process2Service;
#Autowired
private Process3Service;
#Autowired
private Process4Service;
#Autowired
private ApplicationEventPublisher applicationEventPublisher;
#Transactional(rollbackOn = MyException.class)
public void bundledProcess() {
BundleProcessEvent bundleProcessEvent = new BundleProcessEvent();
this.applicationEventPublisher.publishEvent(bundleProcessEvent);
bundleProcessEvent.setCreateFileSuccess = bundprocess1Service.createFileRESTcall();
bundleProcessEvent.setAddFilePermissionsSuccess = process2Service.addFilePermissionsRESTcall();
bundleProcessEvent.setAddFileMetadataSuccess = process3Service.addFileMetadataRESTcall();
bundleProcessEvent.setAddFileTimestampSuccess = process4Service.addFileTimestampRESTcall();
}
#TransactionalEventListener(phase = TransactionPhase.AFTER_ROLLBACK)
public void rollback(BundleProcessEvent bundleProcessEvent) {
// If the last process event is successful, we should not
// be in this rollback method even
//if (bundleProcessEvent.isAddFileTimestampSuccess()) {
// remove timestamp
//}
if (bundleProcessEvent.isAddFileMetadataSuccess()) {
// remove metadata
}
if (bundleProcessEvent.isAddFilePermissionsSuccess()) {
// remove file permissions
}
if (bundleProcessEvent.isCreateFileSuccess()) {
// remove file
}
}
Your operation looks like a transaction, so you can use #Transactional annotation. From your code I can't really tell how you are managing HTTP response calls for each of those operations, but you should consider having your service methods to return them, and then do a rollback depending on response calls. You can create an array of methods like so, but how exactly you want your logic to be is up to you.
private Process[] restCalls = new Process[] {
new Process() { public void call() { process1Service.createFileRESTcall(); } },
new Process() { public void call() { process2Service.addFilePermissionsRESTcall(); } },
new Process() { public void call() { process3Service.addFileMetadataRESTcall(); } },
new Process() { public void call() { process4Service.addFileTimestampRESTcall(); } },
};
interface Process {
void call();
}
#Transactional(rollbackOn = Exception.class)
public void bundledProcess() {
restCalls[0].call();
... // say, see which process returned wrong response code
}
#TransactionalEventListener(phase = TransactionPhase.AFTER_ROLLBACK)
public void rollback() {
// handle rollback according to failed method index
}
Check this article. Might come in handy.
The answer to this question is quite broad. There are various ways to do distributed transactions to go through them all here. However, since you are using Java and Spring, your best bet is to use something like JTA (Java Transaction API), which enables a distributed transactions across multiple services/instances/etc.. Fortunately, Spring Boot supports JTA using either Atomikos or Bitronix. You can read the doc here.
One approach to enable distributed transactions is through a message broker such as JMS, RabbitMQ, Kafka, ActiveMQ, etc. and use a protocol like XA transactions (two-phase commit). In the case of external services that do not support distributed, one approach is to write a wrapper service that understands XA transactions to that external service.
Related
I'm trying to understand what is the right way of implementing post_construct methods in Spring Webflux.
On startup of an application I need to read data from the DB (I have an R2dbcRepository configured), and then perform some logic and save result as Bean's fields).
So I have a findAll() method returning Flux. How should it be done?
I tried using .block(), AtomicBoolean flag, none of these worked
First of all, never use block() method. Use it for tests at most, but there is a better solution out there than StepVerifier. (If you use Kotlin there are await prefixed methods that work like block but not blocking.)
If you need data at launch, that says it is bad design to me because if there is no user, what do you do with it? I think it's illogical. What happens when you use query when you need it, add to cache and reuse it when you need it again. In the case of WebFlux, you can prepare a Mono object that uses a query from the database and use .cache() end of chain. So Spring Bean can contain this Mono object that will be run when you subscribe.
Ofc below example, repo.find will never call if function of Service won't run.
https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html#cache--
#Configuration
public class Config {
private R2dbcRepository repo;
public Config(R2dbcRepository repo) {
this.repo = repo;
}
#Bean
public Mono<Data> myCachedDbData() {
return repo.find(...)
.map(it -> new Data(it))
.cache()
}
}
#Service
public class Service {
private Mono<Data> data;
public Config(Mono<Data> data) {
this.data = data;
}
public Object function() {
return data.flatMap(...)
}
}
I have multiple methods in my codebase annotated with Spring's #transactional with different propgation levels (lets ignore the idea behind choosing the propagation levels). Example -
public class X {
#Transactional(Propagation.NOT_SUPPORTED)
public void A() { do_something; }
#Transactional(Propagation.REQUIRED)
public void B() { do_something; }
#Transactional(Propagation.REQUIRES_NEW)
public void C() { do_something; }
}
Now I have a new use case where I want to perform all these operations in a single transaction (for this specific use case only, without modifying existing behavior), overriding any annotated propagation levels. Example -
public class Y {
private X x;
// Stores application's global state
private GlobalState globalState;
#Transactional
public void newOperation() {
// Set current operation as the new operation in the global state,
// in case this info might be required somewhere
globalState.setCurrentOperation("newOperation");
// For this new operation A, B, C should be performed in the current
// transaction regardless of the propagation level defined on them
x.A();
x.B();
x.C();
}
}
Does Spring provide some way to achieve this ? Is this not possible ?
One way I could think of is to split the original methods
#Transactional(Propagation.NOT_SUPPORTED)
public void A() { A_actual(); }
// Call A_actual from A and newOperation
public void A_actual() { do_something; }
But this might not be as simple to do as this example (there can be a lot of such methods and doing this might not scale). Also it does not look much clean.
Also the use case might also appear counter intuitive, but anyway let's keep that out of scope of this question.
I do believe the only option is to replace TransactionInterceptor via BeanPostProcessor, smth. like:
public class TransactionInterceptorExt extends TransactionInterceptor {
#Override
public Object invoke(MethodInvocation invocation) throws Throwable {
// here some logic determining how to proceed invocation
return super.invoke(invocation);
}
}
public class TransactionInterceptorPostProcessor implements BeanFactoryPostProcessor, BeanPostProcessor, BeanFactoryAware {
#Setter
private BeanFactory beanFactory;
#Override
public void postProcessBeanFactory(#NonNull ConfigurableListableBeanFactory beanFactory) throws BeansException {
beanFactory.addBeanPostProcessor(this);
}
#Override
public Object postProcessBeforeInitialization(#NonNull Object bean, #NonNull String beanName) throws BeansException {
if (bean instanceof TransactionInterceptor) {
TransactionInterceptor interceptor = (TransactionInterceptor) bean;
TransactionInterceptor result = new TransactionInterceptorExt();
result.setTransactionAttributeSource(interceptor.getTransactionAttributeSource());
result.setTransactionManager(interceptor.getTransactionManager());
result.setBeanFactory(beanFactory);
return result;
}
return bean;
}
}
#Configuration
public class CustomTransactionConfiguration {
#Bean
//#ConditionalOnBean(TransactionInterceptor.class)
public static BeanFactoryPostProcessor transactionInterceptorPostProcessor() {
return new TransactionInterceptorPostProcessor();
}
}
However, I would agree with #jim-garrison suggestion to refactor your spring beans.
UPD.
But you favour refactoring the beans instead of following this approach. So for the sake of completeness, can you please mention any issues/shortcomings with this
Well, there are a plenty of things/concepts/ideas in spring framework which were implemented without understanding/anticipating consequences (I believe the goal was to make framework attractive to unexperienced developers), and #Transactional annotation is one of such things. Let's consider the following code:
#Transactional(Propagation.REQUIRED)
public void doSomething() {
do_something;
}
The question is: why do we put #Transactional(Propagation.REQUIRED) annotation above that method? Someone might say smth. like this:
that method modifies multiple rows/tables in DB and we would like to avoid inconsistencies in our DB, moreover Propagation.REQUIRED does not hurt anything, because according to the contract it either starts new transaction or joins to the exisiting one.
and that would be wrong:
#Transactional annotation poisons stacktraces with irrelevant information
in case of exception it marks existing transaction it joined to as rollback-only - after that caller side has no option to compensate that exception
In the most cases developers should not use #Transactional(Propagation.REQUIRED) - technically we just need a simple assertion about transaction status.
Using #Transactional(Propagation.REQUIRES_NEW) is even more harmful:
in case of existing transaction it acquires another one JDBC-connection from connection pool, and hence you start getting 2+ connections per thread - this hurts performance sizing
you need to carefully watch for data you are working with - data corruptions and self-locks are the consequences of using #Transactional(Propagation.REQUIRES_NEW), cause now you have two incarnations of the same data within the same thread
In the most cases #Transactional(Propagation.REQUIRES_NEW) is an indicator that you code requires refactoring.
So, the general idea about #Transactional annotation is do not use it everywhere just because we can, and your question actually confirms this idea: you have failed to tie up 3 methods together just because developer had some assumptions about how those methods should being executed.
I am thinking what would be best solution for following case. Suppose we have at start CRUD app - using Spring Boot. I would like to add read only state for this application - which allows only data read and blocks create, update, delete data operations for admin role. I think about adding aspect (#Aspect) which checks current app state (which is saved in db) and starts if create, update, update operations are invoked. If app is in read-only state - exception will be thrown (handled by #ControllerAdvice)
I wonder if adding aspect is the best option - I dont want modify existing code. Whats your take on that? Moreover - how would you write integration tests for #aspect - testing if aspect starts properly? How could be aspects tested for this case? What are good practises for testing #aspects (integration tests #springboottest)
This honestly seems like an inconvenient way of doing this. Why not just add an Interceptor that checks for this? I did something similar before
#Component
#RequiredArgsConstructor
public class ReadOnlyModeInterceptor implements HandlerInterceptor {
private final ServerProperties serverProperties;
#Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
if (serverProperties.isReadOnlyMode()) {
String method = request.getMethod();
boolean isReadOnlyMethod = "GET".equals(method) || "OPTIONS".equals(method);
String servletPath = request.getServletPath();
boolean isReadOnlyPath = isReadOnlyPath(servletPath);
if (!isReadOnlyMethod && isReadOnlyPath) {
throw new ServiceUnavailableException("Server is in read-only mode.");
}
}
return true;
}
private boolean isReadOnlyPath(String servletPath) {
if (serverProperties.isFullyReadOnly()) {
return true; // wildcard option, entire server is read-only
}
return serverProperties.getReadOnlyPaths().stream().anyMatch(servletPath::contains);
}
}
You also need to register it
#RequiredArgsConstructor
#Configuration
public class WebMvcConfig implements WebMvcConfigurer {
private final ReadOnlyModeInterceptor readOnlyModeInterceptor;
#Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(readOnlyModeInterceptor).order(0);
}
}
I have a spring boot application that uses a few #TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT). I noticed that spring boot doesn't do any exception logging for them when they end up with an exception being thrown.
Because of this I wanted to add some generic logging facility for such exceptions. I found that TransactionalApplicationListener.SynchronizationCallback is the interface I need to implement. However it seems complicated to register these callbacks. I didn't find any call of TransactionalApplicationListener#addCallback in the spring dependencies that would achieve this.
Trying to get a list of TransactionalApplicationListener and the SynchronizationCallback injected and then call addCallback in a #PostConstruct didn't get me further because there were always no listeners injected even though the application did make successful use of them.
So how do I add SynchronizationCallbacks to TransactionalApplicationListeners during spring boot application startup?
The first thing to note is that TransactionalApplicationListeners like all ApplicationListener are not beans in the spring context. They live somewhat outside of it (see org.springframework.context.ConfigurableApplicationContext#addApplicationListener). So injecting them is not possible for the application context.
While debugging and looking through spring sources one finds that these listeners are being created by org.springframework.transaction.event.TransactionalEventListenerFactory. And that is where my solution steps into. We decorate that factory with another one that is aware of SynchronizationCallbacks:
public class SynchronizationCallbackAwareFactory implements EventListenerFactory, Ordered {
private final TransactionalEventListenerFactory delegate;
private final Provider<List<SynchronizationCallback>> synchronizationCallbacks;
private final int order;
public SynchronizationCallbackAwareFactory(TransactionalEventListenerFactory transactionalEventListenerFactory,
Provider<List<SynchronizationCallback>> synchronizationCallbacks,
int order) {
this.delegate = transactionalEventListenerFactory;
this.synchronizationCallbacks = synchronizationCallbacks;
this.order = order;
}
#Override
public boolean supportsMethod(Method method) {
return delegate.supportsMethod(method);
}
#Override
public ApplicationListener<?> createApplicationListener(String beanName, Class<?> type, Method method) {
ApplicationListener<?> applicationListener = delegate.createApplicationListener(beanName, type, method);
if (applicationListener instanceof TransactionalApplicationListener) {
TransactionalApplicationListener<?> listener = (TransactionalApplicationListener<?>) applicationListener;
Collection<SynchronizationCallback> callbacks = this.synchronizationCallbacks.get();
callbacks.forEach(listener::addCallback);
}
return applicationListener;
}
#Override
public int getOrder() {
return order;
}
}
Note that I use a javax.inject.Provider in my case to make the retrieval of the callbacks at the latest possible time.
The decorator has to be Ordered because spring will use the first factory supporting the method it gets across. And therefore the order of an instance of this class has to have higher precedence as the order value 50 of TransactionEventListenerFactory.
I had simmilar problem with code as below
#Transactional(propagation = Propagation.REQUIRES_NEW)
public class SomeListenerFacade {
#TransactionalEventListener
public void onSomething(SomeEvent event) {
throw new RuntimeException("some cause");
}
}
I followed your solution. It worked. On the way I've found an alternative way for at least seeing that exception in the logfile
# application.properties
logging.level.org.springframework.transaction.support.TransactionSynchronizationUtils = DEBUG
tldr: Is there a way to make an internal request (using the method's path) without going to the internet?
--
Why do I need it? I have a project which receives many events. The decision of who will handle each event is made by a Controller. So I have something similar to this:
#RestController
#RequestMapping("/events")
public class EventHandlerAPI {
#Autowired
private EventAHandler eventAhandler;
#Autowired
private EventBHandler eventBhandler;
#PostMapping("/a")
public void handleEventA(#RequestBody EventA event) {
eventAhandler.handle(id, event);
}
#PostMapping("/b")
public void handleEventB(#RequestBody EventB event) {
eventBhandler.handle(id, event);
}
}
We recently added support to receive events through a Queue service. It sends to us the payload and the event class. Our decision is to let both interfaces working (rest and queue). The solution to avoid code duplication was to keep the Controller choosing which handler will take care of the event. The code nowadays is similar to this:
#Configuration
public class EventHandlerQueueConsumer {
#Autowired
private EventHandlerAPI eventHandlerAPI;
private Map<Class, EventHandler> eventHandlers;
#PostConstruct
public void init() {
/* start listen queue */
declareEventHandlers();
}
private void declareEventHandlers() {
eventHandlers = new HashMap<>();
eventHandlers.put(EventAHandler.class, (EventHandler<EventAHandler>) eventHandlerAPI::handleEventA);
eventHandlers.put(EventBHandler.class, (EventHandler<EventBHandler>) eventHandlerAPI::handleEventB);
}
private void onEventReceived(AbstractEvent event) {
EventHandler eventHandler = eventHandlers.get(event.getClass());
eventHandler.handle(event);
}
private interface EventHandler<T extends AbstractEvent> {
void handle(T event);
}
}
This code works, but it doesn't let the controller choose who will handle the event (our intention). The decision is actually being made by the map.
What I would like to do was to invoke the controller method through it's request mapping without going to the internet. Something like this:
#Configuration
public class EventHandlerQueueConsumer {
// MADE UP CLASS TO SHOW WHAT I WANT
#Autowired
private ControllerInkover controllerInvoker;
#PostConstruct
public void init() { /* start listen queue */ }
private void onEventReceived(AbstractEvent event) {
controllerInvoker.post(event.getPath(), new Object[] { event });
}
}
This way is much cleaner and let all the decisions be made by the controller.
I've researched a lot and didn't found a way to implement it. Debugging spring, I found how he routes the request after the DispatcherServlet, but all the spring internals uses HttpServletRequest and HttpServletResponse :(
Is there a way to make an internal request (using the method's path) without going to the internet?
They are classes of the same application
Then it should easy enough.
1) You can call your own API on http(s)://localhost:{port}/api/{path} using RestTemplate utility class. This is preferred way, since you'll follow standard MVC pattern. Something like:
restTemplate.exchange(uri, HttpMethod.POST, httpEntity, ResponseClass.class);
2) If you don't want to invoke network connection at all, then you can either use Spring's internal to find the mapping/method map or use some reflection to build custom
map upon controller's startup. Then you can pass your event/object to the method from the map in a way shown in your mock-up class. Something like:
#RequestMapping("foo")
public void fooMethod() {
System.out.println("mapping = " + getMapping("fooMethod")); // you can get all methods/mapping in #PostContruct initialization phase
}
private String getMapping(String methodName) {
Method methods[] = this.getClass().getMethods();
for (int i = 0; i < methods.length; i++) {
if (methods[i].getName() == methodName) {
String mapping[] = methods[i].getAnnotation(RequestMapping.class).value();
if (mapping.length > 0) {
return mapping[mapping.length - 1];
}
}
}
return null;
}