I need help. I wish to change in runtime the itemWriter based on processor result
#Configuration
public class StepConfig {
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Bean
public Step step(
ItemReader<Sample> itemReader,
ItemProcessor<Sample, Sample> beanProcessor,
ItemWriter<Sample> itemWriter
) {
return stepBuilderFactory.get("our step")
.<Sample, Sample>chunk(1)
.reader(itemReader)
.processor(beanProcessor).listener(new ProcessorListener())
.writer(itemWriter)
.build();
}
}
My processor is a bean validation processor. If the validation succeed, I need a success writer. If it throws exception, I need to execute a exception writer.
How can I do that?
Thanks!
I did that:
I created 2 writers and tie them with a CompositeItemWriter
I pass a DTO with reference to the error for failure or/and the reference data processed with success.
The DTO is only for passing through processor to writers.
Each writer check if DTO has your respective data. If success data is there, work success data, otherwise return. If error data is there, work error data, otherwise return. CompositeItemWriter will always call all writers inside of it.
To be clear, in my case I had a reader that read information from a table. A pool of processors that processed the information, and at the end, populated a DTO. The DTO was only for take information from the processor to the CompositeItemWriter. The DTO had reference to two objects, one for success and another for error. Each writer checked its field on the DTO.
Maybe is possible to define Spring Batch Flow for your case as well. It would be the most elegant way, but you have to study the documentation to see the best way to implement it.
I have solved the problem after days!
First, I got a Classifier Writer. It decide which writer choose based in
my object property.
I highlight that before the classifier, I had to deal with the main problem, which was the BeanValidation processor.
As a bean validation processor, it works with just :
#Bean
public ItemProcessor<Sample,Sample> validate() {
return new BeanValidatingItemProcessor<>();
}
But I need to know if the validation will fail. So I tryed use a try/catch inside the method above and use the .process(sample) to see that. Inexplicably the method throws NPE.
I solved this making a bean:
#Configuration
public class Beans {
#Bean
public ValidatingItemProcessor<Sample> validatorBean() {
return new BeanValidatingItemProcessor<Sample>();
}
}
and injecting it in my processor. Finally it works.
Thank you all.
Related
Considering the snippet below - is it safe to pass simple not thread-safe HashMap to a SomeHandlerFunction that will use it in other autowired beans in SomeHandlerFunction?
#Bean
RouterFunction<ServerResponse> request(SomeHandlerFunction handlerFunction) {
handlerFunction.setSimpleMapProperties(new HashMap());
return route(GET("/dummy"), handlerFunction);
}
#Service
class SomeHandlerFunction implements HandlerFunction {
#Autowired
List<AnotherBeans> anotherBeans;
Mono<T> handle(ServerRequest var1) {
// code...
}
}
I'm a bit aware about multithreading model in WebFlux, but this case made me confusing.
If that HashMap is mutable, it's not a good idea - not even with Spring MVC.
What should you do instead? It really depends on the use case at hand.
If you're trying to setup a local in-memory cache, you could use a library like Caffeine, or simply a ConcurrentMapCache.
If you'd like to share configuration keys with that bean, you could wrap that map with Collections.unmodifiableMap or even better, make that a proper immutable POJO.
If this is meant to carry temporary data that's linked with the request being processed, you should instead use request attributes or Reactor Context for that.
I've been trying to implement #PostConstruct and #PreDestroy methods in an Account class of mine. Neither of them worked in the following situation, but for the sake of brevity, I'll only talk about #PostConstruct.
I'm using Spring Batch's readers to load these accounts from a fixed-length file. So far, so good, except when my reader creates these accounts, it apparently does not call the #PostConstruct method (debug breakpoints are never activated, and log messages are not printed out).
The reader is only custom in the sense that it's a custom class extending FlatFileItemReader<Account> and setting values in the constructor.
Adding the exact same initialization method (that never got called in the Account class) to the reader itself works just fine.
I.e. if the #PostConstruct method should be called upon reader initialization, it works. Just not when the reader itself initializes accounts annotated with #PostConstruct.
If I add a breakpoint or log message in the constructor of Account directly, it also works without an issue.
Is this desired behaviour by Spring Batch? Or could this be caused by any configuration of mine?
Another question's answer mentioned that annotations such as #PostConstruct "only apply to container-managed beans", not if "you are simply calling new BlogEntryDao() yourself".
Is this what's happening here - Spring Batch calling new Account(...) directly, without registering them in the container? After all, I never have these accounts available as beans or anything.
Is your Account class annotated with #Component, #Bean or #Service? If you create objects of account class like Account c = new Account() the nSpring doesn't know about creation of such objects. Because of that Spring doesn't call method annotated with #postConstruct
when my reader creates these accounts, it apparently does not call the #PostConstruct method
#PostConstruct and #PreDestroy methods are called by the Spring container after creating and before destroying the instance of your bean. If your object is not managed by Spring, those methods will not be called.
I'm using Spring Batch's readers to load these accounts from a fixed-length file
In this case, you should have already configured a FieldSetMapper to map fields to an Account instance. If you use the BeanWrapperFieldSetMapper, you can set the PrototypeBeanName which is the name of a bean (for example of type Account) of scope prototype (so that an instance is created for each line). This way, Account instances will be managed by Spring, used by Spring Batch reader, and your method annotated with PostConstruct will be called. Here is an example:
#Bean
#Scope("prototype")
public Account account() {
return new Account();
}
#Bean
public BeanWrapperFieldSetMapper<Account> beanMapper() {
BeanWrapperFieldSetMapper<Account> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
fieldSetMapper.setPrototypeBeanName("account");
return fieldSetMapper;
}
#Bean
public FlatFileItemReader<Account> accountReader() {
return new FlatFileItemReaderBuilder<Account>()
.name("accountsItemReader")
.resource(new ClassPathResource("accounts.txt"))
.fixedLength()
.columns(new Range[] {new Range(1, 1), new Range(2, 4)})
.names(new String[]{"id", "name"})
.fieldSetMapper(beanMapper())
.build();
}
More details about this in the Javadoc.
Hope this helps.
I'm using spring boot. I was new to spring and started a spring project. So I didn't know about pre defined repositories (JPA, CRUD) which can be easily implemented. In case, I wanted to save a bulk data, so I use for loop and save one by one, Its taking more time. So I tried to use #Async. But it doesn't also work, is my concept wrong?
#Async has two limitation
it must be applied to public methods only
self-invocation – calling the async method from within the same class won’t work
1) Controller
for(i=0;i < array.length();i++){
// Other codes
gaugeCategoryService.saveOrUpdate(getEditCategory);
}
2) Dao implementation
#Repository
public class GaugeCategoryDaoImpl implements GaugeCategoryDao {
// Other codings
#Async
#Override
public void saveOrUpdate(GaugeCategory GaugeCategory) {
sessionFactory.getCurrentSession().saveOrUpdate(GaugeCategory);
}
}
After removing #Async , it working normally. But with that annotation it doesn't work. Is there any alternative method for time consuming? Thanks in advance.
the #Async annotation creates a thread for every time you call that method. but you need to enable it in your class using this annotation #EnableAsync
You also need to configure the asyncExecutor Bean.
You can find more details here : https://spring.io/guides/gs/async-method/
In my opinion, there are several issues with your code:
You overwrite the saveOrUpdate() method without any need to do so. A simple call to "super()" should have been enough to make #Async work.
I guess that you somewhere (within your controller class?) declare a transactional context. That one usually applies to the current thread. By using #Async, you might leave this transaction context as (because of the async DAO execution), the main thread may already be finished when saveOrUpdate() is called. And even though I currently don't know it exactly, there is a good change that the declared transaction is only valid for the current thread.
One possble fix: create an additional component like AsyncGaugeCategoryService or so like this:
#Component
public class AsyncGaugeCategoryService {
private final GaugeCategoryDao gaugeCategoryDao;
#Autowired
public AsyncGaugeCategoryService(GaugeCategoryDao gaugeCategoryDao) {
this.gaugeCategoryDao = gaugeCategoryDao;
}
#Async
#Transactional
public void saveOrUpdate(GaugeCategory gaugeCategory) {
gaugeCategoryDao.saveOrUpdate(gaugeCategory);
}
}
Then inject the service instead of the DAO into your controller class. This way, you don't need to overwrite any methods, and you should have a valid transactional context within your async thread.
But be warned that your execution flow won't give you any hint if something goes wrong while storing into the database. You'll have to check the log files to detect any problems.
I've been struggling for a few days to get this working, but it seems that I cannot find a solution to it. That's why I'd like to ask it here.
Short version
I have a multi tenant implementation which works with Spring boot, Spring Data JPA and Hibernate. This works like a charm. But now I'd like to implement a functionality where I switch the database (data source) during a single transaction. For example I use similar code in my service class
#Autowired
private CustomRepository customRepository;
#Autorwired
private CustomTenantIdentifierResolver customResolver;
#Transactional
public Custom getCustom(String name) {
// Set the datasource to "one";
this.customResolver.setIdentifier("one");
Custom result = this.customRepository.findOneByName(name);
//If the result is null, switch datasource to default and try again
this.customResolver.setIdentifier("default");
result = this.customRepository.findOneByName(name);
return result;
}
The problem is, my data source does not switch. It uses the same source for the second request. I guess I'm doing something terribly wrong here.
What is the correct way to switch the data source during a single transaction?
EDIT (07-06-2016)
Since I noticed that switching the data source for a single transaction is not going to work, I'll add a followup.
Would it be possible to switch the data source in between two transactions for a single user request? If so, what would be the correct way to do this?
Long Version
Before moving on, I'd like to mention that my multi tenant implementation is based on the tutorial provided on this blog.
Now, my goal is to use the default data source as a fallback when the dynamic one (chosen by a custom identifier) fails to find a result. All this needs to be done in a single user request. It doesn't make a difference in the solution uses a single or multiple transactional annotated methods.
Until now I tried a couple things, one of them is described above, another includes the use of multiple transaction managers. That implementation uses a configuration file to create two transaction manager beans which each a different data source.
#Configuration
#EnableTransactionManagement
public class TransactionConfig {
#Autowired
private EntityManagerFactory entityManagerFactory;
#Autowired
private DataSourceProvider dataSourceProvider;
#Bean(name = "defaultTransactionManager")
public PlatformTransactionManager defaultTransactionManager() {
JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(entityManagerFactory);
jpaTransactionManager.setDataSource(dataSourceProvider.getDefaultDataSource());
jpaTransactionManager.afterPropertiesSet();
return jpaTransactionManager;
}
#Bean(name = "dynamicTransactionManager")
public PlatformTransactionManager dynamicTransactionManager() {
JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(entityManagerFactory);
jpaTransactionManager.afterPropertiesSet();
return jpaTransactionManager;
}
}
Next I split the service method into two separate ones and added the #Transactional annotation including the right bean name
#Transactional("dynamicTransactionManager")
public Custom getDynamicCustom(String name) {
...stuff...
}
#Transactional("defaultTransactionManager")
public Custom getDefaultCustom(String name) {
...stuff...
}
But it didn't make any difference, the first data source was still used for the second method call (which should use the default transaction manager).
I hope someone can help me find a solution to this.
Thanks in advance.
Spring provides a variation of DataSource, called AbstractRoutingDatasource. It can be used in place of standard DataSource implementations and enables a mechanism to determine which concrete DataSource to use for each operation at runtime. All you need to do is to extend it and to provide an implementation of an abstract determineCurrentLookupKey method.
Keep in mind that determineCurrentLookupKey method will be called whenever TransactionsManager requests a connection. So, if you want to switch DataSource, you just need to open new transaction.
You can find example here
http://fedulov.website/2015/10/14/dynamic-datasource-routing-with-spring/
You can't just move a transaction over to another datasource. While there is a concept of distributed (or XA) transactions, it consists of separate transactions (in separate data sources) that are treated as if they were part of a single (distributed) transaction.
I do not know if it is possible, but i think you should try to avoid switching source during a transaction, for the following reason:
If an error occurs during the second request you will want to roll back the entire transaction, which means switching back to the old source. In order to be able to do that you will need to hold an open connection to that old source: When the transaction is complete you will need to confirm the transaction to that old source.
I would recommend to rethink if you really want this, beside the point if it is possible at all.
I'm using Spring-Batch v3.0.0 for batch imports. There is a StepScope and a JobScope. How can I know which of them is appropriate?
For example, if I define a custom ItemReader or ItemWriter that should use a specific EntityManager, it could look like this:
#Bean
#Scope("step") //#Scope("job") //custom scope required to inject #jobParameters
public JpaItemWriter<T> jpaItemWriter(EntityManagerFactory emf) {
JpaItemWriter<T> writer = new JpaItemWriter<T>();
writer.setEntityManagerFactory(emf);
return writer;
}
But which scope is right here? And why?
Execution with step scope works, but I feel the itemWriters should maybe be of job scope so that they are not recreated on every step.
I tried switching step to job, but that throws following error:
Exception in thread "main" java.lang.IllegalStateException: No Scope registered for scope 'job'
Since Spring-Batch v3.0.1 you can use #JobScope
Marking a #Bean as #JobScope is equivalent to marking it as #Scope(value="job", proxyMode=TARGET_CLASS)
Got it: one has to provide the scope as a bean explicit within the #Configuration file.
#Bean
public JobScope jobScope() {
return new JobScope();
}