Develop Reference

how to skip blank lines while writing to csv file with Spring Batch FlatFileItemWriter

I am trying to write the java data objects to .csv file using the Spring Batch FlatFifileItemWriter exactly as mentioned in the example at below location. http://www.mkyong.com/spring-batch/spring-batch-example-xml-file-to-csv-file/
The writer configuration is like below.
<bean id="cvsFileItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
<!-- write to this csv file -->
<property name="resource" value="file:cvs/report.csv" />
<property name="shouldDeleteIfExists" value="true" />
<property name="lineAggregator">
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter" value="," />
<property name="fieldExtractor">
<bean
class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<property name="names" value="refId, name, age, csvDob, income" />
</bean>
</property>
</bean>
</property>
</bean>
Since the FlatFileItemWriter has following write() method definition in the API, even though the LineAggregator returns empty String,it still writes to file.
StringBuilder lines = new StringBuilder();
int lineCount = 0;
for (T item : items) {
lines.append(lineAggregator.aggregate(item) + lineSeparator);
lineCount++;
}
try {
state.write(lines.toString());
}
is there any way to configure it to skip if the line is blank while writing to csv file in java using spring batch.

You'll want to add an ItemProcessor<YourObject> to your step. In it, you can add a simple YourObject process(YourObject item) method that checks if the object will result in a blank line. If not, return the object. If so, return null.
This will prevent the items you wish to discard from getting to the writer and will increase the FILTER_COUNT when you look at the BATCH_STEP_EXECUTION table for that Step.
More on filtering objects can be found in the Spring Batch docs.
Edit: The tutorial you referenced in your answer includes an example of a filtering processor (FilterReportProcessor) as well.

if memberclass is empty here's another way to writer using composite writer.
Create another writer as below.
public class YourMemberFileItemWriter extends FlatFileItemWriter<YourMainClass>{
#Override
public String doWrite(List<? extends YourMainClass> items) {
StringBuilder lines = new StringBuilder();
for (YourMainClass item : items) {
if ( YourMainClass.getYourMemberClass() != null) {
lines.append(this.lineAggregator.aggregate(item)).append(this.lineSeparator);
}
}
return lines.toString();
}
}
Define your composite writer.
CompositeItemWriter compositeItemWriter = new CompositeItemWriter<>();
compositeItemWriter.setDelegates(Arrays.asList( mainClassCsv()
,memberClassCsv());
return compositeItemWriter;
Add member class methods as below to composite writer:
public FlatFileItemWriter memberClassCsv() {
// TODO Auto-generated method stub
YourMemberFileItemWriter fileWriter = new YourMemberFileItemWriter();
fileWriter.setName("yourMemberClass");
fileWriter.setResource(new FileSystemResource("memberfile.csv"));
fileWriter.setLineAggregator(getMemberClassDelimitedLineAggregator());
return fileWriter;
}
public DelimitedLineAggregator<YourMainClass> getMemberClassDelimitedLineAggregator() {
BeanWrapperFieldExtractor<YourMainClass> beanWrapperFieldExtractor = new BeanWrapperFieldExtractor<YourMainClass>();
beanWrapperFieldExtractor.setNames(new String[] {
"memberClassField.fieldname1"
,"memberClassField.fieldname2"
});
DelimitedLineAggregator<YourMainClass> delimitedLineAggregator = new DelimitedLineAggregator<YourMainClass>();
delimitedLineAggregator.setDelimiter(",");
delimitedLineAggregator.setFieldExtractor(beanWrapperFieldExtractor);
return delimitedLineAggregator;
}

How to handle ThreadPool/ExecutorService in Spring

I want to handle multi Thread in spring mvn model. I have written this code
#RequestMapping("/indialCall")
#ResponseBody
public String indialCall(HttpServletRequest request) {
String result = "FAIL";
try {
Map<String, String> paramList = commonUtilities.getParamList(request);
logger.info("indialCall paramList :::" + paramList);
// System.out.println("indial-call paramList :::" + paramList);
result = inDialHandler.processIndialWork(paramList);
logger.info(result);
} catch (Exception e) {
logger.info("Error :" + e);
}
return result;
}
public String processIndialWork(final Map<String, String> paramList) {
final Boolean sendSms = Boolean.parseBoolean(paramList.get(constantService.getSendSms()));
//assume it is always true
if(true){
Thread thread = new Thread(new Runnable() {
#Override
public void run() {
String sessionId = (String) paramList.get(constantService.getInDialSession());
String msisdn = (String) paramList.get(constantService.getInDialMsisdn());
//This method will save the entry into database
saveMissedCall(callStartDate, sessionId, msisdn, vmnNo, advId, enterpriseId, sendSms, advertiser);
}
});
thread.start();
return "1";
}
}
In this code i am using thread creation on every http request. Which is not good for my case.
because system get 50 request /sec. and when i see the cpu usage it is too high.
I am calling this thread for async communication so that calling party can get response instantly
and later on application do the further processing.
I want to use the Executor service but do not know how to do this. Can some one guide me or can write
few line of code for me to implment the correct thread pool executor.

First define a simple taskExecutor in your config file.
<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="10" />
<property name="WaitForTasksToCompleteOnShutdown" value="true" />
</bean>
Create spring bean with prototype scope (prototype is important as you want to give each thread different data), and they
will run simultaneously.
This spring bean will implement runnable with run method and will have class level variable paramlist for getting vales.
public class MyRunnableBean implements runnable{
private Map<String, String> paramList();
// add setter
public void run(){
// your logic
}
}
Inject task executor (singleton) in your existin bean , get the instances of this runnable bean in your existing bean set the paramlist and add it in executor :-
MyRunnableBEan myRunnableBEan = applicationContext.getBean("myRunnable");
myRunnableBean.setParamList(/* your paramlist*/ );
taskExecutor.execute(myRunnableBean);
Correct the compilation and syntax error, this sample code written on notepad, i don't have java on my machine.

wrap spring bean with another bean

I have some service bean which is accessible by identifier someSpecificService which I need to modify.
Beans are defined in different xml files and are collected together in runtime. So one big xml file is created where all these xmls are imported:
context.xml
....
<import path="spring1.xml" />
<import path="spring2.xml" />
...
So there is following configuration:
<-- definitions from spring1.xml -->
<alias name="defaultSomeSpecificService" alias="someSpecificService" />
<bean id="defaultSomeSpecificService" class="..."/>
....
<!-- definitions from spring2.xml -->
<alias name="myOwnSomeSpecificService" alias="someSpecificService" />
<bean id="myOwnSomeSpecificService" class="..." /> <!-- how to inject previously defined someSpecificService into this new bean? -->
I would like to override someSpecificService from spring1.xml in spring2.xml, however I do need to inject previously defined bean defaultSomeSpecificService and all I know is its alias name someSpecificService which I need to redefine to new bean myOwnSomeSpecificService.
Is it possible to implement?

One solution would be to avoid trying to override the definition, by creating a proxy for the service implementation to intercept all calls towards it.
1) For the sake of the example, suppose the service would be something like:
public interface Service {
public String run();
}
public class ExistingServiceImpl implements Service {
#Override
public String run() {
throw new IllegalStateException("Muahahahaha!");
}
}
2) Implement an interceptor instead of myOwnSomeSpecificService:
import org.aopalliance.intercept.MethodInterceptor;
import org.aopalliance.intercept.MethodInvocation;
public class SomeSpecificServiceInterceptor implements MethodInterceptor {
#Override
public Object invoke(MethodInvocation invocation) throws Throwable {
String status;
try {
// allow the original invocation to actually execute
status = String.valueOf(invocation.proceed());
} catch (IllegalStateException e) {
System.out.println("Existing service threw the following exception [" + e.getMessage() + "]");
status = "FAIL";
}
return status;
}
}
3) In spring2.xml define the proxy creator and the interceptor:
<bean id="serviceInterceptor" class="com.nsn.SomeSpecificServiceInterceptor" />
<bean id="proxyCreator" class="org.springframework.aop.framework.autoproxy.BeanNameAutoProxyCreator">
<property name="beanNames" value="someSpecificService"/>
<property name="interceptorNames">
<list>
<value>serviceInterceptor</value>
</list>
</property>
</bean>
4) Running a small example such as:
public class Main {
public static void main(String[] args) {
Service service = new ClassPathXmlApplicationContext("context.xml").getBean("someSpecificService", Service.class);
System.out.println("Service execution status [" + service.run() + "]");
}
}
... instead of the IllegalStateException stacktrace you'd normally expect, it will print:
Existing service threw the following exception [Muahahahaha!]
Service execution status [FAIL]
Please note that in this example the service instance is not injected in the interceptor as you asked because I had no user for it. However should you really need it, you can easily inject it via constructor/property/etc because the interceptor is a spring bean itself.

Spring Batch ItemReader list processed only once

I'm trying to create a Spring Batch job using a ListItemReader<String>, ItemProcessor<String, String> and ItemWriter<String>.
The XML looks like the following,
<job id="sourceJob" xmlns="http://www.springframework.org/schema/batch">
<step id="step1" next="step2">
<tasklet>
<chunk reader="svnSourceItemReader"
processor="metadataItemProcessor"
writer="metadataItemWriter"
commit-interval="1" />
</tasklet>
</step>
<step id="step2">
<tasklet ref="lastRevisionLoggerTasklet"></tasklet>
</step>
</job>
<bean id="svnSourceItemReader"
class="com.example.repository.batch.SvnSourceItemReader"
scope="prototype">
<constructor-arg index="0">
<list>
<value>doc1.xkbml</value>
<value>doc2.xkbml</value>
<value>doc3.xkbml</value>
</list>
</constructor-arg>
</bean>
<bean id="metadataItemProcessor"
class="com.example.repository.batch.MetadataItemProcessor"
scope="prototype" />
<bean id="metadataItemWriter"
class="com.example.repository.batch.MetadataItemWriter"
scope="prototype" />
The reader, processor and writer are vanilla,
public class SvnSourceItemReader extends ListItemReader<String> {
public SvnSourceItemReader(List<String> list) {
super(list);
System.out.println("Reading data list " + list);
}
#Override
public String read() {
String out = (String) super.read();
System.out.println("Reading data " + out);
return out;
}
}
public class MetadataItemProcessor implements ItemProcessor<String, String> {
#Override
public String process(String i) throws Exception {
System.out.println("Processing " + i + " : documentId " + documentId);
return i;
}
}
public class MetadataItemWriter implements ItemWriter<String> {
#Override
public void write(List<? extends String> list) throws Exception {
System.out.println("Writing " + list);
}
}
The job is started like this, but on a schedule of every 10 seconds.
long nanoBits = System.nanoTime() % 1000000L;
if (nanoBits < 0) {
nanoBits *= -1;
}
String dateParam = new Date().toString() + System.currentTimeMillis()
+ "." + nanoBits;
param = new JobParametersBuilder().addString("date", dateParam)
.toJobParameters();
JobExecution execution = jobLauncher.run(job, param);
When the application starts, I see it read, process and write each of the three items in the list passed to the reader.
Reading data doc1.xkbml
Processing doc1.xkbml : documentId doc1
Writing [doc1.xkbml]
Reading data doc2.xkbml
Processing doc2.xkbml : documentId doc2
Writing [doc2.xkbml]
Reading data doc3.xkbml
Processing doc3.xkbml : documentId doc3
Writing [doc3.xkbml]
Because this sourceJob is on a scheduled timer, every 10 seconds I expected to see that list processed, but instead I see on all subsequent runs.
Reading data null
Does anyone know why this is happening? I'm new to Spring Batch and just can't get my hands around the issue.
Thanks /w

The problem is that you marked your reader as scope="prototype". It should be scope="step".
In Spring-batch there are only two scopes: singleton (the default) and step.
From the javadoc:
StepScope: Scope for step context. Objects in this scope use the
Spring container as an object factory, so there is only one instance
of such a bean per executing step. All objects in this scope are
(no need to decorate the bean definitions).
and
Using a scope of Step is required in order to use late binding since
the bean cannot actually be instantiated until the Step starts, which
allows the attributes to be found.
During the Spring context startup look at your log and you will see this line:
INFO: Done executing SQL script from class path resource
[org/springframework/batch/core/schema-hsqldb.sql] in 9 ms.
Reading data list [doc1.xkbml, doc2.xkbml, doc3.xkbml]
as you can see your reader has already been created and managed as a singleton; dynamic beans in spring-batch context should be managed with the special step scope so that Spring will create a fresh copy of the bean every time a step is executed.
In your reader, ListItemReader.read() is written as:
public T read() {
if (!list.isEmpty()) {
return list.remove(0);
}
return null;
}
In each read items are removed from original list! The reader is constructed once and, on second job execution, the list is empty!

Just an additional information: you can also use JavaConfig instead of the xml config file, and annotate the reader bean declaration with #StepConfig.
ex:
#Configuration
#EnableBatchProcessing
public class MyConfig {
...
#Bean
#StepScope
public ItemReader<HeadingBreakevenAssociation> readerHeadingBreakevenAssociationList(){
ItemReader<Person> itemReader = new ListItemReader<Person>(myList);
return itemReader;
}
}

Spring JDBC DAO

Im learning Spring (2 and 3) and i got this method in a ClientDao
public Client getClient(int id) {
List<Client> clients= getSimpleJdbcTemplate().query(
CLIENT_GET,
new RowMapper<Client>() {
public Client mapRow(ResultSet rs, int rowNum) throws SQLException {
Client client = new ClientImpl(); // !! this (1)
client.setAccounts(new HashSet<Account>()); // !! this (2)
client.setId(rs.getInt(1));
client.setName(rs.getString(2));
return client;
}
},id
);
return clients.get(0);
}
and the following Spring wiring:
<bean id="account" class="client.AccountRON" scope="prototype">
<property name="currency" value = "RON" />
<property name="ammount" value="0" />
</bean>
<bean id="client" class="client.ClientImpl" scope="prototype">
<property name="name" value="--client--" />
<property name="accounts">
<set>
</set>
</property>
</bean>
The things is that i dont like the commented lines of java code (1) and (2).
I'm going to start with (2) which i think is the easy one: is there a way i can wire the bean in the .xml file to tell spring to instantiate a set implementation for the 'accounts' set in ClientImpl? so i can get rid of (2)
Now moving on to (1): what happens if the implementation changes ? do i really need to write another DAO for a different implementation? or do i have to construct a BeanFactory ? or is there another more beautiful solution ?
Thanks!

I'm a bit confused here - why have you defined a ClientImpl bean in your XML, but not using it in your Java?
Your already have most of the solution, you just need to fetch a new ClientImpl from Spring each iterations through the loop:
private #Autowired BeanFactory beanFactory;
public Client getClient(int id) {
List<Client> clients= getSimpleJdbcTemplate().query(
CLIENT_GET,
new RowMapper<Client>() {
public Client mapRow(ResultSet rs, int rowNum) throws SQLException {
Client client = beanFactory.getBean(Client.class);
client.setId(rs.getInt(1));
client.setName(rs.getString(2));
return client;
}
},id
);
return clients.get(0);
}
With this approach, the actual construction and initialization of ClientImpl is done by Spring, not your code.