I need to add the aliases defined in the SQL query while generating the CSV file.
I see some example using FlatFileHeaderCallback but there I don't have a way to pass the aliases
is there any way to get the column aliases in write(List<? extends T> items) method of FlatFileItemWriter
For starters, I think you could simply use a custom FlatFileHeaderCallback which takes a String as a parameter and writes it :
public class CustomHeaderWriter implements FlatFileHeaderCallback {
private String header;
#Override
public void writeHeader(Writer writer) throws IOException {
writer.write(header);
}
public void setHeader(String header) {
this.header = header;
}
}
To use it, declare it in your FlatFileItemWriter and give it a String that contains the name of your columns/aliases separated by your flat file delimiter :
<bean class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
<property name="headerCallback">
<bean class="xx.xx.xx.CustomHeaderWriter">
<property name="header" value="${columns.or.aliases}"></property>
</bean>
</property>
</bean>
Now, I suppose you don't want to write the columns/aliases a second time for the header, and would like to "extract" them from the SQL query. This could be accomplished for example by fiddling with the CustomHeaderWriter :
Instead of passing the columns/aliases directly, you could give it the actual SQL query
Using a Regular Expression or manual parsing, you could then extract the aliases or the columns names (strings beween SELECT and FROM, split with ,, strip quotes, etc.)
You would then need to pass (or use a constant) the delimiter of the FlatFileItemWriter
Finally, write the String you just built
Create a custom class(assuming 5 csv columns):
public class MyFlatFileWriter implements FlatFileHeaderCallback {
#Override
public void writeHeader(Writer writer) throws IOException {
writer.write("Col1,Col2,Col3,Col4,Col5");
}
}
Add bean & its reference in writer bean:
<bean id="flatFileWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
<property name="resource" value="file:csv/outputs/name.csv" />
<property name="headerCallback" ref="headerCallback" />
<property name="lineAggregator">
............
</property>
</bean>
<bean id="headerCallback" class="com.whatever.model.MyFlatFileWriter" />
Related
I have a class
public class DataStore {
public String name;
public String username;
public String password;
public String token;
public String connectionString;
public String type;
public String scheme;
public boolean usesBasicAuth;
public boolean usesBearerAuth;
}
I need to create an bean for it in another project. But i need to fill the fields somehow. The problem is I can not use <constructor-arg ... /> because there is no constructor.
The code below results in BeanCreationException: "Could not resolve matching constructor"
<bean id="dataStore"
class="com.fressnapf.sdk.dataaccess.services.DataStore">
<constructor-arg index="0" value="${spring.datastore.name}"/>
...
</bean>
Assuming, You have public (getters and setters for your) properties, and only the "default (no args) constructor", then You can change your configuration to:
<bean id="dataStore" class="com.fressnapf.sdk.dataaccess.services.DataStore">
<property name="connectionString" value="..."/>
<!-- ... -->
</bean>
Using property instead of constructor-arg.
Docs (Spring 4.2): https://docs.spring.io/spring/docs/4.2.x/spring-framework-reference/html/xsd-configuration.html
Yes, there is a constructor.
If you don't explicitly put one, Java will automatically add a default (non argument) constructor.
Use that one.
It will set all the instance variables to the default value of their type.
In your case: null for every String variable, false for every boolean.
I am using SpringBatch to read from Oracle and write into ElasticSearch.
My code works fine for static queries.
Example: select emp_id, emp_name from employee_table I have a RowMapper class that maps the values from resultSet with the Employee POJO.
My requirement is
The query will be input by the user. So the query might be as follows
select emp_id, emp_name from employee_table
select cust_id, cust_name, cust_age from customer_table
select door_no, street_name, loc_name, city from address_table
Similar queries
My questions are
Is there a way to dynamically create a POJO according to the query given by the user?
Will the RowMapper concept work if the query keeps changing as in my case?
Is there something like a generic rowmapper?
Sample code would be much appreciated.
If you have objects you need to map to...
Consider aliasing your SQL to match your object field names using a custom implementation of RowMapper which actually extends BeanWrapperFieldSetMapper
So if your POJO looks like this:
public class Employee {
private String employeeId;
private String employeeName;
...
// getters and setters
}
Then your SQL can look like this:
SELECT emp_id employeeId, emp_name employeeName from employee_table
Then your wrapped RowMapper would look something like this:
import org.springframework.jdbc.core.RowMapper
import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper
public class BeanWrapperRowMapper<T> extends BeanWrapperFieldSetMapper<T> implements RowMapper<T> {
#Override
public T mapRow(final ResultSet rs, final int rowNum) throws SQLException {
final FieldSet fs = getFieldSet(rs);
try {
return super.mapFieldSet(fs);
} catch (final BindException e) {
throw new IllegalArgumentException("Could not bind bean to FieldSet", e);
}
}
private FieldSet getFieldSet(final ResultSet rs) throws SQLException {
final ResultSetMetaData metaData = rs.getMetaData();
final int columnCount = metaData.getColumnCount();
final List<String> tokens = new ArrayList<>();
final List<String> names = new ArrayList<>();
for (int i = 1; i <= columnCount; i++) {
tokens.add(rs.getString(i));
names.add(metaData.getColumnName(i));
}
return new DefaultFieldSet(tokens.toArray(new String[0]), names.toArray(new String[0]));
}
}
Alternatively...
If you don't have any POJOs to map to, use the out-of-box ColumnMapRowMapper to get get back a map (Map<String,Object>) of column names (let's call them COL_A, COL_B, COL_C) to values. Then if your writer is something like a JdbcBatchItemWriter you can set your named parameters as:
INSERT TO ${schema}.TARGET_TABLE (COL_1, COL_2, COL_3) values (:COL_A, :COL_B, :COL_C)
and then your ItemSqlParameterSourceProvider implementation could look like so:
public class MapItemSqlParameterSourceProvider implements
ItemSqlParameterSourceProvider<Map<String, Object>> {
public SqlParameterSource createSqlParameterSource(Map<String, Object> item) {
return new MapSqlParameterSource(item);
}
}
To answer your questions:
Is there a way to dynamically create a POJO based on the user's query - Even if there was, I'm not sure how much help it would be. For your use case, I'd suggest just using a Map.
Will the RowMapper concept work if the query keeps changing - If you use a Map, you can use the column names as the keys and the column values as the values. You should be able to create a RowMapper implementation that can do this.
Is there something like a generic RowMapper - There is but it's intended for POJO's so you'd need to create your own for this.
You can do it simply like below,
SettingsDto settings = SettingsDao.getById(1, new BeanPropertyRowMapper<>(SettingsDto.class));
In a generic way, you can pass your DTO class, but please note you have to use same name as SQL columns or have to use ALIAS in SQL query according to the DTO.
#Data
public class SettingsDto {
private int id;
private int retryCount;
private int batchSize;
private int retryPeriod;
private int statusInitialDelay;
}
My dao method is below
SettingsDto getById(int id, final RowMapper<OMoneySettingsDto> mapper);
its implementation is below,
#Override
public SettingsDto getById(final int id, final RowMapper<OMoneySettingsDto> mapper) {
return new JdbcTemplate(YourDataSource).queryForObject(QUERY_SETTINGS_BY_ID,new Object[]{id}, mapper);
}
SQL is here, as below you have to use same name in the DTO
private static final String OMONEY_SETTINGS_BY_ID = "SELECT AS id,retry_count AS retryCount FROM setttings WHERE id = ?";
I found a solution to my problem by using Spring's ColumnMapRowMapper. Please find a snippet from the xml configuration file. I didn't generate any POJO class. I managed with a Map and inserted the same into ES. The map's key name should match with the field names present in index.
<step id="slave" xmlns="http://www.springframework.org/schema/batch">
<tasklet>
<chunk reader="pagingItemReader" writer="elasticSearcItemWriter"
processor="itemProcessor" commit-interval="10" />
</tasklet>
</step>
<bean id="pagingItemReader"
class="org.springframework.batch.item.database.JdbcPagingItemReader"
scope="step">
<property name="dataSource" ref="dataSource" />
<property name="queryProvider">
<bean
class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="selectClause" value="*******" />
<property name="fromClause" value="*******" />
<property name="whereClause" value="*******" />
<property name="sortKey" value="*******" />
</bean>
</property>
<!-- Inject via the ExecutionContext in rangePartitioner -->
<property name="parameterValues">
<map>
<entry key="fromId" value="#{stepExecutionContext[fromId]}" />
<entry key="toId" value="#{stepExecutionContext[toId]}" />
</map>
</property>
<property name="pageSize" value="10" />
<property name="rowMapper">
<bean class="org.springframework.jdbc.core.ColumnMapRowMapper" />
</property>
</bean>
And inside my elasticSearcItemWriter class....
public class ElasticSearchItemWriter<T> extends AbstractItemStreamItemWriter<T>
implements ResourceAwareItemWriterItemStream<T>, InitializingBean {
....
....
....
#Override
public void write(List<? extends T> items) throws Exception {
client = jestClient.getJestClient();
if (items.size() > 0) {
for (Object item : items) {
#SuppressWarnings("unchecked")
Map<String, Object> map = (Map<String, Object>) item;
// Asynch index
Index index = new Index.Builder(map).index(Start.prop.getProperty(Constants.ES_INDEX_NAME))
.type(Start.prop.getProperty(Constants.ES_INDEX_TYPE)).build();
client.executeAsync(index, new JestResultHandler<JestResult>() {
public void failed(Exception ex) {
}
public void completed(JestResult result) {
}
});
}
}
}
.....
....
}
I am reading property files using below entry in my Spring xml.
<context:property-placeholder
location="classpath:resources/database1.properties,
classpath:resources/licence.properties"/>
I am injecting this values in variable using xml entry or using #Value annotation.
<bean id="myClass" class="MyClass">
<property name="driverClassName" value="${database.driver}" />
<property name="url" value="${database.url}" />
<property name="name" value="${database.name}" />
</bean>
I want to add a new property file(database2.properties) which has few same variable names as of database1.properties.
database1.properties:
database.driver=com.mysql.jdbc.Driver
database.url=jdbc:mysql://192.168.1.10/
database.name=dbname
database2.properties:
database.url=jdbc:mysql://192.168.1.50/
database.name=anotherdbname
database.user=sampleuser
You can see few property variables have same name like database.url, database.name in both the property files.
Is it possible to inject database.url of database2.properties?
Or I have to change variable names?
Thank you.
You can do it by configuring two PropertyPlaceholderConfigurer. Usually there's only one instance that serves out all the properties, however, if you change the placeholderPrefix you can use two instances, something like
<bean id="firstPropertyGroup" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations" value="classpath:resources/database1.properties,
classpath:resources/licence.properties" />
<property name="placeholderPrefix" value="${db1."/>
</bean>
<bean id="secondPropertyGroup" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations" value="classpath:resources/database2.properties" />
<property name="placeholderPrefix" value="${db2."/>"
</bean>
Then you would access your properties like ${db1.database.url} or ${db2.database.url}
There might be a solution, similar to what's that you want to achieve. Check the second answer to this question: Multiple properties access. It basically explains what to do in order to access the properties of the second file by using another expression, which is defined by you.
Otherwise, the simplest solution would be just changing the key values (the variable names).
You will sooner or later switch to Spring Boot. So with Spring Boot you can do have such POJO:
public class Database {
#NotBlank
private String driver;
#NotBlank
private String url;
#NotBlank
private String dbname;
public String getDriver() {
return driver;
}
public void setDriver(String driver) {
this.driver = driver;
}
public String getUrl() {
return url;
}
public void setUrl(String url) {
this.url = url;
}
public String getDbname() {
return dbname;
}
public void setDbname(String dbname) {
this.dbname = dbname;
}
}
and use #ConfigurationProperties to fill it:
#Bean
#ConfigurationProperties(locations="classpath:database1.properties", prefix="driver")
public Database database1(){
return new Database();
}
#Bean
#ConfigurationProperties(locations="classpath:database2.properties", prefix="driver")
public Database database2(){
return new Database();
}
Downside of this is that it's mutable. With Lombok library, you can eliminate nasty getters and setters.
I try to read an xml file as input for spring batch:
Java Class:
package de.example.schema.processes.standardprocess;
#XmlAccessorType(XmlAccessType.FIELD)
#XmlType(name = "Process", namespace = "http://schema.example.de/processes/process", propOrder = {
"input"
})
public class Process implements Serializable
{
#XmlElement(namespace = "http://schema.example.de/processes/process")
protected ProcessInput input;
public ProcessInput getInput() {
return input;
}
public void setInput(ProcessInput value) {
this.input = value;
}
}
SpringBatch dev-job.xml:
<bean id="exampleReader" class="org.springframework.batch.item.xml.StaxEventItemReader" scope="step">
<property name="fragmentRootElementName" value="input" />
<property name="resource"
value="file:#{jobParameters['dateiname']}" />
<property name="unmarshaller" ref="jaxb2Marshaller" />
</bean>
<bean id="jaxb2Marshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
<property name="classesToBeBound">
<list>
<value>de.example.schema.processes.standardprocess.Process</value>
<value>de.example.schema.processes.standardprocess.ProcessInput</value>
...
</list>
</property>
</bean>
Input file:
<?xml version="1.0" encoding="UTF-8"?>
<process:process xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:process="http://schema.example.de/processes/process">
<process:input>
...
</process:input>
</process:process>
It fires the following exception:
[javax.xml.bind.UnmarshalException: unexpected element (uri:"http://schema.example.de/processes/process", local:"input"). Expected elements are <<{http://schema.example.de/processes/process}processInput>]
at org.springframework.oxm.jaxb.JaxbUtils.convertJaxbException(JaxbUtils.java:92)
at org.springframework.oxm.jaxb.AbstractJaxbMarshaller.convertJaxbException(AbstractJaxbMarshaller.java:143)
at org.springframework.oxm.jaxb.Jaxb2Marshaller.unmarshal(Jaxb2Marshaller.java:428)
If I change to in xml it work's fine. Unfortunately I can change neither the xml nor the java class.
Is there a possibility to make Jaxb2Marshaller map the element 'input' to the class 'ProcessInput'?
I don't believe JAXB allows this. JAXB is a binding API, so it doesn't provide much in the way of customization. That being said, you can use XStream and provide aliases for what you need, allowing you to customize the mapping of XML to object however you want. You can see an XStream example here: https://github.com/spring-projects/spring-batch/blob/master/spring-batch-samples/src/main/resources/jobs/iosample/xml.xml
We are currently using spring framework and are using following XML :-
<bean id="A" class="com.foo.baar.A" >
<property name="attributes">
<set value-type="com.foo.bar.B">
<ref bean="X" />
<ref bean="Y" />
</set>
</property>
</bean>
<bean id="X" class="com.foo.bar.X" />
<bean id="Y" class="com.foo.bar.Y" />
where class X and class Y extend class B
class A has setter as follows :-
public void setAttributes(List<B> attributes) {
this.attributes = attributes;
}
Now, I have to eliminate the above XML and I am setting the beans programatically as following :-
List<Object> beanRefrences = new ArrayList<Object>();
for(String attribute : attributes) {
Object beanReference = new RuntimeBeanReference(attribute);
beanRefrences.add(beanReference);
}
mutablePropertyValues.add(propertyName, beanRefrences);
With above code, I am getting following error :-
nested exception is org.springframework.beans.ConversionNotSupportedException: Failed to convert property value of type 'java.util.ArrayList' to required type 'java.util.List' for property 'attributes';
nested exception is java.lang.IllegalStateException: Cannot convert value of type [org.springframework.beans.factory.config.RuntimeBeanReference] to required type [com.foo.bar.B] for property 'attributes[0]': no matching editors or conversion strategy found
Can anyone give me pointers on how to make it work correctly?
After taking a look at Spring's implementation of BeanDefinitionValueResolver one can see that a traditional, ordinary List is not sufficient. You need to use a ManagedList:
List<Object> beanRefrences = new ManagedList<>();
for(String attribute : attributes) {
Object beanReference = new RuntimeBeanReference(attribute);
beanRefrences.add(beanReference);
}
mutablePropertyValues.add(propertyName, beanRefrences);