In my Spring Batch Application, I am reading, processing and then trying to write with a ItemWriter to the database using stored procedure:
Below is what my CSV file looks like lets say which I want to read, process and write:
Cob Date;Customer Code;Identifer1;Identifier2;Price
20180123;ABC LTD;BFSTACK;1231.CZ;102.00
My ItemWriter:
#Slf4j
public class MyDBWriter implements ItemWriter<Entity> {
private final EntityDAO scpDao;
public MyWriter(EntityDAO scpDao) {
this.scpDao = scpDao;
}
#Override
public void write(List<? extends Entity> items) {
items.forEach(scpDao::insertData);
}
}
My DAO implementation:
#Repository
public class EntityDAOImpl implements EntityDAO {
#Autowired
private JdbcTemplate jdbcTemplate;
private SimpleJdbcCall simpleJdbcCall = null;
#PostConstruct
private void prepareStoredProcedure() {
simpleJdbcCall = new SimpleJdbcCall(jdbcTemplate).withProcedureName("loadPrice");
//declare params
}
#Override
public void insertData(Entity scp) {
Map<String, Object> inParams = new HashMap<>();
inParams.put("Identifier1", scp.getIdentifier1());
inParams.put("Identifier2", scp.getIdentifier1());
inParams.put("ClosingPrice", scp.getClosingPrice());
inParams.put("DownloadDate", scp.getDownloadDate());
simpleJdbcCall.execute(inParams);
}
}
My Stored procedure used to update is as follows:
ALTER PROCEDURE [dbo].[loadPrice]
#Identifier1 VARCHAR(50),
#Identifier1 VARCHAR(50),
#ClosingPrice decimal(28,4),
#DownloadDate datetime
AS
SET NOCOUNT ON;
UPDATE p
SET ClosingPrice = #ClosingPrice,
from Prices p
join Instrument s on s.SecurityID = p.SecurityID
WHERE convert(date, #DownloadDate) = convert(date, DownloadDate)
and s.Identifier1 = #Identifier1
if ##ROWCOUNT = 0
INSERT INTO dbo.Prices
(
sec.SecurityID
, ClosingPrice
, DownloadDate
)
select sec.SecurityID
, #ClosingPrice
, LEFT(CONVERT(VARCHAR, #DownloadDate, 112), 8)
from dbo.Instrument sec
WHERE sec.Identifier1 = #Identifier1
Give I have this setup, one of my requirement is that if I am unable to update/insert to the database using #Identifier1 i.e. there is no SecurityID which matched with Identifier1, I need to THEN update/insert
using the Identifier2. Second level match if you like.
How can I do this in my DAO insertData()? It is business logic and prefer in java code instead of stored proc but I am keen to look at your examples how this can be achieved.
How can I return a result of a row being updated/inserted and take decision as to whether or not to update/insert with second identifier?
For the update I would change the where clause to
WHERE convert(date, #DownloadDate) = convert(date, DownloadDate)
and (s.Identifier1 = #Identifier1 OR s.Identifier2 = #Identifier2)
and for the insert
WHERE sec.Identifier1 = #Identifier1 OR sec.Identifier2 = #Identifier2
That should work even if I haven't verified it myself. I am assuming that the given values for identifier1 and identifier2 can not match two different rows in the Instrument table.
Related
I have a problem with ehcache in my application. I want to store in cache two method that has two diferent queries to db. The problem is that the data of second method is stored in data of first method and when the user make multiple request the data is duplicated everytime.
For example:
First call ->
method 1 return 0 items
method 2 return 2 items
Second call -> The methods are cached and just return the stored data but...
method 1 return 2 items ¿?
method 2 return 2 items
Third call ->
method 1 return 4 items ¿?
method 2 return 2 items
Dao class:
public class DataDAOImpl extends JdbcDaoSupport implements DataDAO {
#Autowired
private JdbcTemplate jdbcTemplate1;
#Autowired
private JdbcTemplate jdbcTemplate2;
#PostConstruct
private void initialize() {
setJdbcTemplate(jdbcTemplate1);
}
#Autowired
private Environment env;
#Cacheable("data_1")
public List<Data> getData1(String data, String start_date, String end_date) {
List<Data> list_data_1 = (List<Data>) jdbcTemplate1.query(
env.getProperty("sql_data_1"),
new BeanPropertyRowMapper<>(Data.class),
data, start_date, end_date);
return list_data_1;
}
#Cacheable("data_2")
public List<Data> getData2(String data, String start_date, String end_date) {
List<Data> list_data_2 = (List<Data>) jdbcTemplate2.query(
env.getProperty("sql_data_2"),
new BeanPropertyRowMapper<>(Data.class),
data, start_date, end_date);
return list_data_2;
}
}
Main class:
List<Data> arrayData = new ArrayList<Data>();
arrayData = dataDAO.getData1(data, start_date, end_date);
arrayData.addAll(dataDAO.getData2(data, start_date, end_date));
Thank you so much!
The caching works fine, the problem is what you do with the result returned.
List<Data> arrayData = new ArrayList<Data>();
arrayData = dataDAO.getData1(data, start_date, end_date);
arrayData.addAll(dataDAO.getData2(data, start_date, end_date));
The code above updates the collection, without doing any defensive copy. Since you are most likely caching on heap, you are effectively modifying the content of what is cached.
So either you do the defensive copy before merging the collections:
List<Data> arrayData = new ArrayList<Data>(dataDAO.getData1(data, start_date, end_date));
arrayData.addAll(dataDAO.getData2(data, start_date, end_date));
or Ehcache has configuration options so that it is the cache doing a copy for you each time something is read from the cache - see documentation for version 2.x and documentation for version 3.x.
Note that the code above is not null safe.
I have a Spring boot application thats committing object to Postgres DB using JPARepository. The code for my repository is as below:
public interface TestObjectDao extends JPARepository<TestObject, Long> {
List<TestObject> findByTestRefIdAndTestType(String testRefId,
Short testType);
TestObject save(TestObject testObject);
}
When I want to create, In my service implementation (implementing above interface) I used "save" method. But, when I try to update, it neither creates entry nor update it.
How can I update records in Postgres database? Below is my code snippet using for update:
#Component
public class TestObjectServiceImpl implements TestObjectService {
#Inject
private TestObjectDao TestObjectDao;
#Override
public TestObjectResponse createTestObject(String testRefId, String testType, TestObject testObject) throws Exception{
--
--
--
try {
//update object
testObject = TestObjectDao.save(testObject);
}
}
catch (Exception ex) {
throw new Exception("Object could not be modified");
}
TestObjectResponse testObjectResponse = new TestObjectResponse();
testObjectResponse.setVal(testObject);
return testObjectResponse;
}
}
I have gone through all related posts but didn't get satisfactory resolution.
Spring detects wether it needs to save or create an entity by checking it's ID.
In your case, you need to select it first, so Spring will populate the entity properly:
try {
testObject = TestObjectDao.findOne(testRefId);
//update object
testObject = TestObjectDao.save(testObject);
}
Please refer section 2.2.1:
http://docs.spring.io/spring-data/jpa/docs/1.4.1.RELEASE/reference/html/jpa.repositories.html
Note that if only some columns are to be updated, it's much more efficient to use:
#Modifying
#Query("update User u set u.firstname = ?1 where u.lastname = ?2")
int setFixedFirstnameFor(String firstname, String lastname);
I am attempting to create an accessor to run slightly more complex queries in cassandra with java. I have no problem with the syntax, and I can get it to work, but my question is this: is there a way to dynamically declare a keyspace in an accessor?
For example, if you create a table map for the MappingManager you would declare the #Table and give it the keyspace and table name like so:
#Table(keypace="mykeyspace", name="orders")
public class Orders {
#PartitionKey
public UUID id;
//blah blah blah, rest of code
}
Now creating an accessor for that specific table is easy enough:
#Accessor
public interface OrdersAccessor {
#Query("SELECT * FROM orders WHERE status = :status")
Result pending(#Param("status") Integer status);
}
Simple. The problem is it demands a keyspace, and I am a huge fan of never hard-coding anything. I realize that I am "hard-coding" the keyspace in the Table definition in the MappingManager class definition, but if need be I only change it there and it updates everything that has to do with that. If I hard-code the keyspace in every single #Query definition inside the Accessor I will have to change, potentially, a bunch of different items if the keyspace gets updated, instead of only changing it one place in the #Table definition.
I have been searching Google for hours and I can't find a single instance of someone dynamically declaring a keyspace with an accessor, only thousands of examples of accessors where they are hard-coding the keyspace into the #Query like so:
#Accessor
public interface OrdersAccessor {
#Query("SELECT * FROM keyspace.orders WHERE status = :status")
Result pending(#Param("status") Integer status);
}
I realize the query I wrote isn't really cause for an accessor, I was just simplifying it for the sake of the example. So I am coming to the community asking for help, I can't find any examples of this anywhere. I can't imagine that I am the first person to ever want to do this, I just can't find any examples of anyone else tackling this problem. Thank you in advance for any help you can give, I can really use it.
#Sudhir Here is the solution I came up with. I am sure there are better ways to handle the connections, but I am still pretty new to cassandra and Java, and this is working well for my needs. I hope this helps...
public class DbInterface {
private Cluster cluster;
private Session session;
private Map<String, Session> dbMap;
private Map<String, Map<String, Mapper<Class>>> mappers = new ConcurrentHashMap<>();
public DbInterface(String host) {
Map<String, Session> connections = createConnection(host);
Session crSession = connections.get("crSession");
Session hppSession = connections.get("hppSession");
cluster = Cluster.builder().addContactPoint(host).build();
Session crSession = cluster.connect("mykeyspace");
Session hppSession = cluster.connect("hpp");
MappingManager crManager = new MappingManager(crSession);
MappingManager hppManager = new MappingManager(hppSession);
mappers.put("mykeyspace", new ConcurrentHashMap<>());
mappers.put("mykeyspace2", new ConcurrentHashMap<>());
Map cr = mappers.get("mykeyspace");
Map hpp = mappers.get("mykeyspace2");
cr.put("status", crManager.mapper(OrderStatus.class));
hpp.put("status", hppManager.mapper(OrderStatus.class));
cr.put("status_accessor", crManager.createAccessor(OrderStatusAccessor.class));
hpp.put("status_accessor", hppManager.createAccessor(OrderStatusAccessor.class));
cr.put("users", crManager.mapper(Users.class));
hpp.put("users", hppManager.mapper(Users.class));
cr.put("orders", crManager.mapper(Orders.class));
hpp.put("orders", hppManager.mapper(Orders.class));
cr.put("order_detail", crManager.mapper(OrderDetail.class));
hpp.put("order_detail", hppManager.mapper(OrderDetail.class));
cr.put("chal_orders", crManager.mapper(ChalOrder.class));
hpp.put("chal_orders", hppManager.mapper(ChalOrder.class));
cr.put("chal_order_detail", crManager.mapper(ChalOrderDetail.class));
hpp.put("chal_order_detail", hppManager.mapper(ChalOrderDetail.class));
cr.put("detail_accessor", crManager.createAccessor(OrderDetailAccessor.class));
hpp.put("detail_accessor", hppManager.createAccessor(OrderDetailAccessor.class));
cr.put("tracking_labels", crManager.mapper(TrackingLabels.class));
hpp.put("tracking_labels", hppManager.mapper(TrackingLabels.class));
}
public Session getConnection(String type) {
if(dbMap.containsKey(type)) {
return dbMap.get(type);
}
if(dbMap.containsKey(type.toLowerCase() +"Session")) {
return dbMap.get(type.toLowerCase() +"Session");
}
return dbMap.get("crSession");
}
public Map<String, Session> createConnection(String host) {
dbMap = new HashMap<>();
cluster = Cluster.builder().addContactPoint(host).build();
Session crSession = cluster.connect("mykeyspace");
Session hppSession = cluster.connect("hpp");
dbMap.put("crSession", crSession);
dbMap.put("hppSession", hppSession);
return dbMap;
}
public Map getDBMap(String client) {
if(mappers.containsKey(client)) {
return mappers.get(client);
}
throw new RuntimeException("Unknown Client: " + client);
}
}
One of the things I was thinking of doing is moving the session creation and Map creation to separate functions, then only connect and build the map for the session that is needed. Like instead of defaulting to connecting to both sessions when the DbInterface() is called, only connect to the session that is requested via the "host" param.
Anywho, I hope this helps you out. If you need it, here is an example of my other library that uses this...
public class MyRestController {
private final DbInterface db = new DbInterface(IPADDRESS);
#CrossOrigin
#RequestMapping("/status")
public String getStatus() {
Map managerMap = db.getDBMap("mykeyspace");
OrderStatusAccessor statuses = (OrderStatusAccessor) managerMap.get("status_accessor");
Result<OrderStatus> allStatuses = statuses.getAll();
//rest of the code here
}
}
In my project I have read multiple tables with different queries and consolidate those results sets in flat files. How do I achieve that. I mean JdbcReader is directly taking 1 select query, how can I customize it.
If JdbcCursorItemReader does not suit your needs, you are always free to implement a custom reader by implementing the ItemReader interface.
public interface ItemReader<T> {
T read() throws Exception, UnexpectedInputException, ParseException;
}
Just write a class that implements this interface and inject a jdbcTemplate to query multiple tables.
public Class MyCompositeJdbcReader implements ItemReader<ConsolidateResult>{
private JdbcTemplate jdbcTemplate;
public ConsolidateResult read()
throws Exception, UnexpectedInputException, ParseException{
ConsolidateResult cr = new ConsolidateResult();
String name= this.jdbcTemplate.queryForObject(
"select name from customer where id = ?",new Object[]{1}, String.class);
String phoneNumber= this.jdbcTemplate.queryForObject(
"select phone from customer_contact where custid = ?",
new Object[]{1},String.class);
cr.setName(name);
cr.setPhone(phoneNumber);
return cr;
}
}
I did not compile the code but I hope it gives an idea.
I'm trying to do a simple thing: call stored procedure which have a object type parameter.
This is what I have in db:
create or replace
TYPE TEST_TYPE AS OBJECT
(
test_field varchar(100)
)
and
CREATE OR REPLACE PROCEDURE TEST_PROC
(
PARAM1 IN TEST_TYPE
) AS
BEGIN
END TEST_PROC;
This is what I have in my java code:
#Embeddable
#Struct(name = "TEST_TYPE", fields = {"TEST_FIELD"})
public class TestStruct
{
private String testField;
public String getTestField() {
return testField;
}
public void setTestField(String testField) {
this.testField = testField;
}
}
and
#PostConstruct
public void init()
{
StoredProcedureCall call = new StoredProcedureCall();
call.setProcedureName("TEST_PROC");
call.addNamedArgument("PARAM1", "PARAM1", Types.STRUCT, "TEST_TYPE", TestStruct.class);
DataReadQuery dataReadQuery = new DataReadQuery(call);
dataReadQuery.addArgument("PARAM1");
TestStruct testStruct = new TestStruct();
List args = new ArrayList();
args.add(testStruct);
Object result = ((EntityManagerImpl)em.getDelegate()).getSession().executeQuery(dataReadQuery,args);
}
this is what I get in runtime:
Internal Exception: java.sql.SQLException: Invalid column type
Error Code: 17004
Call: BEGIN TEST_PROC(PARAM1=>?); END;
bind => [1 parameter bound]
Query: DataReadQuery()
I think I totally don't understand the subject of usage structs with JPA
please help me, good people :)
What is the shortest way to make this working?
Please send complete your code.
For call stored procedures using Spring, you have to extends StoredProcedure class. If you send your complete code, I can help better. sample pseudo code:
class CustomStoredProcedure extends org.springframework.jdbc.object.StoredProcedure
{
CustomStoredProcedure()
{
super([your-data-source], [package-name]);
declareParameter(new SqlParameter([your-struct-name]), Types.STRUCT));
compile();
}
Map<String, Object> execute([your-parameter])
{
return super.execute(inputs);
}
}
for better help, you have explain complete situation.
Your code looks correct.
Ensure that the descriptor was defined for the struct. (i.e. session.getDescrptor(TestStruct.class))
Can you call stored procedures with other types?
What database are you using, have you set your platform correctly to Oracle?
It seems that eclipselink skips descriptors for the #Struct and #Embeddable annotated classes unless they are referenced by some other class. The shortest way to make it working is to use workaround based on this assumption. Put additional class in the jar where your META-INF/persistence.xml is located:
#Entity
public class StructEntitiesWorkaround {
#Id
private String id;
private TestStruct testStruct;
}
You might want to use SimpleJdbcCall with Types.STRUCT.
Here is an example: https://docs.spring.io/spring-data/jdbc/old-docs/2.0.0.M1/reference/html/orcl.datatypes.html