Develop Reference

JPARepository is not saving to DB

Long story, but I had to redesign an application this weekend. From a spring boot app to a spring batch app. The process was always a batch process, but I tried to make this batch engine and it got way too complex and i had to stop what I was doing. I'm sure we've all been there. Anyway everything is working fine!! Except for one piece of code that I tried to keep the original piece of code for. I'm trying to use a JPARepository save method and it's not working!! I am able to call the save method, I feel like the Repo is instantiated because I'm not getting a null pointer exception. In fact, I'm not getting any exceptions thrown. I am just not seeing anything in the DB. And I know this code has worked because I had it running in the previous design. Anyway here are my classes...
Data object:
#Data
#Entity
#Table(name="PAYEE_QUAL_LS")
public class PayeeList {
#EmbeddedId
private PayeeListPK payeeListPK = new PayeeListPK();
#Column(name = "PAYEE_QUAL_CD")
private String payeeQualCode;
#Column(name = "ETL_TS")
private Timestamp etlTimestamp;
}
Primary key data class...
#Data
#Embeddable
public class PayeeListPK implements Serializable {
#Column(name = "PAYEE_NM")
private String payeeName;
#Column(name = "BAT_PROC_DT")
private Date batchProcDate;
}
Repo class...
#Repository
public interface PayeeListRepo extends JpaRepository<PayeeList,String> {}
My Service class...
public class OracleService {
private static final Logger logger = LoggerFactory.getLogger(OracleService.class);
#Autowired
PayeeListRepo payeeListRepo;
public void loadToPayeeListTable(PayeeList payeeList) {
payeeListRepo.save(payeeList);
}
I have an implementation of Tasklet which I am calling from my batch Step...
public class PayeeListTableLoad implements Tasklet {
private static final Logger logger = LoggerFactory.getLogger(PayeeListTableLoad.class);
private java.sql.Date procDt;
private String inputFile;
private Timestamp time;
private int safeRecordCount = 0;
private int blockRecordCount = 0;
private int safeRejectRecordCount = 0;
private int blockRejectRecordCount = 0;
private ArrayList<String> rejectRecordList = new ArrayList<>();
#Autowired
OracleService oracleService;
#Override
public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception {
SimpleDateFormat format = new SimpleDateFormat("yyyyMMdd");
java.util.Date parsed = format.parse(System.getenv("procDt"));
procDt = new java.sql.Date(parsed.getTime());
inputFile = Constants.filePath;
time = new Timestamp(System.currentTimeMillis());
logger.info("Running data quality checks on input file and loading to Oracle");
try (BufferedReader reader = new BufferedReader(new FileReader(inputFile))) {
String line = reader.readLine();
while (line != null) {
if (dataQuality(line)) {
PayeeList payeeList = buildPayeeListObject(line);
oracleService.loadToPayeeListTable(payeeList);
logger.info("Record loaded: " + line);
} else {
rejectRecordList.add(line);
try {
if (line.split("\\|")[1].equals("B")) {
blockRejectRecordCount++;
} else if (line.split("\\|")[1].equals("S")) {
safeRejectRecordCount++;
}
logger.info("Record rejected: " + line);
} catch (ArrayIndexOutOfBoundsException e) {
e.printStackTrace();
}
}
line = reader.readLine();
}
} catch (IOException e) {
e.printStackTrace();
}
logger.info("Safe record count is: " + safeRecordCount);
logger.info("Block record count is: " + blockRecordCount);
logger.info("Rejected records are: " + rejectRecordList);
SendEmail sendEmail = new SendEmail();
sendEmail.sendEmail(Constants.aegisCheckInclearingRecipient,Constants.aegisCheckInclearingSender,Constants.payeeListFileSuccessEmailSubject,Constants.payeeListFileSuccessEmailBodyBuilder(safeRecordCount,blockRecordCount,safeRejectRecordCount,blockRejectRecordCount,rejectRecordList));
logger.info("Successfully loaded to Oracle and sent out Email to stakeholders");
return null;
}
In my batch configuration....
#Bean
public OracleService oracleService() { return new OracleService(); }
#Bean
public PayeeListTableLoad payeeListTableLoad() {
return new PayeeListTableLoad();
}
#Bean
public Step payeeListLoadStep() {
return stepBuilderFactory.get("payeeListLoadStep")
.tasklet(payeeListTableLoad())
.build();
}
#Bean
public Job loadPositivePayFile(NotificationListener listener, Step positivePayLoadStep) {
return jobBuilderFactory.get("loadPositivePayFile")
.incrementer(new RunIdIncrementer())
.listener(listener)
.start(positivePayDataQualityStep())
.next(initialCleanUpStep())
.next(positivePayLoadStep)
.next(metadataTableLoadStep())
.next(cleanUpGOSStep())
.build();
}
Ultimately our step is running an implementation of Tasklet, we are Autowiring out OracleService class, and then that is being called and is then calling the Repo method. I am getting to the Oracle Service class method and I am calling the save method of my Autowired Repository but again nothing is happening!!
EDIT!!!
I have figured out another way to do it and that is with EntityManager and using the persist and flush methods. Below is now my loadToPayeeListTable method in my Oracle Service class...
public void loadToPayeeListTable(PayeeList payeeList) throws ParseException {
EntityManager entityManager = entityManagerFactory.createEntityManager();
EntityTransaction transaction = entityManager.getTransaction();
transaction.begin();
entityManager.persist(payeeList);
entityManager.flush();
transaction.commit();
entityManager.close();
}

Could you have a try to passe the repository with a Spring Test? I have never met this problem, but I am not sure about the DB type. Is it Mysql, Oracle? Because I never used it with #EmbeddedId.
IF you passed the unit test, you ought to check your service logic with debugging. Opposite, you ought to passe the test first.

Change your jpa repository to
#Repository
public interface PayeeListRepo extends JpaRepository<PayeeList, PayeeListPK>

Objects Values Not Persisting. (Spring MVC)

I ma trying to migrate some old code from WebSphere to Tomcat. The older code used Spring 3.2, now I upgraded the JARs to 5.2.2. But somehow object values just do not persist.
My controller class is:
#Controller
#Scope("session")
public class OperationController {
private GUIDataObject guiDO = null;
/**
* Constructor
*/
public OperationController() {
}
#RequestMapping(value="/readDataSource")
#ResponseBody
public String readDataSource() {
try {
String[] sources = guiDO.getDataSources();
.
.
.
Code to work on Array sources
.
.
.
return "ok";
} catch (Exception e) {
return "Error: " + e.getMessage();
}
}
/**
* Set the data sources in the Data Storage Area - these are passed as a "parameter" map
* in the request.
* #param webRequest : WebRequest which parameter map can be pulled from
* #return "ok"
*/
#RequestMapping(value="/setDataSources")
#ResponseBody
public String setDataSources(WebRequest webRequest) {
guiDO.setDatasources(webRequest.getParameterMap());
return "ok";
}
.
.
.
Lots of other code.
.
.
.
}
and the values are stored in the object:
public class GUIDataObject {
private String service;
private String uniqueProcessId;
private String userId;
private String vendor;
// record the data sources to read from
private Map<String, String[]> dataSources = null;
public GUIDataObject(String service, String uniqueProcessId, String userId, String vendor) {
super();
this.service = service;
this.uniqueProcessId = uniqueProcessId;
this.userId = userId;
this.vendor = vendor;
}
public void setDatasources(Map<String, String[]> datasources) {
this.dataSources = datasources;
}
public String[] getDataSources() throws Exception {
if (this.dataSources == null) {
throw new Exception("No datasources have been set from the GUI");
}
if (!this.dataSources.containsKey("source")) {
throw new Exception("No datasources have been set from the GUI");
}
return this.dataSources.get("source");
}
.
.
.
Lots of methods.
.
.
.
}
Now my problem is the dataSources Map is getting set fine. But when fetching the values they return empty. It errors out in the second if-block, so I can say at least its not null. There are other Maps/Strings in the object as well, but I cant really tell if they are being properly set or not, since, this is the first method that is being hit and it errors out after that. I can see that the values that are initialized in the constructor are being retained just fine. So cant really where it is going wrong.
The same code worked fine on WebSphere and Spring 3.2. Now I am not sure if there are any new configurations that I need to in order to get this to work. Since 3.2 is very very old. Any help on this would be appreciated.

The problem was the way webRequest.getParameterMap() works in WebSphere and Tomcat. In WebSphere it returns a concrete HashTable. But in Tomcat, it returns a org.apache.catalina.util.ParameterMap which is a subclass of HashMap. And somehow they just don't mix. Even casting throws a ClassCastException.
I got it to work by changing the dataSources to a HashMap.
private HashMap<String, String[]> dataSources = null;
and the set method to:
public void setDatasources(Map<String, String[]> datasources) {
if (this.dataSources == null) {
this.dataSources = new HashMap<String, String[]>();
this.dataSources.putAll(datasources);
} else {
this.dataSources.putAll(datasources);
}
Probably I could have left the dataSources as a Map and it still would have worked. But I didn't try it out.

DAML Stream all the active contracts using Java Binding - LedgerView

I am new to DAML, I wanted to query all the active contracts using Java binding, Bot API and keep them into DB (or in-memory) for future query.
As per the docs, LedgerView can keep track of active contracts in-memory. However I am not able to successfully stream the active contracts.
You can find my code here, https://github.com/agrawald/daml-java-bot.
The above code have a schedule task which I am not very proud of.
The code for the class where I create DamlLedgerClient and start a schedule job to trigger the Bot. Please note
#Slf4j
#Service
#RequiredArgsConstructor
public class DamlContractSvc implements InitializingBean {
#Value("${daml.host}")
private String host;
#Value("${daml.port}")
private int port;
#Value("${daml.appId}")
private String appId;
#Value("${daml.party}")
private String party;
#Value("${daml.packageId}")
private String packageId;
#Autowired(required = true)
private ContractCache contractCache;
private DamlLedgerClient client;
#Scheduled(fixedDelay = 5000)
public void fetch() {
final TransactionFilter transactionFilter = new FiltersByParty(
Collections.singletonMap(party, NoFilter.instance));
Bot.wire(appId, client, transactionFilter, (ledgerView) -> Flowable.empty(),
contractCache);
}
#Override
public void afterPropertiesSet() throws Exception {
client = DamlLedgerClient.forHostWithLedgerIdDiscovery(host, port, Optional.empty());
client.connect();
}
}
I believe I should be running some Command at (ledgerView) -> Flowable.empty().
contractCache is a class which takes CreatedContract object and load it in the cache.
I may be doing something entirely wrong. please correct me.

I ditched the Bot approach and started using TransactionClient referring to the way Bot.wire method is implemented. Following is what my implementation looks like
#Slf4j
#Service
#RequiredArgsConstructor
public class DamlContractSvc implements InitializingBean {
#Value("${daml.host}")
private String host;
#Value("${daml.port}")
private int port;
#Value("${daml.appId}")
private String appId;
#Value("${daml.party}")
private String party;
#Value("${daml.packageId}")
private String packageId;
#Autowired(required = true)
private ContractRepo contractRepo;
private DamlLedgerClient client;
private final static AtomicReference<LedgerOffset> OFFSET = new AtomicReference<>(
LedgerOffset.LedgerBegin.getInstance());
#Scheduled(fixedDelay = 5000)
public void fetch() {
final TransactionFilter transactionFilter = new FiltersByParty(
Collections.singletonMap(party, NoFilter.instance));
client.getTransactionsClient().getTransactions(OFFSET.get(), transactionFilter, true).flatMapIterable(t -> {
OFFSET.set(new LedgerOffset.Absolute(t.getOffset()));
return t.getEvents();
}).forEach(contractRepo);
}
#Override
public void afterPropertiesSet() throws Exception {
client = DamlLedgerClient.forHostWithLedgerIdDiscovery(host, port, Optional.empty());
client.connect();
}
}
I am keeping track of OFFSET and fetching everything starting from LedgerOffset.LedgerBegin.
Full codebase is here: https://github.com/agrawald/daml-java-bot.

Json Deserialization and save to JPA on boot

Good morning,
I consume API in JSON format, data on the latest exchange rates.
I want this data to be downloaded to me at the beginning of the application and saved in the database. I use spring JPA.
The problem is I do not know how I should write it down.
I have a class responsible for the connection which returns the output in the form of a String.
Another creates de-serialization.
I also have two classes of model that I can use to download data.
I do not want to create a separate class in which the program will pull out each value individually. I was thinking about the map but I do not know how to do it.
Some code:
Model 1
#Data
#AllArgsConstructor
#Entity
public class CurrencyData {
#Id
#GeneratedValue( strategy = GenerationType.AUTO )
private Long id;
#SerializedName("rates")
#Expose
#Embedded
private Rates rates;
#SerializedName("base")
#Expose
#Embedded
private String base;
#SerializedName("date")
#Expose
#Embedded
private String date;
}
Model 2
#Data
#AllArgsConstructor
#Embeddable
public class Rates {
protected Rates(){}
#SerializedName("CAD")
#Expose
private Double cAD;
#SerializedName("HKD")
}
ConnectService with string api output
private static final String REQUEST_CURRENCY = "https://api.exchangeratesapi.io/latest?base=USD";
public String connect() {
String output = null;
try {
System.out.println("URL String : " + REQUEST_CURRENCY);
URL url = new URL(REQUEST_CURRENCY);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new TODO("TODO : ", e.getMessage());
} else {
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
output = response.toString();
}
} catch (Exception e) {
throw new OutputFromApiException("ConnectService CurrencyData-API: output is : ", e.getMessage());
}
return output;
}
GsonConvert- Deserialization
public CurrencyData gsonCurrency(String answer) {
Gson g = new Gson();
CurrencyData currencyData = null;
try {
currencyData = g.fromJson(answer, CurrencyData.class);
} catch (Exception e) {
throw new OutputFromApiException("HistoricalFlight API output is empty ", e.toString());
}
return currencyData;
}
Repository
#Repository
public interface CurrencyRepository extends JpaRepository<CurrencyData, Long> {
}
... And probably I have to write something here..
#Bean
CommandLineRunner runner(CurrencyRepository currencyRepository) {
return args -> {
currencyRepository.save();
};
}

If you are using Spring Boot I think you should define a main class that implements CommandLineRunner instead of defining it as a #Bean. It should be something like:
#SpringBootApplication
public class SpringBootConsoleApplication implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(SpringBootConsoleApplication.class, args);
}
#Autowired
CurrencyRepository currencyRepository;
#Autowired
ConnectService connectionService;
#Override
public void run(String... args) {
String output = connectionService.connect();
CurrencyData currencyData = connectionService.gsonCurrency(output);
currencyRepository.save(currencyData);
}
}
Also I assumed that your jpa configuration is correct and your CurrencyRepository works as expected. If you do not have a manually created database structure than you may consider adding to application.properties file as:
spring.jpa.hibernate.ddl-auto=update
This will provide you that JPA creates or updates the proper database structures on every boot by using your entities configuration.
EDIT:
Sorry I forgot to mention about that you should pass the entity which you want to persist into database. I edited the code as I guess gsonCurrency method is a method inside ConnectionService. Also you can pass a parameter to connectionService.connect() method for base if you want to fetch different data according to different base currencies like this:
CurrencyData currencyDataUSD = connectionService.gsonCurrency(connectionService.connect("USD"));
CurrencyData currencyDataEUR = connectionService.gsonCurrency(connectionService.connect("EUR"));
// and go on if you like

You can use Spring Boot and Rest Template so that you can easily manage the message conversion without having to write the low level HttpConnection. There are two ways to execute a method when an application startup happens in Spring Boot, CommandLineRunner and ApplicationRunner, and here we are using the first as shown below :
#SpringBootApplication
public class Application {
private static final Logger log = LoggerFactory.getLogger(Application.class);
public static void main(String args[]) {
SpringApplication.run(Application.class);
}
#Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
return builder.build();
}
#Bean
public CommandLineRunner run(RestTemplate restTemplate) throws Exception {
return args -> {
Quote quote = restTemplate.getForObject(
"https://gturnquist-quoters.cfapps.io/api/random", Quote.class);
log.info(quote.toString());
};
}
}
Source: https://spring.io/guides/gs/consuming-rest/

java - Autowiring Repository in class that is instantiated in Spring Boot

So I have an API client type class right now, which I am trying to connect to my repository so that I can store data in the MySQL database.
The problem I'm having is that the API client class instantiates a new object of itself, so the Autowiring doesn't work correctly. I've looked around for a workaround for this problem, and I've seen a couple options, but I'm confused on how to apply them to my problem.
For reference, here are parts of some of the relevant files:
GeniusApiClient.java:
#Component
public final class GeniusApiClient {
private final OkHttpClient client = new OkHttpClient();
#Autowired
private ArtistDao artistDao;
public static void main(String[] args) throws Exception {
GeniusApiClient geniusApiClient = new GeniusApiClient();
String artistId = (geniusApiClient.getArtistId("Ugly Duckling"));
ArrayList<String> artistSongIds = geniusApiClient.getArtistSongIds(artistId);
System.out.println(geniusApiClient.getAllSongAnnotations(artistSongIds, artistId));
}
public String getAllSongAnnotations(ArrayList<String> songIds, String artistId) {
Artist artist = new Artist("test name for now", "string123", "223");
artistDao.save(artist);
return "finished";
}
}
ArtistDao.java:
#Transactional
public interface ArtistDao extends CrudRepository<Artist, Long> {
public Artist findByGeniusId(String geniusId);
}
ArtistController.java:
#Controller
public class ArtistController {
#Autowired
private ArtistDao artistDao;
/**
* GET /create --> Create a new artist and save it in the database.
*/
#RequestMapping("/create")
#ResponseBody
public String create(String name, String annotations, String genius_id) {
String userId = "";
try {
genius_id = genius_id.replaceAll("/$", "");
Artist artist = new Artist(name, annotations, genius_id);
artistDao.save(artist);
userId = String.valueOf(artist.getId());
}
catch (Exception ex) {
return "Error creating the artist: " + ex.toString();
}
return "User succesfully created with id = " + userId;
}
/**
* GET /get-by-email --> Return the id for the user having the passed
* email.
*/
#RequestMapping("/get")
#ResponseBody
public String getByEmail(String genius_id) {
String artistId = "";
try {
Artist artist = artistDao.findByGeniusId(genius_id);
artistId = String.valueOf(artist.getId());
}
catch (Exception ex) {
return "User not found";
}
return "The user id is: " + artistId;
}
}
The problem is that in GeniusApiClient.java in the getAllSongAnnotations method, I have a null pointer exception when I try and access the artistDao. I understand that my instantiation of this class is what is messing up the Autowiring, but I'm curious on what the best way to go about fixing this might be.
I considered making all of my methods in the class static so that I wouldn't have to instantiate a new method, but I don't think this would work very well. Any suggestions?
Thanks
EDIT:
Removed some irrelevant code for clarity.
EDIT2:
Added ArtistController.java

To be able to autowire/inject an object, that object must be a Spring bean.
Here you can't autowire ArtistDao because it's not a bean. There are several annotation options to make it bean but the one suits in this case is #Repository annotation. It's just a specialized version of #Component which you used in GeniusApiClient class.
So,
#Repository
#Transactional
public interface ArtistDao extends CrudRepository<Artist, Long> {
public Artist findByGeniusId(String geniusId);
}
should work.
I'd suggest you to read: http://docs.spring.io/spring/docs/current/spring-framework-reference/html/beans.html
If reading reference documentation sounds scary to you, you can also take a look at Core Spring part of Spring in Action.

Don't make GeniusApiClient.class final. Spring will use CGLIB to dynamically extend your class in order to make a proxy. And the requirement for CGLIB to work is to have your classes non-final.
More on this here: Make Spring Service Classes Final?

What you are trying to do in your catch block is not clear to me,you have to correct that and replace it with desired action to be taken on any exception occurrence.