Good morning,
I consume API in JSON format, data on the latest exchange rates.
I want this data to be downloaded to me at the beginning of the application and saved in the database. I use spring JPA.
The problem is I do not know how I should write it down.
I have a class responsible for the connection which returns the output in the form of a String.
Another creates de-serialization.
I also have two classes of model that I can use to download data.
I do not want to create a separate class in which the program will pull out each value individually. I was thinking about the map but I do not know how to do it.
Some code:
Model 1
#Data
#AllArgsConstructor
#Entity
public class CurrencyData {
#Id
#GeneratedValue( strategy = GenerationType.AUTO )
private Long id;
#SerializedName("rates")
#Expose
#Embedded
private Rates rates;
#SerializedName("base")
#Expose
#Embedded
private String base;
#SerializedName("date")
#Expose
#Embedded
private String date;
}
Model 2
#Data
#AllArgsConstructor
#Embeddable
public class Rates {
protected Rates(){}
#SerializedName("CAD")
#Expose
private Double cAD;
#SerializedName("HKD")
}
ConnectService with string api output
private static final String REQUEST_CURRENCY = "https://api.exchangeratesapi.io/latest?base=USD";
public String connect() {
String output = null;
try {
System.out.println("URL String : " + REQUEST_CURRENCY);
URL url = new URL(REQUEST_CURRENCY);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new TODO("TODO : ", e.getMessage());
} else {
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
output = response.toString();
}
} catch (Exception e) {
throw new OutputFromApiException("ConnectService CurrencyData-API: output is : ", e.getMessage());
}
return output;
}
GsonConvert- Deserialization
public CurrencyData gsonCurrency(String answer) {
Gson g = new Gson();
CurrencyData currencyData = null;
try {
currencyData = g.fromJson(answer, CurrencyData.class);
} catch (Exception e) {
throw new OutputFromApiException("HistoricalFlight API output is empty ", e.toString());
}
return currencyData;
}
Repository
#Repository
public interface CurrencyRepository extends JpaRepository<CurrencyData, Long> {
}
... And probably I have to write something here..
#Bean
CommandLineRunner runner(CurrencyRepository currencyRepository) {
return args -> {
currencyRepository.save();
};
}
If you are using Spring Boot I think you should define a main class that implements CommandLineRunner instead of defining it as a #Bean. It should be something like:
#SpringBootApplication
public class SpringBootConsoleApplication implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(SpringBootConsoleApplication.class, args);
}
#Autowired
CurrencyRepository currencyRepository;
#Autowired
ConnectService connectionService;
#Override
public void run(String... args) {
String output = connectionService.connect();
CurrencyData currencyData = connectionService.gsonCurrency(output);
currencyRepository.save(currencyData);
}
}
Also I assumed that your jpa configuration is correct and your CurrencyRepository works as expected. If you do not have a manually created database structure than you may consider adding to application.properties file as:
spring.jpa.hibernate.ddl-auto=update
This will provide you that JPA creates or updates the proper database structures on every boot by using your entities configuration.
EDIT:
Sorry I forgot to mention about that you should pass the entity which you want to persist into database. I edited the code as I guess gsonCurrency method is a method inside ConnectionService. Also you can pass a parameter to connectionService.connect() method for base if you want to fetch different data according to different base currencies like this:
CurrencyData currencyDataUSD = connectionService.gsonCurrency(connectionService.connect("USD"));
CurrencyData currencyDataEUR = connectionService.gsonCurrency(connectionService.connect("EUR"));
// and go on if you like
You can use Spring Boot and Rest Template so that you can easily manage the message conversion without having to write the low level HttpConnection. There are two ways to execute a method when an application startup happens in Spring Boot, CommandLineRunner and ApplicationRunner, and here we are using the first as shown below :
#SpringBootApplication
public class Application {
private static final Logger log = LoggerFactory.getLogger(Application.class);
public static void main(String args[]) {
SpringApplication.run(Application.class);
}
#Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
return builder.build();
}
#Bean
public CommandLineRunner run(RestTemplate restTemplate) throws Exception {
return args -> {
Quote quote = restTemplate.getForObject(
"https://gturnquist-quoters.cfapps.io/api/random", Quote.class);
log.info(quote.toString());
};
}
}
Source: https://spring.io/guides/gs/consuming-rest/
Related
I Have a GET request with some parameters which I handle as an object on the controller, consider it could be several parameters.
The problem is that the values for the properties on the dto are being filled using url encoding which I dont want because it messes up queries to a database later on, ie.: name gets populated with "some%20name" instead of "some name" as I would expect.
How can I avoid this encoding problem?
Bellow is a small scenario that represents my issue:
public class SomeDto {
private String name;
private String hex;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getHex() {
return hex;
}
public void setHex(String hex) {
this.hex = hex;
}
}
#RestController
#RequestMapping("example")
public class RestController {
#GetMapping
public void example(final SomeDto someDto) {
System.out.println(someDto.getName());
System.out.println(someDto.getHex());
}
}
public class ClientApi {
private RestTemplate restTemplate;
private String hostUri;
public ClientApi(RestTemplate restTemplate, String hostUri) {
this.restTemplate = restTemplate;
this.hostUri = hostUri;
}
public void test(SomeDto someDto) {
var uri = UriComponentsBuilder.fromUriString(hostUri + "/example");
if(someDto != null) {
uri.queryParam("name", someDto.getName())
.queryParam("hex", someDto.getHex());
}
restTemplate.exchange(uri.toUriString(), HttpMethod.GET, null, Void.class);
}
}
#SpringBootTest(
classes = DemoApplication.class,
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT
)
class ClientApiTest {
#LocalServerPort
private String port;
private ClientApi clientApi;
#BeforeEach
void before() {
clientApi = new ClientApi(new RestTemplate(), "http://localhost:" + port);
}
#Test
void testMethod() {
SomeDto someDto = new SomeDto();
someDto.setName("some name");
someDto.setHex("#ffffff");
clientApi.test(someDto);
}
}
UPDATE:
I was able to partially fix it by decoding the URL, however it only fixes name "some name" to reach the controller correctly, hex "#ffffff" on the other hand reaches as null.
var decodedUri = URLDecoder.decode(uri.toUriString(), Charset.defaultCharset());
Spring uses some symbols as service symbols.
E.g. you cannot parse param value if it contains a comma.
?someParam=some,value
Would be parsed as two params: some and value. But if receive type is not array or collection then the second value will be ignored. Hence, you'll get someParam=some.
The simplest way to avoid it is URL params base64 encoding.
For me, the convenient way was to encode params as json in Base64.
{
"name": "some name",
"hex": "fffffff"
}
Why json? Because there are many ready-made solutions for parsing JSON into an object.
So, your controller will receive Base64 value which is eyJuYW1lIjoic29tZSBuYW1lIiwgImhleCI6ImZmZmZmZmYifQ==
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
import java.util.Base64;
import java.util.Objects;
#RestController
public class RestController {
#GetMapping("/example")
public void example(String params) {
String decoded = decodeBase64(params);
SomeDto dto = parseTo(decodedFilters, SomeDto.class);
}
public String decodeBase64(String encoded) {
if (Objects.nonNull(encoded)) {
return new String(Base64.getDecoder().decode(encoded));
}
return "";
}
public <T> T parseTo(String jsonAsString, Class<T> classType) {
String toParse = Objects.nonNull(jsonAsString) ? jsonAsString : "{}";
try {
return new ObjectMapper().readValue(toParse, classType);
} catch (IOException e) {
throw new ValidationException(e.getMessage());
}
}
}
I have a small program in spring-boot which through a get call having a #RequestBody returns me a message with all the specifications (in my case of cars)
public class CarsRequest implements Serializable {
private String name;
private String plate ;
private String price;
}
I would like to be able to make sure that if a field is set to null, it can still find the relative message with the other fields having a value, in my case, I wanted to put that the "name" field is optional in the RequestBody, is it possible to do this? I tried setting
public CarsResponse getCars(#RequestBody (required = false) CarsRequest request) throws IOException {
//some code
}
but then when I go to do the get it completely deletes the null field at the time of the get and therefore fails to do it
Just remove the #RequestBody annotation from the function and keep it as it is
public CarsResponse getCars(CarsRequest request) throws IOException {
//some code
}
Now all fields will be converted into query params and all will be optional, because query param by convention are optional
public class CarsRequest implements Serializable {
private String name;
private String plate ;
private String price;
}
And call like this
GET /someEndpoint?name=<value>&plate=null
But still if you want to make some params mandatory, then use javax.annotations or apply validation yourself.
EDIT: As asked in comment, if you are accepting JSON as parameter body then you can do one thing, you can accept it as String and then convert json to object inside function body
public CarsResponse getCars(#RequestParam(required = false) String request) throws IOException {
ObjectMapper mapper = new ObjectMapper();
CarRequest request = mapper.readValue(request,CarRequest.class);
// other code
}
and call it something like this
GET /someEndpoint?request="{ \"name\" : null, \"plate\": \"someValue\" }"
EDIT 2:
You can do one more thing if you want to keep sending json and have it transformed into object, you can declare a binder something like this
// Some controller class
class SomeController {
#Autowired
ObjectMapper mapper;
// Ommited methods here
#GetMapping("/carRequest")
public ResponseEntity<String> testBinder(#RequestParam CarRequest request) {
return ResponseEntity.ok("{\"success\": \"" + request.name+ "\"}");
}
#InitBinder
public void initBinder(WebDataBinder binder) {
binder.registerCustomEditor(CarRequest.class, new CarRequestEditor(mapper));
}
static class CarRequestEditor extends PropertyEditorSupport {
private ObjectMapper objectMapper;
public CarRequestEditor(ObjectMapper objectMapper) {
this.objectMapper = objectMapper;
}
#Override
public void setAsText(String text) throws IllegalArgumentException
{
if (StringUtils.isEmpty(text)) {
setValue(new CarRequest());
} else {
try {
setValue(objectMapper.readValue(text, CarRequest.class));
} catch (JsonProcessingException e) {
throw new IllegalArgumentException(e);
}
}
}
}
}
Please note that the client need to send the json URL encoded like this
http://localhost:8180/carRequest?request=%7B%22name%22%3"test"%7D
Hi you are using #RequestBody (required = false) CarsRequest
that means your CarsRequest object itself is optional
rather than you can use
#NotEmpty
private String plate ;
#NotEmpty
private String price;
You can make a single field optional by making it an Optional, in your case Optional<String>. If the field does not appear in the request body, then the Optional will be empty.
public class CarsRequest implements Serializable {
private String name;
private String plate;
private Optional<String> price;
}
Long story, but I had to redesign an application this weekend. From a spring boot app to a spring batch app. The process was always a batch process, but I tried to make this batch engine and it got way too complex and i had to stop what I was doing. I'm sure we've all been there. Anyway everything is working fine!! Except for one piece of code that I tried to keep the original piece of code for. I'm trying to use a JPARepository save method and it's not working!! I am able to call the save method, I feel like the Repo is instantiated because I'm not getting a null pointer exception. In fact, I'm not getting any exceptions thrown. I am just not seeing anything in the DB. And I know this code has worked because I had it running in the previous design. Anyway here are my classes...
Data object:
#Data
#Entity
#Table(name="PAYEE_QUAL_LS")
public class PayeeList {
#EmbeddedId
private PayeeListPK payeeListPK = new PayeeListPK();
#Column(name = "PAYEE_QUAL_CD")
private String payeeQualCode;
#Column(name = "ETL_TS")
private Timestamp etlTimestamp;
}
Primary key data class...
#Data
#Embeddable
public class PayeeListPK implements Serializable {
#Column(name = "PAYEE_NM")
private String payeeName;
#Column(name = "BAT_PROC_DT")
private Date batchProcDate;
}
Repo class...
#Repository
public interface PayeeListRepo extends JpaRepository<PayeeList,String> {}
My Service class...
public class OracleService {
private static final Logger logger = LoggerFactory.getLogger(OracleService.class);
#Autowired
PayeeListRepo payeeListRepo;
public void loadToPayeeListTable(PayeeList payeeList) {
payeeListRepo.save(payeeList);
}
I have an implementation of Tasklet which I am calling from my batch Step...
public class PayeeListTableLoad implements Tasklet {
private static final Logger logger = LoggerFactory.getLogger(PayeeListTableLoad.class);
private java.sql.Date procDt;
private String inputFile;
private Timestamp time;
private int safeRecordCount = 0;
private int blockRecordCount = 0;
private int safeRejectRecordCount = 0;
private int blockRejectRecordCount = 0;
private ArrayList<String> rejectRecordList = new ArrayList<>();
#Autowired
OracleService oracleService;
#Override
public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception {
SimpleDateFormat format = new SimpleDateFormat("yyyyMMdd");
java.util.Date parsed = format.parse(System.getenv("procDt"));
procDt = new java.sql.Date(parsed.getTime());
inputFile = Constants.filePath;
time = new Timestamp(System.currentTimeMillis());
logger.info("Running data quality checks on input file and loading to Oracle");
try (BufferedReader reader = new BufferedReader(new FileReader(inputFile))) {
String line = reader.readLine();
while (line != null) {
if (dataQuality(line)) {
PayeeList payeeList = buildPayeeListObject(line);
oracleService.loadToPayeeListTable(payeeList);
logger.info("Record loaded: " + line);
} else {
rejectRecordList.add(line);
try {
if (line.split("\\|")[1].equals("B")) {
blockRejectRecordCount++;
} else if (line.split("\\|")[1].equals("S")) {
safeRejectRecordCount++;
}
logger.info("Record rejected: " + line);
} catch (ArrayIndexOutOfBoundsException e) {
e.printStackTrace();
}
}
line = reader.readLine();
}
} catch (IOException e) {
e.printStackTrace();
}
logger.info("Safe record count is: " + safeRecordCount);
logger.info("Block record count is: " + blockRecordCount);
logger.info("Rejected records are: " + rejectRecordList);
SendEmail sendEmail = new SendEmail();
sendEmail.sendEmail(Constants.aegisCheckInclearingRecipient,Constants.aegisCheckInclearingSender,Constants.payeeListFileSuccessEmailSubject,Constants.payeeListFileSuccessEmailBodyBuilder(safeRecordCount,blockRecordCount,safeRejectRecordCount,blockRejectRecordCount,rejectRecordList));
logger.info("Successfully loaded to Oracle and sent out Email to stakeholders");
return null;
}
In my batch configuration....
#Bean
public OracleService oracleService() { return new OracleService(); }
#Bean
public PayeeListTableLoad payeeListTableLoad() {
return new PayeeListTableLoad();
}
#Bean
public Step payeeListLoadStep() {
return stepBuilderFactory.get("payeeListLoadStep")
.tasklet(payeeListTableLoad())
.build();
}
#Bean
public Job loadPositivePayFile(NotificationListener listener, Step positivePayLoadStep) {
return jobBuilderFactory.get("loadPositivePayFile")
.incrementer(new RunIdIncrementer())
.listener(listener)
.start(positivePayDataQualityStep())
.next(initialCleanUpStep())
.next(positivePayLoadStep)
.next(metadataTableLoadStep())
.next(cleanUpGOSStep())
.build();
}
Ultimately our step is running an implementation of Tasklet, we are Autowiring out OracleService class, and then that is being called and is then calling the Repo method. I am getting to the Oracle Service class method and I am calling the save method of my Autowired Repository but again nothing is happening!!
EDIT!!!
I have figured out another way to do it and that is with EntityManager and using the persist and flush methods. Below is now my loadToPayeeListTable method in my Oracle Service class...
public void loadToPayeeListTable(PayeeList payeeList) throws ParseException {
EntityManager entityManager = entityManagerFactory.createEntityManager();
EntityTransaction transaction = entityManager.getTransaction();
transaction.begin();
entityManager.persist(payeeList);
entityManager.flush();
transaction.commit();
entityManager.close();
}
Could you have a try to passe the repository with a Spring Test? I have never met this problem, but I am not sure about the DB type. Is it Mysql, Oracle? Because I never used it with #EmbeddedId.
IF you passed the unit test, you ought to check your service logic with debugging. Opposite, you ought to passe the test first.
Change your jpa repository to
#Repository
public interface PayeeListRepo extends JpaRepository<PayeeList, PayeeListPK>
Please look at the following Mongo DB document:
#Document(collection = CitizenForumMessageDocument.COLLECTION_NAME)
public class ImageDocument {
public static final String COLLECTION_NAME = "images";
#Id
private String id; // autogenerated
private Image data; // data for the client (web, mobile...)
private ImageMeta meta; // for internal application work (uploader ip, etc...)
[...] // getter, setter
}
// send as is to a client
public class Image {
private String id;
[...]
}
Is it possible to apply the document id to the Image id while document creation.
How I'm doing it now:
public void saveUploadedImage(Client client, ImageForm form) {
ImageDocument doc = new ImageDocument();
dao.save(doc); // create document cause we need an id...
try {
doc.setImage(createImage(form, doc.getId()));
doc.setMeta(createMeta(client, form));
} catch(Exception e){
dao.remove(doc);
return; // ugly...
}
dao.update(doc);
}
I could also do it by using some reflection hacks in my dao layer, but I hope there is a better solution for this issue.
You can use Mongo Lifycycle Events for this.
#Component
public class MongoListener extends AbstractMongoEventListener<ImageDocument>
{
private final MongoTemplate mongoTemplate;
#Autowired
public MongoListener(final MongoTemplate mongoTemplate) {
this.mongoTemplate = mongoTemplate;
}
#Override
public void onAfterSave(AfterSaveEvent<ImageDocument> event) {
ImageDocument imageDocument = event.getSource();
if(imageDocument.getData().getId() == null) {
imageDocument.getData().setId(imageDocument.getId());
mongoTemplate.save(imageDocument);
}
}
}
I have to tell, that this is quite ugly, because for every save there will be two database calls.
But I don't see any other way to do this.
I am trying to return {"status": its value}´in the case of routeD!=0 currently I am getting {"status":201,"routes":null} I would get the response in this form {"status":201} without "routes":null at the same time I dont want to lost the response of routeD==0 which is for example {"status":230,"routes":[1,9,3]}
I appeciate any help.
Receiver class:
#Path("/data")
public class Receiver {
#POST
#Consumes(MediaType.APPLICATION_JSON)
public Response storeData(Data data) {
Database db = new Database();
String macD = data.getMac();
int routeD = data.getRoute();
double latD = data.getLatitude();
double longD = data.getLongitude();
double speedD = data.getSpeed();
// Jackson class to wrapper the data in JSON string.
SDBean bean = new SDBean();
if (routeD != 0) {
bean.status = db.insertData(macD, routeD, latD, longD);
return Response.status(bean.status).entity(bean.toJson()).build();
} else {
bean.routes = db.detectRoute(latD, longD);
return Response.status(230).entity(bean.toJson()).build();
}
}
}
SDBean class:
public class SDBean {
public int status;
public ArrayList<Integer> routes;
public SDBean(){
status = 230;
}
public String toJson() {
ObjectMapper mapper = new ObjectMapper();
String json = null;
try {
json = mapper.writeValueAsString(this);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
System.out.println(json);
return json;
}
}
Just use #JsonInclude(JsonInclude.Include.NON_NULL)
Annotation used to indicate when value of the annotated property (when used for a field, method or constructor parameter), or all properties of the annotated class, is to be serialized. Without annotation property values are always included, but by using this annotation one can specify simple exclusion rules to reduce amount of properties to write out.
import com.fasterxml.jackson.annotation.JsonInclude;
[...]
#JsonInclude(JsonInclude.Include.NON_NULL)
public class SDBean {