I am trying to add the BusinessInformation data in my existing JSON.
{
"Original": "Web Form",
"SubmitterNetworkName": "null",
"SourceName": "Contact Request Form",
"SourceKind": "Web Form",
** "BusinessInformation": {
"BusinessContactName": null,
“AccountNumber”:null,
},**
"EmployeeName": null,
"EmployeeDOB": null,
}
So I have a Spring Batch app that exports data from a database to another DB and during that run some JSON needs to be created mapping multiple columns. I am using lombok and jackson mainly for the creation of JSON.
The model sample that I have. I also tried creating BusinessInformation class.
package model
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
#Data
#AllArgsConstructor
#NoArgsConstructor
#Builder
#JsonIgnoreProperties(ignoreUnknown = true)
public class ColumnNameForTheDBImImportingTo implements Serializable {
#JsonProperty("Origin")
private String Origin;
and so on..
}
And then I have the Service for it.
import org.codehaus.jackson.map.ObjectMapper;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import lombok.extern.slf4j.Slf4j;
import twc.batch.extcbk.model.*;
import java.io.IOException;
#Component
#Slf4j
public class ColumnNameForTheDBImImportingToService {
private ColumnNameForTheDBImImportingTo columnNameForTheDBImImportingTo (some codes) {
ColumnNameForTheDBImImportingTo columnNameForTheDBImImportingTo = ColumnNameForTheDBImImportingTo .builder().build();
columnNameForTheDBImImportingTo.setOrigin("Web Form");
}
**Then I have a method for the object mapper.**
private String getColumnNameForTheDBImImportingTo (ColumnNameForTheDBImImportingTo columnNameForTheDBImImportingTo ) {
String columnNameForTheDBImImportingToStr = null;
try {
ObjectMapper mapper = new ObjectMapper();
columnNameForTheDBImImportingToStr = mapper.writeValueAsString(columnNameForTheDBImImportingTo );
log.debug("columnNameForTheDBImImportingToStr {}", columnNameForTheDBImImportingToStr );
} catch (IOException e) {
log.error("getColumnNameForTheDBImImportingTo " + e.getMessage());
}
return columnNameForTheDBImImportingToStr ;
}
**Then another class builds the db columns and sets the values.**
I've tried following the Baeldung but I don't quite understand it.
I tried creating a new class for the business information and i was thinking of inserting it in the ColumnNameForTheDBImImportingTo.
Please refer me to any useful information I can follow for this problem.
Thanks!
Data transformation is a typical use case for an item processor.
The way I would implement this with Spring Batch is configure a chunk-oriented step defined as follows:
A JsonItemReader to read the input file
An item processor that adds the business information
A JsonItemWriter to write the output file
In my Java program, I am trying to parse data that I get from Strava.com's API. One of the JSON payloads, I receive from there looks as follows:
[
{"type": "altitude","data": [519.1,519.3,519.3,519.4,519.5],"series_type": "distance","original_size": 5,"resolution": "high"},
{"type": "latlng","data": [[46.01234,6.01234],[46.11234,6.11234],[46.21234,6.21234],[46.31234,6.31234],[46.41234,6.41234]],"series_type": "distance","original_size": 5,"resolution": "high"},
{"type": "velocity_smooth","data": [0.0,0.0,0.0,5.5,5.2],"series_type": "distance","original_size": 5,"resolution": "high"},
{"type": "distance","data": [0.0,8.6,11.8,16.6,20.8],"series_type": "distance","original_size": 5,"resolution": "high"},
{"type": "time","data": [0,1,2,3,4],"series_type": "distance","original_size": 5,"resolution": "high"}
]
Basically, four of these entries (altitude, velocity_smooth, distance and time) have the same structure (their data field is an array of doubles (or ints that can be parsed as doubles)), but the second entry (latlng) has a slighlty different structure for the data field (it is a an array of arrays of double).
I am familiar with the Jackson library to convert between JSON and POJOs if all the content is named, but do not see how I can model the above data structure to deserialise it.
Let's say that instead of the data above, it looked as follows:
{
"altitude": {"data": [519.1,519.3,519.3,519.4,519.5],"series_type": "distance","original_size": 5,"resolution": "high"},
"latlng": {"data": [[46.01234,6.01234],[46.11234,6.11234],[46.21234,6.21234],[46.31234,6.31234],[46.41234,6.41234]],"series_type": "distance","original_size": 5,"resolution": "high"},
"velocity_smooth": {"data": [0.0,0.0,0.0,5.5,5.2],"series_type": "distance","original_size": 5,"resolution": "high"},
"distance": {"data": [0.0,8.6,11.8,16.6,20.8],"series_type": "distance","original_size": 5,"resolution": "high"},
"time": {"data": [0,1,2,3,4],"series_type": "distance","original_size": 5,"resolution": "high"}
}
Then I could define the following three classes
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.AllArgsConstructor;
import lombok.Value;
import java.util.List;
#Getter
#NoArgsConstructor
public class Holder {
DoubleData altitude;
CoordinateData latlng;
#JsonProperty("velocity_smooth") DoubleData velocitySmooth;
DoubleData distance;
DoubleData time;
}
#Getter
#NoArgsConstructor
public class DoubleData {
List<Double> data;
#JsonProperty("series_type") String seriesType;
#JsonProperty("original_size") Integer originalSize;
String resolution;
}
#Getter
#NoArgsConstructor
public class CoordinateData {
List<List<Double>> data;
#JsonProperty("series_type") String seriesType;
#JsonProperty("original_size") Integer originalSize;
String resolution;
}
And then use
objectMapper.readValue(jsonString, Holder.class);
to read in that object. However, as the data received from Strava is an array instead of an object, I am failing. I have read Baeldung's article on how to unmarshal to collections/arrays but that assumes that all classes in the array/collection are the same.
I though about defining an interface which would be extended by the two classes that could be found in the array and then use that mechanism:
public interface Data {
}
#Getter
#NoArgsConstructor
public class DoubleData implements Data {
String type;
List<Double> data;
#JsonProperty("series_type") String seriesType;
#JsonProperty("original_size") Integer originalSize;
String resolution;
}
#Getter
#NoArgsConstructor
public class CoordinateData implements Data {
String type;
List<List<Double>> data;
#JsonProperty("series_type") String seriesType;
#JsonProperty("original_size") Integer originalSize;
String resolution;
}
Data[] array = objectMapper.readValue(jsonString, Data[].class);
But that doesn't work, as I would need to find some way to let it find out when to use a DoubleData class and when to use a CoordinateData class.
I am sure, I am not the first person trying to use Strava data in Java. Can this be done?
If possible, you should definitely use their's client. Strava API v3 shows many examples how to use this API together with theirs model.
If you want to implement your own model you should consider inheritance and com.fasterxml.jackson.annotation.JsonTypeInfo, com.fasterxml.jackson.annotation.JsonSubTypes annotations. Also, JSON Object with type latlng contains list of objects which are represented in JSON in form of array. We can handle this using com.fasterxml.jackson.annotation.JsonFormat annotation. All together gives:
import com.fasterxml.jackson.annotation.JsonFormat;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonSubTypes;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.File;
import java.io.IOException;
import java.util.List;
import java.util.StringJoiner;
public class StravaApp {
public static void main(String[] args) throws IOException {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
ObjectMapper mapper = new ObjectMapper();
mapper.readValue(jsonFile, new TypeReference<List<Data>>() {}).forEach(System.out::println);
}
}
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME,
include = JsonTypeInfo.As.EXISTING_PROPERTY,
visible = true,
property = "type")
#JsonSubTypes({
#JsonSubTypes.Type(name = "altitude", value = DoubleData.class),
#JsonSubTypes.Type(name = "latlng", value = CoordinateData.class),
#JsonSubTypes.Type(name = "velocity_smooth", value = DoubleData.class),
#JsonSubTypes.Type(name = "distance", value = DoubleData.class),
#JsonSubTypes.Type(name = "time", value = DoubleData.class)
})
abstract class Data<T> {
private String type;
#JsonProperty("series_type")
private String seriesType;
#JsonProperty("original_size")
private Integer originalSize;
private String resolution;
private List<T> data;
// getters, setters, toString
}
class DoubleData extends Data<Double> {
}
class CoordinateData extends Data<Coordinates> {
}
#JsonFormat(shape = JsonFormat.Shape.ARRAY)
class Coordinates {
private double lat;
private double lng;
// getters, setters, toString
}
Above code prints:
Data[type='altitude', seriesType='distance', originalSize=5, resolution='high', data=[519.1, 519.3, 519.3, 519.4, 519.5]]
Data[type='latlng', seriesType='distance', originalSize=5, resolution='high', data=[Coordinates[lat=46.01234, lng=6.01234], Coordinates[lat=46.11234, lng=6.11234], Coordinates[lat=46.21234, lng=6.21234], Coordinates[lat=46.31234, lng=6.31234], Coordinates[lat=46.41234, lng=6.41234]]]
Data[type='velocity_smooth', seriesType='distance', originalSize=5, resolution='high', data=[0.0, 0.0, 0.0, 5.5, 5.2]]
Data[type='distance', seriesType='distance', originalSize=5, resolution='high', data=[0.0, 8.6, 11.8, 16.6, 20.8]]
Data[type='time', seriesType='distance', originalSize=5, resolution='high', data=[0.0, 1.0, 2.0, 3.0, 4.0]]
You should also take a look on Google Dev Group and consult this solution.
Edit: I found a related question here, but the only 2 answers contradict each other, and there was not enough information to address my use case.
I am trying to use Spring Data Mongo to load records from a collection. One of the fields within those records is an Enum, defined as such:
#AllArgsConstructor
#Getter
#JsonFormat(shape = JsonFormat.Shape.OBJECT)
public enum Action {
APPROVED("Approved"),
SAVED("Saved"),
CORRECTED("Corrected");
private String name;
#JsonCreator
static Action findValue(#JsonProperty("name") String name) {
return Arrays.stream(Action.values()).filter(v -> v.name.equals(name)).findFirst().get();
}
}
This should define enums to be serialized and deserialized according to a JSON representation: {"name": "Saved"} for example.
Jackson seems to be working fine, since I threw an API call at it and told it to expect an Action type, and it read the enum without any issues.
public void save(#RequestBody #Valid Action action) {
System.out.println(action.getName());
} // successfully prints the name of whatever Action I give
However, when I try to read an object with an Action field using Spring Data Mongo, I get the following:
Expected to read Document Document{{name=Corrected}} into type class package.structure.for.some.proprietary.stuff.constants.Action but didn't find a PersistentEntity for the latter!
So I'm thinking Spring Data Mongo just can't make heads or tails of these enums for whatever reason. But I'm not sure how to help it register that as a PersistentEntity. The main class of my Spring Boot app is in package package.structure.for.some.proprietary.stuff and is annotated as such:
#ComponentScan("package.structure")
#EnableTransactionManagement
#EnableAutoConfiguration
#SpringBootApplication
The object in particular I'm trying to read is defined by this POJO:
import java.util.Date;
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.mapping.Document;
import lombok.Data;
import lombok.NonNull;
import package.structure.for.some.proprietary.stuff.constants.Action;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonPropertyOrder;
#JsonInclude(JsonInclude.Include.NON_NULL)
#JsonPropertyOrder({
"timeStamp",
"action",
})
#Data
#Document(collection = "sample_coll")
public class Sample {
#Id
#JsonIgnore
private String id = null;
#JsonProperty("timeStamp")
#NonNull
private Date timeStamp;
#JsonProperty("action")
#NonNull
private Action action;
}
and is queried from the collection with a MongoRepository:
public interface SampleRepository extends MongoRepository<Sample, String> {
}
using SampleRepository.findAll();
So my big question is, how do I get Spring Data Mongo to recognize this enum Action as a PersistentEntity?
Try #Enumerated
#Enumerated
#JsonProperty("action")
#NonNull
private Action action;
Fellow SO-er's:
I've been puzzling over this one for a couple of days, and, as of yet, don't have a solution ...
I'm building a Spring Boot web app and what I'd like to be able to do is to activate/deactivate encryption of data fields in my datastore (using the facilities provided by jasypt+spring+hibernate) via activating/deactivating configuration profiles. So that - for development work - I can have data fields stored as clear text, while for production, they would be encrypted.
Currently, I'm doing this via a rather inelegant approach. Specifically, I comment/uncomment code in my package-info.java file where I define two #TypeDefs annotation blocks for the type used for the datastore field- one of which is commented and the other of which is active. Thus, my current package-info.java file is written as follows:
//////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Use this #TypeDefs annotation when dataencrypt configuration profile is active
//#TypeDefs({ #TypeDef(name = com.castlehillgaming.gameshare.model.Ticket.ENCRYPTED_STRING_TYPENAME, typeClass = EncryptedStringType.class, parameters = {
// #Parameter(name = "encryptorRegisteredName", value = com..evilcorp.evilproject.config.EncryptionConfig.REGISTERED_NAME) }) })
//////////////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Use this #TypeDefs annotation when dataencrypt configuration profile is not active
#TypeDefs({
#TypeDef(name = com.evilcorp.evilproject.model.Ticket.ENCRYPTED_STRING_TYPENAME, typeClass = String.class, parameters = {
#Parameter(name = "encryptorRegisteredName", value = com..evilcorp.evilproject.config.EncryptionConfig.REGISTERED_NAME) }) })
//////////////////////////////////////////////////////////////////////////////////////////////////////////////
package com..evilcorp.evilproject.model;
import org.hibernate.annotations.Parameter;
import org.hibernate.annotations.TypeDef;
import org.hibernate.annotations.TypeDefs;
import org.jasypt.hibernate4.type.EncryptedStringType;
And my #Entity Ticket class contains the following:
#Entity
#EqualsAndHashCode(of = { "ticketId" })
#NoArgsConstructor(access = AccessLevel.PRIVATE, force = true)
public class Ticket implements Serializable {
...
#Column(unique = true, nullable = false)
#Type(type = ENCRYPTED_STRING_TYPENAME)
private #Getter String ticketId;
...
}
I'm hoping that I can devise something that will allow me to reduce my package-info.java file to the following:
#TypeDefs({
#TypeDef(name = com.evilcorp.evilproject.model.Ticket.ENCRYPTED_STRING_TYPENAME, typeClass = com.evilcorp.evilproject.config.MyTicketDataFieldString.class, parameters = {
#Parameter(name = "encryptorRegisteredName", value = com..evilcorp.evilproject.config.EncryptionConfig.REGISTERED_NAME) }) })
package com..evilcorp.evilproject.model;
import org.hibernate.annotations.Parameter;
import org.hibernate.annotations.TypeDef;
import org.hibernate.annotations.TypeDefs;
And define two distinct versions of MyTicketDataFieldString class based on the state of Spring Boot Configuration Profiles. E.g.,
#Configuration
#Profile("dataencrypt")
public class MyTicketDataFieldString extends EncryptedStringType {}
and
#Configuration
#Profile("!dataencrypt")
public class MyTicketDataFieldString implements CharSequence { ... }
where the CharSequence implementation behaves like a vanilla java.lang.String.
But, this won't work because I'll have to define the same class twice in a specific package.
Any ideas on how this can be done (or something equivalent) would be much appreciated.