I am reading from a stream that provides updates to an order book used to calculate market depth. Each includes a list of new entries for the order book. Each entry contains three properties.
Market side (e.g. buy or sell)
Quantity transacted of the commodity
Unit price of the commodity in this transaction
These entries are represented in JSON as array nodes. Here is an example of how entries might be provided.
{
"Changes": [
{ "entry": ["buy","470.84724800000004","16.14963"] },
{ "entry": ["buy","470.787392","0.01"] },
{ "entry": ["sell","473.112752","9.325423"] },
{ "entry": ["sell","473.052608","11.80723"] }
],
...some more fields; not relevant to this question...
}
As you can see, the indices of each entry array are used as field names. The position of each array element defines what property it represents. The side is at index 0, the unit price is at index 1, and the quantity is at index 2.
How can I de/serialize these using arrays Jackson annotations in Java 8? I am only asking about the innermost arrays. I don't need help with the object structure in general.
I tried making a class similar to the following.
public class OrderBookEntry {
final String side;
final BigDecimal price;
final BigDecimal quantity;
#JsonCreator
public OrderBookEntry(#JsonProperty(index = 0, required = true) String side,
#JsonProperty(index = 1, required = true) BigDecimal price,
#JsonProperty(index = 2, required = true) BigDecimal quantity) {
this.side = side;
this.price = price;
this.quantity = quantity;
}
}
I have tried specifying #JsonFormat.Shape.ARRAY on the class. Every time I try to deserialize a sample string, I get an InvalidDefinitionException.
Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidDefinitionException:
Invalid type definition for type com.example.OrderBookEntry: Argument #0 has no property name, is not Injectable: can not use as Creator [constructor for com.example.OrderBookEntry, annotations: {interface com.fasterxml.jackson.annotation.JsonCreator=#com.fasterxml.jackson.annotation.JsonCreator(mode=DEFAULT)}]
at [Source: (String)"["buy","470.84724800000004","16.14963"]"; line: 1, column: 1]
Is there not a way to do this with only annotations?
P.S. (rant)
I'd just like to add, this is an absurd data structure. It makes no sense. The purpose of using array indices instead of object field names would be to reduce the size of messages. This is a financial data stream, and any improvement to the network latency of financial data is desirable. But, around each of these arrays is a completely superfluous wrapper object with a single named field. This adds at least 10 bytes of unnecessary traffic per entry. The data structure has a very poor design.
There is more than one way to do this but I always prefer the combination of a dedicated class with a dedicated serializer. Other options would be:
registering the serializer with ObjectMapper – explicit coding instead of (meta level) annotations
use a general set method (with Map<String, Object>) – hides the serializer's aspect in a lengthy method
mapping JsonNode with setter in parent class – like #2
Instead:
#JsonDeserialize(using = OrderBookEntryDeserializer.class)
public class OrderBookEntry {
// no further Jackson-annotations necessary
}
And the serializer:
class OrderBookEntryDeserializer extends StdDeserializer<OrderBookEntry> {
OrderBookEntryDeserializer() {
super(OrderBookEntry.class);
}
#Override
public OrderBookEntry deserialize(JsonParser p, DeserializationContext ctxt)
throws IOException, JsonProcessingException {
final JsonNode node = p.getCodec().readTree(p);
// now follow Jackson API to turn the node into
// an ArrayNode, there are plenty of is-methods
// and it requires a hard cast.
}
My example uses a package private deserializer class which works absolutely fine and locks it in its package (preferrably next to the OrderBookEntry).
Related
I have two classes (entity and DTO)
public class Deliver {
private Long id;
private String uri;
private Instant moment;
private DeliverStatus status; // enum PENDING,ACCEPTED,REJECTED
private String feedback; // feedback about received task
private Integer correctCount; // nr of correct questions
private Enrollment enrollment;
private Lesson lesson;
// constructors, getters and setters..
public class DeliverRevisionDto {
private DeliverStatus status;
private String feedback;
private Integer correctCount;
// constructors, getters and setters..
The goal is pretty simple, update the entity fields conveyed by Dto class I have the following code at Service layer (Spring Boot version 2.4.4):
#Service
public class DeliverService {
#Autowired
private DeliverRepository deliverRepository;
#Autowired
private ModelMapper modelMapper;
#Transactional
public void saveRevision(Long id, DeliverRevisionDto dto) {
Deliver deliver = deliverRepository.getOne(id);
System.out.println("BEFORE MAPPING: " + deliver.toString()); // # debug purpose
deliver = modelMapper.map(dto, Deliver.class);
// # debug purpose
TypeMap<DeliverRevisionDto, Deliver> tm = modelMapper.getTypeMap(DeliverRevisionDto.class, Deliver.class);
List<Mapping> list = tm.getMappings();
for (Mapping m : list)
{
System.out.println(m);
}
System.out.println("AFTER MAPPING: " + deliver.toString()); // # debug purpose
deliverRepository.save(deliver);
}
}
The console output is:
BEFORE MAPPING: Deliver [id=1, uri=``https://github/someone.com``, moment=2020-12-10T10:00:00Z, status=PENDING, feedback=null, correctCount=null, enrollment=com.devsuperior.dslearnbds.entities.Enrollment#7e0, lesson=com.devsuperior.dslearnbds.entities.Task#23]`
`PropertyMapping[DeliverRevisionDto.correctCount -> Deliver.correctCount]`
`PropertyMapping[DeliverRevisionDto.feedback -> Deliver.feedback]`
`PropertyMapping[DeliverRevisionDto.status -> Deliver.status]`
`AFTER MAPPING: Deliver [id=null, uri=null, moment=null, status=ACCEPTED, feedback=Muito bem cabra, tarefa aceita., correctCount=5, enrollment=null, lesson=null]
The mapping of the 3 fields in DTO is done correctly, BUT all the other fields of my entity are set to null. I know that I can skip fields according http://modelmapper.org/user-manual/property-mapping/
The problem is that I don´t want to couple the code with specific field names/getters/setters, that´s the reason I´m using ModelMapper. I wonder if there is any configuration that, upon mapping the modelmapper object says "Hey, the TARGET class have way more fields than the SOURCE class, I will left them untouched unconditionally (meaning I don´t need to say what fields are).
I'm trying to map fields between 2 classes with different set of fields (some are the same), and when I map the class with smaller set of fields to the one with bigger set of fields, the mapper set fields that don´t match with "null", I want these fields untouched (with original values) without I telling which one they are, after all, the mapper knows which ones match.
ModelMapper documentation is not the best part of that framework. Let us see what happens in your code.
Here you fetch the entity to be updated from the repo:
Deliver deliver = deliverRepository.getOne(id);
and log it having all the fields as should be. However this line:
deliver = modelMapper.map(dto, Deliver.class);
does a re-assignment to your variable deliver. This method creates a new instance of Deliver class and assigns it to variable deliver so discarding the entity fetched from repo.
This new instance will have all the fields that are not existing or not set in DTO null.
This is the API doc that my IDE provides, fotr these two different methods:
String org.modelmapper.ModelMapper.map(Object source, Class destinationType)
Maps source to an instance of destinationType. Mapping is performed according to the corresponding TypeMap. If no TypeMap exists for source.getClass() and destinationType then one is created.
Versus
void org.modelmapper.ModelMapper.map(Object source, Object destination)
Maps source to destination. Mapping is performed according to the corresponding TypeMap. If no TypeMap exists for source.getClass() and destination.getClass() then one is created.
It might not be clearly stated that the first method actually creates a new instance based on the type (Class) passed but it should be clear that ModelMapper cannot alter some arbitrary variable just by knowing the type. You need to pass the variable to alter as method parameter.
I have an API which is being extended to be consumed by another system. Pre-req's exist whereby each frontend has different expectations for one json field.
Example:
The response field 'amount' must be either a String or an int, depending on which value I receive from downstream. So for some instances I will return a string value in the json, while in others I will return int.
Expected json outputs:
{
"amount": 21
}
or
{
"amount": "21"
}
I have done the following:
class Response {
#JsonProperty("amount")
private int amount;
#JsonProperty("amount")
private String amountString;
// Getters and setters
Hoping that I would be able to return either an int or String value for the json field 'amount' but I'm getting the following error:
com.fasterxml.jackson.databind.JsonMappingException: Conflicting getter definitions for property "amount"
Any help would be appreciated
You can not do that. The element in your JSON is called amount, you can not bind it to 2 different variables.
You are getting this exception because the #JsonProperty("amount") is used on 2 seperate class fields.
It is best to create 2 different response:
Example:
class FrontEndResponse1 {
#JsonProperty("amount")
private int amount;
//getter/setter
}
class FrontEndResponse2 {
#JsonProperty("amount")
private String amount;
//getter/setter
}
Yes, this won't work with typed return values.
Unless you're fine with providing 2 endpoints with different return types you could return a plain Map<String, Object> instead of Response:
public Map<String, Object> awkwardMethod(boolean wantString) {
Response response = new Response();
... // do business logic
return Map.of("amount", wantString ? String.value(response.amount) : response.amount);
}
To be clear: I'm not encouraging you to do this as you'll loose type information for the return value and introduce error prone manual mapping. I'm just providing a solution for a problem that should be fixed by providing a clear API and clients sticking to this API.
I am coming from Java background and trying to understand how to model Domain classes/POJOs in Scala.
I am trying to Deserialize JSON response from a RestAPI and my Java POJOs are as follows:
#Data
public class ColumnResponse {
private String id;
private String name;
private String type;
...
}
k
#Data
public class DataSetGetResponse {
private String id;
private List<ColumnResponse> columns;
...
}
Now I have created following Case Classes
case class DataSetGetResponse (id: String,
columns: List[ColumnResponse]
.... )
case class ColumnResponse (id: String,name: String ...)
I am trying to use https://sttp.readthedocs.io/en/latest/json.html#json4s library for HTTP communication and json4s for deserialization.
Questions:
1) In the DataSetGetResponse case class, field "columns" is a List.By default this is an immutable list. How the Deserialization library add new DataColumnGetResponse objects to this immutable list? Do I have to declare this as mutable ?
2) There is a field called 'type' field in the ColumnResponse POJO. In Scala 'type' is a reserved keyword.How to handle this case?
Answer the first one:
An immutable object can be mutated with the copy function:
dataSet.copy(columns = newResp :: dataSet.columns)
For more complex tasks you can use Lenses see for example here: enter link description here
Answer the second one:
If it is a reserved word you can do it like
case class ColumnResponse (id: String, name: String, `type`: String)
This answer addresses the following aspect of the question:
How the Deserialization library add new DataColumnGetResponse objects
to this immutable list?
Let us consider a simplified version of the problem:
JsonMethods.parse("""[1,2,3]""").extract[List[Int]]
How does json4s deserialise [1,2,3] into immutable List[Int]? First it parses the raw string into an intermediary AST (abstract syntax tree) data structure where it represents the list like so
case class JArray(arr: List[JValue]) extends JValue
We see here that arr is an immutable list. The key line that builds it up after parse executes is in JsonParser
def newValue(v: JValue): Unit = {
...
case a: JArray => vals.replace(JArray(v :: a.arr))
...
}
Note how the operator :: in v :: a.arr adds an element at the beginning of this list and returns a new list with v added in. This means since there are three elements in [1,2,3] the following three lists are created by json4s in the process of deserialisation
JArray(List(JInt(1))
JArray(List(JInt(2), JInt(1)))
JArray(List(JInt(3), JInt(2), JInt(1)))
Again note these are three separate lists.
Next, after internal AST is created, actual deserialisation to List[Int] takes place by calling extract[List[Int]]. The key component that does this for lists is CollectionBuilder
private class CollectionBuilder(json: JValue, tpe: ScalaType)(implicit formats: Formats) {
...
val array: Array[_] = json match {
case JArray(arr) => arr.map(extractDetectingNonTerminal(_, typeArg)).toArray
...
}
Note how we simply map over AST arr built up during parsing step and convert each element to the model of type typeArg, which in our simple case is Int but in your case would be DataColumnGetResponse.
tl;dr
Atempting to add an ArrayList in which Object may be an ArrayList to Persistance.
Tried to add an AttributeConverter > Failed
Plz Help
I have no idea what I am doing.
How stupid am I?
The Problem
Dependencies
spring-boot-starter-data-jpa 2.0.0
spring-boot-starter-data-mongodb 2.0.0
eclipselink 2.7.1 <- Probably don't need this one, not sure.
So here is my problem I am trying to add persistence in a Spring Boot Application for a MongoDB in this case I am using tables, the problem comes exactly on the TableRaw bean (a striped down version of Table just for persistance).
Document(collection = "rule_tables")
public class TableRaw {
#Id
private String _id;
private String key;
private String name;
private String returns;
private ArrayList<AxisRaw> axis;
private ArrayList<Object> values = new ArrayList<>();
}
Everything else is just the default constructor (without _id) and getsetters.
So everything works fine with the exception of the values ArrayList. It works fine if it just a simple ArrayList with number and whatnot however in my case I want something like what I am inserting into the database (this is done every time it runs for testing purposes and the values inserted are using the MongoRepository, it works fine)
{
"_id":"5ac20c8b8ee6e6360c8947be",
"key":"1",
"name":"Table 1",
"returns":"Number",
"axis":[
{
"name":"potato",
"values":[
{
"_id":"BottomEdge","value":0
},{
"_id":"Range",
"value":[1,2]
},{
"_id":"TopEdge",
"value":3
}
]
}
],
"values":[
[1,2,3],
[1,2,3],
[1,2,3]
],
"_class":"pt.i2s.gm.gm.rulehandler.tables.model.TableRaw"
}
(For usage in the code the axis length and number of axis matters but in this case it is completely irrelevant.)
Anyway as stated previously it inserts fine into MongoDB but when attempting to get the value the following error is presented.
org.springframework.data.mapping.MappingException: Cannot convert [1, 2, 3] of type class java.util.ArrayList into an instance of class java.lang.Object! Implement a custom Converter<class java.util.ArrayList, class java.lang.Object> and register it with the CustomConversions. Parent object was: [empty]
First thing first I don't exactly know what Parent object was: [empty] means.
Second I tried creating an AttributeConverter as such:
#Component
#Converter(autoApply = true)
public class ArrayList2ObjectConverter implements
AttributeConverter<ArrayList<Object>,Object> {
#Override
public Object convertToDatabaseColumn(ArrayList<Object> attribute) {
return attribute;
}
#SuppressWarnings("unchecked") //If you don't like it suppress it
#Override
public ArrayList<Object> convertToEntityAttribute(Object dbData) {
System.out.println("Converting...");
return (ArrayList<Object>)dbData;
}
}
And adding #Convert(converter = ArrayList2ObjectConverter.class) above the values attribute. However this wasn't even called.
For some reason I couldn't find any answers to this problem, possibly due to my bad coding and making something that is just stupid to do so nobody would do it like this cause it doesn't work.
So how do I do this? And thank you for reading.
Update regarding the Axis and Value amounts
thomi sugested something that would work if I knew from the get go what type of values the table added. I apreciate the answere however some clarification should be made regarding this.
I do not know how many Axis, and therefore nested arrays I will have, it may be 1 it may be 30.
I do not know what the class type of objects will be, it may be numbers, Strings, Booleans, dates, etc. the options are limited but still extensive.
Possible Solution Which I Do Not don't want to use
I could simply create an Object that held a string and an ArrayList which would probably work fine, however I wanted to avoid this resolution, as I don't want to add irrelevant information to the database.
Adopted Solution
By request of #user_531 I will add the solution to this problem.
As this was not working I altered my aproach to the utilization of a new object called ValueList which is simply a wrapper class for a single Object
private ArrayList<ValueList> values;
ValueList Class
public class ValueList {
public Object value;
}
This allows me to add any type of object I want to the list, this does result however in tables looking like this:
{
"key":1,
...... (Same as above)
"values": [
{
"value": [
{
"value":1
},
{
"value":2
}
]
},
{
"value": [
{
"value":3
},
{
"value":4
}
]
}
]
}
Which does look hidious but it doesn't fail anymore and allows me to read values relativelly consistently by calling the "getValue()" method or "getValueList()" method acording to the result from "isValueList()".
I think you should not map something to an object. In your DB, you will surely have an idea of what datatype there will be in your Array, In your case, try and replace with:
#Document(collection = "rule_tables")
public class TableRaw {
#Id
private String _id;
private String key;
private String name;
private String returns;
private ArrayList<AxisRaw> axis;
private List<List<Integer>> values; // no initialization.
}
This should map your structure just fine.
I have an entity with Integer attributes that looks like this in proto code:
class MyEntity:
String name
#Choices({1, "BSD", 2, "Apache", 3, "GPL"}
Integer frequency
#ChoicesSegment({1000, 2000, 3000}, {"BSD", "Apache", "GPL"})
Integer type
String getFrequency()
return getChoice("frequency", frequency)
String getType()
return getChoice("type", type)
maybe this solution is more feasible:
class MyEntity:
String name
final static private Something? frequencyChoices = {1000, 2000, 3000}, {"BSD", "Apache", "GPL"}
Integer frequency
final static private String[] typeChoices = new String[] {"BSD", "Apache", "GPL"}
Integer type
#Choices(MyEntity.frequencyChoices)
String getFrequency()
return frequency)
#IntervalChoices(MyEntity.typeChoices)
String getType()
return type
*get** accessors return strings according to this table.
value(type) HumanReadableString(type)
1 BSD
2 Apache
3 GPL
min frequency max frequency HumanReadableString(frequency)
0 1000 rare
1000 2000 frequent
2001 3000 sexy
It should be possible to get all possible values that an attribute can take, example:
getChoices(MyEntity, "type") returns ("rare", "frequent", "sexy")
It should be possible to get the bound value from the string:
getValue(MyEntity, "frequency", "sexy") returns (2000,3000)
edit: purpose of all of this This methods should simplify the generation of forms and requests (of course this should not be view implementation bound)
edit: added how I would like to tell Java that some attributes are spécial so that it can generate get* accessors acordingly.
edit: added how to submit in the code the choices
edit: the only thing that I store in the db is integers, when I want to print them they should be converted somehow to their human readable string.
You can have additional info in enums:
public enum License {
GPL("GPL"),
APACHE("Apache License");
public License(String displayName) {
this.displayName=displayName;
}
String displayName;
}
Additional functions as required, but have a close look which functions are already provided by the Enum classes.
You can do it without any hassle (but note, that the value in the DB will be the ordinal() value of the enums. So:
public enum License { GPL, APACHE, BSD }
FrequencyChoices could go into an #ElementCollection.
If you need human readable values, you may want to convert your Enum to an ordinary class, and persist it as a separate table, so you can add new licenses more easily to the list...
#Entity
public class License {
#Id long id;
String name;
}
I have not tried to persist this but you can try following http://marekhalmo.blogspot.sk/2012/09/cool-java-enums.html
I would stick to enums.