I have a model
data class SomeComplexObject<T>(val name: String, val value: T)
#Document("model")
data class SomeModel<T>(val id: String? = null, val map: Map<SomeComplexObject<T>, Int>)
Which I save to Mongo via the save method of:
#Repository
interface SomeRepo<T>: MongoRepository<SomeModel<T>, String>
On it's own, this would throw a MappingException: Cannot use a complex object as a key value. error so i'm trying to find a work around using Converters where my Converters change the Map to a List before persisting and back from List -> Map on Read.
#WritingConverter
class Map2ListConverter: Converter<Map<SomeComplexObject<*>, Int>, List<Pair<SomeComplexObject<*>, Int>>> {
override fun convert(domainModel: Map<SomeComplexObject<*>, Int>): List<Pair<SomeComplexObject<*>, Int>> {
return domainModel.map { (key, value) -> Pair(key, value) }
}
}
#ReadingConverter
class List2MapConverter: Converter<List<Pair<SomeComplexObject<*>, Int>>, Map<SomeComplexObject<*>, Int>> {
override fun convert(dbModel: List<Pair<SomeComplexObject<*>, Int>>): Map<SomeComplexObject<*>, Int> {
return dbModel.toMap()
}
}
Which I register with
#Bean
fun customConversions(): MongoCustomConversions {
val converters = ArrayList<Converter<*, *>>()
converters.add(Map2ListConverter())
converters.add(List2MapConverter())
return MongoCustomConversions(converters)
}
This however does not work & I get a CodecConfigurationException: Can't find a codec for class kotlin.Pair.. It looks like spring tries to send a Document containing my Pair class direct to Mongo which it understandably doesn't know what to do with.
Is there a way around this? Or do I need to admit defeat and just store my Maps as Sets everywhere?
Related
I have a POJO model as follows -
public class Model implements Serializable {
private Map<String,Set<String>> map;
}
which I am trying to write to a parquet file using Spark. The code for the same is follows:
JavaRDD<Context> dataSet = generate();
JavaPairRDD<Model, Model> outputRDD = dataSet.mapToPair((PairFunction<Context, Model, Model>)
context -> {
return new Tuple2<>(context.getMoel1(), context.getModel2());
});
Dataset<Tuple2<Model, Model>> outputDS = sqlContext.createDataset(JavaPairRDD.toRDD(outputRDD),
Encoders.tuple(Encoders.bean(Model.class), Encoders.bean(Model.class)));
outputDS.coalesce(numPartitions).write().mode(SaveMode.Overwrite).parquet(outputPath + "v2/");
It gives me the following exception because of using a Set<> inside the map.
Caused by: org.apache.parquet.schema.InvalidSchemaException: Cannot write a schema with an empty group: optional group value {
}
at org.apache.parquet.schema.TypeUtil$1.visit(TypeUtil.java:27)
at org.apache.parquet.schema.GroupType.accept(GroupType.java:255)
at org.apache.parquet.schema.TypeUtil$1.visit(TypeUtil.java:31)
at org.apache.parquet.schema.GroupType.accept(GroupType.java:255)
at org.apache.parquet.schema.TypeUtil$1.visit(TypeUtil.java:31)
So, I tried making it a Map<String, List<String>> and it worked fine.
However, since this model is used all over the code base, there will be many repurcussions for changing it.
Why is this happening? And how to resolve this?
Thanks in advance!
P.S. I am using Spark 3.1.2
I am trying to get application property object by value, i already did this in Java, but from some reason using Kotlin i can not manage to do it.
So basically what i have is list of application properties that looks like this:
ee.car.config.audi.serial=1
ee.car.config.audi.base=platform1
ee.car.config.bmw.serial=2
ee.car.config.bmw.base=platform2
so as you can see car identifiers (audi,bmw,peugeot,etc..) are dynamic, and i need simply by serial value to get object that represents the specific car and by car key(audi, bmw) to get all other properties.
And what i did is simple, i created configuration properties class like this:
#Configuration
#ConfigurationProperties(prefix = "ee.car")
data class FesEngineKeys(
val config: HashMap<String, EeCarConfigParam> = HashMap()
) {
fun getOrDefaultEEConfig(engineKey: String): EeCarConfigParam? {
return config.getOrDefault(engineKey, config["audi"])
}
And then object to map keys after dynamic value:
data class EeCarConfigParam {
var serial: String,
var base: String
}
But problem here is, in FesEngineKeys class, config property is empty, it looks like EeCarConfigParam can not be mapped.
Also interesting part is when i change:
val config: HashMap<String, EeCarConfigParam> = HashMap() to
val config: HashMap<String, String> = HashMap()
then i can see that config param is populated with all the values.
This code already works in Java and it looks like this:
#Configuration
#Getter
#Setter
#ConfigurationProperties(prefix = "ee.car")
public class FESEngineKeys {
private Map<String, EeCarConfigParam> config = new HashMap<>();
public EeCarConfigParam getOrDefaultEEConfig(String engineKey) {
return config.getOrDefault(engineKey, config.get("audi"));
}
public EeCarConfigParam findBySerial(String serial) {
return config.values().stream().filter(cfg -> cfg.getSerial().equalsIgnoreCase(serial)).findFirst().orElse(null);
}
}
#Data
public class EeCarConfigParam {
private String serial;
private String base;
}
I really don't know why in the Kotlin case it is not working, i probably made some very basic mistake, and i would really appreciate if anyone can explain what is happening here
Okay i got it.
According to that: https://docs.spring.io/spring-boot/docs/2.0.x/reference/html/boot-features-kotlin.html the support for what you want is very limited.
I made it working like that - not pretty nice :-( :
#ConfigurationProperties(prefix = "ee.car")
class FesEngineKeyProperties() {
var config: MutableMap<String, EeCarConfigParam?>? = mutableMapOf()
fun getBase(serial: String): String {
if(config == null) return ""
return config!!["audi"]?.base ?: ""
}
}
class EeCarConfigParam() {
lateinit var serial: String
lateinit var base: String
}
#SpringBootApplication
#EnableConfigurationProperties(FesEngineKeyProperties::class)
class SandboxApplication
fun main(args: Array<String>) {
runApplication<SandboxApplication>(*args)
}
I was able to handle this issue, it is somehow related to kotlin, because once when i instead of this:
data class EeCarConfigParam {
var serial: String,
var base: String
}
used "norma" Java class, everything started working, so all code from my question stays the same, only difference is this: instead of Kotlin EeCardConfigParam i created Java class like this:
public class EeCarConfigParam {
private String publicUrl;
private String base;
}
Note: with all default getters, setters, equals, hash and toString methods.
I have a spring JPA entity with numeric properties which should be serialized as string values that are the result of a lookup in a code-list.
#Entity
class Test {
String name ;
int status ;
}
Should be serialized by looking up the numeric value in a code-list like so:
{ "name" : "mytest" , "status" : "APPROVED" }
The code-list is implemented as another entity and can be accessed using a spring-boot JPA repository.
The solution I am looking for must be scalable in that
the code-list cannot be loaded from the database again for each serialization or new object
the serialization code must be generic so that it can be re-used in other entities.
This is, other entities also have numeric properties and their own corresponding code-list and repository.
I understand one could either use a custom Jackson serializer, or implement the lookup as part of the entity. However, neither seems to satisfy the conditions above. A custom jackson serializer can only have the repository autowired if I share it or the map using a static field. The static field makes it hard to re-use the serializer implementation for other entities.
Implementing the lookup as part of the entity, say as custom getter, would make the code hard to re-use, especially since the map for the code-list lookup must be shared across instances.
Update: A third strategy would be to add JPA relationships to the code-list in my entities and to expose the value in a custom getter. For the deserialization rather inefficient lookups would be required, though.
This works, but the static map prevents re-using the code.
#Component
public class NewApprovalSerializer extends JsonSerializer<Integer>
{
#Autowired
ApprovalStatusRefRepository repo;
static Map<Integer, String> map = new HashMap<Integer,String>() ;
#PostConstruct
public void init() {
for ( TGoApprovalStatusRef as : repo.findAll() ) {
Integer key = new Integer( as.getApprovalStatusId() );
String val = as.getNameTx();
map.put( key , val );
}
}
public NewApprovalSerializer() {
SpringBeanAutowiringSupport.processInjectionBasedOnCurrentContext(this);
}
public void serialize(Integer value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
gen.writeObject( map.get( value ) );
}
}
I could change the entity like such, but I again have a static map and the code is even harder to re-use in another entity.
#Entity
class Test {
String name ;
#JsonIgnore
int status ;
static Map<Integer, String> map = new HashMap<Integer,String>() ;
public Test() {
... init static map
}
public String getStatus() {
return map.get(this.status);
}
}
What is the standard way to implement a lookup of values uppon serialization (and vice versa in deserialization)?
I am trying to use Jackson to serialize the same DTO object in 2 different ways, depending on the #JsonView.
I want to use 2 different names for the same field. In one case I want to name the json property myField (just like the class field name), in the other I want it to be named myInternalApiField.
As a result I would like to see outcomes similar to the presented below:
Usage 1 (External API View):
{
"myField": "value1",
"myOtherField": "otherValue"
}
Usage 2 (Internal API View):
{
"myInternalApiField": "value1",
"myOtherField": "otherValue"
}
In my implementation in Java to achieve that I used the combination of custom getters, setters and #JsonView annotation as below:
public class CustomDTO {
#JsonView(Views.ExternalApiView)
private String myField;
// Other fields here
#JsonView(Views.InternalApiView)
public String getMyInternalApiField() { return myField; }
#JsonView(Views.InternalApiView)
public void setMyInternalApiField(String value) { this.myField = value; }
#JsonView(Views.ExternalApiView)
public String getMyField() { return myField; }
#JsonView(Views.ExternalApiView)
public void setMyField(String value) { this.myField = value }
}
However I don't know how to properly achieve the same result in Kotlin.
I was thinking about using something like:
data class CustomDTO(
#get:[JsonView(Views.ExternalApiView) JsonProperty("myField")]
#get:[JsonView(Views.InternalApiView) JsonProperty("myInternalApiField")]
#set:[JsonView(Views.InternalApiView) JsonProperty("myField")]
#set:[JsonView(Views.InternalApiView) JsonProperty("myInternalApiField")]
var myField: String,
val myOtherField: String,
val myDifferentField: String
)
But this is not allowed in Kotlin.
Do you have any suggestions how to utilize the #JsonView in Kotlin in the similar way as I did it in Java?
How about something like:
data class CustomDTO(
#JsonView(ExternalApiView::class)
var myField: String,
val myOtherField: String,
val myDifferentField: String
) {
val myExternalField: String
#JsonView(InternalApiView::class)
get() {
return myField
}
}
It looks like there are ways that don't require creating computed properties in the DTO, like:
Using Jackson Mixins
Creating a custom serializer for a particular invocation
Combining a custom serializer with custom annotations
But these have their own complexity, even if that complexity isn't in the DTO class. I'm not sure these are much more appealing to me but you could see if they appeal to you.
sealed class Entity
data class Bacteria(
val uid: String,
val rank: String,
val division: String,
val scientificname: String,
val commonname: String
): Entity()
data class CTDDisease(
val diseaseId: String,
val name: String,
val altDiseaseIds: List<String>,
val parentIds: List<String>,
val definition: String?
) : Entity()
And then I want to define my document as
#Document(collection = "annotations")
data class Annotation(
#Id val id: String,
...
val spans: List<AnnotationSpan>
)
data class AnnotationSpan(
val startIndex: Int,
val endIndex: Int,
val entity: Entity? // Can be Bacteria or Disease or null
)
I also accept these 2 classes within the RequestBody from the client from time to time e.g.
#PutMapping("/annotations")
fun submitAnnotations(#RequestBody submission: Submissions): Mono<Void> { ... }
data class Submission(val annotations: List<AnnotationSpan>, ...) // AnnotationSpan contains Entity
but I get
java.lang.IllegalAccessException: class kotlin.reflect.jvm.internal.calls.CallerImpl$Constructor cannot access a member of class com.package.name.Entity with modifiers "private"
If I change the class Entity to an abstract class
abstract class Entity
then I don't get an error but my query operations keep going on forever.
Bacteria and Disease both have different fields so they should be distinguishable.
I tried using a hacky converter
#ReadingConverter
class NormalizedEntityReaderConverter: Converter<DBObject, NormalizedEntity> {
override fun convert(source: DBObject): NormalizedEntity? {
println(source.toString())
val gson = Gson()
return gson.runCatching { fromJson(source.toString(), CTDDisease::class.java) }.getOrNull()
?: gson.runCatching { fromJson(source.toString(), Bacteria::class.java) }.getOrNull()
}
}
and then registering it like
#Configuration
class MongoConverterConfig {
#Bean
fun customConversions(): MongoCustomConversions {
val normalizedEntityReaderConverter = NormalizedEntityReaderConverter()
val converterList = listOf<Converter<*, *>>(normalizedEntityReaderConverter)
return MongoCustomConversions(converterList)
}
}
My converter seems to work when called manually but for some reason, Spring still isn't picking it up.
I'm new to Spring. I would achieve this functionality in my Node.js server by using union types in TypeScript e.g.
interface AnnotationSpan {
startIndex: number;
endIndex: number;
entity?: Bacteria | Disease;
}
How can I achieve this behavior?