Spring Data DynamoDB - Respository performing scan instead of query - java

Expected Behavior
Repository perform a query over a method that only uses hashKey and rangeKey attributes, and result this:
"{"TableName":"music","KeyConditionExpression":"artist = :_artist and begins_with(title, :_title)","ExpressionAttributeValues":{":_artist":{"S":"Djavan"},":_title":{"S":"Eu te devoro"}}}"
Actual Behavior
Repository perform a scanFilter over a method that only uses hashKey and rangeKey attributes, and result this:
"{"TableName":"music","ScanFilter":{"artist":{"AttributeValueList":[{"S":"Djavan"}],"ComparisonOperator":"EQ"},"title":{"AttributeValueList":[{"S":"Eu te devoro"}],"ComparisonOperator":"BEGINS_WITH"}}}"
Steps to Reproduce the Problem
Using a entity named Music
#DynamoDBTable(tableName = "music")
data class Music(
#field:Id
#DynamoDBIgnore
val id: MusicId = MusicId(),
var genre: String? = null
) {
#DynamoDBHashKey(attributeName = "artist")
fun getArtist() = id.artist
fun setArtist(artist: String) {
id.artist = artist
}
#DynamoDBHashKey(attributeName = "title")
fun getTitle() = id.title
fun setTitle(title: String) {
id.title = title
}
}
#DynamoDBDocument
data class MusicId(
#field:DynamoDBHashKey(attributeName = "artist")
var artist: String? = null,
#field:DynamoDBRangeKey(attributeName = "title")
var title: String? = null
) : Serializable
And a repository
#EnableScan //I know that if I remove this annotation, enables won't be permitted, but the problem is that the implementation code doesn't recognize my method as a key query and if I remove this annotation, the method falls on invocation
interface MusicRepository : CrudRepository<Music, MusicId> {
fun findByArtistAndTitleStartingWith(artista: String, sortKey: String): List<Music>
}
And when i invoke:
#PostConstruct
fun init() {
println(musicRepository.findByArtistAndTitleStartingWith("Djavan", "Eu te devoro").joinToString())
}
the log show's me the call to AWS as i showed above
Specifications
Lib: https://github.com/boostchicken/spring-data-dynamodb
Spring Data DynamoDB Version: 5.2.5
Spring Data Version: Doesn't used
Spring Boot Starter Web Version: 2.3.4.RELEASE
AWS SDK Version: 1.11.573
Java Version: 11
Platform Details: Windows
Github issue
did I something wrong? Or is the other approach that spring data create the correct query to aws?

After trying random changes in entity. I got the expected result in the repository method.
Correct way to map a Entity with composite key
#DynamoDBTable(tableName = "music")
data class Music(
#get:DynamoDBHashKey(attributeName = "artist")
var artist: String? = null,
#get:DynamoDBRangeKey(attributeName = "title")
var title: String? = null,
var genre: String? = null
) {
#Id
private var id: MusicId? = null
get() = MusicId(artist, title)
}
#DynamoDBDocument
data class MusicId(
#field:DynamoDBHashKey(attributeName = "artist")
var artist: String? = null,
#field:DynamoDBRangeKey(attributeName = "title")
var title: String? = null
) : Serializable
After change the entity and invoke:
musicRepository.findByArtistAndTitleStartingWith("Djavan", "Eu te devoro")
I got the corret invocation of AWS api.
"{"TableName":"music","ConsistentRead":true,"KeyConditions":{"artist":{"AttributeValueList":[{"S":"Djavan"}],"ComparisonOperator":"EQ"},"title":{"AttributeValueList":[{"S":"Eu te devoro"}],"ComparisonOperator":"BEGINS_WITH"}},"ScanIndexForward":true}"

Related

Exception while computing database live data with Enum

I have added a data class and try to save it into Room. I went through stackoverflow and didn't find an answer.
So, the error is:
Caused by: com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was BEGIN_ARRAY at line 1 column 2 path $
Caused by: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was BEGIN_ARRAY at line 1 column 2 path $
I am using Room 2.4.2 so enum is supposed to be supported.
The model I am using is :
#Entity(tableName = "userpreferencestable")
class UserPreferencesEntity (
#PrimaryKey()
var vin: String,
#ColumnInfo(name = "control")
var command: List<CommandTile?>
)
and CommandTile is defined as below:
data class CommandTile(
#SerializedName("name")
var name: DashboardTile.Name,
#SerializedName("state")
var state: DashboardTile.State
)
State and Name are enum and defined as below:
enum class Name {
NAME1,
NAME2...
}
enum class State {
TAT,
TOT
}
I have tried to add a DataConverter but it's not working.
#TypeConverter
fun fromName(name: Name): String {
return name.name
}
#TypeConverter
fun toName(name: String): Name {
return Name.valueOf(name)
}
#TypeConverter
fun fromState(state: State): String {
return state.name
}
#TypeConverter
fun toState(state: String):State {
return State.valueOf(state)
}
It still not working. I cannot figure out how to save the List of data class with enum.
Any idea ?
You issue is not the Enums, rather it is with the command List<CommandTile> (according to the disclosed code).
TypeConverters are for converting from/to a column for the data to be stored/retrieved.
As you have no #Entity annotation for the CommandTile class BUT instead have List<CommandTile?> as a column in the UserPrefrences class, which does have #Entity annotation, then Room will want to convert the List of CommandTiles to an acceptable type (in SQLite along with Room's restrictions this would have to be a type that resolves to one of TEXT (String), INTEGER (Int, Long ...), REAL (Double, Float ....) or BLOB (ByteArray).
types in parenthesise are Kotlin Types, they are examples and are not fully comprehensive.
As an example, overcoming issue that you may encounter using List, consider the following:-
A new class CommandTileList
data class CommandTileList(
val commandTileList: List<CommandTile>
)
to avoid a column that is a List
A modified UserPreferencesEntity class to use a CommandTileList rather than List<CommandTile>
#Entity(tableName = "userpreferencestable")
class UserPreferencesEntity (
#PrimaryKey()
var vin: String,
#ColumnInfo(name = "control")
var command: CommandTileList
)
the TypeConverters class, with appropriate TypeConverters
class TypeConverters {
#TypeConverter
fun fromCommandTileToString(commandTileList: CommandTileList): String {
return Gson().toJson(commandTileList)
}
#TypeConverter
fun fromStringToCommandTile(string: String): CommandTileList {
return Gson().fromJson(string,CommandTileList::class.java)
}
}
A suitable #Dao annotated class AllDAO (for demo)
#Dao
interface AllDAO {
#Insert(onConflict = OnConflictStrategy.IGNORE)
fun insert(userPreferencesEntity: UserPreferencesEntity): Long
#Query("SELECT * FROM userpreferencestable")
fun getAllUserPreferences(): List<UserPreferencesEntity>
}
A suitable #Database annotated class TheDatabase (for demo) noting the TypeConverters class being defined with full scope via the #TypeConverters annotation (not the plural rather than singular form)
#TypeConverters(value = [TypeConverters::class])
#Database(entities = [UserPreferencesEntity::class], version = 1, exportSchema = false)
abstract class TheDatabase: RoomDatabase() {
abstract fun getAllDAO(): AllDAO
companion object {
var instance: TheDatabase? = null
fun getInstance(context: Context): TheDatabase {
if (instance == null) {
instance = Room.databaseBuilder(context,TheDatabase::class.java,"the_database.db")
.allowMainThreadQueries()
.build()
}
return instance as TheDatabase
}
}
}
.allowMainThreadQueries for convenience/brevity
Finally putting it all into action witin an Activty MainActivity
class MainActivity : AppCompatActivity() {
lateinit var db: TheDatabase
lateinit var dao: AllDAO
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
val c1 = CommandTile(DashboardTile.Name.NAME1,DashboardTile.State.TAT)
val c2 = CommandTile(DashboardTile.Name.NAME2,DashboardTile.State.TOT)
db = TheDatabase.getInstance(this)
dao = db.getAllDAO()
dao.insert(userPreferencesEntity = UserPreferencesEntity("VIN1", CommandTileList(listOf(
c1,c2))))
for(u in dao.getAllUserPreferences()) {
Log.d("DBINFO","VIV = ${u.vin} CommandTiles in Command = ${u.command.commandTileList.size} They Are:-")
for (ct in u.command.commandTileList) {
Log.d("DBINFO","\tName = ${ct.name} State = ${ct.state}")
}
}
}
}
Result
The log includes:-
D/DBINFO: VIV = VIN1 CommandTiles in Command = 2 They Are:-
D/DBINFO: Name = NAME1 State = TAT
D/DBINFO: Name = NAME2 State = TOT
The Database, via App Inspection :-
As you can see the list of 2 CommandTiles (a CommandTileList) has been converted to a String (SQLite type TEXT) and stored and subsequently retrieved.
Note from a database perspective this is not ideal it
limits/complicates the usefulness of the stored data.
For example (simple) if you wanted to select all States that start with A then SELECT * FROM userpreferencestable WHERE command LIKE 'A%' would find all columns you would have to use some like SELECT * FROM userpreferencestable WHERE 'state":A%'.
introduces bloat with all that extra data and thus introduces inefficiencies.
breaks normalisation as the same values are stored multiple times
The database way would be to have a table based upon the CommandTile incorporating the suitable relationship between CommandTiles and the UserPreferences.
Then there would be no need for the TypeConverter.
To convert enum value you have to do something like this
#TypeConverter
fun fromName(name: Name): String {
return name.name
}
#TypeConverter
fun toName(name: String): Name {
return enumValueOf<Name>(name)
}

Jackson XML serialization does not produce desired result

The goal is to create a class to serialize to the following xml:
<ParticipantID code="AA">participant name</PArticipantID>
I would expect the following class to work (code shown in kotlin):
data class ParticipantID(
#JacksonXmlProperty(isAttribute = true)
var code:String,
#JacksonXmlText
var value:String
)
yet serializing produces
<ParticipantID> <code>AA</code> participant name</PArticipantID>
It turns out that it is possible to make this work using the following class:
class ParticipantID(code:String, value:String) {
#JacksonXmlProperty(isAttribute = true)
private var code:String = code
get() = field
#JacksonXmlText
private var value:String = value
get() = field
}

Spring MongoDB - use abstract/sealed class field

sealed class Entity
data class Bacteria(
val uid: String,
val rank: String,
val division: String,
val scientificname: String,
val commonname: String
): Entity()
data class CTDDisease(
val diseaseId: String,
val name: String,
val altDiseaseIds: List<String>,
val parentIds: List<String>,
val definition: String?
) : Entity()
And then I want to define my document as
#Document(collection = "annotations")
data class Annotation(
#Id val id: String,
...
val spans: List<AnnotationSpan>
)
data class AnnotationSpan(
val startIndex: Int,
val endIndex: Int,
val entity: Entity? // Can be Bacteria or Disease or null
)
I also accept these 2 classes within the RequestBody from the client from time to time e.g.
#PutMapping("/annotations")
fun submitAnnotations(#RequestBody submission: Submissions): Mono<Void> { ... }
data class Submission(val annotations: List<AnnotationSpan>, ...) // AnnotationSpan contains Entity
but I get
java.lang.IllegalAccessException: class kotlin.reflect.jvm.internal.calls.CallerImpl$Constructor cannot access a member of class com.package.name.Entity with modifiers "private"
If I change the class Entity to an abstract class
abstract class Entity
then I don't get an error but my query operations keep going on forever.
Bacteria and Disease both have different fields so they should be distinguishable.
I tried using a hacky converter
#ReadingConverter
class NormalizedEntityReaderConverter: Converter<DBObject, NormalizedEntity> {
override fun convert(source: DBObject): NormalizedEntity? {
println(source.toString())
val gson = Gson()
return gson.runCatching { fromJson(source.toString(), CTDDisease::class.java) }.getOrNull()
?: gson.runCatching { fromJson(source.toString(), Bacteria::class.java) }.getOrNull()
}
}
and then registering it like
#Configuration
class MongoConverterConfig {
#Bean
fun customConversions(): MongoCustomConversions {
val normalizedEntityReaderConverter = NormalizedEntityReaderConverter()
val converterList = listOf<Converter<*, *>>(normalizedEntityReaderConverter)
return MongoCustomConversions(converterList)
}
}
My converter seems to work when called manually but for some reason, Spring still isn't picking it up.
I'm new to Spring. I would achieve this functionality in my Node.js server by using union types in TypeScript e.g.
interface AnnotationSpan {
startIndex: number;
endIndex: number;
entity?: Bacteria | Disease;
}
How can I achieve this behavior?

Issue with replacing Java DTO classes with Kotlin Data classes

I read about Kotlin data classes and thought that they could be pretty useful in cases with describing data transfer objects (DTOs). In my Java project I already has DTO classes written on Java, something like:
public class Tweet {
private String id;
private String profileId;
private String message;
public Tweet() {}
public String getId() {
return id;
}
public String getProfileId() {
return profileId;
}
public String getMessage() {
return message;
}
public void setId(String id) {
this.id = id;
}
public void setProfileId(String profileId) {
this.profileId = profileId;
}
public Tweet setMessage(String message) {
this.message = message;
return this;
}
}
These DTO classes are stored in separate artifact which I add as a dependency to other artifacts. So, I decided to replace it with Kotlin classes and rewrote mentioned Tweet class on Kotlin, so it started to looks like:
data class Tweet(var id: String? = null,
var profileId: String? = null,
var message: String? = null)
It's my first experience with Kotlin, so possibly there are something that can looks ugly, but my main issue is - when I try to rebuild artifacts which use my DTOs as dependencies, I get such exception:
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2992)
at io.vertx.core.json.Json.decodeValue(Json.java:117)
at gk.tweetsched.api.repository.TweetRepository.get(TweetRepository.java:51)
at gk.tweetsched.api.repository.TweetRepositoryTest.testGet(TweetRepositoryTest.java:68)
Caused by: java.lang.ClassNotFoundException:
kotlin.jvm.internal.DefaultConstructorMarker
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 67 more
As I see according that stacktrace Jackson couldn't deserialize JSON to Tweet Kotlin class.
Here is my Java method where I get that exception:
public Tweet get(String id) {
try (Jedis jedis = pool.getResource()) {
return Json.decodeValue(jedis.hget(TWEETS_HASH, id), Tweet.class);
} catch (Exception e) {
LOGGER.error(e);
}
return null;
}
Where Json class is from 'io.vertx.core.json' package.
How can I fix that issue? Which additional configurations should I make in my Java projects to use Kotlin classes?
By default Jackson needs a parameterless constructor to deserialize JSON to a class - Kotlin data classes do not have one, so you need to add a Jackson module to handle this:
jackson-module-kotlin
Edit:
I've read the source for io.vertx.core.json.Json class and it seems that both object mappers used by the class are stored in public static fields.
So to register jackson-module-kotlin you need to include this snippet in your application initialization code (or anywhere else really as long as it is executed before you attempt to deserialize any Kotlin data classes):
Json.mapper.registerModule(new KotlinModule())
Json.prettyMapper.registerModule(new KotlinModule())
In my case, I create kotlin class DTO instance in java for consuming RESTful Api.
Now I have 2 solutions tested:
Use parameterless constructor in data class.
The reference kotlin says:
On the JVM, if all of the parameters of the primary constructor have
default values, the compiler will generate an additional parameterless
constructor which will use the default values. This makes it easier to
use Kotlin with libraries such as Jackson or JPA that create class
instances through parameterless constructors.
So I have a DTO in kotlin like this:
data class Dto (
var id: Int?=null,
var version: Int?=null,
var title: String?=null,
var firstname: String?=null,
var lastname: String?=null,
var birthdate: String?=null
)
Then, I create class instance DTO in java:
Dto dto = new Dto();
dto.setId(javadto.getId());
...
Use plugin jackson-module-kotlin
import com.fasterxml.jackson.annotation.JsonProperty
data class Dto (
#JsonProperty("id") var id: Int?,
#JsonProperty("version") var version: Int?,
#JsonProperty("title") var title: String?,
#JsonProperty("firstname") var firstname: String?,
#JsonProperty("lastname") var lastname: String?,
#JsonProperty("birthdate") var birthdate: String?,
)
Then, I create class instance DTO in java:
Dto dto = new Dto(null, null, null, null, null, null);
dto.setId(javadto.getId());
...

Grails mongodb map type gives illegalStateException

I'm trying to add a Map property to a User Domain
class User {
String id
String username
String password
....
Map<String, Follower> followers
//i've tried also without embeded and got the same error. also tried follower insead of followers
static embedded = ['followers']
}
class Follower {
String name
List<String> interests
}
I Have a restful controller the implements the save method
#Transactional
#Secured(['permitAll'])
def save(User user){
user.id = new ObjectId().encodeAsBase64()
user = user.insert(flush: true)
respond
}
Sadly i'm getting an exception:
java.lang.IllegalStateException: Cannot convert value of type [com.google.gson.JsonObject] to required type [Follower] for property 'followers[532dbe3b8fef86ebe3e64789]': no matching editors or conversion strategy found Around line 26 ofUserController.groovy
line 26 is : user = user.insert(flush: true)
example json request:
{
username : "roy",
password : "123456",
followers : {
"532dbe3b8fef86ebe3e64789" : {
name : "Roy",
interests : ["math"]
}
}
}
Any help will be greatly appreciated
Thanks!
Roy
you are trying to save JSONObject's as Follower instances. The straight forward way to solve the issue would be to convert those into Follower instances manually:
def save(User user){
user.id = new ObjectId().encodeAsBase64()
user.followers = user.followers.collect{ new Follower( it ) }
user = user.insert(flush: true)
respond
}
if you have more of such cases, you should register a property editor for conversion JSON <-> Follower

Categories

Resources