I am trying to call a method where it's signature includes a parameter of Class<T>
below is the sample code in kotlin
val response: ResponseEntity<ResponseObject<*>> = testRestTemplate.postForEntity("/users", user, ResponseObject::class.java)
what i am trying to achieve is to get rid of the <*> in responseObject and let it be
val response: ResponseEntity<ResponseObject<User>> = ???
but i am not sure on what is the correct syntax to provide to satisfy the Class<T> requirement
i tried
ResponseObject<User::class.java>::class.java
but that is not a valid syntax. any pointers?
The real problem is if i use * i don't know how exactly to infer the User instance from there correctly.
ok I managed to solve my problem using type casting using when
#Test
fun testCreateUser() {
val user = User(id = null)
val response = testRestTemplate.postForEntity("/users", user, ResponseObject::class.java)
val responseObject = response.body
when (val returnedUser = responseObject.model) {
is User -> {
assertNotNull(returnedUser.id)
assertEquals(UserStatus.active, returnedUser.status)
}
}
}
If you can change the signature of the method then you may try something similar to the following:
class ResponseEntity<T : Any>(val body: T)
class ResponseObject<T : Any>(val model: T)
data class User(val id: Long, val status: String)
fun <M : Any, K : ResponseObject<M>> postForEntity(paht: String, model: M): ResponseEntity<K> {
return TODO()
}
val response: ResponseEntity<ResponseObject<User>> = postForEntity("/users", User(1, "good"))
You could use
#Suppress("UNCHECKED_CAST")
val response = testRestTemplate.postForEntity("/users", user, ResponseObject::class.java as Class<ResponseObject<User>>)
Or a helper function if you need it more than once for different parameters
inline fun <reified T> classOf<T>() = T::class.java
val response = testRestTemplate.postForEntity("/users", user, classOf<ResponseObject<User>>())
(in both cases the type ResponseEntity<ResponseObject<User>> should be inferred)
Related
I have a kotlin data class OfflineDataRequestInfo which I want to convert to Json using Gson, but it always returns an empty object
data class OfflineDataRequestInfo (
#SerializedName("status") val status: String,
#SerializedName("userId") val userId: String?,
#SerializedName("fulOrderId") val fulOrderId: String,
#SerializedName("timeStamp") val timeStamp: String,
#SerializedName("fulOrder") val fulOrder: String,
#SerializedName("checks") val checks: String?
)
some of the values could be null so I tried the bellow code too which returned {}
gson.toJson(OfflineDataRequestInfo("a","b","c","d", "e", "f"))
Here is a bit more info just in case that is an issue
#Entity
data class OfflineData (
#PrimaryKey(autoGenerate = true) val id: Int = 0,
#ColumnInfo(name="request_code") val requestCode: String?,
#Embedded
val requestInfoJson: OfflineDataRequestInfo
)
this is my actual function
fun postFulOurderData(offlineData: OfflineData) {
if (offlineData != null) {
val mainRepository = MainRepository(ApiHelper(RetrofitBuilder.apiService))
val builder = GsonBuilder()
builder.serializeNulls()
val gson = builder.create()
launch {
val postFulOrder = mainRepository.postFulOrderOfflineData(gson.toJson(offlineData.requestInfoJson), tokenResult.access_token)
}
}
}
I also tried using GsonBuilder as shown above and also the default Gson, but no luck
also tried gson.toJson(offlineData.requestInfoJson, OfflineDataRequestInfo::class.java)
can any one suggest please where I am doing it wrong
your help will be very much appreciated
thanks
R
Probably you have new GsonBuilder().excludeFieldsWithoutExposeAnnotation().create() setting for gson. If so you just need to add #Expose annotation to each field you want serialize in json:
data class OfflineDataRequestInfo (
#Expose #SerializedName("status") val status: String,
#Expose #SerializedName("userId") val userId: String?,
#Expose #SerializedName("fulOrderId") val fulOrderId: String,
#Expose #SerializedName("timeStamp") val timeStamp: String,
#Expose #SerializedName("fulOrder") val fulOrder: String,
#Expose #SerializedName("checks") val checks: String?
)
I don't know that much about Kotlin, but I do know that you can mix Java and Kotlin pretty well, and since Gson was made with Java use in mind I think you should use Java. You can create a Java class, and then a method like this:
String toJson(OfflineDataRequestInfo o) {
return new Gson().toJson(o, OfflineDataRequestInfo.class);
}
and then call that method from Kotlin.
I'm not saying that you should do your entire project in Java, just use Java for this one method.
Hope this helps, and please tell me if this wasn't what you had in mind :)
I just tested the following code using gson:2.2.4 and it works:
data class OfflineDataRequestInfo (
#SerializedName("status") val status: String,
#SerializedName("userId") val userId: String?,
#SerializedName("fulOrderId") val fulOrderId: String,
#SerializedName("timeStamp") val timeStamp: String,
#SerializedName("fulOrder") val fulOrder: String,
#SerializedName("checks") val checks: String?
)
fun main() {
println(Gson().toJson(OfflineDataRequestInfo("a","b","c","d", "e", "f")))
}
Output:
{"status":"a","userId":"b","fulOrderId":"c","timeStamp":"d","fulOrder":"e","checks":"f"}
I didn't try it on android, but it should also work.
I suggest you try to update your gson version and also to write a unit test using it.
I've no experience with Kotlin and I'm trying to write an app fetching data from diffrent financial APIs using Gson. I have created two classes implementing an interface and I'd like to instantiate it in generic function. Right now I have two diffrent methods to operate on each API and I'd like to make it more decent.
EDIT:
I want to make a generic function out of two given functions:
Interface and two classes:
interface TickerEntity{
val tickers: Array<String>
data class MainData (
val Bid: Double,
val Ask: Double
)
}
object API1TickerEntity : TickerEntity {
val tickers = arrayOf<String>("BTC-LTC", "BTC-DOGE", "BTC-POT", "BTC-USD")
data class MainData(
val success: Boolean,
val message: String,
val result: ResultData
)
data class ResultData (
val Bid: Double,
val Ask: Double,
val Last: Double
)
}
object API2TickerEntity : TickerEntity {
val tickers = arrayOf<String>("LTCBTC", "BTCDOGE", "BTCPOT", "BTCUSD")
data class MainData(
val max : Double,
val min : Double,
val last : Double,
val bid : Double,
val ask : Double,
val vwap : Double,
val average : Double,
val volume : Double
)
}
My functions to manage Json I want to be generic:
data class BuySell( val stockName: String, val buy: Double = 0.0, val sell: Double = 0.0)
fun getAPI1BuySell(): () -> BuySell {
val currency = API1TickerEntity.tickers[0]
val response = sendRequest("somesite.com")
val gson = Gson()
val ticker: API1Entity.MainData = gson.fromJson(response.body, API1TickerEntity.MainData::class.java)
println(currency)
return { BuySell("API1", ticker.result.Ask, ticker.result.Bid) }
}
fun getAPI2BuySell(): () -> BuySell {
val currency = API2TickerEntity.tickers[0]
val response = sendRequest("someothersite.com")
val gson = Gson()
val ticker: API2TickerEntity.MainData = gson.fromJson(response.body, API2TickerEntity.MainData::class.java)
return { BuySell("API2", ticker.ask, ticker.bid) }
}
So far I have tried:
fun <T : TickerEntity> getStockBuySell(url: String, stockName: String): () -> BuySell {
val tickerEntity : T = T
val currency = tickerEntity.tickers[0]
val response = sendRequest(url.replace("{}", currency))
val gson = Gson()
val ticker: tickerEntity.MainData = gson.fromJson(response.body, tickerEntity.MainData::class.java)
println(currency)
return { BuySell (stockName, ticker.Ask, ticker.Bid) }
}
}
But I can't instantiate the interface alone, and also it seems I can't override data class alone since it is not a value.
JSON files to manage:
API1:
{"success":true,"message":"","result":{"Bid":0.00596655,"Ask":0.00597554,"Last":0.00597933}}
API2:
{"max":0.00606939,"min":0.0059345,"last":0.00595134,"bid":0.00594972,"ask":0.00599205,"vwap":0.00595134,"average":0.00595134,"volume":29.60407718}
All help appreciated
I am trying to register the PubicKey interface for data conversion in KTor so I can easily receive a public key like this:
data class StoreRequest(
val publicKey: PublicKey
)
...
val publicKey: PublicKey = call.receive<StoreRequest>().publicKey
To achieve this I used this page: https://ktor.io/servers/features/data-conversion.html
I registered this data converter:
convert<PublicKey> {
decode { values, _ ->
// When I add a breakpoint here it won't be reached.
values.singleOrNull()?.let { key ->
val encryptedKey = Base64.getDecoder().decode(key.split(" ")[1])
val inputStream = DataInputStream(ByteArrayInputStream(encryptedKey))
val format = String(ByteArray(inputStream.readInt()).also(inputStream::readFully))
if (format != "ssh-rsa") throw RuntimeException("Unsupported format")
val publicExponent = ByteArray(inputStream.readInt()).also(inputStream::readFully)
val modulus = ByteArray(inputStream.readInt()).also(inputStream::readFully)
val spec = RSAPublicKeySpec(BigInteger(modulus), BigInteger(publicExponent))
val keyFactory = KeyFactory.getInstance("RSA")
keyFactory.generatePublic(spec)
}
}
}
But for some reason Gson is complaining because I'm using an interface:
java.lang.RuntimeException: Unable to invoke no-args constructor for interface java.security.PublicKey. Register an InstanceCreator with Gson for this type may fix this problem`
So I'm guessing that I need to create an InstanceCreator for an initial value.
This is the initial PublicKey class I created:
class PkTest : PublicKey {
override fun getAlgorithm(): String = ""
override fun getEncoded(): ByteArray = ByteArray(0)
override fun getFormat(): String = ""
}
...
install(ContentNegotiation) {
gson {
setPrettyPrinting()
registerTypeAdapter(PublicKey::class.java, InstanceCreator<PublicKey> { PkTest() // This is called when I add a breakpoint })
}
}
But this also doesn't work! This is the exception I am getting:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 2 column 16 path $.publicKey
For some reason it's expecting the provided key to be a JSON object while I'm providing this:
{
"publicKey": "ssh-rsa AAAAB3NzaC1yc2EABAADAQABAAACAQDBPL2s+25Ank3zS6iHUoVk0tS63dZM0LzAaniiDon0tdWwq4vcL4+fV8BsAEcpMeijS92JhDDc9FccXlHbdDcmd6c4ITOt9h9xxhIefGsi1FTVJ/EjVtbqF5m0bu7ruIMGvuP1p5s004roHx9y0UdHvD/yNWLISMhy4nio6jLailIj3FS53Emj1WRNsOrpja3LzPXzhuuj6YnD9yfByT7iGZipxkmleaXrknChPClLI9uhcqtAzBLdd0NVTJLOt/3+d1cSNwdBw9e53wJvpEmH+P8UOZd+oV/y7cHIej4jQpBXVvpJR1Yaluh5RuxY90B0hSescUAj5g/3HVPpR/gE7op6i9Ab//0iXF15uWGlGzipI4lA2/wYEtv8swTjmdCTMNcTDw/1huTDEzZjghIKVpskHde/Lj416c7eSByLqsMg2OhlZGChKznpIjhuNRXz93DwqKuIKvJKSnhqaJDxmDGfG7nlQ/eTwGeAZ6VR50yMPiRTIpuYd767+Nsg486z7p0pnKoBlL6ffTbfeolUX2b6Nb9ZIOxJdpCSNTQRKQ50p4Y3S580cUM1Y2EfjlfIQG1JdmTQYB75AZXi/cB2PvScmF0bXRoj7iHg4lCnSUvRprWA0xbwzCW/wjNqw6MyRX42FFlvSRrmfaxGZxKYbmk3TzBv+Fp+CADPqQm3OQ== dynmem#memmen.frl"
}
How can I 'trick' GSON in accepting a string for PublicKey? Or is there something else I'm doing wrong?
I think GSON wants to serialize a JSON object to a PublicKey. But I want it to accept a String. I think this should be possible because classes like UUID and Date work just fine...
I solved it! Instead of using an InstanceCreator I used a JsonDeserializer:
install(ContentNegotiation) {
gson {
setPrettyPrinting()
registerTypeAdapter(PublicKey::class.java, JsonDeserializer<PublicKey> { json, _, _ ->
// TODO some type checking.
val key = json.asString
val encryptedKey = Base64.getDecoder().decode(key.split(" ")[1])
val inputStream = DataInputStream(ByteArrayInputStream(encryptedKey))
val format = String(ByteArray(inputStream.readInt()).also(inputStream::readFully))
if (format != "ssh-rsa") throw RuntimeException("Unsupported format")
val publicExponent = ByteArray(inputStream.readInt()).also(inputStream::readFully)
val modulus = ByteArray(inputStream.readInt()).also(inputStream::readFully)
val spec = RSAPublicKeySpec(BigInteger(modulus), BigInteger(publicExponent))
val keyFactory = KeyFactory.getInstance("RSA")
keyFactory.generatePublic(spec)
})
}
}
This part can fully be removed:
convert<PublicKey> {
decode { values, _ ->
...
}
}
I want to pass a type to a function in Scala.
Problem in detail
First iteration
I have the following Java classes (coming from an external source):
public class MyComplexType {
public String name;
public int number;
}
and
public class MyGeneric<T> {
public String myName;
public T myValue;
}
In this example I want MyComplexType to be the the actual type of MyGeneric; in the real problem there are several possibilities.
I want to deserialize a JSON string using a Scala code as follows:
import org.codehaus.jackson.map.ObjectMapper
object GenericExample {
def main(args: Array[String]) {
val jsonString = "{\"myName\":\"myNumber\",\"myValue\":{\"name\":\"fifteen\",\"number\":\"15\"}}"
val objectMapper = new ObjectMapper()
val myGeneric: MyGeneric[MyComplexType] = objectMapper.readValue(jsonString, classOf[MyGeneric[MyComplexType]])
val myComplexType: MyComplexType = myGeneric.myValue
}
}
it compiles fine but runtime error occurs:
java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to MyComplexType
at GenericExample$.main(GenericExample.scala:9)
Second iteration
Working solution to the problem:
val jsonString = "{\"myName\":\"myNumber\",\"myValue\":{\"name\":\"fifteen\",\"number\":\"15\"}}"
val objectMapper = new ObjectMapper()
val myGeneric: MyGeneric[MyComplexType] = objectMapper.readValue(jsonString, classOf[MyGeneric[MyComplexType]])
myGeneric.myValue = objectMapper.readValue(objectMapper.readTree(jsonString).get("myValue").toString, classOf[MyComplexType])
val myComplexType: MyComplexType = myGeneric.myValue
Not nice but works. (If anybody knows how to make it better, that would also welcome.)
Third iteration
The lines in the solution of second iteration occur in the real problem several times, therefore I want to create a function. The altering variables are the JSON formatted string and the MyComplexType.
I want something like this:
def main(args: Array[String]) {
val jsonString = "{\"myName\":\"myNumber\",\"myValue\":{\"name\":\"fifteen\",\"number\":\"15\"}}"
val myGeneric = extractMyGeneric[MyComplexType](jsonString)
val myComplexType: MyComplexType = myGeneric.myValue
}
private def extractMyGeneric[T](jsonString: String) = {
val objectMapper = new ObjectMapper()
val myGeneric = objectMapper.readValue(jsonString, classOf[MyGeneric[T]])
myGeneric.myValue = objectMapper.readValue(objectMapper.readTree(jsonString).get("myValue").toString, classOf[T])
myGeneric
}
This does not work (compiler error). I've already played around with various combinations of Class, ClassTag, classOf but none of them helped. There were compiler and runtime errors as well. Do you know how to pass and how to use such a type in Scala? Thank you!
When you use jackson to parse json, you can use TypeReference to parse generic type. Example:
val jsonString = "{\"myName\":\"myNumber\",\"myValue\":{\"name\":\"fifteen\",\"number\":\"15\"}}"
val objectMapper = new ObjectMapper()
val reference = new TypeReference[MyGeneric[MyComplexType]]() {}
val value: MyGeneric[MyComplexType] = objectMapper.readValue(jsonString, reference)
if you still want to use Jackson, I think you can create a parameter with TypeReference type. like:
implicit val typeReference = new TypeReference[MyGeneric[MyComplexType]] {}
val value = foo(jsonString)
println(value.myValue.name)
def foo[T](jsonStr: String)(implicit typeReference: TypeReference[MyGeneric[T]]): MyGeneric[T] = {
val objectMapper = new ObjectMapper()
objectMapper.readValue(jsonStr, typeReference)
}
Using your approach, I think this is how you can get classes that you need using ClassTags:
def extractMyGeneric[A : ClassTag](jsonString: String)(implicit generic: ClassTag[MyGeneric[A]]): MyGeneric[A] = {
val classOfA = implicitly[ClassTag[A]].runtimeClass.asInstanceOf[Class[A]]
val classOfMyGenericOfA = generic.runtimeClass.asInstanceOf[Class[MyGeneric[A]]]
val objectMapper = new ObjectMapper()
val myGeneric = objectMapper.readValue(jsonString, classOfMyGenericOfA)
myGeneric.myValue = objectMapper.readValue(objectMapper.readTree(jsonString).get("myValue").toString, classOfA)
myGeneric
}
I am not familiar with jackson but in play-json you could easily define Reads for your generic class like this
import play.api.libs.functional.syntax._
import play.api.libs.json._
implicit def genReads[A: Reads]: Reads[MyGeneric[A]] = (
(__ \ "myName").read[String] and
(__ \ "myValue").read[A]
)((name, value) => {
val e = new MyGeneric[A]
e.myName = name
e.myValue = value
e
})
Having this, and provided that instance of Reads for MyComplexType exists, you can implement your method as
def extractMyGeneric[A: Reads](jsonString: String): MyGeneric[A] = {
Json.parse(jsonString).as[MyGeneric[A]]
}
the issue here is that you need to provide Reads for all of your complex types, which would be as easy as
implicit complexReads: Reads[MyComplexType] = Json.reads[MyComplexType]
if those were case classes, otherways I think you would have to define them manually in simillar way to what I've done with genReads.
I am building an application in Spark, and would like to use the SparkContext and/or SQLContext within methods in my classes, mostly to pull/generate data sets from files or SQL queries.
For example, I would like to create a T2P object which contains methods that gather data (and in this case need access to the SparkContext):
class T2P (mid: Int, sc: SparkContext, sqlContext: SQLContext) extends Serializable {
def getImps(): DataFrame = {
val imps = sc.textFile("file.txt").map(line => line.split("\t")).map(d => Data(d(0).toInt, d(1), d(2), d(3))).toDF()
return imps
}
def getX(): DataFrame = {
val x = sqlContext.sql("SELECT a,b,c FROM table")
return x
}
}
//creating the T2P object
class App {
val conf = new SparkConf().setAppName("T2P App").setMaster("local[2]")
val sc = new SparkContext(conf)
val sqlContext = new SQLContext(sc)
val t2p = new T2P(0, sc, sqlContext);
}
Passing the SparkContext as an argument to the T2P class doesn't work since the SparkContext is not serializable (getting a task not serializable error when creating T2P objects). What is the best way to use the SparkContext/SQLContext inside my classes? Or perhaps is this the wrong way to design a data pull type process in Spark?
UPDATE
Realized from the comments on this post that the SparkContext was not the problem, but that I was using a using a method within a 'map' function, causing Spark to try to serialize the entire class. This would cause the error since SparkContext is not serializable.
def startMetricTo(userData: ((Int, String), List[(Int, String)]), startMetric: String) : T2PUser = {
//do something
}
def buildUserRollup() = {
this.userRollup = this.userSorted.map(line=>startMetricTo(line, this.startMetric))
}
This results in a 'task not serializable' exception.
I fixed this problem (with the help of the commenters and other StackOverflow users) by creating a separate MetricCalc object to store my startMetricTo() method. Then I changed the buildUserRollup() method to use this new startMetricTo(). This allows the entire MetricCalc object to be serialized without issue.
//newly created object
object MetricCalc {
def startMetricTo(userData: ((Int, String), List[(Int, String)]), startMetric: String) : T2PUser = {
//do something
}
}
//using function in T2P
def buildUserRollup(startMetric: String) = {
this.userRollup = this.userSorted.map(line=>MetricCalc.startMetricTo(line, startMetric))
}
I tried several options, this is what worked eventually for me..
object SomeName extends App {
val conf = new SparkConf()...
val sc = new SparkContext(conf)
implicit val sqlC = SQLContext.getOrCreate(sc)
getDF1(sqlC)
def getDF1(sqlCo: SQLContext): Unit = {
val query1 = SomeQuery here
val df1 = sqlCo.read.format("jdbc").options(Map("url" -> dbUrl,"dbtable" -> query1)).load.cache()
//iterate through df1 and retrieve the 2nd DataFrame based on some values in the Row of the first DataFrame
df1.foreach(x => {
getDF2(x.getString(0), x.getDecimal(1).toString, x.getDecimal(3).doubleValue) (sqlCo)
})
}
def getDF2(a: String, b: String, c: Double)(implicit sqlCont: SQLContext) : Unit = {
val query2 = Somequery
val sqlcc = SQLContext.getOrCreate(sc)
//val sqlcc = sqlCont //Did not work for me. Also, omitting (implicit sqlCont: SQLContext) altogether did not work
val df2 = sqlcc.read.format("jdbc").options(Map("url" -> dbURL, "dbtable" -> query2)).load().cache()
.
.
.
}
}
Note: In the above code, if I omitted (implicit sqlCont: SQLContext) parameter from getDF2 method signature, it would not work. I tried several other options of passing the sqlContext from one method to the other, it always gave me NullPointerException or Task not serializable Excpetion. Good thins is it eventually worked this way, and I could retrieve parameters from a row of the DataFrame1 and use those values in loading the DataFrame 2.