I'm a beginner of Scala, and I have JSON data formatted like below:
{
"index_key": {
"time":"12938473",
"event_detail": {
"event_name":"click",
"location":"US"
}
}
}
I'm trying to get the content of "index_key" and extract the content of second level as a new JSON objet and initiate a class based on the second level data.
{
"time":"12938473",
"event_detail": {
"event_name":"click",
"location":"US"
}
}
I tried to use json4s to extract from the above json to be a Event class, but how to get rid of the "index_key" which is the first level key?
case class Detail(event_name: String, location: String)
case class Event(time: String, event_detail: Detail)
json.extract[Event]
I've read json4s documentation, and also http://www.scala-lang.org/api/2.10.3/index.html#scala.util.parsing.json.JSON$, but still don't quite get it, as it seems the pre-defined json should be fit for the parser?
Could anyone please tell me how to get the second level data (or any lower level) of the json structure?
You can use \ to get to the object you want to extract:
val json = parse(str) \ "index_key"
json.extract[Event]
Related
I am taking a JSON file as input for a class and parsing the values using gson through respective data classes.
I want to call a function that takes a String value as an argument.
The string value allowed is decided from the values parsed from JSON file. Can I somehow check for that string value passed to the function at compile-time & give an error at compile-time?
Or If I can allow only certain values in the argument for the function based on the values from JSON
Detailed Explanation of use case:
I am building a SDK in which a the person using sdk inputs json String. The json is standardised and is parsed in my code.
{
"name": "Test",
"objects": [
{
"name": "object1",
"type": "object1"
}
]
}
Here name values and other values may vary based on the input by the developer using it but key remains same. But we need to call a function using the value in objects name parameter.
fun testMethod(objectName:String)
So developer calls the testMethod as testMethod(object1).
I need to validate object1 parameter based on json but is there any way possible restricting the test method parameter to object1 only & give error at compile time if the developer calls testMethod(obj1)
Right now I parse JSON & have checks inside the testMethod()
Sure it's possible to do, but somehow in different way, that you described. First of all, as you already mentioned this behavior could be done easily. For this purpose we have Objects.requireNotNull() or Guava.Preconditions(). At the same way you can define you checking but this will work on runtime only.
To do in compile time, you need to create Annotation Preprocessor. The same, as did in different libraries, and one of them, could be Lombok, with their NotNull and Nullable. Android annotation just provide mark and bound for IDE warning, but in their case they adding NotNull checking and throw exception for every annotation usage during compile time.
It's not an easy way, but it's what you are looking for.
No, it's impossible check it in compiler time. It's string handling, as numeric calculation.
In my app, I convert string to JSON and JSON to string, passing class descriptor. My aim is record JSON string in a text file to load in SQLite database. This code I've run in my desktop computer not in Android.
data class calcDescr (
...
)
val calc = CalcDescr(...)
// toJson: internal Kotlin data to JSON
val content = Gson().toJson(calc)
//==================
// Testing validity
// ================
// fromJson: JSON to internal Kotlin data.
// It needs pass the class descriptor. Uses *Java* token, but it's *Kotlin*
var testModel = Gson().fromJson(content, CalcDescr::class.java)
// toJson: internal Kotlin data to JSON again
var contentAgain = Gson().toJson(testModel)
// shoul be equal!
if (content == contentAgain) println("***ok***")
In my file, I write the variable content in a file
I have a text file which contains a line like
players={"Messi":{"Details":{"Goals":500},"Country":"Argentina"},"Neymar":{"Clubs":["Santos", "FC barcelona", "Paris saint German"], "Country":"Brazil"}}
Now I am used a regex for extract the
{"Messi":{"Details":{"Goals":500},"Country":"Argentina"},"Neymar":{"Clubs":["Santos", "FC barcelona", "Paris saint German"],"Country":"Brazil"}}
from the text file and pass it in to a case class which accepts the value as a String.
and I am making a Dataframe using this case class.
In my case every line may be different in the contents with in the JSON String.So I am looking for a general solution to Convert any complex Json string to Map values.
When checking dataframe.printSchema, I am getting the players column as a String type.
But I need it to be as a Map type which holds a Key and value as a Struct type.
I tried method referred in this link
How can I convert a json string to a scala map?
when I used this way,I got error
"org.json4s.package$MappingException: Do not know how to convert JObject(List((Details,JObject(List((Goals,JString(500))))), (Country,JString(Argentina)))) into class java.lang.String "
and I used following solutions
Converting JSON string to a JSON object in Scala
But these too won't worked for me.
This is my case class
case class caseClass (
Players :String = ""
)
I am Extracting the json string using a user defined function.
Simply my requirement is that I have a complex Json String which contains keys and values as struct,list etc..
so I want to make the string to its corresponding JSON which holds a proper schema with respect to its contents.
Kindly expecting Valuable solutions.
If you also can live with JsValue as value instead of String it looks a bit simpler:
import play.api.libs.json._
case class CaseClass (
Players :Option[JsValue]
)
object CaseClass {
implicit val jsonFormat = Json.format[CaseClass ]
}
I saw some problems with your Json - so you would need to have something like:
val json = Json.parse("""{
"Players":{
"Messi":{"Details":{"Goals":500},"Country":"Argentina"},
"Neymar":{"Clubs":["Santos", "FC barcelona", "Paris saint German"], "Country":"Brazil"}
}
}"""
)
To get a String out of this you can use:
json.validate[CaseClass] match {
case JsSuccess(cc, _) => cc.Players.toString
case JsError(errors) => // handle errors
}
I got another solution which I think More easier.
I Made an own schema for the JSON and Used from_json method with the schema,and it worked well.
from_json(col("Players"),ownschema).as("new_Json")
and my ownschema contains the structure of the Json String.
For any doubts, Comment.
Happy Coding.
Here's what I wanna do
I have a json, like:
{
"demoNumber":123,
"demoText":"asdasdasd"
}
and I wanna make a simple String array from it, which should be
["demoNumber","demoText"]
In the app we're making the user can add any type of data, so we can't do data models for everything, that's not an option
I have added json to my Gradle:
dependencies {
implementation 'org.json:json:20180130'
}
But it still can't find the method.
Assuming you have the JSON as a string, this example uses the JSON-java library:
JSONObject jo = new JSONObject(myJsonStr);
Set<String> keys = jo.toMap().keySet();
// You should be able to extract an array from the set of keys
See also https://www.baeldung.com/java-org-json
In my case, there might be different data type of same json field. Example:
"need_exp":1500
or
"need_exp":"-"
How to process this case? I know it can be processed by parse or use custom encoders, but this is a very complex json text, is there any way to solve that without rewriting the whole decoder (for example, just "tell" the decoder to convert all Int to String in need_exp field)?
It is called a disjunction which can be encoded with the Scala standard Either class.
Simply map the json that to the following class:
case class Foo(need_exp: Either[String, Int])
My solution is to use a custom decoder. Rewrite a little part of the JSON can be fine.
For example, there is a simple JSON:
{
/*many fields*/
"hotList":[/* ... many lists inside*/],
"list":[ {/*... many fields*/
"level_info":{
"current_exp":11463,
"current_level":5,
"current_min":10800,
"next_exp":28800 //there is the problem
},
"sex":"\u4fdd\u5bc6"},/*...many lists*/]
}
In this case, I don't need to rewrite the whole JSON encoder, just write a custom encoder of level_info:
implicit val decodeUserLevel: Decoder[UserLevel] = (c: HCursor) => for
{
current_exp <- c.downField("current_exp").as[Int]
current_level <- c.downField("current_level").as[Int]
current_min <- c.downField("current_min").as[Int]
next_exp <- c.downField("next_exp").withFocus(_.mapString
{
case """-""" => "-1"
case default => default
}).as[Int]
} yield
{
UserLevel(current_exp, current_level, current_min, next_exp)
}
and it worked.
I'm starting out to collect logs with logstash. The current setup consist of a Java server using logback as logging mechanism and logstash-logback-encoder, outputting the data in a neat JSON representation. The basics work just fine.
I would like to separate additional data in JSON format in separate fields (so each key of the JSON ends up in its own field). logstash-logback-encoder provides a mechanism for that to output such data in a json_mesage field. However this JSON string is placed into a JSON array. See here a sample formatted for better reading.
{
"#timestamp":"2014-03-25T19:34:11.586+01:00",
"#version":1,
"message":"Message{\"activeSessions\":0}",
"logger_name":"metric.SessionMetrics",
"thread_name":"scheduler-2",
"level":"INFO",
"level_value":20000,
"HOSTNAME":"stage-01",
"json_message":["{\"activeSessions\":0}"],
"tags":[]
}
I tried to parse the incoming JSON using a simple JSON filter. See here my configuration:
input {
lumberjack {
<snipped>
codec => "json"
}
}
filter {
json {
source => "json_message"
}
}
output {
elasticsearch {
<snipped>
}
}
However this leads to following error in the logstash log. The JSON string in an array simply can't be handled.
{:timestamp=>"2014-03-25T19:43:13.232000+0100",
:message=>"Trouble parsing json",
:source=>"json_message",
:raw=>["{\"activeSessions\":0}"],
:exception=>#<TypeError: can't convert Array into String>,
:level=>:warn}
Is there a way to extract the JSON string from the array prior to parsing? Any help is greatly appreciated. Thanks.
Actually, it is quite simple and plays along the lines of common programming languages. Though, I did not find the answer in the docs.
Just add an index to the field in the filter:
filter {
json {
source => "json_message[0]"
}
}