Goal:
I am trying to parse the postman_echo collection json and persist the result into a new json copy on disk, resulting the same file as the original one.
I prefer built-in data structure from the language, but using json library should be good too. Not sure Antlr4 is a better way.
follow-up question
is it possible to allow any valid nested json in the body of post request?
update:
https://github.com/chakpongchung/postman-parser
In the end we come up with this satisfactory solution.
An alternative to what zoran mentioned is to create a case class if the structure is not too dynamic (with Play JSON). This would make it easier to compare the result.
case class MyObject(
queryString: List[KeyValue],
method: String,
url: String,
httpVersion: String
) ... and so on
object MyObject {
implicit val format: Format[MyObject] = Json.format
}
case class KeyValue(name: String, value: String)
object KeyValue {
implicit val format: Format[KeyValue] = Json.format
}
Then, you just need to do:
object JsonParser extends App {
val postman_collections = "./scala_input.json"
val jsonifiedString = scala.io.Source.fromFile(postman_collections).mkString
val myJsonData = Try(Json.parse(jsonifiedString)).map(_.as[MyObject])
myJsonData match {
case Success(myValue) => // compare your case class here
case Failure(err) => println("none")
}
}
I'm not sure if I understand your question well, but if you are trying to iterate over json string, you might try something like this:
import play.api.libs.json.{JsObject, JsValue, Json}
import scala.util.{Failure, Success, Try}
object JsonParser extends App {
val postman_coolections = "./resources/scala_input.json"
val jsonifiedString = scala.io.Source.fromFile(postman_coolections).mkString
val json: JsValue = Try(Json.parse(jsonifiedString)) match {
case Success(js) => js
case Failure(ex) => throw new Exception("Couldn't parse json", ex)
}
json.asInstanceOf[JsObject].fields.foreach{
case (key: String, value: JsValue)=>
println(s"Key:$key value:${value.toString}")
writeFile(s"$key.json", Json.prettyPrint(value))
}
//writing the whole postman input as a single file
writeFile("postmanInputFormatted.json", Json.prettyPrint(json))
writeFile("postmanInput.json", Json.stringify(json))
// To access individual property one option is to use this approach
val lookedValue = json \ "postData" \ "params" \ 1 \ "hello" \ "test"
lookedValue match {
case JsDefined(value) => println(s"Test value is $value")
case JsUndefined() => println("Didn't find test value")
}
// or
val lookedValueAlt = (json \ "postData" \ "params" \ 1 \ "hello" \ "test").getOrElse(throw SomeException)
There are multiple problems in your parser and most of them are that you are trying to use default parser to handle Json object as string. For example, in Request you are handling header as Seq[String] when it's actually Seq of (key, value) pairs. For this particular case, you should change it to something like this:
case class Request(
method: String,
header: Seq[HeaderItem], // header: []
url: Option[Url] = None,
description: String = ""
)
object Request {
implicit val format: Format[Request] = Json.using[Json.WithDefaultValues].format
case class HeaderItem(key: String, value: String)
object HeaderItem {
implicit val format: Format[HeaderItem] = Json.format
}
You can convert header to Seq[String] if you want, but you will have to write custom Read for that.
In the above case you also have cases when description is missing, so you want to handle that with default values.
You have such problems to handle in few other places as well, e.g. "response".
Another problem that I've noticed is a way how you handle "type" property from Json string. Type is reserved keyword, and you can handle it by wrapping it in ``, e.g.
case class Script(
`type`: String,
exec: Seq[String]
)
A satisfactory solution is posted in the github link above in the question.
Related
I want to store a randomized integer that has been called out in the request body and store it in test case Property so that it can be passed as a body parameter in the next request.
for example:
Request1:
id_num = randomNumeric(10)
Properties:
id_num = 1234567890
Request2:
trfered_IDNum = ${#TestCase#id_num}
The Structure looks like below,
Project
|---TestSuite
|-------TestCase
|------RestRequestTestStep1
|------RestRequestTestStep2
|------GroovyScript
Let's start with some assumptions.
RestRequestTestStep1 Response Body has below fields :
{"Resp1Field1Key":"Resp1Field1Value",
"Resp1Field2Key":"Resp1Field2Value"}
RestRequestTestStep2 Request Body has below fields :
{"Resp2Field1Key":"Resp2Field1Value",
"Resp2Field2Key":"Resp2Field2Value"}
Resp1Field1Value and Resp1Field2Value from first response will be substituted to Resp2Field1Value and Resp2Field2Value of second request.
RestRequestTestStep2 body should be as below as we will be substituting values from the testCase Property that will be set in the groovy script once first request is completed.
{
"Resp2Field1Key":"${#TestCase#Resp2Field1Value}",
"Resp2Field2Key":"${#TestCase#Resp2Field2Value}"
}
The Code..rather Script : The groovy script can be placed under the same Test Case and should do below,
import groovy.json.JsonSlurper
//Substitute with appropriate testSuiteName,testCaseName,testStepName1 and testStepName1 as per the Project Structure you have.
def testSuite = testRunner.testCase.testSuite.project.getTestSuiteByName("testSuiteName")
def testCase = testSuite.getTestCaseByName("testCaseName")
def testStep1 = testCase.getTestStepByName("testStepName1")
def testStep2 = testCase.getTestStepByName("testStepName2")
// Call the first REST Request
testStep1.run(testRunner, context)
def response = testStep1.testRequest.response.responseContent
def jsonSlurper = new JsonSlurper().parseText(response)
//Assign it to a testCase Property to grab for second Rest Request
if (jsonSlurper.size() > 0) {
testCase.setPropertyValue("Resp1Field1Value",Resp1Field1Value)
testCase.setPropertyValue("Resp1Field2Value",Resp1Field2Value)
);
//Call the second Rest Request
testStep2.run(testRunner, context)
def response = testStep2.testRequest.response.responseContent
def jsonSlurper = new JsonSlurper().parseText(response)
// Perform Validation/assertion as desired
Use these 2 lines in your groovy script before second request in the same test case
def id_num = (new Random().nextInt(100000000)).toString()
testRunner.testCase.setPropertyValue( "id_num", id_num )
log.info id_num
Now since you have saved the value in a testcase properties in a soap step, you can use like below in the next request within same testcase
${#TestCase#id_num}
This way it will automatically replace the value once you run. To see the values replacement in soap UI , you can see the Raw Tab
check this out below.. the value got replaced
I am trying to deserialize a JSON String using Jackson 2 with RestAssured (java tool for IT tests).
I have a problem. The String I am trying to deserialize is :
{"Medium":{"uuid":"2","estimatedWaitTime":0,"status":"OPEN_AVAILABLE","name":"Chat","type":"CHAT"}}
There is the object type "Medium" at the begining of the String. This cause Jackson failing during deserialization:
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "Medium"
I've set the "IGNORE_ON_UNKNOWN_PROPERTIES" to false and then I got no exception during deserialisation. However, all of my properties are 'null' in java.
Response getAvailability -> {"Medium":{"uuid":"2","estimatedWaitTime":0,"status":"OPEN_AVAILABLE","name":"Chat","type":"CHAT"}}
### MEDIUM name -> null
### MEDIUM uuid -> null
### MEDIUM wait time -> null
### MEDIUM wait time -> null
### MEDIUM status -> null
Does anyone can help me ? (note: I can't change my input JSON string).
{
"Medium": {
"uuid": "2",
"estimatedWaitTime": 0,
"status": "OPEN_AVAILABLE",
"name": "Chat",
"type": "CHAT"
}
}
as you can see uuid and other params are part of medium object , so class in which it can be deserialized is.
class Medium
{
string name;
// specify other params also.
}
class BaseObject
{
Medium Medium;
}
and then use jackson.deserialize('json', BaseObject.class)
above i had given pseudo code
You need to put annotation
#JsonRootName("Medium")
on your bean class and configure object mapper to
mapper.enable(DeserializationFeature.UNWRAP_ROOT_VALUE).
You need a way to remove the Object name that is the part of the input JSON. Since you cannot change the input string, Use this code to change this input string to a tree and get the value of "Medium" node.
ObjectMapper m = new ObjectMapper();
JsonNode root = m.readTree("{\"Medium\":{\"uuid\":\"2\",\"estimatedWaitTime\":0,\"status\":\"OPEN_AVAILABLE\",\"name\":\"Chat\",\"type\":\"CHAT\"}}");
JsonNode obj = root.get("Medium");
Medium medium = m.readValue(obj.asText, Medium.class);
The title talks by itself, I have a Config object (from https://github.com/typesafehub/config) and I want to pass it the a constructor which only supports java.util.Properties as argument.
Is there an easy way to convert a Config to a Properties object ?
Here is a way to convert a typesafe Config object into a Properties java object. I have only tested it in a simple case for creating Kafka properties.
Given this configuration in application.conf
kafka-topics {
my-topic {
zookeeper.connect = "localhost:2181",
group.id = "testgroup",
zookeeper.session.timeout.ms = "500",
zookeeper.sync.time.ms = "250",
auto.commit.interval.ms = "1000"
}
}
You can create the corresponding Properties object like that:
import com.typesafe.config.{Config, ConfigFactory}
import java.util.Properties
import kafka.consumer.ConsumerConfig
object Application extends App {
def propsFromConfig(config: Config): Properties = {
import scala.collection.JavaConversions._
val props = new Properties()
val map: Map[String, Object] = config.entrySet().map({ entry =>
entry.getKey -> entry.getValue.unwrapped()
})(collection.breakOut)
props.putAll(map)
props
}
val config = ConfigFactory.load()
val consumerConfig = {
val topicConfig = config.getConfig("kafka-topics.my-topic")
val props = propsFromConfig(topicConfig)
new ConsumerConfig(props)
}
// ...
}
The function propsFromConfig is what you are mainly interested in, and the key points are the use of entrySet to get a flatten list of properties, and the unwrapped of the entry value, that gives an Object which type depends on the configuration value.
You can try my scala wrapper https://github.com/andr83/scalaconfig. Using it convert config object to java Properties is simple:
val properties = config.as[Properties]
As typesafe config/hocon supports a much richer structure than java.util.propeties it will be hard to get a safe conversion.
Or spoken otherwise as properties can only express a subset of hocon the conversion is not clear, as it will have a possible information loss.
So if you configuration is rather flat and does not contain utf-8 then you could transform hocon to json and then extract the values.
A better solution would be to implement a ConfigClass and populate the values with values from hocon and passing this to the class you want to configure.
It is not possible directly through typesafe config. Even rending the entire hocon file into json does provide a true valid json:
ex:
"play" : {
"filters" : {
"disabled" : ${?play.filters.disabled}[
"play.filters.hosts.AllowedHostsFilter"
],
"disabled" : ${?play.filters.disabled}[
"play.filters.csrf.CSRFFilter"
]
}
}
That format is directly from Config.render
as you can see, disabled is represented twice with hocon style syntax.
I have also had problems with rendering hocon -> json -> hocon
Example hocon:
http {
port = "9000"
port = ${?HTTP_PORT}
}
typesafe config would parse this to
{
"http": {
"port": "9000,${?HTTP_PORT}"
}
}
However if you try to parse that in hocon - it throws a syntax error. the , cannot be there.
The hocon correct parsing would be 9000${?HTTP_PORT} - with no comma between the values. I believe this is true for all array concatenation and substitution
I have a template foo.mustache saved in {{ES_HOME}}/config/scripts.
POST to http://localhost:9200/forward/_search/template with the following message body returns a valid response:
{
"template": {
"file": "foo"
},
"params": {
"q": "a",
"hasfilters": false
}
}
I want to translate this to using the java API now that I've validated all the different components work. The documentation here describes how to do it in java:
SearchResponse sr = client.prepareSearch("forward")
.setTemplateName("foo")
.setTemplateType(ScriptService.ScriptType.FILE)
.setTemplateParams(template_params)
.get();
However, I would instead like to just send a plain string query (i.e. the contents of the message body from above) rather than build up the response using the java. Is there a way to do this? I know with normal queries, I can construct it like so:
SearchRequestBuilder response = client.prepareSearch("forward")
.setQuery("""JSON_QUERY_HERE""")
I believe the setQuery() method wraps the contents into a query object, which is not what I want for my template query. If this is not possible, I will just have to go with the documented way and convert my json params to Map<String, Object>
I ended up just translating my template_params to a Map<String, Object> as the documentation requires. I utilized groovy's JsonSlurper to convert the text to an object with a pretty simple method.
import groovy.json.JsonSlurper
public static Map<String,Object> convertJsonToTemplateParam(String s) {
Object result = new JsonSlurper().parseText(s);
//Manipulate your result if you need to do any additional work here.
//I.e. Programmatically determine value of hasfilters if filters != null
return (Map<String,Object>) result;
}
And you could pass in the following as a string to this method:
{
"q": "a",
"hasfilters": true
"filters":[
{
"filter_name" : "foo.untouched",
"filters" : [ "FOO", "BAR"]
},
{
"filter_name" : "hello.untouched",
"list" : [ "WORLD"]
}
]
}
Let's say I have a function
writeToFileOrStdout(fname: String = Nil) = { ... }
If the user passes a string value for fname, then I'd like to open a file with that name and write to it; otherwise, I'd like to print to stdout. I could always just write an if statement to take care of this, but how would I write a case statement on fname and open the correct corresponding outputStream?
val outStream = fname match {
case Nil => ???
case _ => new java.io.FileOutputStream(new java.io.File(fname))
}
outStream.write( ... )
Thanks!
Why not rewrite the function as:
def writeToFileOrStdout(fname: Option[String] = None) = {
val outStream = fname match{
case Some(name) => new java.io.FileOutputStream(new java.io.File(name))
case None => System.out
}
...
}
It's always a good idea to use Option for an optional input as opposed to using null. That's basically what it's there for. In good scala code, you will not see explicit references to null.
In fact, your code doesn't even compile for me. Nil is used to represent an empty list, not a null or non supplied String.
To augment cmbaxter's response...
Mapping a String with a possible null value to Option[String] is trivial: Option(stringValue) will return None where stringValue is null, and Some(stringValue) where non-null.
Thus, you can either:
writeToFileOrStdout(Option(stringValue)), or
If you're stuck on String (and possibly a null value) as the parameter to writeToFileOrStdout, then internally use Option(fname) and match to what it returns::
def writeToFileOrStdout(fname: String = null) = {
val outStream = Option(fname) match{
case Some(name) => new java.io.FileOutputStream(new java.io.File(name))
case None => System.out
}
...
}
To further augment cmbaxter's response, you might consider writing this:
def filenameToOutputStream(name: String) =
new java.io.FileOutputStream(new java.io.File(name))
def writeToFileOrStdout(fname: Option[String] = None) = {
val outStream = fname map filenameToOutputStream getOrElse System.out
...
}
As the post Idiomatic Scala: Your Options Do Not Match suggests, this might be more idiomatic Scala.