Anyone know how to efficiently set json in groovy with variable paths?
Context: I am working with soapui, a testing tool. Some tests are candidates to be data-driven. I have alot of variables. To make something sustainable that is easily implementable in similar circumstances, I would like a Groovy script that enables me to set variables.
I would name the variables 'parent.subParent.child'.
What I found:
http://groovy-lang.org/json.html
Referencing groovy variable as part of JSON path
I did find other things, but did not record them all.
The straight-forward thing I found was evaluation. With evaluation it was possible to get the values, but not the set them.
Eval.x(jsonbuilder, 'x.content.' + path) = 'newValue'
will return an error. But like I said, no problem retrieving the values in the json this way.
What I tried:
I have got an implementation which works for one level.
I can say:
jsonbuilder.content.parent.subParent[child] = 'newValue'
This will set the value of the requested entity.
Then I tried to expand this to an undefined number of levels.
//Assuming there is a jsonbuilder initialized
def jsonString = "{"parent":{"subParent":{"child":"oldValue"}}}"
def json = new JsonSlurper().parseText(jsonString)
def jsonbuilder = new JsonBuilder(json)
def path = 'parent.subParent.child'
def listPath = path.split("\\.")
def element = jsonbuilder.content
for(int i = 0; i < listPath.size(); i++) {
element = element[listPath[i]]
}
element = 'newValue'
assert jsonbuilder.toString() == "{"parent":{"subParent":{"child":"newValue"}}}"
The issue: the value in the original json is not updated. Likely because I leave the jsonbuilder variable once I assign it to 'element' and continue with that entity.
That leaves me with two questions:
How do I get the element value in the original json?
More general: How do I update json with a variable path?
The rudimentary JSON assign function with jsonbuilder like this: jsonbuilder.content.parent.subParent.child = 'newValue' as given in one of the answers below is not what I am eyeing for. I am looking for a way to make the entire thing dynamic. I don't want to build a simple assignment, that already exists and works well. I am looking to build a machine that does the assignment for me, with the variable names parsed as the paths. Preferably within the groovy.json.* environment, but if I have to involve external libraries, so be it.
I was staring myself blind on a specific implementation of Eval. My solution was actually simple if I would have read the docs from the start.
You can find the docs for Eval here: http://docs.groovy-lang.org/2.4.7/html/api/groovy/util/Eval.html
Instead of trying to assign a value to an evaluated method/function, which is not logical now I think of if, you need to integrate everything into the evaluated expression. For what I find, you can have up to three variables you can use in you Eval function.
I only need two. I need the jsonbuilder object to be able to get the source of information. And I need to get the value to set. The path itself can be used as it exists because it is already what it needs to be with respect to the evaluation: a String.
The code:
import groovy.json.*
def jsonString = '{"parent":{"child":"oldValue"}}'
def newValue = 'newValue'
def stringPath = 'parent.child'
def json = new JsonSlurper().parseText(jsonString)
def jsonbuilder = new JsonBuilder(json)
Eval.xy(jsonbuilder, newValue, 'x.content.' + stringPath + '= y')
System.out.println(jsonbuilder.toString()=='{"parent":{"child":"newValue"}}')
System.out.println(jsonbuilder.content.parent.child == 'newValue')
By using Eval.xy(objectOne, objectTwo, StringExpression), I am telling that I am passing a string to be evaluated as an expression, in which x represents objectOne and y represents objectTwo.
The code can be viewed in an online groovy script engine here: https://groovyconsole.appspot.com/edit/5202721384693760
Small disclaimer: I can't imagine using an evaluated expression in a code base that lets variables be randomly manipulated by the outside world. This expression, if used, will sit comfortably inside the context of my SoapUI project.
Since you are willing to use library, json-path does that.
Credits to #kalle from here
Download the zip files from here
Extract the libraries and its dependencies from above zip
Copy them under SOAPUI_HOME/bin/ext directory
Restart SoapUI
Here you go:
import com.jayway.jsonpath.Configuration
import com.jayway.jsonpath.JsonPath
import com.jayway.jsonpath.spi.json.JacksonJsonNodeJsonProvider
import com.jayway.jsonpath.spi.mapper.JacksonMappingProvider
Configuration configuration = Configuration.builder()
.jsonProvider(new JacksonJsonNodeJsonProvider())
.mappingProvider(new JacksonMappingProvider())
.build()
//You need to prepend $. before the path which becomes valid jsonpath
def path = '$.parent.subParent.child'
def originalJson = """{
"parent": {
"subParent": {
"child": "oldValue"
}
}
}"""
def updatedJson = JsonPath.using(configuration).parse(originalJson).set(path, 'newValue').json()
println(updatedJson.toString())
Here you go:
import groovy.json.JsonSlurper
import groovy.json.JsonBuilder
def jsonString = """{ "parent": {
"subParent": {
"child": "oldValue"
}
}
}"""
def json = new JsonSlurper().parseText(jsonString)
def jsonbuilder = new JsonBuilder(json)
//Assign the value for child with new value
jsonbuilder.content.parent.subParent.child = 'newValue'
println jsonbuilder.toPrettyString()
You can try online Demo
Related
I want to use a single YAML file which contains several different objects - for different applications. I need to fetch one object to get an instance of MyClass1, ignoring the rest of docs for MyClass2, MyClass3, etc. Some sort of selective de-serializing: now this class, then that one... The structure of MyClass2, MyClass3 is totally unknown to the application working with MyClass1. The file is always a valid YAML, of course.
The YAML may be of any structure we need to implement such a multi-class container. The preferred parsing tool is snakeyaml.
Is it sensible? How can I ignore all but one object?
UPD: replaced all "document" with "object". I think we have to speak about the single YAML document containing several objects of different structure. More of it, the parser knows exactly only 1 structure and wants to ignore the rest.
UDP2: I think it is impossible with snakeyaml. We have to read all objects anyway - and select the needed one later. But maybe I'm wrong.
UPD2: sample config file
---
-
exportConfiguration781:
attachmentFieldName: "name"
baseSftpInboxPath: /home/user/somedir/
somebool: false
days: 9999
expected:
- ABC w/o quotes
- "Cat ABC"
- "Some string"
dateFormat: yyyy-MMdd-HHmm
user: someuser
-
anotherConfiguration:
k1: v1
k2:
- v21
- v22
This is definitely possible with SnakeYAML, albeit not trivial. Here's a general rundown what you need to do:
First, let's have a look what loading with SnakeYAML does. Here's the important part of the YAML class:
private Object loadFromReader(StreamReader sreader, Class<?> type) {
Composer composer = new Composer(new ParserImpl(sreader), resolver, loadingConfig);
constructor.setComposer(composer);
return constructor.getSingleData(type);
}
The composer parses YAML input into Nodes. To do that, it doesn't need any knowledge about the structure of your classes, since every node is either a ScalarNode, a SequenceNode or a MappingNode and they just represent the YAML structure.
The constructor takes a root node generated by the composer and generates native POJOs from it. So what you want to do is to throw away parts of the node graph before they reach the constructor.
The easiest way to do that is probably to derive from Composer and override two methods like this:
public class MyComposer extends Composer {
private final int objIndex;
public MyComposer(Parser parser, Resolver resolver, int objIndex) {
super(parser, resolver);
this.objIndex = objIndex;
}
public MyComposer(Parser parser, Resolver resolver, LoaderOptions loadingConfig, int objIndex) {
super(parser, resolver, loadingConfig);
this.objIndex = objIndex;
}
#Override
public Node getNode() {
return strip(super.getNode());
}
private Node strip(Node input) {
return ((SequenceNode)input).getValue().get(objIndex);
}
}
The strip implementation is just an example. In this case, I assumed your YAML looks like this (object content is arbitrary):
- {first: obj}
- {second: obj}
- {third: obj}
And you simply select the object you actually want to deserialize by its index in the sequence. But you can also have something more complex like a searching algorithm.
Now that you have your own composer, you can do
Constructor constructor = new Constructor();
// assuming we want to get the object at index 1 (i.e. second object)
Composer composer = new MyComposer(new ParserImpl(sreader), new Resolver(), 1);
constructor.setComposer(composer);
MyObject result = (MyObject)constructor.getSingleData(MyObject.class);
The answer of #flyx was very helpful for me, opening the way to workaround the library (in our case - snakeyaml) limitations by overriding some methods. Thanks a lot! It's quite possible there is a final solution in it - but not now. Besides, the simple solution below is robust and should be considered even if we'd found the complete library-intruding solution.
I've decided to solve the task by double distilling, sorry, processing the configuration file. Imagine the latter consisting of several parts and every part is marked by the unique token-delimiter. For the sake of keeping the YAML-likenes, it may be
---
#this is a unique key for the configuration A
<some YAML document>
---
#this is another key for the configuration B
<some YAML document
The first pass is pre-processing. For the given String fileString and String key (and DELIMITER = "\n---\n". for example) we select a substring with the key-defined configuration:
int begIndex;
do {
begIndex= fileString.indexOf(DELIMITER);
if (begIndex == -1) {
break;
}
if (fileString.startsWith(DELIMITER + key, begIndex)) {
fileString = fileString.substring(begIndex + DELIMITER.length() + key.length());
break;
}
// spoil alien delimiter and repeat search
fileString = fileString.replaceFirst(DELIMITER, " ");
} while (true);
int endIndex = fileString.indexOf(DELIMITER);
if (endIndex != -1) {
fileString = fileString.substring(0, endIndex);
}
Now we feed the fileString to the simple YAML parsing
ExportConfiguration configuration = new Yaml(new Constructor(ExportConfiguration.class))
.loadAs(fileString, ExportConfiguration.class);
This time we have a single document that must co-respond to the ExportConfiguration class.
Note 1: The structure and even the very content of the rest of configuration file plays absolutely no role. This was the main idea, to get independent configurations in a single file
Note 2: the rest of configurations may be JSON or XML or whatever. We have a method-preprocessor that returns a String configuration - and the next processor parses it properly.
I just started exploring JsonPath today. I want to explore not just what's possible to do with it, but some effective strategies.
For instance, let's say I have to iterate through an array contained within one element in the json string.
I'm using the "store" example from https://github.com/jayway/JsonPath#path-examples .
To get the list of books itself, I would imagine I could do something like this:
List<?> allBooks = JsonPath.<List<?>>read(context, "$.store.book");
Does it make sense to think about it this way?
It's the logic for iterating through this that I'm uncertain about.
I would have thought I could define a "Book" pojo and then do something like this:
for (int ctr = 0; ctr < allBooks.size(); ++ ctr) {
Book book = JsonPath.<Book>read(context, ".[" + ctr + "]");
System.out.println("book[" + book + "]");
}
However, this doesn't work. The "read" method at this point returns a JSONArray.
The last line in the code sample at https://github.com/jayway/JsonPath#what-is-returned-when is close to what I'm looking at, but this requires parsing the json in every iteration. It seems like the "DocumentContext" class has "read" methods that can take a type parameter, but not "JsonPath".
What are some reasonable strategies for navigating something like this?
JSON path will just return you a list of Maps as you've no doubt already seen. You need a way to tell it how to map these values to an object - for this you will need a custom configuration. There are other providers like Gson etc., but I've only used Jackson.
Configuration configuration = Configuration
.builder()
.jsonProvider(new JacksonJsonProvider())
.mappingProvider(new JacksonMappingProvider())
.build();
The second step is to specify generic type information with a TypeRef and pass it along when reading the tag.
List<Book> allBooks = JsonPath.using(configuration)
.parse(context)
.read("$.store.book", new TypeRef<List<Book>>() {});
As a result you get a nice list of Book objects.
I am planning to implement a JavaScript parser in java. I know that there are several ways to do it. There are view frameworks/engines/parsers which could help to do it right, like:
ANTLR 3/4:
it seems like there is only a js grammer for v3
Mozilla Rhino: atm i can parse variable names on initital (top-) namespace. but i am not able to parse nested scopes e.g. object members.. hm..
Nashorn: maybe i should give it a try..?
Maybe:
closure-compiler: IMHO this is very nice. but not for "non-google" js-code :) e.g. you have to apply several coding conventions to your javascript sources to get it working properly..
maybe it is possible to adapt Packer to do it? Is there a Java implementation of Packer???
There is EcmaScript 5.1 related to this article. it seems to be very comfortable. But this is not exactly what I´am looking for.. And still no java :)
My question is:
What could/would be the best way to parse JavaScript for:
(object-)function names
(object-)member names e.g. variables
Is it even possible to do it?
What would be your approach? For me it is not essential to parse ALL special markups of JavaScript.. The important factor would be to parse function/variables in a consistent context for the typical markups like this:
// Avoid `console` errors in browsers that lack a console.
function Object() {
var method;
var noop = function() {
};
var methods = ['assert', 'clear', 'count', 'debug', 'dir', 'dirxml', 'error', 'exception', 'group', 'groupCollapsed', 'groupEnd', 'info', 'log', 'markTimeline', 'profile', 'profileEnd', 'table', 'time', 'timeEnd', 'timeStamp', 'trace', 'warn'];
var length = methods.length;
var console = (window.console = window.console || {});
while (length--) {
method = methods[length];
// Only stub undefined methods.
if (!console[method]) {
console[method] = noop;
}
}
};
var obj = new Object();
var test = "Hello World";
The parse should be able to deliver this information:
Node: Object
Node: Object.method
Node: Object.noop
Node: Object.length
Node: Object.console
Node: Object
Node: obj
Node: test
There is no direct need of any determination if the node is a function/variable.
I need to do a static analysis of Javascript files using Java. Here I need to check whether the Javascript file has any function calls to document.write() or reference to properties like innerHTML etc. Can I use javax.script.* package to achieve this? or Which Java api do I need to use for Parsing? Also can you provide examples for the same?
You can't statically analyze Javascript in the way you intend because Javascript is not a statically typed language.
You can check for document.write() but what if my code was this:
var whatever = document; whatever.write()
Or do you want to reject any function named write() even if it didn't write to the document?
Furthermore, Javascript has an eval function so you could always do:
var m = "ment";
eval("docu" + m + ".wri" + "te('hahahaha')");`.
How are you going to check for that?
Similarly, property access can be done in many ways.
Imagine this piece of code:
var x = document.children[0];
x.innerHTML = ...;
x["inner" + "HTML"] = ...;
var y = "inner";
x[y + "HTML"] = ...;
You're not going to be able to detect all those variants, and the hundreds more variants that you could make, using static analysis.
I am trying to call a constructor for a custom collection object. This custom object takes in a parameter of type Class.
In java, this is done like this:
ICollection col = new PersistentCollection(ContentX.class);
This is my first dive into rhino, and I haven't been able to figure out quite how to pass this parameter. I figured out that "class" is a reserved word and therefor not usable.
I figured that I could get the Class from Class.forName like this:
importPackage(Packages.something.collections);
importPackage(Packages.something.content4);
var col = new PersistentCollection(Class.forName(ContentX));
But it just tosses ClassNotFoundException - with the fully qualified path something.content4.ContentX! So obviously it found the class or it wouldn't have known the path to it.
Am I doing it wrong? Sadly, I'm not in any position to change the java library right now, I need to fix the data without a new deploy.
Googling for javascript class just yields DOM/CSS problems.
I think you simply need to do:
var col = new PersistentCollection(ContentX);
Or, if your class name is a string:
var col = new PersistentCollection(
java.lang.Class.forName('something.content4.ContentX'));