Develop Reference

Looping through SerenityRest Response in JAVA - java

I am trying to get a count of all models from the cars object, which is part of a SerenityRest response.
Response response = SerenityRest.rest()
.contentType("application/json")
.when()
.get("/api/");
if (response.statusCode() == 200) {
int numUniqueModels = response.body().path("cars.size()"); // 3
}
Response:
"cars": {
"Acura": [
"ILX",
"MDX",
"TLX"
],
"Audi": [
"A3",
"A4",
"A6",
"A7"
],
"BMW": [
"x",
"y"
]
}
For example,
response.body().path("cars.size()") = 3,
but i need the sum of cars.Acura.size() + cars.Audi.size() + cars.BMW.size() to get all models. However, i don't know if the exact names Acura, Audi or BMW will exist in the response, since vehicles may change dynamically. To solve this, i will need to do some kind of a loop, where:
sum = 0;
for (int i = 0; i < response.body().path("cars.size()"); i++) {
sum += response.body().path("cars.[i].size()");
}
The sum should give a total number of car models = 9.
The problem is that this syntax: path("cars.[i].size()") is not correct. What is the correct call?

If you want to make complex request with rest-assured you have to follow the synthax described here groovy gpath as mentionned here rest-assured doc:
Note that the JsonPath implementation uses Groovy's GPath syntax and is not to be confused with Jayway's JsonPath implementation.
So you have to play with some groovy synthax:
int total = JsonPath.from("{ "
+ " \"cars\": {\n"
+ " \"Acura\": [\n"
+ " \"ILX\",\n"
+ " \"MDX\",\n"
+ " \"TLX\"\n"
+ " ],\n"
+ " \"Audi\": [\n"
+ " \"A3\",\n"
+ " \"A4\",\n"
+ " \"A6\",\n"
+ " \"A7\"\n"
+ " ],\n"
+ " \"BMW\": [\n"
+ " \"x\",\n"
+ " \"y\"\n"
+ " ]\n"
+ " }"
+ "}")
.getInt("cars.collect { it.value.size() }.sum()")
So this expression should make the job cars.collect { it.value.size() }.sum(). The collect method is like a map method in functional programming. So you map the collection cars HashMap with the size() of its values and you collect the sum()!
Edit
So you just have to do:
Response response = SerenityRest.rest()
.contentType("application/json")
.when()
.get("/api/");
if (response.statusCode() == 200) {
int numUniqueModels = response.body().path("cars.collect { it.value.size() }.sum()"); // 9
}

Related

This question already has answers here:
How to parse JSON in Java
(36 answers)
How to read all keys in Json without specifying keyname using java
(2 answers)
Closed 1 year ago.
I would like to read the data from the JSON file, but not only the values but also the fields in this file, i.e. "_id", "name", "surname" etc. For example, I have a file like the one below and the problem is that the files will contain different data and it is not one and the same the file itself so the fields will change and not be the same all the time.
[
{
"_id": 1,
"name": "Adam",
"surname": "Smith",
"course": "IT",
"grades": [
{
"maths": 4,
"physics": 4,
"programming": 5
},
{
"maths": 3,
"physics": 5,
"programming": 4
}
]
},
{
"_id": 2,
"name": "Robert",
"surname": "Brown",
"course": "IT",
"grades": [
{
"maths": 5,
"physics": 5,
"angielski": 5
},
{
"maths": 4,
"physics": 4,
"programming": 4
}
]
}
]
I thought about parsing the file into a string and reading it character by character, but that would be time consuming. And here is my question, how to read not only values but also fields in a JSON file.

I think you want to iterate over all Objects/Arrays in this json.
If you are fine using a library, below one will be helpful
Maven:
<dependency>
<groupId>org.json</groupId>
<artifactId>org.json</artifactId>
<version>chargebee-1.0</version>
</dependency>
This is a sample code for reading your string
import java.io.IOException;
import java.util.Iterator;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
public class Test {
public static void main(String[] args) throws IOException, JSONException {
String json_str = "[{\r\n" +
" \"_id\": 1,\r\n" +
" \"name\": \"Adam\",\r\n" +
" \"surname\": \"Smith\",\r\n" +
" \"course\": \"IT\",\r\n" +
" \"grades\": [\r\n" +
" {\r\n" +
" \"maths\": 4,\r\n" +
" \"physics\": 4,\r\n" +
" \"programming\": 5\r\n" +
" },\r\n" +
" {\r\n" +
" \"maths\": 3,\r\n" +
" \"physics\": 5,\r\n" +
" \"programming\": 4\r\n" +
" }]\r\n" +
" },{\r\n" +
" \"_id\": 2,\r\n" +
" \"name\": \"Robert\",\r\n" +
" \"surname\": \"Brown\",\r\n" +
" \"course\": \"IT\",\r\n" +
" \"grades\": [\r\n" +
" {\r\n" +
" \"maths\": 5,\r\n" +
" \"physics\": 5,\r\n" +
" \"angielski\": 5\r\n" +
" },\r\n" +
" {\r\n" +
" \"maths\": 4,\r\n" +
" \"physics\": 4,\r\n" +
" \"programming\": 4\r\n" +
" }]\r\n" +
"}]";
JSONArray jsonArray = new JSONArray(json_str);
int length = jsonArray.length();
for(int i=0; i<length; i++) {
JSONObject jsonObject = jsonArray.getJSONObject(i);
Iterator<String> keys = jsonObject.keys();
while(keys.hasNext()) {
//this will give 1st level keys - "surname,name,course,_id,grades"
String key = keys.next();
if (jsonObject.get(key) instanceof JSONObject) {
//build either a recursive function or required logic to iterate over inner json objects similar to this
} else if (jsonObject.get(key) instanceof JSONArray) {
//build either a recursive function or required logic to iterate over inner json arrays similar to this
} else {
/* Output:
key is = surname ==> value is = Smith
key is = name ==> value is = Adam
key is = course ==> value is = IT
key is = _id ==> value is = 1
key is = surname ==> value is = Brown
key is = name ==> value is = Robert
key is = course ==> value is = IT
key is = _id ==> value is = 2
*/
System.out.println("key is = "+key+" ==> value is = "+jsonObject.get(key));
}
}
}
}
}

I'm trying ElasticSearch 7.9 and wanted to do a benchmark on 1M documents. I use the 'single node' docker image.
I use the high level java client to index documents using BulkRequest. I consistenly get a Too Many Requests exception after 360k requests, even if I put some sleep(1000) statements after each 10k docs.
I tried increasing the memory in the jvm.options from 1G to 8G but that did not affect it.
Is there an option to increase this number of requests?
My laptop has 4 cores and 16GB and docker is not limited in any way.
Error details:
{"error":{"root_cause":[{"type":"es_rejected_execution_exception","reason":"rejected execution of coordinating operation [coordinating_and_primary_bytes=0, replica_bytes=0, all_bytes=0, coordinating_operation_bytes=108400734, max_coordinating_and_primary_bytes=107374182]"}],"type":"es_rejected_execution_exception","reason":"rejected execution of coordinating operation [coordinating_and_primary_bytes=0, replica_bytes=0, all_bytes=0, coordinating_operation_bytes=108400734, max_coordinating_and_primary_bytes=107374182]"},"status":429}
Indexing code
CreateIndexRequest createIndexRequest = new CreateIndexRequest(index);
createIndexRequest.mapping(
"{\n" +
" \"properties\": {\n" +
" \"category\": {\n" +
" \"type\": \"keyword\"\n" +
" },\n" +
" \"title\": {\n" +
" \"type\": \"keyword\"\n" +
" },\n" +
" \"naam\": {\n" +
" \"type\": \"keyword\"\n" +
" }\n" +
" }\n" +
"}",
XContentType.JSON);
CreateIndexResponse createIndexResponse = client.indices().create(createIndexRequest, RequestOptions.DEFAULT);
for (int b=0;b<100; b++) {
List<Book> bookList = new ArrayList<>();
for (int i = 0; i < 10_000; i++) {
int item = b*100_000 + i;
bookList.add(new Book("" + item,
item % 2 == 0 ? "aap" : "banaan",
item % 4 == 0 ? "naam1" : "naam2",
"Rob" + item,
"The great start" + item/100,
item));
}
bookList.forEach(book -> {
IndexRequest indexRequest = new IndexRequest().
source(objectMapper.convertValue(book, Map.class)).index(index).id(book.id());
bulkRequest.add(indexRequest);
});
System.out.println("Ok, batch: " + b);
bulkRequest.timeout(TimeValue.timeValueSeconds(20));
try {
Thread.sleep(1_000);
} catch (InterruptedException e) {
e.printStackTrace();
}
try {
client.bulk(bulkRequest, RequestOptions.DEFAULT);
System.out.println("Ok2");
} catch (IOException e) {
e.printStackTrace();
// System.out.println(objectMapper.convertValue(book, Map.class));
}
}

Ok I found it. I just kept adding request to the BulkRequest instead of clearing it.

I'm using the Closure Compiler in Google. I'm a beginner at this. When I run my code, I get an error.
I want to know what is the reason or is there any demo used.
my code：
public class JsClosureCompiler {
public static String compileJs(String code){
Compiler compiler = new Compiler();
CompilerOptions options = new CompilerOptions();
// Simple mode is used here, but additional options could be set, too.
CompilationLevel.ADVANCED_OPTIMIZATIONS.setOptionsForCompilationLevel(options);
// To get the complete set of externs, the logic in
// CompilerRunner.getDefaultExterns() should be used here.
SourceFile extern = SourceFile.fromCode("externs.js",
"function textDiv(text){};");
// The dummy input name "input.js" is used here so that any warnings or
// errors will cite line numbers in terms of input.js.
SourceFile input = SourceFile.fromCode("input.js", code);
// compile() returns a Result, but it is not needed here.
compiler.compile(extern, input, options);
// The compiler is responsible for generating the compiled code; it is not
// accessible via the Result.
if(compiler.getErrorCount() > 0){
StringBuilder erroInfo = new StringBuilder();
for(JSError jsError: compiler.getErrors()) {
erroInfo.append(jsError.toString());
}
}
return compiler.toSource();
}
public static void main(String[] args) {
String code = "function makeNoteDom(noteTitle, noteContent, noteContainer) {\n" +
" // Create DOM structure to represent the note.\n" +
" var headerElement = textDiv(noteTitle);\n" +
" var contentElement = textDiv(noteContent);\n" +
"\n" +
" var newNote = document.createElement('div');\n" +
" newNote.appendChild(headerElement);\n" +
" newNote.appendChild(contentElement);\n" +
"\n" +
" // Add the note's DOM structure to the document.\n" +
" noteContainer.appendChild(newNote);\n" +
"}\n" +
"\n" +
"/**\n" +
" * Iterates over a list of note data objects and creates a DOM\n" +
" */\n" +
"function makeNotes(data, noteContainer) {\n" +
" for (var i = 0; i < data.length; i++) {\n" +
" makeNoteDom(data[i].title, data[i].content, noteContainer);\n" +
" }\n" +
"}\n" +
"\n" +
"function main() {\n" +
" var noteData = [\n" +
" {title: 'Note 1', content: 'Content of Note 1'},\n" +
" {title: 'Note 2', content: 'Content of Note 2'}];\n" +
" var noteListElement = document.getElementById('notes');\n" +
" makeNotes(noteData, noteListElement);\n" +
"}\n" +
"\n" +
"main();";
String s = compileJs(code);
System.out.println(s);
}
}
--------------------------------error--error--------------------------------
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.collect.ImmutableList.toImmutableList()Ljava/util/stream/Collector;
at com.google.javascript.jscomp.deps.DependencyInfo$Require.asSymbolList(DependencyInfo.java:60)
at com.google.javascript.jscomp.deps.DependencyInfo$Base.getRequiredSymbols(DependencyInfo.java:163)
at com.google.javascript.jscomp.Compiler.findModulesFromInput(Compiler.java:1901)
at com.google.javascript.jscomp.Compiler.findModulesFromEntryPoints(Compiler.java:1857)
at com.google.javascript.jscomp.Compiler.parseInputs(Compiler.java:1666)
at com.google.javascript.jscomp.Compiler.parseForCompilationInternal(Compiler.java:939)
at com.google.javascript.jscomp.Compiler.lambda$parseForCompilation$4(Compiler.java:922)
at com.google.javascript.jscomp.CompilerExecutor$2.call(CompilerExecutor.java:102)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

put guava into pom.xml
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>22.0</version>
</dependency>

The json looks like this :
"hour_totals": {
"382423": {
"imp": 126,
"clk": 1,
"spend": "$0.03",
"conv": 0,
"cpm": "$0.22",
"cpc": "$0.03",
"ctr": "0.79%",
"cpa": "$Inf"
},
"382424": {
"imp": 209,
"clk": 1,
"spend": "$0.05",
"conv": 0,
"cpm": "$0.23",
"cpc": "$0.05",
"ctr": "0.48%",
"cpa": "$Inf"
}}
I read more than 20 answers, but unable to find how to deserialize such a structure, please help on what would the class look like since the hour is not a fixed string.

To parse this JSON with Gson you need two steps.
Define this classes:
public class Total {
Map<String, HourData> hour_totals;
#Override
public String toString() {
return "Total [hour_totals=" + hour_totals + "]";
}
}
where HourData is
public class HourData {
Integer imp;
Integer clk;
String spend;
Integer conv;
String cpm;
String cpc;
String cpa;
#Override
public String toString() {
return "HourData [imp=" + imp + ", clk=" + clk + ", spend=" + spend
+ ", conv=" + conv + ", cpm=" + cpm + ", cpc=" + cpc + ", cpa="
+ cpa + "]";
}
}
Hack a bit your "Json string" since it's not a valid Json (see more details below). You just need to add braces like this code:
public class Q19201300 {
public static void main(String[] args) {
String json = "\"hour_totals\": { "
+ " \"382423\": { "
+ " \"imp\": 126, "
+ " \"clk\": 1, "
+ " \"spend\": \"$0.03\", "
+ " \"conv\": 0, "
+ " \"cpm\": \"$0.22\", "
+ " \"cpc\": \"$0.03\", "
+ " \"ctr\": \"0.79%\", "
+ " \"cpa\": \"$Inf\" "
+ "}, "
+ "\"382424\": { "
+ " \"imp\": 209, "
+ " \"clk\": 1, "
+ " \"spend\": \"$0.05\", "
+ " \"conv\": 0, "
+ " \"cpm\": \"$0.23\", "
+ " \"cpc\": \"$0.05\", "
+ " \"ctr\": \"0.48%\", "
+ " \"cpa\": \"$Inf\" "
+ "}} ";
Total t = new Gson().fromJson("{" + json + "}", Total.class);
System.out.println(t);
}
}
This will give you:
Total [hour_totals={382423=HourData [imp=126, clk=1, spend=$0.03,
conv=0, cpm=$0.22, cpc=$0.03, cpa=$Inf], 382424=HourData [imp=209,
clk=1, spend=$0.05, conv=0, cpm=$0.23, cpc=$0.05, cpa=$Inf]}]
About your string. From JSON official grammar (http://www.ietf.org/rfc/rfc4627.txt):
JSON Grammar
A JSON text is a sequence of tokens. The set of tokens includes
six structural characters, strings, numbers, and three literal
names.
A JSON text is a serialized object or array.

I am using boilerpipe and it seems great, but I want to output JSON. I am using the Java version and testing in NetBeans as follows:
final URL url = new URL("http://mashable.com/2012/09/26/worlds-best-father-kickstarter-calendar");
System.out.println(ArticleExtractor.INSTANCE.getText(url));
Can anyone tell me how I go about this?

Boilerpipe does not come with a JSON serializer.
You can, however, do this (assuming you already extracted all data):
public String articleTextToJson(String article, String title, String sourceUrl) {
if (null == article) {
return "{ \"error\" : { " +
" \"message\" : \"Article did not extract\", " +
" \"code\" : 1 " +
" }, " +
" \"status\" : \"error\" " +
"}";
}
return "{ \"response\" : { " +
" \"title\" : \"" + title + "\" " +
" \"content\" : \"" + article + "\", " +
" \"source\" : \"" + sourceUrl + "\" " +
" }, " +
" \"status\" : \"success\" " +
"}"
}
The tricky part will be of course getting the title...
Or better yet use some JSON serializer like JSONObject.
Hope that helps.