Develop Reference

Merge two JSON into one JSON using Java and validate in karate feature file

Json1 {"key1" :"one","key2":"two"}
Json2 {"FN": "AB","LN":"XY"}
I wish to have Json3 {"key1" :"one","key2":"two","FN": "AB","LN":"XY"}
I have used below code but it does not work:
JSONObject mergedJSON = new JSONObject();
try {
mergedJSON = new JSONObject(json1, JSONObject.getNames(json1));
for (String Key : JSONObject.getNames(json2)) {
mergedJSON.put(Key, json2.get(Key));
}
} catch (JSONException e) {
throw new RuntimeException("JSON Exception" + e);
}
return mergedJSON;
}
* call defaultCOM {ID: "COM-123"}
* def defaultResponse = response.data.default
* def jMap = mergeJSON.toMap(defaultResponse)
Here error comes (language: Java, type: com.intuit.karate.graal.JsMap) to Java type 'org.json.JSONObject': Unsupported target type

All I'll say is that the recommended way to merge 2 JSONs is given in the documentation: https://github.com/karatelabs/karate#json-transforms
* def foo = { a: 1 }
* def bar = karate.merge(foo, { b: 2 })
* match bar == { a: 1, b: 2 }
I'll also say that when you use custom Java code, you should stick to using Map or List: https://github.com/karatelabs/karate#calling-java
And if you use things like JSONObject whatever that is, you are on your own - and please consider that not supported by Karate.
When you have to mix Java and the Karate-style JS code (and this something you should try to avoid as far as possible) you need to be aware of some caveats: https://github.com/karatelabs/karate/wiki/1.0-upgrade-guide#js-to-java

Well, if you just don't care about key collisions this should work:
String jsons01 = "{\"key1\" :\"one\",\"key2\":\"two\"}";
String jsons02 = "{\"FN\": \"AB\",\"LN\":\"XY\"}";
JSONObject jsono01 = new JSONObject(jsons01);
JSONObject jsono02 = new JSONObject(jsons02);
JSONObject merged = new JSONObject(jsono01, Object.getNames(jsono01));
for (String key : JSONObject.getNames(jsono02)) {
merged.append(key, jsono02.get(key));
}
System.out.println(merged);
Result is: {"key1":"one","FN":["AB"],"key2":"two","LN":["XY"]}

How create a (key, value) in JsonArray

I have a JSONObject, like the output in this link:
https://hadoop.apache.org/docs/r1.0.4/webhdfs.html#GETFILESTATUS
I woul dlike to get the pathSuffix (file names) and the modificationTime (Dates) values in a JSON Array, like this:
[
{ file1, date
},
{ file1, date
},
{ file1, date
},
.
.
.
]
My code is the following:
JsonObject fileStatuses = jsonObject.getJsonObject("FileStatuses");
JsonArray fileStatus = (JsonArray) fileStatuses.getJsonArray("FileStatus");
for (int i = 0; i < fileStatus.size(); i++) {
JsonObject rec = fileStatus.getJsonObject(i);
String pathSuffix = rec.getString("pathSuffix");
String modificationTime = rec.getJsonNumber("modificationTime").toString();
long a = Long.parseLong(modificationTime);
Date modificationTimeDate = new Date(a);
JsonObject jo = Json.createObjectBuilder()
.add("list", Json.createArrayBuilder()
.add(Json.createObjectBuilder()
.add(pathSuffix, (JsonValue) modificationTimeDate)
))
.build();
logger.info("JSON object is '{}'", jo);
I got this Exception:
Exception in thread "main" java.lang.ClassCastException: java.util.Date incompatible with javax.json.JsonValue
How can I create a JsonArray that contain the values of Pathsuffix and ModificationTime like (key, value) ? Thanks

If you want to store a JSON-Object filename:modificationTime you can accomplish this by constructing a Map that holds your key and value pairs:
JsonArrayBuilder builder = Json.createArrayBuilder();
for (int i = 0; i < fileStatus.size(); i++) {
JsonObject rec = fileStatus.getJsonObject(i);
String timeStamp = rec.get("modificationTime").toString();
Map<String, Object> jsonMapping = Map.of(rec.getString("pathSuffix"),
timeStamp);
builder.add(Json.createObjectBuilder(jsonMapping));
}
After that, for the json from your Link, builder.build() will yield a JsonArray like this:
[
{
"a.patch":"1320171722771" // FileStatus.pathSuffix : FileStatus.modificationTime
},
{
"bar":"1320895981256"
}
]
Unless you need to construct a certain Date from the timestamp I stronlgy advice to store the acquired timestamp as is. This is because creating any LocalDate (or worse: Date) from a timestamp will yield inaccurate dates unless you consider the TimeZones of both where the file modification happened and where the modificationTime value is read.

json does not support a time type, that is the reason for the error. What you need to do is to change that into a type json can use. That might be a string that represents the time (choose the formating yourself, so you are sure, that when reading it out again you have consistent data) or easier you just keep the long value used.
Here you cansee what json can use:
https://www.json.org/json-en.html

try this json structure
[
{
fileName:"test.java",
modificationTime:"2021-03-02 xx:xx:xx"
}，
{
fileName:"test2.java",
modificationTime:"2021-03-01 xx:xx:xx"
}
]

Reading data from Android App using bluetooth

I have Java code to receive data in Android App via Bluetooth like the attached code
Java Code
so readMessage will equal = {\"Pin\":\"A4\",\"Value\":\"20\"},{\"Pin\":\"A5\",\"Value\":\"925\"},{\"Pin\":\"A0\",\"Value\":\"30\"}
So I want to take only the values after string \"Value\" from received data so
Can anyone suggest how to make do that?
Thanks

you can parse the readMessage with JSON format
example:
String[] pinValueArr = readMessage.split(",")
for (String pinValue : pinValueArr) {
try {
JSONObject pinValueJSON = new JSONObject(pinValue);
String pin = pinValueJSON.optString("pin", ""); // opt means if parse failed, return default value what is ""
int pin = pinValueJSON.optInt("Value", 0); // opt means if parse failed, return default value what is "0"
} catch (JSONParsedException e) {
// catch exception when parse to JSONObject failed
}
}
And if you want to manage them, you can make a List and add them all.
List<JSONObject> pinValueList = new ArrayList<JSONObject>();
for (String pinValue : pinValueArr) {
JSONObject pinValueJSON = new JSONObject(pinValue);
// ..
pinValueList.add(pinValueJSON);
}

You can use Gson to convert Json to Object.
(https://github.com/google/gson)
Create Model Class
data class PinItem(
#SerializedName("Pin")
val pin: String? = null,
#SerializedName("Value")
val value: String? = null
)
Convert your json.
val json = "[{"Pin":"A4","Value":"20"},{"Pin":"A5","Value":"925"},{"Pin":"A0","Value":"30"}]"
val result = Gson().fromJson(this, object : TypeToken<List<PinItem>>() {}.type)
So now you having list PinItem and you can get all info off it.

ElasticSearch Indexing 100K documents with BulkRequest API using Java RestHighLevelClient

Am reading 100k plus file path from the index documents_qa using scroll API. Actual files will be available in my local d:\drive. By using the file path am reading the actual file and converting into base64 and am reindex with the base64 content (of a file) in another index document_attachment_qa.
My current implementation is, am reading filePath, convering the file into base64 and indexing document along with fileContent one by one. So its taking more time for eg:- indexing 4000 documents its taking more than 6 hours and also connection is terminating due to IO exception.
So now i want to index the documents using BulkRequest API, but am using RestHighLevelClient and am not sure how to using BulkRequest API along with RestHighLevelClient.
Please find my current implementation, which am indexing one by one document.
jsonMap = new HashMap<String, Object>();
jsonMap.put("id", doc.getId());
jsonMap.put("app_language", doc.getApp_language());
jsonMap.put("fileContent", result);
String id=Long.toString(doc.getId());
IndexRequest request = new IndexRequest(ATTACHMENT, "doc", id ) // ATTACHMENT is the index name
.source(jsonMap) // Its my single document.
.setPipeline(ATTACHMENT);
IndexResponse response = SearchEngineClient.getInstance3().index(request); // increased timeout
I found the below documentation for BulkRequest.
https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-docs-bulk.html
But am not sure how to implement BulkRequestBuilder bulkRequest = client.prepareBulk(); client.prepareBulk() method when and using RestHighLevelClient.
UPDATE 1
Am trying to indexing all 100K documents in one shot. so i creating one JSONArray and put all my JSONObject into the array one by one. Finally am trying to build BulkRequest and add all my documents (JSONArray) as a source to the BulkRequest and trying to index them.
Here am not sure, how to convert my JSONArray to List of String.
private final static String ATTACHMENT = "document_attachment_qa";
private final static String TYPE = "doc";
JSONArray reqJSONArray=new JSONArray();
while (searchHits != null && searchHits.length > 0) {
...
...
jsonMap = new HashMap<String, Object>();
jsonMap.put("id", doc.getId());
jsonMap.put("app_language", doc.getApp_language());
jsonMap.put("fileContent", result);
reqJSONArray.put(jsonMap)
}
String actionMetaData = String.format("{ \"index\" : { \"_index\" : \"%s\", \"_type\" : \"%s\" } }%n", ATTACHMENT, TYPE);
List<String> bulkData = // not sure how to convert a list of my documents in JSON strings
StringBuilder bulkRequestBody = new StringBuilder();
for (String bulkItem : bulkData) {
bulkRequestBody.append(actionMetaData);
bulkRequestBody.append(bulkItem);
bulkRequestBody.append("\n");
}
HttpEntity entity = new NStringEntity(bulkRequestBody.toString(), ContentType.APPLICATION_JSON);
try {
Response response = SearchEngineClient.getRestClientInstance().performRequest("POST", "/ATTACHMENT/TYPE/_bulk", Collections.emptyMap(), entity);
return response.getStatusLine().getStatusCode() == HttpStatus.SC_OK;
} catch (Exception e) {
// do something
}

You can just new BulkRequest() and add the requests without using BulkRequestBuilder, like:
BulkRequest request = new BulkRequest();
request.add(new IndexRequest("foo", "bar", "1")
.source(XContentType.JSON,"field", "foobar"));
request.add(new IndexRequest("foo", "bar", "2")
.source(XContentType.JSON,"field", "foobar"));
...
BulkResponse bulkResponse = myHighLevelClient.bulk(request, RequestOptions.DEFAULT);

In addition to #chengpohi answer. I would like to add below points:
A BulkRequest can be used to execute multiple index, update and/or delete operations using a single request.
It requires at least one operation to be added to the Bulk request:
BulkRequest request = new BulkRequest();
request.add(new IndexRequest("posts", "doc", "1")
.source(XContentType.JSON,"field", "foo"));
request.add(new IndexRequest("posts", "doc", "2")
.source(XContentType.JSON,"field", "bar"));
request.add(new IndexRequest("posts", "doc", "3")
.source(XContentType.JSON,"field", "baz"));
Note: The Bulk API supports only documents encoded in JSON or SMILE.
Providing documents in any other format will result in an error.
Synchronous Operation:
BulkResponse bulkResponse = client.bulk(request, RequestOptions.DEFAULT);
client will be High-Level Rest Client and execution will be synchronous.
Asynchronous Operation(Recommended Approach):
client.bulkAsync(request, RequestOptions.DEFAULT, listener);
The asynchronous execution of a bulk request requires both the BulkRequest instance and an ActionListener instance to be passed to the asynchronous method.
Listener Example:
ActionListener<BulkResponse> listener = new ActionListener<BulkResponse>() {
#Override
public void onResponse(BulkResponse bulkResponse) {
}
#Override
public void onFailure(Exception e) {
}
};
The returned BulkResponse contains information about the executed operations and allows to iterate over each result as follows:
for (BulkItemResponse bulkItemResponse : bulkResponse) {
DocWriteResponse itemResponse = bulkItemResponse.getResponse();
if (bulkItemResponse.getOpType() == DocWriteRequest.OpType.INDEX
|| bulkItemResponse.getOpType() == DocWriteRequest.OpType.CREATE) {
IndexResponse indexResponse = (IndexResponse) itemResponse;
} else if (bulkItemResponse.getOpType() == DocWriteRequest.OpType.UPDATE) {
UpdateResponse updateResponse = (UpdateResponse) itemResponse;
} else if (bulkItemResponse.getOpType() == DocWriteRequest.OpType.DELETE) {
DeleteResponse deleteResponse = (DeleteResponse) itemResponse;
}
}
The following arguments can optionally be provided:
request.timeout(TimeValue.timeValueMinutes(2));
request.timeout("2m");
I hope this helps.

Parsing unnamed nested arrays with minimal-json?

So I'm working on a fairly simple Java program which grabs market data from cryptocurrency exchanges and displays information to the user. I am using the minimal-json library.
Here is my current code:
public class Market {
static JsonArray arrayBittrex;
public static void startTimer(){
Timer timer = new Timer();
timer.scheduleAtFixedRate(new TimerTask() {
#Override
public void run() {
String url = "https://bittrex.com/api/v1.1/public/getmarketsummaries";
try {
URL url2 = new URL(url);
URLConnection con = url2.openConnection();
InputStream in = con.getInputStream();
String encoding = "UTF-8";
String body = IOUtils.toString(in, encoding);
arrayBittrex = Json.parse(body).asObject().get("result").asArray();
}
catch(MalformedURLException e) {}
catch(IOException e) {}
}
}, 0,5000);
}
public static float getPrice(String exchange, String market) {
for (JsonValue item : arrayBittrex) {
float last = item.asObject().getFloat("Last", 0);
System.out.println(last);
return last;
}
return 0;
}
}
This code works with simple json, for example (from https://bittrex.com/api/v1.1/public/getmarketsummary?market=btc-ltc):
{
"success" : true,
"message" : "",
"result" : [{
"MarketName" : "BTC-LTC",
"High" : 0.01350000,
"Low" : 0.01200000,
"Volume" : 3833.97619253,
"Last" : 0.01349998
}
]
}
It will properly return the "Last" value in the array.
However, this cant work when the json has multiple arrays (like in https://bittrex.com/api/v1.1/public/getmarketsummaries):
{
"success" : true,
"message" : "",
"result" : [{
"MarketName" : "BTC-888",
"High" : 0.00000919,
"Low" : 0.00000820,
"Volume" : 74339.61396015,
"Last" : 0.00000820
}, {
"MarketName" : "BTC-A3C",
"High" : 0.00000072,
"Low" : 0.00000001,
"Volume" : 166340678.42280999,
"Last" : 0.00000005
}
]
}
So my question is: how can I get the "Last" value by searching for the array by the "MarketName" value?

Here is a direct & null-safe way to tackle this using Java 8 library Dynamics. We're going to parse the json into a Map, read that map dynamically to what we want.
So first we can use Jackson, Gson or something to convert json -> map.
// com.fasterxml.jackson.core:jackson-databind json -> map
Map jsonMap = new ObjectMapper()
.enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS)
.readValue(jsonStringOrInputSourceEtc, Map.class);
We can now get a Dynamic instance. And, for example, grab the BTC-A3C - Last value.
Dynamic json = Dynamic.from(jsonMap);
BigDecimal a3cLast = json.get("result").children()
.filter(data -> data.get("MarketName").asString().equals("BTC-A3C"))
.findAny()
.flatMap(data -> data.get("Last").maybe().convert().intoDecimal())
.orElse(BigDecimal.ZERO);
// 5E-8
Or perhaps convert the whole lot into a map of MarketName -> Last value
Map<String, BigDecimal> marketNameLastValue = json.get("result").children()
// assume fields are always present, otherwise see #maybe() methods
.collect(toMap(
data -> data.get("MarketName").asString(),
data -> data.get("Last").convert().intoDecimal()
));
// {BTC-A3C=5E-8, BTC-888=0.00000820}
See more examples https://github.com/alexheretic/dynamics