I need help in the logic for transforming one json file to another json file. I am trying to achieve this in mule without dataweave in the custom java component.
I want to convert a linear json to nested json, The input data is a linear json containing the details of all files and directory in particular FTP server. The output Json file should be able to nest the files and directory based on the root directory. Here is the example of input and output json.
{
"InputJson": [
{
"type": "dir",
"id": "RootDir",
"name": "abcd",
"Dir": "/abcd"
},
{
"type": "dir",
"name": "Folder1",
"Dir": "/abcd/Folder1",
"id": "XXXXX"
},
{
"type": "file",
"name": "Folder1SubFolder1",
"Dir": "/abcd/Folder1/Folder1SubFolder1",
"id": "XXXXXX"
},
{
"type": "dir",
"name": "Folder2",
"Dir": "/abcd/Folder2"
"id": "XXXXXX"
},
{
"type": "dir",
"name": "Folder2SubFolder1",
"Dir": "/abcd/Folder2/Folder2SubFolder1"
"id": "XXXXXX"
},
{
"type": "file",
"name": "Folder2SubFolder1SubFolder1",
"Dir": "/abcd/Folder2/Folder2SubFolder1/Folder2SubFolder1SubFolder1"
"id": "XXXXXX"
}
]
}
Output
{
"id": "RootDir",
"value": "Files",
"type": "folder"
"OutData": [{
"value": "Folder1",
"OutData": [{
"value": "Folder1SubFolder1"
}
]
}
]
"OutData": [{
"value": "Folder2",
"OutData": [{
"value": "Folder2SubFolder1",
"OutData":[{
"value": "Folder2SubFolder1SubFolder1",
}]
}
]
}
]
the logic
1. CREATE (java) `outputElemtsList` = []
2. FOR EACH (json) `inputElement` IN `InputJson`
3. CREATE (java) `outputElemt`
4. ADD `outputElement` TO `outputElemtsList`
5. IF `outputElement` HAS `parent`
6. ADD `outputElement` TO `parent`.outData
7. CONVERT `outputElemtsList`[0] TO `Json`
assuming, the list in InputJson, is ordered in the same as the sample, (the child never come before their parent)
if not, you'll need to add some checks as:
3. create `outputElement` if not in `outputElemtsList`; else continue
6. create `parent` if not in `outputElemtsList`
in practice
you can use a Json parser, such jakson, to:
// parse InputJson, to Java Objects
Map<String, Object> rootNode = mapper.readValue(jsonString, Map.class);
// ... implement the logic ...
// serialize a java Object into Json
String outputJson = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(routOutputs);
the code
updated to distinguish between file and folders(dir)
1- OutputElement class
public class OutputElement {
String id, value, type;
public void addOutData(OutputElement outputElement) {}
// constructor, accessors ...
}
1.2- class OutputDir extends OutputElement
public class OutputElement {
List<OutputElement> outData = new ArrayList<>();
#Override
public void addOutData(OutputElement outputElement) {
this.outData.add(outputElement);
}
}
2- Main class : LinearToNestedJson
method to check if outputElements List contains an outputElement
public static boolean contains(List<OutputElement> outputElements, String value) {
for (OutputElement outputElement : outputElements) {
if (outputElement.getValue().equals(value))
return true;
}
return false;
}
main method
public static void main(String args[]) {
JacksonTester tester = new JacksonTester();
try {
ObjectMapper mapper = new ObjectMapper();
String jsonString = IN_JSON;
#SuppressWarnings("unchecked")
Map<String, Object> rootNode = mapper.readValue(jsonString, Map.class);
#SuppressWarnings("unchecked")
List<Map<String, Object>> inputElemnts =
(ArrayList<Map<String, Object>>) rootNode.getOrDefault("InputJson", null);
List<OutputElement> outputElements = new ArrayList<>();
for (Map inputElemnt : inputElemnts) {
String fullpath = (String) inputElemnt.get("Dir");
String[] tree = fullpath.substring(1).split("/");
final int deepth = tree.length;
String dirName = tree[deepth - 1];
final String value = (String) inputElemnt.get("name");
final String id = (String) inputElemnt.get("id");
String type = (String) inputElemnt.get("type");
OutputElement outputElement;
if (type != null && type.equals("dir")) {
outputElement = new OutputDir();
} else {
if(type==null) type = "file";
outputElement = new OutputElement();
}
outputElement.setValue(value);
outputElement.setId(id);
outputElement.setType(type);
if (!contains(outputElements, value)) {
outputElements.add(outputElement);
}
if (deepth > 1) {
String parentName = tree[deepth - 2];
for (OutputElement element : outputElements) {
if (element.getValue().equals(parentName)) {
element.addOutData(outputElement);
}
}
}
// for (int i = 0; i < deepth -1; i++) {
// System.out.println(tree[i]);
// }
}
OutputElement routOutputs = outputElements.get(0);
String outputJson = mapper.writerWithDefaultPrettyPrinter()
.writeValueAsString(routOutputs);
System.out.println(outputJson);
} catch (JsonParseException | JsonMappingException e) {
} catch (IOException e) {
}
}
it's output, for the given input (after validation)
{
"id" : "RootDir",
"value" : "abcd",
"type" : "dir",
"outData" : [ {
"id" : "XXXXX",
"value" : "Folder1",
"type" : "dir",
"outData" : [ {
"id" : "XXXXXX",
"value" : "Folder1SubFolder1",
"type" : "file"
} ]
}, {
"id" : "XXXXXX",
"value" : "Folder2",
"type" : "dir",
"outData" : [ {
"id" : "XXXXXX",
"value" : "Folder2SubFolder1",
"type" : "dir",
"outData" : [ {
"id" : "XXXXXX",
"value" : "Folder2SubFolder1SubFolder1",
"type" : "file"
} ]
} ]
} ]
}
Use groupBy - that's exactly what you need.
Here is code:
%dw 1.0
%output application/json
---
items : payload.InputJson groupBy $.id pluck {
id: $$,
values: $
}
Here is result:
Related
I´m new to the world of json format. I have Json info stored in a json object and I only want to extract name key values in a list. At least I have one user and sometimes more than one user. Extraction using Java or Groovy.
{
"reviewers": [
{
"user": {
"name": "name1.n1",
"emailAddress": "example#example.com"
},
"role": "REVIEWER"
},
{
"user": {
"name": "name2.n2",
"emailAddress": "example2#example.com"
},
"role": "REVIEWER"
}
]
}
basic groovy+json doc here: https://groovy-lang.org/json.html
import groovy.json.JsonSlurper
def json = '''{
"reviewers": [
{
"user": {
"name": "name1.n1",
"emailAddress": "example#example.com"
},
"role": "REVIEWER"
},
{
"user": {
"name": "name2.n2",
"emailAddress": "example2#example.com"
},
"role": "REVIEWER"
}
]
}
'''
def obj = new JsonSlurper().parseText(json)
println obj.reviewers.collect{ it.user.name } // v1
println obj.reviewers*.user.name // the same as above but shorter
Using Java with library org.json.JSONObject;
JSONObject json =new JSONObject(YOUR_JSON_HERE );
JSONArray array = json.getJSONArray("reviewers" );
for(int i=0;i<array.length();i++){
JSONObject user =array.getJSONObject(i);
System.out.println(user.getJSONObject("user").get("name"));
}
}
You can get a list of names like this, using just Groovy:
jason = '''{
"reviewers": [
{
"user": {
"name": "name1.n1",
"emailAddress": "example#example.com"
},
"role": "REVIEWER"
},
{
"user": {
"name": "name2.n2",
"emailAddress": "example2#example.com"
},
"role": "REVIEWER"
}
]
}
'''
import groovy.json.JsonSlurper
def jsonslurper = new JsonSlurper()
def object = jsonslurper.parseText(jason)
List names = object.findAll { it.value instanceof List }
.values()
.flatten()
.collect { it.user.name }
println names
I have an object nested inside another object in Json file. I want to map this object with fields to a Model class.
{
"code": 200,
"time": "2019-09-05T07:09:44.228+0000",
"data": {
"statuses": [
{
"statusType": "IN_PROGRESS",
"statusTimestamp":"019-09-05T17:04:54+1000"
},
{
"statusType": "SENT",
"statusTimestamp":"2019-09-05T21:04:55+1000"
},
{
"statusType": "OPENED",
"statusTimestamp":"2019-09-05T23:04:55+1000"
},
{
"statusType": "INTERACTION_ID_RECEIVED",
"statusTimestamp":"2019-09-06T00:04:55+1000"
}
]
},
"status": 200,
"message": null,
"errors": null,
}
I want to map the statusType and TimeStamp to a custom model class.
Model Class:
public class Model{
private String statusType;
private DateTime statusTimestamp;
public Model(String statusType, String statusTimestamp) {
this.statusType=statusType;
this.statusTimestamp=new DateTime(statusTimestamp);
}
public String getStatusType() {
return statusType;
}
public void setStatusType(String statusType) {
this.statusType = statusType;
}
public DateTime getStatusTimestamp() {
return statusTimestamp;
}
public void setStatusTimestamp(String statusTimestamp) {
this.statusTimestamp = new DateTime(statusTimestamp);
}
}
I want to map the statuses to this model class and store these objects in a link something like this
List statuses = ParsedJson.read("$..['statuses'][*]", List.class)
If you don't want to model the entire response, you could use Jackson to parse the JSON into tree nodes and then map only the parts you care about:
ObjectMapper mapper = new ObjectMapper();
JsonNode rootNode = mapper.readTree(json);
JsonNode statusesNode = rootNode.path("data").path("statuses");
List<Status> statuses = mapper.convertValue(statusesNode,
new TypeReference<List<Status>>(){});
I saw 2 errors in the json string fixing it should help you to create Object using java or any other language easily.
line 12: " was missing
line 26: invalid placement of ,
Json after fixing issues
{
"code": 200,
"time": "2019-09-05T07:09:44.228+0000",
"data": {
"statuses": [
{
"statusType": "IN_PROGRESS",
"statusTimestamp":"019-09-05T17:04:54+1000"
},
{
"statusType": "SENT",
"statusTimestamp":"2019-09-05T21:04:55+1000"
},
{
"statusType": "OPENED",
"statusTimestamp":"2019-09-05T23:04:55+1000"
},
{
"statusType": "INTERACTION_ID_RECEIVED",
"statusTimestamp":"2019-09-06T00:04:55+1000"
}
]
},
"status": 200,
"message": null,
"errors": null
}
I have a JSON file like this:
{
"Resources": {
"HelloWorldFunction": {
"Type": "AWS::Serverless::Function",
"Properties": {
"Handler": "index.handler",
"Runtime": "nodejs8.10",
"Events": {
"HelloWorldApi": {
"Type": "Api",
"Properties": {
"Path": "/",
"Method": "GET"
}
}
},
"Policies": [
{
"SNSPublishMessagePolicy": {
"TopicName": {
"Fn::GetAtt": [
"HelloWorldTopic",
"TopicName"
]
}
}
}
],
"Environment": {
"Variables": {
"SNS_TOPIC_ARN": {
"Ref": "HelloWorldTopic"
}
}
},
"CodeUri": "nothing"
}
},
"HelloWorldTopic": {
"Type": "AWS::SNS::Topic",
"Properties": {
"Subscription": [
{
"Endpoint": "nothing",
"Protocol": "email"
}
]
}
}
}
}
I am using the Jackson YAMLFactory to parse a YAML-file that is equivalent to this JSON. How can I parse this in a way that all the content inside "Resources" is stored in a single String? (I want to keep this as a separate YAML/JSON for further analysis)
ObjectMapper mapper = new ObjectMapper();
String resources = mapper.readTree(new FileReader(path_to_your_json_file).at("/Resources").asText()
Or something like this.
I have an example Avro schema:
{
"type": "record",
"name": "wpmcn.MyPair",
"doc": "A pair of strings",
"fields": [
{"name": "left", "type": "string"},
{"name": "right", "type": "string"}
]
}
In Java, this would be a way to get all the field names:
public static void main(String[] args) throws IOException {
Schema schema =
new Schema.Parser().parse(AvroTest.class.getClassLoader().
getResourceAsStream("pair.avsc"));
//Collect all field values to an array list
List<String> fieldValues = new ArrayList<>();
for(Field field : schema.getFields()) {
fieldValues.add(field.name());
}
//Iterate the arraylist
for(String value: fieldValues) {
System.out.println(value);
}
}
How do I do the same using Scala?
import collection.JavaConverters._
avroSchema.getFields.asScala.map(_.name())) //forEach
val myAvroSchemaText= io.Source.fromInputStream(getClass.getResourceAsStream("pair.avsc")).mkString
val avroSchema = new Schema.Parser().parse(myAvroSchemaText)
avroSchema.getFields().foreach {
f =>
println(f.name)
}
I have a json such as below
{
"apiVersion": "v1",
"metadata": {
"status": {
"statusCode": 0
},
},
"stuff": [
{
"name": {
"text": "red"
},
"properties": [
{
"attributes": {
"shade": "dark"
},
"component": {
"id": "BA1",
}
"type": "Color"
}
]
},
{
"name": {
"text": "Toyota Camry"
},
"properties": [
{
"attributes": {},
"component": {
"id": "MS",
},
"type": "Vehicle"
}
]
},
]
}
I'm using GSON to parse the results like this:
Gson gson = new Gson();
JsonObject json = (JsonObject) gson.fromJson(in, JsonObject.class);
System.out.println(json.get("apiVersion").getAsString());
I can get the apiVersion but don't know how to get elements that are inside the json tree. For example, type...what if I want to output all the different type..in this case Color and Vehicle
I must be missing something here, but why can't you nest calls to getJsonObject? For example, to get the status code:
System.out.println(json.getAsJsonObject("metadata")
.getAsJsonObject("status")
.get("statusCode").getAsInt());
You can create an object in that matter and to parse the json to it (with GSON):
ParsedObject parsedObject = new Gson().fromJson(json, ParsedObject.class);
public class ParsedObject {
#SerializedName(value = "apiVersion")
private String mApiVersion;
#SerializedName(value = "metadata")
private Metadata mMetadata;
}