I need to put the values inside a pat as the values of my XML file for Ex:
Map<String, String> props = new HashMap<>();
props.put("role", "Admin");
props.put("externalId", "2ew1Q");
props.put("Property", "internal");
props.put("Execution", "internal");
My expected output should be:
<role>Admin</role>
<externalId>2ew1Q</externalId>
<Property>internal</Property>
<Execution>internal</Execution>
But instead of it, I'm getting
<entry string="role">Admin</entry>
<entry string="Execution">internal</entry>
<entry string="externalId">2ew1Q</entry>
<entry string="Property">internal</entry>
I have to do it with Simple XML, and this is my code:
#Root
public class Data {
#ElementMap(entry = "property", key = "key", attribute = true, inline = true)
private Map<String, String> customProps;
public Map<String, String> getData() {
return customProps;
}
public void setData(Map<String, String> data) {
this.customProps = data;
}
}
public static void main(String[] args) throws Exception {
Map<String, String> props = new HashMap<>();
props.put("role", "Admin");
props.put("externalId", "2ew1Q");
props.put("Property", "internal");
props.put("Execution", "internal");
Data customProps = new Data();
customProps.setData(props);
Serializer serializer = new Persister();
File result = new File("example.xml");
serializer.write(customProps, result);
}
Try:
#ElementMap(entry = "property", key = "key", attribute = false, inline = true)
Related
I'm trying to retrieve subscribers to a price object. Here's my approach, which works:
public void getSubscriptions() throws StripeException {
Stripe.apiKey = KEY;
Map<String, Object> params = new HashMap<>();
Iterable<Subscription> subscrips = Subscription.list(params).autoPagingIterable();
for(Subscription s : subscrips){
if((s.getItems().getData().get(0).getPlan().getId()).equals("price_ABC123")){
System.out.println(s);
}
}
//System.out.println(subscriptions);
}
The issue with this code is that its a bit slow. Is there a more efficient way to retrieve this info?
Thanks
Yes you can filter by price id as shown here https://site-admin.stripe.com/docs/api/subscriptions/list#list_subscriptions-price
public void getSubscriptions() throws StripeException {
Stripe.apiKey = KEY;
Map<String, Object> params = new HashMap<>();
params["price"] = "price_ABC123";
Iterable<Subscription> subscrips = Subscription.list(params).autoPagingIterable();
for(Subscription s : subscrips){
System.out.println(s);
}
}
I want one JSONObject response after passing URI through RESTTemplate
Test case is passing but the code coverage is still 0%
I have to return accountDetails object in JSON format
how do we Pass URI which takes account ID and given response entity in JSON Format this is what I have to figure out.
Test method:
void scheduleOnDemand() throws Exception {
AccountDTO accountDTO = new AccountDTO();
accountDTO.setId(1);
accountDTO.setTimeZone("Asia/Kolkata");
accountDTO.setPlatformType("AZURE");
accountDTO.setEnvironmentName("test");
accountDTO.setName("azureAccount");
accountDTO.setNextScheduleDate("2021-09-13");
accountDTO.setEnvironmentId(1);
HashMap<String, Object> accountDetails = new HashMap<>();
accountDetails.put("account_Id", "1");
accountDetails.put("TimeZone", "Asia/Kolkata");
accountDetails.put("AgentStatus", "Initiated");
accountDetails.put("Account_Platform", "AZURE");
accountDetails.put("Schedule_Time", "13:30:50.000");
accountDetails.put("Environment_Name", "test");
accountDetails.put("Account_Name", "azureAccount");
accountDetails.put("History_Id", "109");
accountDetails.put("Schedule_Date", "2021-09-13");
accountDetails.put("Environment_Id", "1");
Mockito.when(restTemplate.postForEntity("configure/accountDetails?Account_Id=1",null, JSONObject.class))
.thenReturn(new ResponseEntity((accountDetails), HttpStatus.OK));
}
Actual Method:
#Override
public JSONObject scheduleOnDemand(String accountId) throws Exception {
JSONObject object = null;
// PlatformHistoryDetails phistory = null;
HashMap<String, Object> accountDetails = new HashMap<>();
accountDetails = utilService.getAccountDetails(Integer.parseInt(accountId));
if (((String) accountDetails.get("scantype")).equalsIgnoreCase("infra")||((String) accountDetails.get("scantype")).equalsIgnoreCase("all")) {
URI postUri = UriComponentsBuilder.fromPath("/").pathSegment("api/scheduleOnDemand")
.queryParam("requestId", MDC.get("requestId")).queryParam("service", "scan")
.queryParam("Account_Id", accountId).build().toUri();
PlatformHistoryDetails phistory = modelMapper.map(apiClient.postOperation(postUri, Object.class),
PlatformHistoryDetails.class);
phistory.getHistory().setUser("admin");
object = utilService.processOneAccount(phistory);
} else {
throw new Exception("Account is not of type INFRA or ALL but of type " + accountDetails.get("scantype"));
}
return object;
}
accountDetails Implementation:
#Override
#SuppressWarnings({ "unchecked", "rawtypes" })
public HashMap<String, Object> getAccountDetails(int accountId) {
URI getUri = UriComponentsBuilder.fromPath("/").pathSegment("configure/accountDetails")
.queryParam("Account_Id", accountId).build().toUri();
HashMap<String, Object> account = (LinkedHashMap) apiClient.getAccountDetails(getUri, Object.class);
return account;
}
I have a List<Map<String, Object>> input like below:
[{
CURRENCY = USD,
STATUS = NEW,
PUBLISH_REGION = DEL,
SOURCE = ALADDIN,
RECON_STATUS = null,
JOB_ID_COUNT = 783
}, {
CURRENCY = USD,
STATUS = IN_PROGRESS,
PUBLISH_REGION = DEL,
SOURCE = ALADDIN,
RECON_STATUS = null,
JOB_ID_COUNT = 462
}, {
CURRENCY = USD,
STATUS = NEW,
PUBLISH_REGION = DEL,
SOURCE = GROUP,
RECON_STATUS = null,
JOB_ID_COUNT = 4
}]
I am trying to create another List<Map<String, Object>> by grouping on CURRENCY, PUBLISH_REGION, SOURCE and RECON_STATUS columns. And add all unique STATUS values as pivot to the output map and use JOB_ID_COUNT to summarize/aggregate the count.
List<String> groups = new ArrayList<>(asList("SOURCE", "RECON_STATUS", "PUBLISH_REGION", "CURRENCY"));
List<Map<String, Object>> = input.stream()
.collect(groupingBy(row -> row.get(groups.get(0)), mapping(map -> map.get(groups.get(0)), toList())));
I am expecting below response:
Output:
[{
CURRENCY = USD,
PUBLISH_REGION = DEL,
SOURCE = ALADDIN,
RECON_STATUS = null,
NEW = 783,
IN_PROGRESS = 462
}, {
CURRENCY = USD,
PUBLISH_REGION = DEL,
SOURCE = GROUP,
RECON_STATUS = null,
NEW = 4,
IN_PROGRESS = 0
}]
I am getting compile time error when trying to group by multiple map fields. Single field groupingBy is working fine. Any help is greatly appriciated.
Without Using Custom Class
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class MultipleFieldSorting2 {
private static Map<String, Object> map, map1, map2;
private static List<Map<String, Object>> lst = new ArrayList<>();
static {
map = new HashMap<>();
map.put("CURRENCY", "USD");
map.put("STATUS", "NEW");
map.put("PUBLISH_REGION", "DEL");
map.put("SOURCE", "ALADDIN");
map.put("RECON_STATUS", null);
map.put("JOB_ID_COUNT", "783");
map1 = new HashMap<>();
map1.put("CURRENCY", "USD");
map1.put("STATUS", "IN_PROGRESS");
map1.put("PUBLISH_REGION", "DEL");
map1.put("SOURCE", "ALADDIN");
map1.put("RECON_STATUS", null);
map1.put("JOB_ID_COUNT", "462");
map2 = new HashMap<>();
map2.put("CURRENCY", "USD");
map2.put("STATUS", "NEW");
map2.put("PUBLISH_REGION", "DEL");
map2.put("SOURCE", "GROUP");
map2.put("RECON_STATUS", null);
map2.put("JOB_ID_COUNT", "4");
lst.add(map);
lst.add(map1);
lst.add(map2);
}
public static Map<String, Object> mapper(Map<String, Object> e){
String key = e.get("CURRENCY") + "-" + e.get("PUBLISH_REGION") + "-" + e.get("SOURCE") + "-" + e.get("RECON_STATUS");
Map<String, Object> groupedValue = res.get(key);
if(groupedValue!=null){
groupedValue.put((String) e.get("STATUS"), groupedValue.get("STATUS")!=null ? groupedValue.get("STATUS")+","+e.get("JOB_ID_COUNT") : e.get("JOB_ID_COUNT"));
if(groupedValue.get("NEW")==null){
groupedValue.put("NEW", 0);
}
if(groupedValue.get("IN_PROGRESS")==null){
groupedValue.put("IN_PROGRESS", 0);
}
}else{
groupedValue = new HashMap<>();
res.put(key, groupedValue);
groupedValue.put("CURRENCY", e.get("CURRENCY"));
groupedValue.put("PUBLISH_REGION", e.get("PUBLISH_REGION"));
groupedValue.put("SOURCE", e.get("SOURCE"));
groupedValue.put("RECON_STATUS", e.get("RECON_STATUS"));
groupedValue.put((String) e.get("STATUS"), e.get("JOB_ID_COUNT"));
}
return groupedValue;
}
static Map<String, Map<String, Object>> res = new HashMap<>();
public static void main(String[] args) {
List<Map<String, Object>> finalResult = new ArrayList<>();
lst.stream()
.map(MultipleFieldSorting2::mapper)
.forEach(result -> {
if(!finalResult.contains(result))
finalResult.add(result);
});
System.out.println(finalResult);
}
}
Tried this solution and it is working
Stream the source List
Map each value of map in the list to Class MapWrapper(a pojo where each key is a field)
GroupBy using the groupByKey defined in MapWrapper(uses CURRENCY, PUBLISH_REGION, SOURCE and RECON_STATUS columns)
3.a The result is a Map<String, List<MapWrapper>>
4.Stream through the entry set
map - and get the value alone from (Map<String, List<MapWrapper>>)
Map - convert from List<MapWrapper> to Map<String, Object> using MapWrapper::map
Collect to a list
In Short the solution is
List<Map<String, Object>> value = lst.stream()
.map(map -> new MapWrapper(map))
.collect(groupingBy(MapWrapper::groupByKey))
.entrySet()
.stream()
.map(e -> e.getValue())
.map(MapWrapper::map).collect(toList());
Working Code
public class MultipleFieldSorting {
private static Map<String, Object> map, map1, map2;
private static List<Map<String, Object>> lst = new ArrayList<>();
static {
map = new HashMap<>();
map.put("CURRENCY", "USD");
map.put("STATUS", "NEW");
map.put("PUBLISH_REGION", "DEL");
map.put("SOURCE", "ALADDIN");
map.put("RECON_STATUS", null);
map.put("JOB_ID_COUNT", "783");
map1 = new HashMap<>();
map1.put("CURRENCY", "USD");
map1.put("STATUS", "IN_PROGRESS");
map1.put("PUBLISH_REGION", "DEL");
map1.put("SOURCE", "ALADDIN");
map1.put("RECON_STATUS", null);
map1.put("JOB_ID_COUNT", "462");
map2 = new HashMap<>();
map2.put("CURRENCY", "USD");
map2.put("STATUS", "NEW");
map2.put("PUBLISH_REGION", "DEL");
map2.put("SOURCE", "GROUP");
map2.put("RECON_STATUS", null);
map2.put("JOB_ID_COUNT", "4");
lst.add(map);
lst.add(map1);
lst.add(map2);
}
public static void main(String[] args) {
List<Map<String, Object>> value = lst.stream()
.map(map -> new MapWrapper(map))
.collect(groupingBy(MapWrapper::groupByKey))
.entrySet()
.stream()
.map(e -> e.getValue())
.map(MapWrapper::map).collect(toList());
System.out.println(value);
}
}
class MapWrapper {
private String currency;
private String status;
private String publish;
private String source;
private String recon_status;
private String job_id;
public MapWrapper(Map<String, Object> map) {
this.currency = (String) map.get("CURRENCY");
this.status = (String) map.get("STATUS");
this.publish = (String) map.get("PUBLISH_REGION");
this.source = (String) map.get("SOURCE");
this.recon_status = (String) map.get("RECON_STATUS");
this.job_id = (String) map.get("JOB_ID_COUNT");
}
String groupByKey() {
return new StringBuilder().append(this.getCurrency()).append("-").append(this.publish).append("-")
.append(this.source).append("-").append(this.recon_status).toString();
}
public static Map<String, Object> map(List<MapWrapper> lst){
Map<String, Object> res = new HashMap<>();
res.put("CURRENCY",lst.get(0).getCurrency());
res.put("PUBLISH_REGION",lst.get(0).getPublish());
res.put("SOURCE",lst.get(0).getSource());
res.put("RECON_STATUS",lst.get(0).getRecon_status());
for(MapWrapper m : lst){
res.put(m.getStatus(), m.getJob_id());
}
if(res.get("NEW")==null){
res.put("NEW", 0);
}
if(res.get("IN_PROGRESS")==null){
res.put("IN_PROGRESS", 0);
}
return res;
}
String getCurrency() {
return currency;
}
void setCurrency(String currency) {
this.currency = currency;
}
String getStatus() {
return status;
}
void setStatus(String status) {
this.status = status;
}
String getPublish() {
return publish;
}
void setPublish(String publish) {
this.publish = publish;
}
String getSource() {
return source;
}
void setSource(String source) {
this.source = source;
}
String getJob_id() {
return job_id;
}
void setJob_id(String job_id) {
this.job_id = job_id;
}
String getRecon_status() {
return recon_status;
}
void setRecon_status(String recon_status) {
this.recon_status = recon_status;
}
}
First of all let me describe the scenario.
Step 1. I have to read from a file, line by line. The file is a .json and each line has the following format:
{
"schema":{Several keys that are to be deleted},
"payload":{"key1":20001,"key2":"aaaa","key3":"bbbb","key4":"USD","key5":"100"}
}
Step 2. Delete schema object and end up with (added more examples for the sake of the next steps):
{"key1":20001,"key2":"aaaa","key3":"bbbb","key4":"USD","key5":"100"}
{"key1":20001,"key2":"aaaa","key3":"bbbb","key4":"US","key5":"90"}
{"key1":2002,"key2":"cccc","key3":"hhhh","key4":"CN","key5":"80"}
Step 3. Split these values into key and value by making them json in memory and use the strings as keys and values with map
{"key1":20001,"key2":"aaaa","key3":"bbbb"} = {"key4":"USD","key5":"100"}
{"key1":20001,"key2":"aaaa","key3":"bbbb"} = {"key4":"US","key5":"90"}
{"key1":2002,"key2":"cccc","key3":"hhhh"} = {"key4":"CN","key5":"80"}
Step 4, and the one I can't work out due to my lack of knowledge in Pcollections. I need to grab all the lines read and do a GroupByKey so that it would end up like:
{"key1":20001,"key2":"aaaa","key3":"bbbb"} = [
{"key4":"USD","key5":"100"},
{"key4":"US","key5":"90"} ]
{"key1":2002,"key2":"cccc","key3":"hhhh"} = {"key4":"CN","key5":"80"}
Righ now my code looks like this:
static void runSimplePipeline(PipelineOptionsCustom options) {
Pipeline p = Pipeline.create(options);
p.apply("ReadLines", TextIO.read().from(options.getInputFile()))
.apply("TransformData", ParDo.of(new DoFn<String, String>() {
#ProcessElement
public void processElement(ProcessContext c) {
Gson gson = new GsonBuilder().create();
ObjectMapper oMapper = new ObjectMapper();
JSONObject obj_key = new JSONObject();
JSONObject obj_value = new JSONObject();
List<String> listMainKeys = Arrays.asList(new String[]{"Key1", "Key2", "Key3"});
HashMap<String, Object> parsedMap = gson.fromJson(c.element().toString(), HashMap.class);
parsedMap.remove("schema");
Map<String, String> map = oMapper.convertValue(parsedMap.get("payload"), Map.class);
for (Map.Entry<String,String> entry : map.entrySet()) {
if (listMainKeys.contains(entry.getKey())) {
obj_key.put(entry.getKey(),entry.getValue());
} else {
obj_value.put(entry.getKey(),entry.getValue());
}
}
KV objectKV = KV.of(obj_key.toJSONString(), obj_value.toJSONString());
System.out.print(obj_key.toString() + " : " + obj_value.toString() +"\n");
}
})); <------- RIGHT HERE
p.run().waitUntilFinish();
}
Now the obvious part is that on where it says "RIGHT HERE" I should have another apply with CountByKey however that requires a full PCollection and that's what I do not really understand.
Here's the code, thanks to Guillem Xercavins's linked Github:
static void runSimplePipeline(PipelineOptionsCustom options) {
Pipeline p = Pipeline.create(options);
PCollection<Void> results = p.apply("ReadLines", TextIO.read().from(options.getInputFile()))
.apply("TransformData", ParDo.of(new DoFn<String, KV<String, String>>() {
#ProcessElement
public void processElement(ProcessContext c) {
Gson gson = new GsonBuilder().create();
ObjectMapper oMapper = new ObjectMapper();
JSONObject obj_key = new JSONObject();
JSONObject obj_value = new JSONObject();
List<String> listMainKeys = Arrays
.asList(new String[] { "EBELN", "AEDAT", "BATXT", "EKOTX", "Land1", "WAERS" });
HashMap<String, Object> parsedMap = gson.fromJson(c.element().toString(), HashMap.class);
parsedMap.remove("schema");
Map<String, String> map = oMapper.convertValue(parsedMap.get("payload"), Map.class);
for (Map.Entry<String, String> entry : map.entrySet()) {
if (listMainKeys.contains(entry.getKey())) {
obj_key.put(entry.getKey(), entry.getValue());
} else {
obj_value.put(entry.getKey(), entry.getValue());
}
}
KV objectKV = KV.of(obj_key.toJSONString(), obj_value.toJSONString());
c.output(objectKV);
}
})).apply("Group By Key", GroupByKey.<String, String>create())
.apply("Continue Processing", ParDo.of(new DoFn<KV<String, Iterable<String>>, Void>() {
#ProcessElement
public void processElement(ProcessContext c) {
System.out.print(c.element());
}
}));
p.run().waitUntilFinish();
}
I am trying to insert a row to BigQuery using java.
The entity I was inserting has a field which is double nested.
Generating the entity suitable for BigQuery:
ObjectMapper mapper = new ObjectMapper();
String barcodeDetailsJSON = order.getBarcodeDetailsJSON();
List<StateForBQ> stateForBQList = new ArrayList<StateForBQ>();
for (State state : order.getStates()) {
StateForBQ stateForBQ = new StateForBQ(state);
stateForBQ.setSetOn(new Date(stateForBQ.getSetOn().getTime()/1000));
stateForBQList.add(stateForBQ);
}
List<BarcodeDetailForBQ> barcodeDetailForBQList = getBarcodeDetailsFromBarcodeDetailsJSON(barcodeDetailsJSON, order.getIsGrouped());
Without the following, state is getting set as null. (State is nested entity)
List<Map<String, Object>> stateMap =
mapper.convertValue(stateForBQList, new TypeReference<List<Map<String, Object>>>() {});
Without the following, barcodeDetails is getting set as null. (BarcodeDetails is double nested entity)
List<Map<String, Object>> barcodeMapList =
mapper.convertValue(barcodeDetailForBQList, new TypeReference<List<Map<String, Object>>>() {});
Without the follwing, productPriceDetails, productDetails, cgst, sgst are getting set as null
for (Map<String, Object> barcodeMap : barcodeMapList) {
barcodeMap.put("productPriceDetails", mapper.convertValue(barcodeMap.get("productPriceDetails"), new TypeReference<Map<String, Object>>() {}));
barcodeMap.put("productDetails", mapper.convertValue(barcodeMap.get("productDetails"), new TypeReference<Map<String, Object>>() {}));
barcodeMap.put("cgst", mapper.convertValue(barcodeMap.get("cgst"), new TypeReference<Map<String, Object>>() {}));
barcodeMap.put("sgst", mapper.convertValue(barcodeMap.get("sgst"), new TypeReference<Map<String, Object>>() {}));
}
Preparing the rowcontent
Map<String, Object> rowContent = new HashMap<>();
rowContent.put("orderId", order.getOrderId());
rowContent.put("customerId", order.getCustomerId());
rowContent.put("barcodeDetails", barcodeMapList);
rowContent.put("states", stateMap);
Inserting to BigQuery
Gson gson = new Gson();
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
String datasetName = "Latest_Data";
String tableName= "ORDER_TEMP";
// [START bigquery_table_insert_rows]
TableId tableId = TableId.of(datasetName, tableName);
// Values of the row to insert
String barcodeDetailsJSON = order.getBarcodeDetailsJSON();
List<BarcodeDetailForBQ> barcodeDetailForBQList = new OrderForBQ().getBarcodeDetailsFromBarcodeDetailsJSON(barcodeDetailsJSON, order.getIsGrouped());
String recordsContentString = gson.toJson(rowContent);
InsertAllResponse response =
bigquery.insertAll(
InsertAllRequest.newBuilder(tableId)
.addRow(""+orderId, rowContent)
// More rows can be added in the same RPC by invoking .addRow() on the builder
.build());
if (response.hasErrors()) {
// If any of the insertions failed, this lets you inspect the errors
for (Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
// inspect row error
}
}
Following is the response I am getting.
{
insertErrors: {
0: [
{
reason: "invalid",
location: "",
message: "Repeated record added outside of an array."
}
]
}
}