I'm wondering if we can perform Batch Write/Update with low-level API for DynamoDB for java.
Thanks in advance!
Yes. Something like this:
Map<String, List<WriteRequest>> writeRequestItems = new HashMap<String, List<WriteRequest>>();
Map<String, AttributeValue> userItem1 = new HashMap<String, AttributeValue>();
userItem1.put("userId", new AttributeValue().withS("1"));
userItem1.put("name", new AttributeValue().withS("Alex"));
Map<String, AttributeValue> userItem2 = new HashMap<String,AttributeValue>();
userItem2.put("userId", new AttributeValue().withS("2"));
userItem2.put("name", new AttributeValue().withS("Jonh"));
List<WriteRequest> userList = new ArrayList<WriteRequest>();
userList.add(new WriteRequest().withPutRequest(new PutRequest().withItem(userItem1)));
userList.add(new WriteRequest().withPutRequest(new PutRequest().withItem(userItem2)));
writeRequestItems.put("User", userList);
BatchWriteItemRequest batchWriteItemRequest = new BatchWriteItemRequest(writeRequestItems);
BatchWriteItemResult batchWriteItemResult = dynamoDBClient.batchWriteItem(batchWriteItemRequest);
Yes. You can use AmazonDynamoDB class to perform these operations.
Check http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/dynamodbv2/AmazonDynamoDB.html#batchWriteItem-com.amazonaws.services.dynamodbv2.model.BatchWriteItemRequest-
Related
I was trying to implemented stripe payment gateway. Every thing is going fine. I was able to create connected account for the user in my stripe dashboard, but the problem is I'm missing following parameter
Website
SSN
Industry
Now I want to know how to add these parameters while creating account.
I have add the screen shot from stripe dashboard and here is the code:
Map<String, Object> dob =
new HashMap<>();
dob.put("day", "12");
dob.put("month", "1");
dob.put("year", "1991");
Map<String, Object> address =
new HashMap<>();
Map<String, Object> address_pram =
new HashMap<>();
address_pram.put("city", "Baton Rouge");
address_pram.put("line1", "1 Calais Ave");
address_pram.put("postal_code", "70806");
address_pram.put("state", "Louisiana");
address.put("address", address_pram);
address.put("dob", dob);
address.put("email", "ahmad#example.com");
address.put("first_name", "ahmad");
address.put("last_name", "bajwa");
address.put("phone", "+12015551023");
//address.put("website", "www.goldenkeystone.com");
//address.put("industry", "");
// address.put("ssn", "000000000");
Map<String, Object> acceptance =
new HashMap<>();
acceptance.put("date", System.currentTimeMillis() / 1000L);
acceptance.put("ip", ipString);
Map<String, Object> cardPayments =
new HashMap<>();
cardPayments.put("requested", true);
Map<String, Object> transfers = new HashMap<>();
transfers.put("requested", true);
Map<String, Object> capabilities =
new HashMap<>();
capabilities.put("card_payments", cardPayments);
capabilities.put("transfers", transfers);
Map<String, Object> params = new HashMap<>();
params.put("type", "custom");
params.put("country", "US");
params.put("tos_acceptance", acceptance);
params.put("business_type", "individual");
params.put("individual", address);
params.put("capabilities", capabilities);
Account account = Account.create(params, requestOptions);
Note: If still question is unclear, I would be glad if you add your contribution.
That data and other sensitive information would be collected by Stripe during the onboarding via Account Links: stripe.com/docs/connect/connect-onboarding
It's not something that you can pass to the Accounts API. See here for more information: https://stripe.com/docs/connect/collect-then-transfer-guide?platform=web#create-an-account-link
I am trying to get the data from DynamoDB based on a few filters. (e.g.: get me a record where the productNumber and colorwayNumber is 'A', 'B'.) Following is the code snippet:
AmazonDynamoDB amazonDynamoDB = DynamoDBClient.getInstance().getConnection();
Map<String, HashMap<String, String>> attrVal = new HashMap<String, HashMap<String, String>>();
Map<String,String> expressionAttributesNames = new HashMap<>();
expressionAttributesNames.put("#colorwayNumber","colorwayNumber");
expressionAttributesNames.put("#productNumber","productNumber");
Map<String,AttributeValue> expressionAttributeValues = new HashMap<>();
expressionAttributeValues.put(":colorwayNumber",new AttributeValue().withS(colorwayID));
expressionAttributeValues.put(":productNumber",new AttributeValue().withS(productNumber));
QueryRequest queryRequest = new QueryRequest()
.withTableName(ConverseConstants.PLM_PRODUCT_TABLE)
.withKeyConditionExpression("#colorwayNumber = :colorwayNumber")
.withKeyConditionExpression("#productNumber = :productNumber")
.withExpressionAttributeNames(expressionAttributesNames)
.withExpressionAttributeValues(expressionAttributeValues);
QueryResult queryResult1 = amazonDynamoDB.query(queryRequest);
List<Map<String, AttributeValue>> results1 = queryResult1.getItems();
Following is the exception:
Caused by: com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: Value provided in ExpressionAttributeNames unused in expressions: keys: {#colorwayNumber} (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException;
Hope I provided enough information. Thanks!
The problem seems to be here:
.withKeyConditionExpression("#colorwayNumber = :colorwayNumber")
.withKeyConditionExpression("#productNumber = :productNumber")
Here you overwrite the first expression (#colorwayNumber = :colorwayNumber) with the second one (#productNumber = :productNumber). This way, colorwayNumber is actually unused in the expressions.
I have tried the below way and its working for me.
.withKeyConditionExpression("#colorwayNumber = :colorwayNumber and #productNumber = :productNumber")
or you can try below way which should work with FilterExpression and you can build it with DynamoDBScanExpression:
.withFilterExpression(("#colorwayNumber = :colorwayNumber and #productNumber = :productNumber")
The followings works, but how can I collect multiple MapSqlParameterSource and insert them all in one batch?
new SimpleJdbcInsert(ds).withTableName(TABLENAME);
MapSqlParameterSource entry = new MapSqlParameterSource()
.addValue("id", report.queryId, Types.INTEGER)
.addValue("firstname", report.reportDate, Types.DATE)
.addValue("age", report.completionRatio, Types.INTEGER);
insert.execute(entry);
Luckily SimpleJdbcInsert can take an array (not a list) of MapSqlParameterSource. So it's possible as follows:
List<MapSqlParameterSource> entries = new ArrayList<>();
entries.add(entry);
MapSqlParameterSource[] array = entries.toArray(new MapSqlParameterSource[entries.size()]);
insert.executeBatch(array);
There is a better way of doing it with SqlParameterSourceUtils
private final List<Map<String, Object>> records = new LinkedList<>();
final SimpleJdbcInsert statement = new SimpleJdbcInsert(dataSource)
.withTableName("stats")
.usingGeneratedKeyColumns("id")
.usingColumns("document", "error", "run", "celex");
statement.executeBatch(SqlParameterSourceUtils.createBatch(records));
I'm trying to use Spark (Java API) to take an in-memory Map (that potentially contains other nested Maps as its values) and convert it into a dataframe. I think I need something along these lines:
Map myMap = getSomehow();
RDD myRDD = sparkContext.makeRDD(myMap); // ???
DataFrame df = sparkContext.read(myRDD); // ???
But I'm having a tough time seeing the forest through the trees here...any ideas? Again this might be a Map<String,String> or a Map<String,Map>, where there could be several nested layers of maps-inside-of-maps-inside-of-maps, etc.
So I tried something, not sure if this is the most efficient option to do it, but I do not see any other right now.
SparkConf sf = new SparkConf().setAppName("name").setMaster("local[*]");
JavaSparkContext sc = new JavaSparkContext(sf);
SQLContext sqlCon = new SQLContext(sc);
Map map = new HashMap<String, Map<String, String>>();
map.put("test1", putMap);
HashMap putMap = new HashMap<String, String>();
putMap.put("1", "test");
List<Tuple2<String, HashMap>> list = new ArrayList<Tuple2<String, HashMap>>();
Set<String> allKeys = map.keySet();
for (String key : allKeys) {
list.add(new Tuple2<String, HashMap>(key, (HashMap) map.get(key)));
};
JavaRDD<Tuple2<String, HashMap>> rdd = sc.parallelize(list);
System.out.println(rdd.first());
List<StructField> fields = new ArrayList<>();
StructField field1 = DataTypes.createStructField("String", DataTypes.StringType, true);
StructField field2 = DataTypes.createStructField("Map",
DataTypes.createMapType(DataTypes.StringType, DataTypes.StringType), true);
fields.add(field1);
fields.add(field2);
StructType struct = DataTypes.createStructType(fields);
JavaRDD<Row> rowRDD = rdd.map(new Function<Tuple2<String, HashMap>, Row>() {
#Override
public Row call(Tuple2<String, HashMap> arg0) throws Exception {
return RowFactory.create(arg0._1, arg0._2);
}
});
DataFrame df = sqlCon.createDataFrame(rowRDD, struct);
df.show();
In this scenario I assumed that the Map in the Dataframe is of Type (String, String). Hope this helps!
Edit: Obviously you can delete all the prints. I did this for visualization purposes!
I have below list, i want to convert list to map.
SurveyAllocationUsers user1 = new SurveyAllocationUsers();
user1.setSurveyorId("1");
user1.setSurveyorTypeCode("LSR");
SurveyAllocationUsers user2 = new SurveyAllocationUsers();
user2.setSurveyorId("1");
user2.setSurveyorTypeCode("SR");
SurveyAllocationUsers user3 = new SurveyAllocationUsers();
user3.setSurveyorId("2");
user3.setSurveyorTypeCode("LSR");
SurveyAllocationUsers user4 = new SurveyAllocationUsers();
user4.setSurveyorId("2");
user4.setSurveyorTypeCode("SR");
SurveyAllocationUsers user5 = new SurveyAllocationUsers();
user5.setSurveyorId("2");
user5.setSurveyorTypeCode("BG");
List<SurveyAllocationUsers> list = new ArrayList<SurveyAllocationUsers>();
list.add(user1);list.add(user2);list.add(user3);list.add(user4);list.add(user5);
want to convert list to map like below.
Map<String,List<String>> usersMap = new HashMap<String, List<String>>();
map key will be SurveyorId and values will be correspondiing List of SurveyorTypeCode.
Thanks for the help in advance!!!
It should be something like below
userMap.put(user1.getSurveyorId(), new ArrayList<>().add(user1.getSurveyorTypeCode));
userMap.put(user2.getSurveyorId(), new ArrayList<>().add(user2.getSurveyorTypeCode));
It should be something like this:
List<SurveyAllocationUsers> list = new ArrayList<SurveyAllocationUsers>();
list.add(user1);list.add(user2);list.add(user3);list.add(user4);list.add(user5);
Map<String,List<String>> usersMap = new HashMap<String, List<String>>();
for(SurveyAllocationUsers sau: list){
if(!userMap.contains(sau.getSurveyorId())){
userMap.put(sau.getSurveyorId(), new ArrayList<>().add(sau.getSurveyorTypeCode));
}else{
userMap.get(sau.getSurveyorId()).add(sau.getSurveyorTypeCode);
}
}
Note: Code in not complied.
A little bit ugly but solves your problem:
Map<String, List<String>> map = list.stream().collect(toMap(SurveyAllocationUsers::getSurveyorId, p -> new ArrayList(singletonList(p.getSurveyorTypeCode())), (s, a) -> {
s.addAll(a);
return s;
}));
You can try this:
Map<String, List<String>> usersMap = new HashMap<String, List<String>>();
for (SurveyAllocationUsers user : list) {
List<String> typeCodes = new ArrayList<>();
typeCodes.add(user.getSurveyorTypeCode());
usersMap.put(user.getSurveyorId(), typeCodes);
}
Thank you everyone for your reply, I got the solution as below.
List <String>newUserList = null;
Map<String,List<String>> usersMap = new HashMap<String, List<String>>();
for(SurveyAllocationUsers sau: list){
if(usersMap.containsKey(sau.getSurveyorId())){
List <String>myList = usersMap.get(sau.getSurveyorId());
myList.add(sau.getSurveyorTypeCode());
usersMap.put(sau.getSurveyorId(), myList);
}else{
if(sau.getSurveyorId() != null){
newUserList = new ArrayList<String>();
newUserList.add(sau.getSurveyorTypeCode());
usersMap.put(sau.getSurveyorId(), newUserList);
}
}
}
Answer as expected
1 [LSR, SR]
2 [LSR, SR, BG]
It should be like this using Java8
resMap = list.stream().collect(Collectors.groupingBy(SurveyAllocationUsers::getSurveyorId,
Collectors.mapping(SurveyAllocationUsers::getSurveyorTypeCode,Collectors.toList())));
Output: {1=[LSR, SR], 2=[LSR, SR, BG]}