#Override
public Long createPost(Request request) {
Base.open();
ObjectMapper mapper = new ObjectMapper();
try {
Post newPost = mapper.readValue(request.body(), Post.class);
// Map values = ... initialize map
// newPost.saveIt();
} catch (IOException ex) {
Logger.getLogger(PostServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
}
Base.close();
return 1L;
}
From the official docs, this Map values = ... initialize map is not clear. I can do newPost.set("first_name", "Sam") but is there a better way instead of setting values likes this?
I'm not familiar with Spark (I'm the author of ActiveWeb ), but you can use filters to open/close connections instead of polluting your service classes:
http://sparkjava.com/documentation.html#filters
Additionally, if you can convert your request parameters into a java.util.Map, you then do this:
Post post = new Post();
post.fromMap(parameters);
if(post.save()){
//display success
}else{
Errors errors = post.errors();
//display errors
}
This is an example from ActiveWeb, but will help you with Spark too:
https://github.com/javalite/activeweb-simple/blob/master/src/main/java/app/controllers/BooksController.java
If I'm understanding correctly, you are looking at how to take values from a POST request and then use ActiveJDBC to save those values. I'm quite new as well and we are in the beginning stages of our app, but we are using SparkJava with ActiveJDBC.
The example is actual code, I didn't have time to simplify it. But basically we created a POJO for the model class. We originally extended the org.javalite.activejdbc.Model but we needed to handle audit fields (create, update user/time) and help translate from JSON, so we extended this with a custom class called CecilModel. But CecilModel extends the Model class.
We have a controller that receives the request. The request comes in as JSON that matches the field names of our model class. In our custom CecilModel class we map the JSON to a Map which then we use Model.fromMap method to hyrdrate the fields and puts it into our custom model POJO. We don't need the getters or setters, it's more for convenience. We just need our JSON request to have the same names as in our model.
Below is our code but maybe you can peek through it to see how we are doing it.
Our table model pojo.
package com.brookdale.model;
import java.sql.Timestamp;
import org.javalite.activejdbc.Model;
import org.javalite.activejdbc.annotations.BelongsTo;
import org.javalite.activejdbc.annotations.BelongsToParents;
import org.javalite.activejdbc.annotations.IdGenerator;
import org.javalite.activejdbc.annotations.IdName;
import org.javalite.activejdbc.annotations.Table;
import com.brookdale.model.activejdbc.CecilModel;
// This class handles mapping of data from the database to objects
// and back, including custom selection queries.
#Table("RECURRINGITEMSCHEDULE")
#BelongsToParents({
#BelongsTo(foreignKeyName="itemID",parent=Item.class),
#BelongsTo(foreignKeyName="contactAgreementID",parent=ContactAgreement.class),
#BelongsTo(foreignKeyName="unitOfMeasureCode",parent=UnitOfMeasure.class)
})
#IdGenerator("SQ_RECURRINGITEMSCHEDULE.nextVal")
#IdName("recurringItemScheduleID")
public class RecurringItem extends CecilModel {
public Long getRecurringItemScheduleID() {
return getLong("recurringItemScheduleID");
}
public void setRecurringItemScheduleID(Long recurringItemScheduleID) {
set("recurringItemScheduleID",recurringItemScheduleID);
}
public Long getContactAgreementID() {
return getLong("contactAgreementID");
}
public void setContactAgreementID(Long contactAgreementID) {
set("contactAgreementID",contactAgreementID);
}
public Long getItemID() {
return getLong("itemID");
}
public void setItemID(Long itemID) {
set("itemID",itemID);
}
public Double getUnitChargeAmt() {
return getDouble("unitChargeAmt");
}
public void setUnitChargeAmt(Double unitChargeAmt) {
set("unitChargeAmt",unitChargeAmt);
}
public Integer getUnitQty() {
return getInteger("unitQty");
}
public void setUnitQty(Integer unitQty) {
set("unitQty",unitQty);
}
public String getUnitOfMeasureCode() {
return getString("unitOfMeasureCode");
}
public void setUnitOfMeasureCode(String unitOfMeasureCode) {
set("unitOfMeasureCode",unitOfMeasureCode);
}
public Timestamp getLastGeneratedPeriodEndDate() {
return getTimestamp("lastGeneratedPeriodEndDate");
}
public void setLastGeneratedPeriodEndDate(Timestamp lastGeneratedPeriodEndDate) {
set("lastGeneratedPeriodEndDate",lastGeneratedPeriodEndDate);
}
public Timestamp getEffEndDate() {
return getTimestamp("effEndDate");
}
public void setEffEndDate(Timestamp effEndDate) {
set("effEndDate",effEndDate);
}
public Timestamp getEffStartDate() {
return getTimestamp("effStartDate");
}
public void setEffStartDate(Timestamp effStartDate) {
set("effStartDate",effStartDate);
}
#Override
public void validate() {
validatePresenceOf("unitofmeasurecode","itemid","unitqty","effstartdate","unitChargeAmt","contactAgreementID");
validateNumericalityOf("itemid","unitQty","contactAgreementID");
// check to make sure this is an update operation
if (!this.isNew()) {
RecurringItem ridb = RecurringItem.findById(this.getId());
if (ridb.getLastGeneratedPeriodEndDate() != null) {
if (this.getItemID() != ridb.getItemID())
this.addError("itemid", "Item can not be updated once a charge has been created.");
if (!this.getEffStartDate().equals(ridb.getEffStartDate()))
this.addError("effstartdate", "Effective start date can not be updated once a charge has been created.");
if (this.getUnitChargeAmt() != ridb.getUnitChargeAmt())
this.addError("unitchargeamt", "Unit charge amount can not be updated after last generated period end date has been set.");
if (this.getUnitQty() != ridb.getUnitQty())
this.addError("unitqty", "Unit quantity can not be updated after last generated period end date has been set.");
if (!this.getUnitOfMeasureCode().equals(ridb.getUnitOfMeasureCode()))
this.addError("", "Unit of measure can not be updated after last generated period end date has been set.");
}
}
if (this.getEffEndDate() != null && this.getEffStartDate().compareTo(this.getEffEndDate()) >= 0) {
this.addError("effenddate", "Effective end date can not come before the start date.");
}
}
}
Extends our custom Model class. This will extend the actual ActiveJDBC Model class.
package com.brookdale.model.activejdbc;
import java.io.IOException;
import java.sql.Timestamp;
import java.text.SimpleDateFormat;
import java.time.LocalDateTime;
import java.time.ZonedDateTime;
import java.util.Collection;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import org.codehaus.jackson.map.ObjectMapper;
import org.codehaus.jackson.type.TypeReference;
import org.javalite.activejdbc.Model;
import org.javalite.activejdbc.validation.ValidationBuilder;
import org.javalite.activejdbc.validation.ValidatorAdapter;
import com.brookdale.core.CLArgs;
import com.brookdale.security.bo.User;
public abstract class CecilModel extends Model {
private static final transient TypeReference<HashMap<String, Object>> mapType = new TypeReference<HashMap<String, Object>>() {};
private static final transient TypeReference<LinkedList<HashMap<String, Object>>> listMapType = new TypeReference<LinkedList<HashMap<String, Object>>>() {};
private static final transient SimpleDateFormat jsonDateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'");
public Timestamp getUpdateDateTime() {
return getTimestamp("updateDateTime");
}
public void setUpdateDateTime(LocalDateTime updateDateTime) {
set("updateDateTime",updateDateTime == null ? null : Timestamp.valueOf(updateDateTime));
}
public Timestamp getCreateDateTime() {
return getTimestamp("createDateTime");
}
public void setCreateDateTime(LocalDateTime createDateTime) {
set("createDateTime",createDateTime == null ? null : Timestamp.valueOf(createDateTime));
}
public String getUpdateUsername() {
return getString("updateUsername");
}
public void setUpdateUsername(String updateUsername) {
set("updateUsername",updateUsername);
}
public String getCreateUsername() {
return getString("createUsername");
}
public void setCreateUsername(String createUsername) {
set("createUsername",createUsername);
}
public Long getUpdateTimeId() {
return getLong("updateTimeId");
}
public void setUpdateTimeId(Long updateTimeId) {
set("updateTimeId",updateTimeId);
}
public boolean save(User user) {
String userId = (CLArgs.args.isAuthenabled()) ? user.getUserid() : "TEST_MODE";
// insert
java.sql.Timestamp now = java.sql.Timestamp.valueOf(java.time.LocalDateTime.now());
if (this.getId() == null || this.getId().toString().equals("0")) {
this.setId(null);
this.set("createDateTime", now);
this.set("createUsername", userId);
this.set("updateDateTime", now);
this.set("updateUsername", userId);
this.set("updateTimeId", 1);
}
// update
else {
Long updatetimeid = this.getLong("updateTimeid");
this.set("updateDateTime", now);
this.set("updateUsername", userId);
this.set("updateTimeId", updatetimeid == null ? 1 : updatetimeid + 1);
}
return super.save();
}
public boolean saveIt(User user) {
String userId = (CLArgs.args.isAuthenabled()) ? user.getUserid() : "TEST_MODE";
// insert
java.sql.Timestamp now = java.sql.Timestamp.valueOf(java.time.LocalDateTime.now());
if (this.isNew()) {
this.setId(null);
this.set("createDateTime", now);
this.set("createUsername", userId);
this.set("updateDateTime", now);
this.set("updateUsername", userId);
this.set("updateTimeId", 1);
}
// update
else {
Long updatetimeid = this.getLong("updateTimeid");
this.set("updateDateTime", now);
this.set("updateUsername", userId);
this.set("updateTimeId", updatetimeid == null ? 1 : updatetimeid + 1);
}
return super.saveIt();
}
public boolean saveModel(User user, boolean insert) {
if(insert){
this.insertIt(user);
}else{
this.updateIt(user);
}
return super.saveIt();
}
public boolean insertIt(User user) {
// insert
java.sql.Timestamp now = java.sql.Timestamp.valueOf(java.time.LocalDateTime.now());
this.setId(null);
this.set("createDateTime", now);
this.set("createUsername", user.getUserid());
this.set("updateDateTime", now);
this.set("updateUsername", user.getUserid());
this.set("updateTimeId", 1);
return super.saveIt();
}
public boolean updateIt(User user) {
// insert
java.sql.Timestamp now = java.sql.Timestamp.valueOf(java.time.LocalDateTime.now());
Long updatetimeid = this.getLong("updateTimeid");
this.set("updateDateTime", now);
this.set("updateUsername", user.getUserid());
this.set("updateTimeId", updatetimeid == null ? 1 : updatetimeid + 1);
return super.saveIt();
}
// Convert a single ActiveJdbc Object to a json string
#SuppressWarnings("unchecked")
public String toJson() {
Map<String, Object> objMap = this.toMap();
try {
if (objMap.containsKey("parents")) {
Map<String, ?> parentsMap = (Map<String, ?>) objMap.get("parents");
for (String key: parentsMap.keySet()) {
objMap.put(key, parentsMap.get(key));
}
objMap.remove("parents");
}
if (objMap.containsKey("children")) {
Map<String, ?> childrenMap = (Map<String, ?>) objMap.get("children");
for (String key: childrenMap.keySet()) {
objMap.put(key, childrenMap.get(key));
}
objMap.remove("children");
}
ObjectMapper mapper = new ObjectMapper();
mapper.setDateFormat(jsonDateFormat);
return mapper.writerWithDefaultPrettyPrinter().writeValueAsString(objMap);
} catch (Exception e) { throw new RuntimeException(e); }
}
// Converts a single json object into an ActiveJdbc Model
public <T extends CecilModel> T toObj(String json) {
try {
Map<String, Object> objMap = toMap(json);
convertDatesToTimestamp(objMap);
return this.fromMap(objMap);
} catch (Exception e) { throw new RuntimeException(e); }
}
// STATIC CONVERTERS FOR COLLECTIONS OF OBJECTS
// Convert an ActiveJdbc LazyList Collection to a JSON string
public static <T extends Model> String toJson(Collection<T> objCollection) {
//objCollection.load();
StringBuffer json = new StringBuffer("[ ");
for (T obj: objCollection) {
if (CecilModel.class.isInstance(obj)) {
json.append(((CecilModel)obj).toJson() + ",");
} else {
try {
json.append(new ObjectMapper().writeValueAsString(obj));
} catch (Exception e) {
e.printStackTrace();
}
}
}
return json.substring(0, json.length()-1) + "]";
}
// Converts an array of json objects into a list of ActiveJdbc Models
public static <T extends Model> List<T> toObjList(String json, Class<T> cls) {
List<T> results = new LinkedList<T>();
try {
List<Map<String, Object>> objMapList = toMaps(json);
for (Map<String, Object> objMap: objMapList) {
convertDatesToTimestamp(objMap);
T obj = cls.newInstance().fromMap(objMap);
results.add(obj);
}
} catch (Exception e) { throw new RuntimeException(e); }
return results;
}
// Converts a single json object into a map of key:value pairs
private static Map<String, Object> toMap(String json) {
ObjectMapper mapper = new ObjectMapper();
try {
return mapper.readValue(json, mapType);
} catch (IOException e) { throw new RuntimeException(e); }
}
// Converts an array of json objects into a list of maps
private static List<Map<String, Object>> toMaps(String json) {
ObjectMapper mapper = new ObjectMapper();
try {
return mapper.readValue(json, listMapType);
} catch (IOException e) { throw new RuntimeException(e); }
}
// checks for Strings that are formatted in 'yyyy-MM-ddTHH:mm:ss.SSSZ' format
// if found it will replace the String value with a java.sql.Timestamp value in the objMap
private static final Map<String,Object> convertDatesToTimestamp(Map<String, Object> objMap) {
// attempt to convert all dates to java.sql.Timestamp
for (String key: objMap.keySet()) {
Object value = objMap.get(key);
System.out.println("Checking if '" + key + "=" + (value == null ? "null" : value.toString()) +"' is a date...");
if (value instanceof String && ((String) value).matches("^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}\\.\\d{3}Z$")) {
String valuestr = (String) value;
System.out.println("DATE FOUND FOR '" + key + "' " + value);
objMap.put(key, Timestamp.valueOf(ZonedDateTime.parse(valuestr).toLocalDateTime()));
}
}
return objMap;
}
public static ValidationBuilder<?> validateAbsenceOf(String ... attributes) {
return validateWith(new ValidatorAdapter() {
#Override
public void validate(Model m) {
boolean containsAttribute = false;
for(String attribute:attributes) {
if(m.attributeNames().contains(attribute)) {
//model contains attribute, invalidate now !
m.addValidator(this, attribute);
break;
}
}
}
});
}
}
Our Controller
package com.brookdale.controller;
import static spark.Spark.get;
import static spark.Spark.post;
import static spark.Spark.put;
import org.codehaus.jackson.map.ObjectMapper;
import com.brookdale.model.RecurringItem;
import com.brookdale.model.activejdbc.CecilModel;
import com.brookdale.security.bo.User;
import com.brookdale.service.RecurringItemService;
public class RecurringItemController {
public RecurringItemController(final RecurringItemService service) {
// get list of recurring items based on the agreementid
get("/api/recurring/list/:agreementid", (req,res)-> {
String all = req.queryParams("all");
Long agreementid = Long.parseLong(req.params(":agreementid"));
//return RecurringItemAPI.searchByAgreement(Long.parseLong(req.params(":agreementid"))).toJson(true);
return CecilModel.toJson(service.findByAgreementId(agreementid, all != null));
});
// insert
post("/api/recurring/save", (req,res)-> {
//RecurringItem ri = ActiveJdbcJson.toObj(req.body(), RecurringItem.class);
RecurringItem ri = new RecurringItem().toObj(req.body());
// set unitqty to '1' since the field is not nullable, but not used
if (ri.getUnitQty() == null)
ri.setUnitQty(1);
System.out.println("ri to insert: " + new ObjectMapper().writeValueAsString(ri));
return service.saveRecurringItem(ri, (User) req.attribute("user")).toJson();
});
// update
put("/api/recurring/save", (req,res)-> {
//RecurringItem ri = ActiveJdbcJson.toObj(req.body(), RecurringItem.class);
RecurringItem ri = new RecurringItem().toObj(req.body());
System.out.println("ri to update: " + new ObjectMapper().writeValueAsString(ri));
return service.saveRecurringItem(ri, (User) req.attribute("user")).toJson();
});
}
}
Which calls our service layer to do the save.
package com.brookdale.service;
import org.javalite.activejdbc.LazyList;
import org.javalite.activejdbc.Model;
import com.brookdale.model.RecurringItem;
import com.brookdale.security.bo.User;
public class RecurringItemService {
public RecurringItemService() { }
public LazyList<Model> findByAgreementId(Long agreementId, boolean includeAll) {
if (includeAll)
return RecurringItem.find("contactAgreementID = ?", agreementId).orderBy("effstartdate desc");
else
return RecurringItem.find("contactAgreementID = ? and effenddate is null", agreementId).orderBy("effstartdate desc");
}
public RecurringItem saveRecurringItem(RecurringItem ri, User user) {
ri.saveIt(user);
return ri;
}
}
Related
I need to create a custom UDF in hive to convert array<map<string, string>> into array<struct<key:string, value:string>>
I am trying with the following class:
import java.util.List;
import java.util.Map;
import com.google.common.collect.Lists;
import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
import org.apache.hadoop.hive.ql.exec.UDFArgumentLengthException;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.udf.generic.GenericUDF;
import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
import org.apache.hadoop.hive.serde2.objectinspector.MapObjectInspector;
import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory;
import org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector;
import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
import org.apache.hadoop.io.Text;
public class ArrayOfMapToArrayOfStructUdf extends GenericUDF {
private static final String UDF_NAME = "convertArrayMapToArrayStruct";
#Override
public String getUdfName() {
return UDF_NAME;
}
#Override
public ObjectInspector initialize(ObjectInspector[] objectInspectors) throws UDFArgumentException {
if (objectInspectors.length != 1) {
throw new UDFArgumentLengthException(UDF_NAME + " takes 1 argument of type array<map<key, value>>");
}
if (!(validateArgumentType(objectInspectors))) {
throw new IllegalArgumentException("Code should never reach this section!");
}
return createReturnObjectInspector();
}
private boolean validateArgumentType(ObjectInspector[] objectInspectors) throws UDFArgumentException {
if (!(objectInspectors[0] instanceof ListObjectInspector)) {
throw new UDFArgumentException("the argument must be of type: array<map<key, value>>");
}
ListObjectInspector listObjectInspector = (ListObjectInspector) objectInspectors[0];
if (!(listObjectInspector.getListElementObjectInspector() instanceof MapObjectInspector)) {
throw new UDFArgumentException("the array contents must be of type: map<key, value>");
}
return true;
}
private ObjectInspector createReturnObjectInspector() {
List<String> structFieldNames = Lists.newArrayList("key", "value");
List<ObjectInspector> structFieldObjectInspectors =
Lists.newArrayList(PrimitiveObjectInspectorFactory.javaStringObjectInspector,
PrimitiveObjectInspectorFactory.javaStringObjectInspector);
StructObjectInspector structObjectInspector =
ObjectInspectorFactory.getStandardStructObjectInspector(structFieldNames, structFieldObjectInspectors);
return ObjectInspectorFactory.getStandardListObjectInspector(structObjectInspector);
}
#Override
public Object evaluate(DeferredObject[] deferredObjects) throws HiveException {
if (deferredObjects == null || deferredObjects.length < 1) {
return null;
}
List<Map<String, String>> arrayOfMap = (List<Map<String, String>>) deferredObjects[0].get();
if (arrayOfMap == null) {
return null;
}
List<Object> arrayOfStruct = Lists.newArrayList();
for (Map<String, String> map : arrayOfMap) {
Object[] object = new Object[2];
object[0] = new Text(map.get("key"));
object[1] = new Text(map.get("value"));
arrayOfStruct.add(object);
}
return arrayOfStruct;
}
#Override
public String getDisplayString(String[] strings) {
return UDF_NAME;
}
}
And I'm getting the following error:
Failed with exception java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating convertArrayMapToArrayStruct
I dont know how to build the object to return in the evaluate method.
The column im trying to transform has data as follows:
[{"key": "key1", "value": "value1"}, {"key": "key2", "value": "value2"}, ..., {"key": "keyN", "value": "valueN"}]
Thanks!
This worked:
#Override
public Object evaluate(DeferredObject[] deferredObjects) throws HiveException {
if (deferredObjects == null || deferredObjects.length < 1) {
return null;
}
LazyArray lazyArray = (LazyArray) deferredObjects[0].get();
if (lazyArray == null) {
return null;
}
List<Object> lazyList = lazyArray.getList();
List<Object> finalList = Lists.newArrayList();
for (Object o : lazyList) {
LazyMap lazyMap = (LazyMap) o;
String key = "";
String value = "";
for (Map.Entry<?, ?> entry : lazyMap.getMap().entrySet()) {
if (entry.getKey().toString().equals("key")) {
key = entry.getValue().toString();
} else if (entry.getKey().toString().equals("value")) {
value = entry.getValue().toString();
}
}
finalList.add(Lists.newArrayList(key, value));
}
return finalList;
}
I came across this question and answer, showing how to push data from DynamoDB to Elasticsearch for full-text search indexing. Our application, however, is not making use of Lambdas. Instead, we're using Apache Camel to capture DynamoDB Streams events and want to push the records to Elasticsearch from there.
Since we are using AWS SDK v2, we're not capturing a DynamodbEvent class or corresponding DynamodbStreamRecord record class containing the DynamoDB record. Instead, we are receiving a software.amazon.awssdk.services.dynamodb.model.Record object. Given that, how can we serialize and subsequently index this data in Elasticsearch? In the other question referenced, the record is converted to a JSON string and then sent over to Elasticsearch. Is there a way to do this with the v2 Record class? The ItemUtils class mentioned in the answer no longer exists, so I was unaware of another way to serialize it.
Any help you can give is greatly appreciated!!
Similar to the example you provided, you can try something like the following:
public void processRecord(Record record, String index, String type, RestHighLevelClient esClient) throws Exception {
// Get operation
final OperationType operationType = record.eventName();
// Obtain a reference to actual DynamoDB stream record
final StreamRecord streamRecord = record.dynamodb();
// Get ID. Assume single numeric attribute as partition key
final Map<String,AttributeValue> keys = streamRecord.keys();
final String recordId = keys.get("ID").n();
switch (operationType) {
case INSERT:
if (!streamRecord.hasNewImage()) {
throw new IllegalArgumentException("No new image when inserting");
}
Map<String,AttributeValue> newImage = streamRecord.newImage();
// Where toJson is defined here https://github.com/aaronanderson/aws-java-sdk-v2-utils/blob/master/src/main/java/DynamoDBUtil.java
// and included below
JsonObject jsonObject = toJson(newImage);
IndexRequest indexRequest = new IndexRequest(index, type, recordId);
indexRequest.source(jsonObject.toString(), XContentType.JSON);
IndexResponse indexResponse = esClient.index(indexRequest, RequestOptions.DEFAULT);
System.out.println("New content successfully indexed: " + indexResponse);
break;
case MODIFY:
if (!streamRecord.hasNewImage()) {
throw new IllegalArgumentException("No new image when updating");
}
Map<String,AttributeValue> newImage = streamRecord.newImage();
JsonObject jsonObject = toJson(newImage);
UpdateRequest updateRequest = new UpdateRequest(index, type, recordId);
request.doc(jsonObject.toString(), XContentType.JSON);
UpdateResponse updateResponse = esClient.update(updateRequest, RequestOptions.DEFAULT);
System.out.println("Content successfully updated: " + updateResponse);
break;
case REMOVE:
DeleteRequest deleteRequest = new DeleteRequest(index, type, recordId);
DeleteResponse deleteResponse = esClient.delete(deleteRequest, RequestOptions.DEFAULT);
System.out.println("Successfully removed: " + deleteResponse);
break;
default:
throw new UnsupportedOperationException("Operation type " + opetationType + " not supportd");
}
}
The toJson method is defined is this class: https://github.com/aaronanderson/aws-java-sdk-v2-utils/blob/master/src/main/java/DynamoDBUtil.java
The source code is reproduced here:
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.math.BigDecimal;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.zip.DataFormatException;
import java.util.zip.Deflater;
import java.util.zip.Inflater;
import javax.json.Json;
import javax.json.JsonArray;
import javax.json.JsonArrayBuilder;
import javax.json.JsonNumber;
import javax.json.JsonObject;
import javax.json.JsonObjectBuilder;
import javax.json.JsonString;
import javax.json.JsonStructure;
import javax.json.JsonValue;
import software.amazon.awssdk.core.SdkBytes;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;
/** This is a utility for converting DynamoDB AttributeValues to and from Java JSON-P objects */
public class DynamoDBUtil {
public static void addList(String key, JsonObjectBuilder objectBuilder, List<JsonObject> items) {
if (!items.isEmpty()) {
JsonArrayBuilder builder = Json.createArrayBuilder();
items.forEach(i -> builder.add(i));
objectBuilder.add(key, builder.build());
}
}
public static JsonArray toJson(List<AttributeValue> attributeValues) {
if (attributeValues == null) {
return null;
}
JsonArrayBuilder valueBuilder = Json.createArrayBuilder();
for (AttributeValue a : attributeValues) {
add(toJson(a), valueBuilder);
}
return valueBuilder.build();
}
public static JsonObject toJson(Map<String, AttributeValue> attributeValues) {
if (attributeValues == null) {
return null;
}
JsonObjectBuilder valueBuilder = Json.createObjectBuilder();
for (Map.Entry<String, AttributeValue> a : attributeValues.entrySet()) {
add(a.getKey(), toJson(a.getValue()), valueBuilder);
}
return valueBuilder.build();
}
public static void add(String key, Object value, JsonObjectBuilder object) {
if (value instanceof JsonValue) {
object.add(key, (JsonValue) value);
// with json-p 1.0 can't create JsonString or JsonNumber so simply setting JsonValue not an option.
} else if (value instanceof String) {
object.add(key, (String) value);
} else if (value instanceof BigDecimal) {
object.add(key, (BigDecimal) value);
} else if (value instanceof Boolean) {
object.add(key, (Boolean) value);
} else if (value == null || value.equals(JsonValue.NULL)) {
object.addNull(key);
}
}
public static void add(Object value, JsonArrayBuilder array) {
if (value instanceof JsonValue) {
array.add((JsonValue) value);
} else if (value instanceof String) {
array.add((String) value);
} else if (value instanceof BigDecimal) {
array.add((BigDecimal) value);
} else if (value instanceof Boolean) {
array.add((Boolean) value);
} else if (value.equals(JsonValue.NULL)) {
array.addNull();
}
}
public static Object toJson(AttributeValue attributeValue) {
// with json-p 1.1 Json.createValue() can be used.
if (attributeValue == null) {
return null;
}
if (attributeValue.s() != null) {
return attributeValue.s();
}
if (attributeValue.n() != null) {
return new BigDecimal(attributeValue.n());
}
if (attributeValue.bool() != null) {
// return attributeValue.bool() ? JsonValue.TRUE : JsonValue.FALSE;
return attributeValue.bool();
}
if (attributeValue.b() != null) {
// return Base64.getEncoder().encodeToString(attributeValue.b().array());
return null;
}
if (attributeValue.nul() != null && attributeValue.nul()) {
return JsonValue.NULL;
}
if (!attributeValue.m().isEmpty()) {
return toJson(attributeValue.m());
}
if (!attributeValue.l().isEmpty()) {
return toJson(attributeValue.l());
}
if (!attributeValue.ss().isEmpty()) {
return attributeValue.ss();
}
if (!attributeValue.ns().isEmpty()) {
return attributeValue.ns();
}
if (!attributeValue.bs().isEmpty()) {
//return attributeValue.bs();
return null;
}
return null;
}
public static Map<String, AttributeValue> toAttribute(JsonObject jsonObject) {
Map<String, AttributeValue> attribute = new HashMap<>();
jsonObject.entrySet().forEach(e -> {
attribute.put(e.getKey(), toAttribute(e.getValue()));
});
return attribute;
}
public static List<AttributeValue> toAttribute(JsonArray jsonArray) {
List<AttributeValue> attributes = new LinkedList<>();
jsonArray.forEach(e -> {
attributes.add(toAttribute(e));
});
return attributes;
}
public static AttributeValue toAttribute(JsonValue jsonValue) {
if (jsonValue == null) {
return null;
}
switch (jsonValue.getValueType()) {
case STRING:
return AttributeValue.builder().s(((JsonString) jsonValue).getString()).build();
case OBJECT:
return AttributeValue.builder().m(toAttribute((JsonObject) jsonValue)).build();
case ARRAY:
return AttributeValue.builder().l(toAttribute((JsonArray) jsonValue)).build();
case NUMBER:
return AttributeValue.builder().n(((JsonNumber) jsonValue).toString()).build();
case TRUE:
return AttributeValue.builder().bool(true).build();
case FALSE:
return AttributeValue.builder().bool(false).build();
case NULL:
return AttributeValue.builder().nul(true).build();
}
return null;
}
public static AttributeValue compress(Map<String, AttributeValue> attributeValues) throws IOException {
return compress(toJson(attributeValues));
}
public static AttributeValue compress(List<AttributeValue> attributeValues) throws IOException {
return compress(toJson(attributeValues));
}
public static AttributeValue compress(JsonStructure jsonStructure) throws IOException {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
Json.createWriter(outputStream).write(jsonStructure);
outputStream.close();
byte[] jsonBinary = outputStream.toByteArray();
outputStream = new ByteArrayOutputStream();
Deflater deflater = new Deflater();
deflater.setInput(jsonBinary);
deflater.finish();
byte[] buffer = new byte[1024];
while (!deflater.finished()) {
int count = deflater.deflate(buffer); // returns the generated code... index
outputStream.write(buffer, 0, count);
}
outputStream.close();
jsonBinary = outputStream.toByteArray();
return AttributeValue.builder().b(SdkBytes.fromByteArray(jsonBinary)).build();
}
public static JsonStructure decompress(AttributeValue attributeValue) throws IOException, DataFormatException {
Inflater inflater = new Inflater();
byte[] jsonBinary = attributeValue.b().asByteArray();
inflater.setInput(jsonBinary);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(jsonBinary.length);
byte[] buffer = new byte[1024];
while (!inflater.finished()) {
int count = inflater.inflate(buffer);
outputStream.write(buffer, 0, count);
}
outputStream.close();
byte[] output = outputStream.toByteArray();
ByteArrayInputStream bis = new ByteArrayInputStream(output);
return Json.createReader(bis).read();
}
}
This class is an updated version of the originally introduced in this gist.
This post also provide a link to a Jackson's AtributeValue serializer if your prefer to use that library for JSON serialization.
I have a spring project, and I need to use cache in my service class, like this:
import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
#Service
public class WebAuthnServer {
private final Cache<String, RegistrationRequest> registerRequestStorage = newCache();
private static <K, V> Cache<K, V> newCache() {
return CacheBuilder.newBuilder()
.maximumSize(100)
.expireAfterAccess(10, TimeUnit.MINUTES)
.build();
}
public Either<String, RegistrationRequest> startRegistration(String username, String displayName, String credentialNickname, boolean requireResidentKey) {
if (userStorage.getRegistrationsByUsername(username).isEmpty()) {
RegistrationRequest request = new RegistrationRequest(...);
registerRequestStorage.put(request.getRequestId(), request);
} else {
return new Left("The username \"" + username + "\" is already registered.");
}
}
}
I have registerRequestStorage cache, and I put some data in cache using the method startRegistration. But when I try to get this data in another method, the cache is empty.
public Either<List<String>, SuccessfulRegistrationResult> finishRegistration(String responseJson) {
RegistrationResponse response = null;
try {
response = jsonMapper.readValue(responseJson, RegistrationResponse.class);
} catch (IOException e) {
return Left.apply(Arrays.asList("Registration failed!", "Failed to decode response object.", e.getMessage()));
}
RegistrationRequest request = registerRequestStorage.getIfPresent(response.getRequestId());
registerRequestStorage.invalidate(response.getRequestId());
if (request == null) {
logger.info("fail finishRegistration responseJson: {}", responseJson);
return Left.apply(Arrays.asList("Registration failed!", "No such registration in progress."));
} else {
Try<RegistrationResult> registrationTry = rp.finishRegistration(
request.getPublicKeyCredentialCreationOptions(),
response.getCredential(),
Optional.empty()
);
}
}
I am looking for dynamic approach in order to set the values in setter method, in which I fetch the total number of column from ResultSetMetaData, on based of those result, I want to set the values in corresponding setter method. I already prepared the snippet in my utility class.
public Integer extractData(ResultSet rs) throws SQLException, DataAccessException {
ResultSetMetaData rsmd = rs.getMetaData();
int columnCount = rsmd.getColumnCount();
for(int i = 1 ; i <= columnCount ; i++){
SQLColumn column = new SQLColumn();
String columnName = rsmd.getColumnName(i);
}
return columnCount;
}
Now the scenario is I want to set Pay_codes Of Employees, now there are total 95 paycode, but it differs in case of employees.
For example, Regular employees has 13 paycode, where as contactual employee has 6 codes, similarly society employees has 8 paycodes.
Now I need to show the list of paycodes on behalf of employee.
I passed EmployeeCode and I got the result.
Now The problem
whether I set all the column in my RowMapper like below
Paycode pc=new PayCode();
if(rs.getString("ESIC") != null){
pc.setESICCode(rs.getString("ESIC"));
}
if(rs.getString("EPF") != null){
pc.setEPFCode(rs.getString("EPF"));
}
if(rs.getString("Allowance") != null){
pc.setAllowance(rs.getString("Allowance"));
}
and many more columns........till 300+ lines
This seems very bad approach, because, Paycode can be increase or decrease as per client request.
So, I am looking for 2-3 line of dynamic code which set the values in setter according to the column name inside "ResultSetMetaData".So Kindly suggest me the best approach and help me to achieve the same.
However I am thinking of generic implementation, is generic do some magic, If yes > then how? please suggest me.
Use this code:
private void mapFields(PayCode p, String setterName, Object value) {
Method[] methods = PayCode.class.getDeclaredMethods();
for(Method method: methods){
if(method.getName().contains(setterName)){
try {
method.invoke(p, value);
break;
} catch (IllegalAccessException e) {
e.printStackTrace();
} catch (InvocationTargetException e) {
e.printStackTrace();
}
}
}
}
So you use Java reflection API with PayCode.class.getDeclaredMethods to get class methods, and then you iterate through method names searching for method that contains name of the property.
Once you find it you set the value to the object with method.invoke(p, value), and exit the loop.
You can make things faster if you store PayCode.class.getDeclaredMethods(), and then make a hash set with method name as a key, and method as a value Map<String, Method>:
static Map<String, Method> mapFields(Class clazz) {
Method[] methods = clazz.getDeclaredMethods();
Map<String, Method> result = new HashMap<>();
for(Method method: methods){
String methodName = method.getName();
if(methodName.indexOf("set") == 0){
result.put(methodName.substring(3), method);
}
}
return result;
}
and then use it:
Map<String, Method> fields = mapFields(PayCode.class);
if(fields.containsKey(name)){
try {
fields.get(name).invoke(p, value);
} catch (IllegalAccessException e) {
e.printStackTrace();
} catch (InvocationTargetException e) {
e.printStackTrace();
}
}
You can use reflection
package com.test;
import java.lang.reflect.Method;
class PayCode {
private String esic;
private String epf;
public String getEsic() {
return esic;
}
public void setEsic(String esic) {
this.esic = esic;
}
public String getEpf() {
return epf;
}
public void setEpf(String epf) {
this.epf = epf;
}
#Override
public String toString() {
return "PayCode [esic=" + esic + ", epf=" + epf + "]";
}
}
public class DynamicSetter {
public static void main(String[] args) {
PayCode payCode = new PayCode();
try {
/**
* Here you can take "set" hardcoded and generate suffix "Esic"
* dynamically, may be using apache StringUtils or implement using
* Substring, based on your class setter methods.
*/
Method esicMethod = PayCode.class.getMethod("set" + "Esic", String.class);
esicMethod.invoke(payCode, "Test Paycode");
System.out.println("Check paycode in object : " + payCode);
} catch (NoSuchMethodException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
}
You can use HashMap<> to add all pay code in that. You can also implement gtter for each pay code to read from HashMap.
package com.test;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
import java.util.HashMap;
import java.util.Map;
class Employee {
private Map<String, String> payCodes = new HashMap<>();
private String epf;
public String getEsic() {
return payCodes.get("ESIC");
}
public String getEpf() {
return payCodes.get("EPF");
}
public Map<String, String> getPayCodes() {
return payCodes;
}
}
public class DynamicSetter {
public static void main(String[] args) {
}
public static Integer extractData(ResultSet rs) throws SQLException, DataAccessException {
while(rs.next()) {
Employee e = new Employee();
Map<String, String> payCodes = e.getPayCodes();
//set app paycode in payCodes, you can use column name as key and fetch value from result set.
}
}
}
I'm trying to get the JSON values from Distance24 JSON output via Google GSON.
But I can't figure out what and where the Exception comes from (I'm using Google AppEngine with Java).
Here's the class from which i send and get the request and response.
package de.tum.in.eist.distance;
import java.io.IOException;
import javax.inject.Inject;
import java.net.URL;
import com.google.appengine.api.urlfetch.HTTPResponse;
import com.google.appengine.api.urlfetch.URLFetchService;
import com.google.gson.JsonObject;
import de.tum.in.eist.JsonHelper;
import de.tum.in.eist.URLFetchServiceHelper;
public class Distance24Client {
private final URLFetchService service;
#Inject
public Distance24Client(URLFetchService service) {
this.service = service;
}
public Distance24 getDistanceAPI(String source, String destination) throws IOException {
URL url = new URL("http://www.distance24.org/route.json?stops=" + source + "|" + destination);
HTTPResponse response = service.fetch(url);
String jsonString = URLFetchServiceHelper.toString(response);
try {
JsonObject json = JsonHelper.parse(jsonString);
return toDistance24(json);
} catch (Exception e) {
throw new IOException("Error ocurred in getDistanceAPI(): " + e.getMessage());
}
}
private Distance24 toDistance24(JsonObject response) {
if (!(response.get("stops").getAsJsonObject().getAsJsonArray().size() != 0)) {
throw new IllegalArgumentException("No Status set from Distance24 API");
} else {
JsonObject distances = response.get("distances").getAsJsonObject();
return new Distance24(distances);
}
}
}
And here's the Distance24 Object:
package de.tum.in.eist.distance;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
public class Distance24 {
private int[] distances;
private int totalDistance;
private Double sourceLat;
private Double sourceLon;
private Double destLat;
private Double destLong;
public Distance24(JsonObject distances) {
this.setDistances(getIntArray(distances));
this.setTotalDistance(getSum(this.distances));
this.setSourceLat(distances.get("stops").getAsJsonObject().getAsJsonArray().get(0).getAsJsonObject().get("latitude").getAsDouble());
this.setSourceLon(distances.get("stops").getAsJsonObject().getAsJsonArray().get(0).getAsJsonObject().get("longitude").getAsDouble());
this.setDestLat(distances.get("stops").getAsJsonObject().getAsJsonArray().get(1).getAsJsonObject().get("latitude").getAsDouble());
this.setDestLong(distances.get("stops").getAsJsonObject().getAsJsonArray().get(1).getAsJsonObject().get("longitude").getAsDouble());
}
private int[] getIntArray(JsonObject array) {
JsonArray distances = array.getAsJsonArray();
int[] result = new int[distances.size()];
for(int i = 0; i < distances.size(); i++) {
result[i] = distances.get(i).getAsInt();
}
return result;
}
private int getSum(int[] array) {
int sum = 0;
for(int element : array) {
sum += element;
}
return sum;
}
private void setDistances(int[] distances) {
this.distances = distances;
}
public int getTotalDistance() {
return totalDistance;
}
public void setTotalDistance(int totalDistance) {
this.totalDistance = totalDistance;
}
public Double getSourceLat() {
return sourceLat;
}
public void setSourceLat(Double sourceLat) {
this.sourceLat = sourceLat;
}
public Double getSourceLon() {
return sourceLon;
}
public void setSourceLon(Double sourceLon) {
this.sourceLon = sourceLon;
}
public Double getDestLat() {
return destLat;
}
public void setDestLat(Double destLat) {
this.destLat = destLat;
}
public Double getDestLong() {
return destLong;
}
public void setDestLong(Double destLong) {
this.destLong = destLong;
}
}
As a result, I get the whole JSON Object as a String output for e.getMessage(). So I guess the information retrieving works, even though it's on the wrong part of the code.
Plus in the same try-catch-block of the code (Distance24Client, method "toDistance24") it says, the error ocurred in line 30, which is the return statement of the "toDistance24" method.
(clickable)
Running http://www.distance24.org/route.json?stops=detroit|dublin from my browser gives me
{"stops":[{"region":"Michigan ...
"distances":[5581]}
So distances is an array and not an object.
So your line:
JsonObject distances = response.get("distances").getAsJsonObject();
is wrong. Read distances as a JsonArray.
Create a method to handle array or no-array
public static JsonElement toJsonElement(String jsonString) {
JsonParser parser = new JsonParser();
JsonElement jsonElement = parser.parse(jsonString);
JsonElement result = null;
if (jsonElement instanceof JsonObject) {
result = jsonElement.getAsJsonObject();
} else if (jsonElement instanceof JsonArray) {
result = jsonElement.getAsJsonArray();
} else {
throw new IllegalArgumentException(jsonString + " is not valid JSON stirng");
}
return result;
}