I have a use case as below.
List<Document> from mongo
For each Document
Get a field
Add to a set
Alter the field
Check altered field against two other sets
If not present convert to a pojo
add pojo to a list
I have something as below.
Optional.ofNullable(documentList).orElseGet(Collections::emptyList).forEach(doc -> {
// I am doing all the operations in java7 way
})
I thought something similar as below.
Iterate document list - add field to the set
Iterate the set from step1 - check field not contains on two other collections - then form pojo from original document - add pojo to list
But here I need to iterate twice. Moreover I need to maintain my original doc also. If my list is huge, iteration time is added unnecessarily.
How can I do this in standard way?
documentList.forEach(doc -> {
String deviceId = doc.getString("deviceId");
deviceSet.add(deviceId);
String alteredId = "IMEI" + deviceId;
if(!clearedDevices.contains(alteredId) && !superSetDevices.contains(deviceId)) {
//Pojo from Document
//add pojo to a list
}
})
Well it depends on the type of the concerned field. If it is a mutable object, then Java 8 stream is perfect. Something like:
documentList.stream()
.map(d->d.getString("deviceId") // stream of Stringified field
.peek(deviceSet::add) // add field to set
.map("IMEI"::concat) // alter the field
.filter(mf->!clearedDevices.contains(mf)&&!superSetDevices.contains(mf)) // filter...
.map(f->XXX.convert2Pojo(f)) // convert to pojo
.forEach(pojoList::add) // add to pojolist, or collect?
Related
Feeling a bit stupid, but I have a simple architecture where the repositories are the only ones accessing ~Record classes and the services work on POJOs.
So basic flow is
repository fetches into POJO
service modifies POJO
repository receives POJO to update DB
repository matches updated POJO to record
repository stores (insert or update) the record
repository maps updated record (may have received generated values from insert) back to POJO
service receives updated POJO
i.e. something like
fun save(set: MySet): MySet {
set.description = set.description ?: ""
val record = ctx.newRecord(MY_SET, set).apply {
store()
}
// "When store() performs an INSERT statement, jOOQ attempts to load any generated keys from the database back into the record."
// cf. https://www.jooq.org/doc/latest/manual/sql-execution/crud-with-updatablerecords/simple-crud/
return record.into(MySet::class.java)
}
This fails because to quote documentation for newRecord:
Create a new pre-filled Record that can be inserted into the corresponding table.
This performs roughly the inverse operation of Record.into(Class)
The resulting record will have its internal "changed" flags set to true for all values. This means that UpdatableRecord.store() will perform an INSERT statement. If you wish to store the record using an UPDATE statement, use executeUpdate(UpdatableRecord) instead.
I CAN, of course, check if I have an id, and then either fetch the record from the database or create a new one
fun save(set: MySet): MySet {
set.description = set.description ?: ""
val record = when (val setId = set.id) {
null -> ctx.newRecord(MY_SET, set)
else -> ctx.selectFrom(MY_SET).where(MY_SET.ID.eq(setId)).fetchSingle()
}
//TOOD: update record manually from `set`
record.store()
// "When store() performs an INSERT statement, jOOQ attempts to load any generated keys from the database back into the record."
// cf. https://www.jooq.org/doc/latest/manual/sql-execution/crud-with-updatablerecords/simple-crud/
return record.into(MySet::class.java)
}
But that kind of is a lot of boilerplate code.
I DO have access to the MySetDao but that one just has insert and update, there's no store or upsert, as far as I can see.
Is there a way to turn a POJO into an UpdatableRecord directly or is this fetch-and-manual-update the way to go?
(Worth noting: the MySet POJO used here was generated by jOOQ.)
I am using Dynamodb Item - getItem API to get records from DynamoDB table. But it returns Item object and I want to retrieve specific attribute value from the Item object. How can we do it in Java? I couldn't find references.
Table table = dynamoDB.getTable(tableName);
Item item = table.getItem(hashKeyFieldName, hashKeyFieldValue);
The item contains the following fields:
HashKey, TimeStamp, NumRetries
I want to get the specific NumRetries value from item above. Is it something that is possible? something like int numRetries = item.get("NumRetries");?
You should be able to do that with a Projection Expression:
GetItemSpec spec = new GetItemSpec().withPrimaryKey("primaryKey", primaryKey)
.withProjectionExpression("HashKey, TimeStamp, NumRetries");
Item outcome = table.getItem(spec);
A names map may be necessary.
You can use Projection Expressions to get certain attributes from an item but do keep in mind that using projection expressions does not reduce the usage and cost of RCUs that are used in retrieving the object.
Code example,
GetItemSpec spec = new GetItemSpec()
.withPrimaryKey("YourPrimaryKey", value)
.withProjectionExpression("NumRetries");
Item item = table.getItem(spec);
System.out.println(item.toJSONPretty());
More code examples can be found here.
I've seen a lot of examples of using UpdateExpression to update attributes using the updateItem method. However, I still don't understand how to update multiple attributes in DynamoDB at the same time dynamically.
I am trying to update AND rename multiple attributes in the same updateItem call.
I understand that this requires a REMOVE of the old name and a SET of the new name. I have these names in hashedId's of objects, but won't have them until runtime. So my question is how do I use UpdateExpression with variables and not a hard-coded String?
All the examples I have seen use hard-coded UpdateExpressions.
can't update item in DynamoDB
Dynamo DB : UpdateItemSpec : Multiple Update Expression - Not Working
DynamoDB update Item multi action
How to rename DynamoDB column/key
I am working in Java.
It seems very odd to me that I haven't been able to find an example of this... which leads me to believe I am doing something wrong.
Thanks for the help!
You have to build the update expression string dynamically based on the attribute names and values that you receive at runtime. I do exactly this. I'm not working in Java, but here is some pseudo code (with a Ruby bias) example for you that dynamically builds the update expression string, the expression attribute names hash, and the expression attribute values hash. You can then plug in these 3 things into the update_item method:
update_exp_set = [] //array of remove expression snippets
update_exp_remove = [] //array of update expression snippets
exp_attribute_names = {} //hash of attribute names
exp_attribute_values = {} //hash of attribute values
// Iterate through all your fields and add things as needed to your arrays and hashes.
// Two examples are below for a field with the name <fieldName> and value <fieldValue>.
// You'll need to convert this to Java and update it to make sense with the AWS Java SDK.
// For a given field that needs to be updated:
update_exp_set << "#<fieldName> = :fieldValue" //add to array of set expression snippets
exp_attribute_names["#<fieldName>"] = "<fieldName>" //add to hash of attribute names
exp_attribute_values[":<fieldValue>"] = "<fieldValue>" //add to hash of attribute values
// For a given field that needs to be removed:
update_exp_remove << "#<fieldName>"
exp_attribute_names["#<fieldName>"] = "<fieldName>" //add to hash of attribute names
// Use your snippets to create your full update expression:
update_exp_set_clause = ""
update_exp_remove_clause = ""
if update_exp_set.length != 0 //check if you have something to set
update_exp_set_clause = "SET " + update_exp_set.join(',')
end
if update_exp_remove.length != 0 //check if you have something to remove
update_exp_remove_clause = "REMOVE" + update_exp_remove.join(',')
end
final_update_exp = update_exp_set_clause + " " + update_exp_remove_clause
Does this help?
I have a list of objects which I want to insert into a collection. The mongoTemplate.insert(list); works fine but now I want to modify it to upsert(); as my list can contain duplicate objects which are already inserted into a collection. So what I want is insert entire list and on the go check if the item is already present in the collection then skip it else insert it.
You can try out continueOnError or ordered flag like this:
db.collection.insert(myArray, {continueOnError: true})
OR,
db.collection.insert(myArray, {ordered: false})
You need to create a unique index field of your object's id(if there is no unique constraint). So that it will make error while you try to insert using same id.
Using the unique constraint you insert array or using BulkInsert
For using insert you can set a flag continueOnError: true which can continue insertion whenever error found in case of error because of unique constraint while inserting existing id of object.
The only way to do a bulk-upsert operation is the method MongoCollection.bulkWrite (or at least: the only way I know... ;-))
To use it, you have to convert your documents to the appropriate WriteModel: for upserts on a per-document basis, this is UpdateOneModel.
List<Document> toUpdate = ...;
MongoCollection coll = ...;
// Convert Document to UpdateOneModel<Document>
List<UpdateOneModel<Document>> bulkOperationList = toUpdate.stream()
.map(doc -> new UpdateOneModel<Document>(
Filters.eq("_id", doc.get("_id")), // identify by same _id
doc,
new UpdateOptions().upsert(true)))
.collect(Collectors.toList());
// Write to DB
coll.bulkWrite(bulkOperationList);
(Disclaimer: I only typed this code, I never ran it)
I have a XML file containing metadata like a field's maximum length. I have to use drools to build rules to validate this metadata against a list of facts. I don't want to hardcode the name of each field that may or may not be specified in the XML.
I tried to do this :
when
$metadata: Metadata(maxLength != null);
$obj: Object(eval($metadata.getFieldName()).length > $metadata.maxLength);
then
// TODO
end
It does not work and I get the following error :
java.lang.IllegalStateException: Errors while building rules : Unable
to Analyse Expression $metadata.getFieldName() > $metadata.maxLength:
[Error: Comparison operation requires compatible types. Found class
java.lang.String and class java.lang.Integer] [Near : {...
$metadata.getFieldName() > $metadata.maxLength ....}]
Is it possible to dynamically get a field name and compare its maximum length? Will I have to create a java object to accomplish this?
Thank you
You talk of XML and metadata. Can you distinguish all entities? For example, if it is about orders, can you extract each order, and attributes of each order?
I solved a similar problem with using maps to store each attribute.
public class Order{
private int id;
private Map<String, Integer> num_attribute_map = new HashMap<String, Integer>();
public Map getNumAttributeMap(){
return this.num_attribute_map;
}
If an order has customer_satisfaction = 5,
order_obj.getNumAttributeMap().put("customer_satisfaction" , 5);
And thus you have created Orders with their attributes stored in the numAttributeMap.
For implementing a rule on an Order
$ord : Order(
getNumAttributeMap[$attribute] >= $value
)
where $attribute would be "customer_satisfaction", of course. The [] notation is used to access elements of a list, given index or values of a map, given the key.
Hope you "get" the concepts of maps. Also, do look up Drools language support for list and map access.
I have also implemented maps of lists of strings to perform an "is in" operation, in addition to maps of integers that do comparison operations. Please refer https://stackoverflow.com/a/9241089/604511 too
Finally, I have decided to generate my drools file dynamically from my XML using rule templates.