Develop Reference

UPDATE-FROM clause in jOOQ throws an expecption for CTE field

I am trying to convert following PostgreSQL query into jOOQ:
UPDATE book
SET amount = bat.amount
FROM (
VALUES (2, 136),(5, 75)
) AS bat(book_id, amount)
WHERE book.book_id = bat.book_id;
VALUES inside of FROM-clause are being created from Map<Long, Integer> bookIdsAmountMap parameter and I am trying to perform that this way:
class BookUtilHelper {
#SuppressWarnings("unchecked")
static Table<Record2<Long, Integer>> batTmp(DSLContext dsl, Map<Long, Integer> bookIdAmountMapUpdated) {
Row2<Long,Integer> array[] = new Row2[bookIdAmountMapUpdated.size()];
int i = 0;
for (Map.Entry<Long, Integer> pair : bookIdAmountMapUpdated.entrySet()) {
array[i]=DSL.row(pair.getKey(), pair.getValue());
i++;
}
Table<Record2<Long, Integer>> batTmp = DSL.values(array);
batTmp.fields("book_id", "amount");
return batTmp;
}
}
Then, I try to also create fields which can be accessed like in this example
Field<Long> bookIdField = DSL.field(DSL.name("bat", "book_id"), Long.class);
Field<Integer> amountField = DSL.field(DSL.name("bat", "amount"), Integer.class);
Table<Record2<Long, Integer>> batTmp = BookUtilHelper.batTmp(dsl, bookIdAmountMapUpdated);
// ctx variable is of type DSLContext
ctx.update(BOOK).set(BOOK.AMOUNT, amountField).from(batTmp.as("bat"))
.where(BOOK.BOOK_ID.eq(bookIdField));
When I try to update book I get following exception:
column bat.book_id does not exist
Any advice on how to solve this issue would be greatly appreciated. :)

This doesn't have any effect:
batTmp.fields("book_id", "amount");
Whereas this only renames the table, not the columns:
batTmp.as("bat")
Write this instead:
batTmp.as("bat", "book_id", "amount")

RavenDB query returns null

I'm on RavenDB 3.5.35183. I have a type:
import com.mysema.query.annotations.QueryEntity;
#QueryEntity
public class CountryLayerCount
{
public String countryName;
public int layerCount;
}
and the following query:
private int getCountryLayerCount(String countryName, IDocumentSession currentSession)
{
QCountryLayerCount countryLayerCountSurrogate = QCountryLayerCount.countryLayerCount;
IRavenQueryable<CountryLayerCount> levelDepthQuery = currentSession.query(CountryLayerCount.class, "CountryLayerCount/ByName").where(countryLayerCountSurrogate.countryName.eq(countryName));
CountryLayerCount countryLayerCount = new CountryLayerCount();
try (CloseableIterator<StreamResult<CountryLayerCount>> results = currentSession.advanced().stream(levelDepthQuery))
{
while(results.hasNext())
{
StreamResult<CountryLayerCount> srclc = results.next();
System.out.println(srclc.getKey());
CountryLayerCount clc = srclc.getDocument();
countryLayerCount = clc;
break;
}
}
catch(Exception e)
{
}
return countryLayerCount.layerCount;
}
The query executes successfully, and shows the correct ID for the document I'm retrieving (e.g. "CountryLayerCount/123"), but its data members are both null. The where clause also works fine, the country name is used to retrieve individual countries. This is so simple, but I can't see where I've gone wrong. The StreamResult contains the correct key, but getDocument() doesn't work - or, rather, it doesn't contain an object. The collection has string IDs.
In the db logger, I can see the request coming in:
Receive Request # 29: GET - geodata - http://localhost:8888/databases/geodata/streams/query/CountryLayerCount/ByName?&query=CountryName:Germany
Request # 29: GET - 22 ms - geodata - 200 - http://localhost:8888/databases/geodata/streams/query/CountryLayerCount/ByName?&query=CountryName:Germany
which, when plugged into the browser, correctly gives me:
{"Results":[{"countryName":"Germany","layerCount":5,"#metadata":{"Raven-Entity-Name":"CountryLayerCounts","Raven-Clr-Type":"DbUtilityFunctions.CountryLayerCount, DbUtilityFunctions","#id":"CountryLayerCounts/212","Temp-Index-Score":0.0,"Last-Modified":"2018-02-03T09:41:36.3165473Z","Raven-Last-Modified":"2018-02-03T09:41:36.3165473","#etag":"01000000-0000-008B-0000-0000000000D7","SerializedSizeOnDisk":164}}
]}
The index definition:
from country in docs.CountryLayerCounts
select new {
CountryName = country.countryName
}
AFAIK, one doesn't have to index all the fields of the object to retrieve it in its entirety, right ? In other words, I just need to index the field(s) to find the object, not all the fields I want to retrieve; at least that was my understanding...
Thanks !

The problem is related to incorrect casing.
For example:
try (IDocumentSession sesion = store.openSession()) {
CountryLayerCount c1 = new CountryLayerCount();
c1.layerCount = 5;
c1.countryName = "Germany";
sesion.store(c1);
sesion.saveChanges();
}
Is saved as:
{
"LayerCount": 5,
"CountryName": "Germany"
}
Please notice we use upper case letters in json for property names (this only applies to 3.X versions).
So in order to make it work, please update json properties names + edit your index:
from country in docs.CountryLayerCounts
select new {
CountryName = country.CountryName
}
Btw. If you have per country aggregation, then you can simply query using:
QCountryLayerCount countryLayerCountSurrogate =
QCountryLayerCount.countryLayerCount;
CountryLayerCount levelDepthQuery = currentSession
.query(CountryLayerCount.class, "CountryLayerCount/ByName")
.where(countryLayerCountSurrogate.countryName.eq(countryName))
.single();

Java Mongo DB Unable to get valid next id

I am trying insert an item in MongoDB using Java MongoDB driver.Before inserting I am trying to get nextId to insert,but not sure why I am always getting nextId as 4 .I am using below given method to get nextId before inserting any item in Mongo.
private Long getNextIdValue(DBCollection dbCollection) {
Long nextSequenceNumber = 1L;
DBObject query = new BasicDBObject();
query.put("id", -1);
DBCursor cursor = dbCollection.find().sort(query).limit(1);
while (cursor.hasNext()) {
DBObject itemDBObj = cursor.next();
nextSequenceNumber = new Long(itemDBObj.get("id").toString()) + 1;
}
return nextSequenceNumber;
}
I have total 13 record in my mongodb collection.What I am doing wrong here?

Please don't do that. You don't need create a bad management id situation as the driver already do this in the best way, just use the right type and annotation for the field:
#Id
#ObjectId
private String id;
Then write a generic method to insert all entites:
public T create(T entity) throws MongoException, IOException {
WriteResult<? extends Object, String> result = jacksonDB.insert(entity);
return (T) result.getSavedObject();
}
This will create a time-based indexed hash for id's which is pretty much more powerful than get the "next id".
https://www.tutorialspoint.com/mongodb/mongodb_objectid.htm

How can you perform Arithmetic operations like +1 to String
nextSequenceNumber = new Long(itemDBObj.get("id").toString()) + 1;
Try to create a Sequence collection like this.
{"id":"MySequence","sequence":1}
Then use Update to increment the id
// Query for sequence collection
Query query = new Query(new Criteria().where("id").is("MySequence"));
//Increment the sequence by 1
Update update = new Update();
update.inc("sequence", 1);
FindAndModifyOptions findAndModifyOptions = new FindAndModifyOptions();
findAndModifyOptions.returnNew(true);
SequenceCollection sequenceCollection = mongoOperations.findAndModify(query, update,findAndModifyOptions, SequenceCollection.class);
return sequenceModel.getSequence();

I found the work around using b.collection.count().I simply find the total count and incremented by 1 to assign id to my object.

MongoDB update with layered JSON - how to efficiently? Dot notation?

Mongodb has an update function, where it can increment pre-existing fields. However, I found that it could only update flat JSON. Whenever there's a JSONObject inside of a JSONObject, with a value I want to increment, I can't actually seem to do it. It will return this error:
com.mongodb.WriteConcernException: Write failed with error code 14 and error message
'Cannot increment with non-numeric argument: {laneQty: { BOTTOM: 1 }}'
As you can see, I tried update incrementing laneQty.BOTTOM by 1. I don't want to write an algorithm to change every single layered json field into dot notation(like laneQty.BOTTOM), so is there a way to either turn the JSON into dot notation pre-upsert?
For now my general upsert function looks like this:
public boolean incrementJson(BasicDBObject json, String colName, ArrayList<String> queryParams, ArrayList<String> removeParams){
/*make sure the game id AND the main player id can't both be the same.
If either/or, it's fine. We don't want duplicates.
*/
BasicDBObject query = new BasicDBObject();
DBCollection collection = db.getCollection(colName);
for(int i = 0; i < queryParams.size(); i++){
String param = queryParams.get(i);
query.put(param, json.get(param));
}
for(String param : removeParams){
json.remove(param);
}
return collection.update(query, new BasicDBObject("$inc", json), true, false).isUpdateOfExisting();
}
Is there any suggested upgrades to this code that could make it easily update layered json as well? Thank you!
By the way, it'll be very hard for me to hardcode this. There are a ton of layered objects and that would take me forever. Also, I am not in complete control of which fields are populated in the layers, so I can't just say laneQty.BOTTOM every single time because it will not always exist. Prior to upserting, the BasicDBObject json was actually a java bean parsed into BasicDBObject. This is its constructor if it's of any help:
public ChampionBean(int rank, int division, int assists, int deaths, int kills, int qty, int championId,
HashMap<String, Integer> laneQty, HashMap<String, Integer> roleQty,
ParticipantTimelineDataBean assistedLaneDeathsPerMinDeltas,
ParticipantTimelineDataBean assistedLaneKillsPerMinDeltas, ParticipantTimelineDataBean creepsPerMinDeltas,
ParticipantTimelineDataBean csDiffPerMinDeltas, ParticipantTimelineDataBean damageTakenDiffPerMinDeltas,
ParticipantTimelineDataBean damageTakenPerMinDeltas, ParticipantTimelineDataBean goldPerMinDeltas,
ParticipantTimelineDataBean xpDiffPerMinDeltas, ParticipantTimelineDataBean xpPerMinDeltas, int wins,
int weekDate, int yearDate) {
super();
this.rank = rank;
this.division = division;
this.assists = assists;
this.deaths = deaths;
this.kills = kills;
this.qty = qty;
this.championId = championId;
this.laneQty = laneQty;
this.roleQty = roleQty;
this.assistedLaneDeathsPerMinDeltas = assistedLaneDeathsPerMinDeltas;
this.assistedLaneKillsPerMinDeltas = assistedLaneKillsPerMinDeltas;
this.creepsPerMinDeltas = creepsPerMinDeltas;
this.csDiffPerMinDeltas = csDiffPerMinDeltas;
this.damageTakenDiffPerMinDeltas = damageTakenDiffPerMinDeltas;
this.damageTakenPerMinDeltas = damageTakenPerMinDeltas;
this.goldPerMinDeltas = goldPerMinDeltas;
this.xpDiffPerMinDeltas = xpDiffPerMinDeltas;
this.xpPerMinDeltas = xpPerMinDeltas;
this.wins = wins;
this.weekDate = weekDate;
this.yearDate = yearDate;
}
The participantTimelineDataBean is another bean with 4 int fields inside of it. I want to increment those fields (so yes it's only 2 layers deep, so if there's a solution with 2 layers deep availability I'll take that too).

Use the dot-notation:
new BasicDBObject("$inc", new BasicDBObject("laneQty.BOTTOM", 1) )
Alternative quick&dirty solution: Just collection.save the whole document under the same _id.

Use this library:
https://github.com/rhalff/dot-object
For example if you have an object like this:
var jsonObject = {
info : {
firstName : 'aamir',
lastName : 'ryu'
email : 'aamiryu#gmail.com'
},
}
then your node.js code would be like this:
var dot = require('dot-object');
var jsonObject = // as above ;-);
var convertJsonObjectToDot = dot.dot(jsonObject);
console.log(convertJsonObjectToDot);
Output will be as shown below:
{
info.firstName : 'aamir',
info.lastName : 'ryu',
info.email : 'aamiryu#gmail.com
}
Please bear with me, this is my first answer on stackoverflow ever, since i was searching for the same thing and i found one solution to it, hope it helps you out.

scan count returns significantly less number for a dynamodb table

I'm running a sample java program to query a dynamodb table, the table has about 90000 items but when i get the scan count from java it shows only 1994 items
ScanRequest scanRequest = new ScanRequest().withTableName(tableName);
ScanResult result = client.scan(scanRequest);
System.out.println("#items:" + result.getScannedCount());
the program outputs #items:1994
but the detail from amazon aws console shows:
Item Count*: 89249
any idea?
thanks

scanning or querying dynamodb only returns maximum of 1MB of data.
the count is the number of return items fit in 1MB. in order to get the whole table, you should aggressively scan the database until the value LastEvaluatedKey is null

Set your book object with correct hash key value, and use DynamoDBMapper to get the count.
DynamoDBQueryExpression<Book> queryExpression = new DynamoDBQueryExpression<Book>()
.withHashKeyValues(book);
dynamoDbMapper.count(Book.class, queryExpression);

This should help . Worked for me
AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()
.withRegion("your region").build();
DynamoDB dynamoDB = new DynamoDB(client);
TableDescription tableDescription = dynamoDB.getTable("table name").describe();
tableDescription.getItemCount();

Based on answer from nightograph
private ArrayList<String> fetchItems() {
ArrayList<String> ids = new ArrayList<>();
ScanResult result = null;
do {
ScanRequest req = new ScanRequest();
req.setTableName("table_name");
if (result != null) {
req.setExclusiveStartKey(result.getLastEvaluatedKey());
}
result = amazonDynamoDBClient.scan(req);
List<Map<String, AttributeValue>> rows = result.getItems();
for (Map<String, AttributeValue> map : rows) {
AttributeValue v = map.get("rangeKey");
String id = v.getS();
ids.add(id);
}
} while (result.getLastEvaluatedKey() != null);
System.out.println("Result size: " + ids.size());
return ids;
}

I agreed with nightograph. I thinks this link is useful.
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScan.html
I just tested with this example. Anyway this is the Dyanamodb v2.
final ScanRequest scanRequest = new ScanRequest()
.withTableName("table_name");
final ScanResult result = dynamoDB.scan(scanRequest);
return result.getCount();