Develop Reference

How to properly format generated routine result of JSON type by using formatJSON(JSONFormat) in jOOQ?

I'm having issue with properly formatting my JSON result when querying generated Routine by using jOOQ code generator. I'm trying to perform SELECT-clause on my get_all_orders() method defined in PL/pgSQL (mentioned in this question) which returns result of json type. This is my code for performing jOOQ-fied query:
DSLContext create = DSL.using(connection, SQLDialect.POSTGRES);
Result<Record1<String>> resultR1S = create.select(Routines.getAllOrders()).fetch();
final String strResultFinal = resultR1S.formatJSON(
new JSONFormat().header(false).recordFormat(RecordFormat.ARRAY)
);
...and this is output I get on console (bit truncated at the end because result output is waaaay too long to fit in):
[["{\"orders\" : [{\"order_id\" : 1, \"total_price\" : 29.98, \"order_date\" : \"2019-08-22T10:06:33\", \"user\" : {\"user_id\" : 1, \"username\" : \"test\"}, \"order_items\" : [{\"order_item_id\" : 1, \"amount\" : 1, \"book\" : {\"book_id\" : 1, \"title\" : \"Harry Potter and the Philosopher's Stone\", \"price\" : 29.98, \"amount\" : 400, \"is_deleted\" : false, \"authors\" : [{\"author_id\":4,\"first_name\":\"JK\",\"last_name\":\"Rowling\"}], \"categories\" : [{\"category_id\":2,\"name\":\"Lyric\",\"is_deleted\":false}]}, \"order_id\" : 1, \"total_order_item_price\" : 29.98}]}, {...}"]]
What I'm trying to achieve is to get rid off double angle brackets (at beginning and end of output) and backslash characters so it looks something like this:
{"orders" : [{"order_id" : 1, "total_price" : 29.98, "order_date" : "2019-08-22T10:06:33", "user\" : {"user_id" : 1, "username\" : "test"}, ...]}
I can't seem to find a fix for this, so is there any proper way to achieve that by using formatJSON(JSONFormat) method...or some other method?
Any help/advice is greatly appreciated.

There's a missing feature to allow for combining the use of JSON/JSONB columns with Result.formatJSON() (or of XML columns with Result.formatXML()): https://github.com/jOOQ/jOOQ/issues/10361
As a workaround, you'll have to do this work yourself, manually, and avoid the formatJSON() method.

After doing some research for appropriate JSON library to achieve what I want I've decided to do it this way (until more convenient method is available in jOOQ 3.13.1):
String strResultFinal = resultR1S.formatJSON(
new JSONFormat()
.header(false)
.recordFormat(RecordFormat.ARRAY)
);
final String fixedJSONString = strResultFinal
.substring(3, strResultFinal.length() - 3)
.replaceAll("\\\\n", "") // for some reason '\n' is being part of String (I presume for new row) and needs to be removed for proper JSON format...
.replaceAll("\\\\", ""); //...as well as escaping backslash character
Now I get desired JSON format like this (BTW, it's trimmed :) ):
{"orders" : [{"order_id" : 1, "total_price" : 29.98, "order_date" : "2019-08-22T10:06:33", "user" : {"user_id" : 1, "username" : "test"}, ..}]}

SpagoBI multi value parameter

I'm trying to create a multi-value parameter in SpagoBI.
Here is my data set query whose last line appears to be causing an issue.
select C."CUSTOMERNAME", C."CITY", D."YEAR", P."NAME"
from "CUSTOMER" C, "DAY" D, "PRODUCT" P, "TRANSACTIONS" T
where C."CUSTOMERID" = T."CUSTOMERID"
and D."DAYID" = T."DAYID"
and P."PRODUCTID" = T."PRODUCTID"
and _CITY_
I created before open script in my dataset which looks like this:
this.queryText = this.queryText.replace(_CITY_, " CUSTOMER.CITY in ( "+params["cp"].value+" ) ");
My parameter is set as string, display type dynamic list box.
When I run the report I'm getting that error.
org.eclipse.birt.report.engine.api.EngineException: There are errors evaluating script "
this.queryText = this.queryText.replace(_CITY_, " CUSTOMER.CITY in ( "+params["cp"].value+" ) ");
":
Fail to execute script in function __bm_beforeOpen(). Source:
Could anyone please help me?

Hello I managed to solve the problem. Here is my code:
var substring = "" ;
var strParamValsSelected=reportContext.getParameterValue("citytext");
substring += "?," + strParamValsSelected ;
this.queryText = this.queryText.replace("'xxx'",substring);
As You can see the "?" is necessary before my parameter. Maybe It will help somebody. Thank You so much for Your comments.

If you are using SpagoBI server and High charts (JFreeChart Engine) / JSChat Engine you can just use ($P{param_url}) in query,
or build dynamic query using Java script / groovy Script
so your query could also be:
select C."CUSTOMERNAME", C."CITY", D."YEAR", P."NAME"
from "CUSTOMER" C, "DAY" D, "PRODUCT" P, "TRANSACTIONS" T
where C."CUSTOMERID" = T."CUSTOMERID"
and D."DAYID" = T."DAYID"
and P."PRODUCTID" = T."PRODUCTID"
and CUSTOMER."CITY" in ('$P{param_url}')

Call MongoDB function from Java

I'm trying to call a stored JavaScript function from the MongoDB Java driver.
I have been following this guide to store the function on the DB server and I'm able to call the function from the mongo shell and have the result returned.
However I cannot figure out how to call the same function in Java?
According to this http://api.mongodb.org/java/current/com/mongodb/DB.html#doEval-java.lang.String-java.lang.Object...- there's a method called doEval
I have also tried to use it with this method:
public static String callFunction() {
try (MongoClient client = new MongoClient("localhost")) {
com.mongodb.DB db = client.getDB("TestDB");
return db.doEval("echoFunction", 3).toString();
}
}
But when I call the method this is what I get:
{ "retval" : { "$code" : "function (x) {\n return x;\n}"} , "ok" : 1.0}
and I would expect to get the number 3 back in this case.
Another problem with the above code is that the method client.getDB() is deprecated. As I understand it the new method to call is client.getDatabase() and it returns a MongoDatabase object, but according to the API there is no method to execute a function.
So my question is: Is it possible to execute a stored JavaScript function on the database server from Java and get back the result of that function? And if it is possible, I would appreciate some help on how to do it?
Thank you.
Edit:
According to a comment on Calling server js function on mongodb from java:
"It seems like getNextSequence is a function written in the mongo
javascript shell. Neither the database (mongod) nor the Java side
knows this function exists and neither is able to interprete the
Javascript code the function contains. You will have to reimplement it
in Java. "
The function I'm trying to implement is a bit more complex than the example above - it's supposed to return a collection of documents and that does not seems to be working using the db.doEval method.
So I guess the comment is correct?

You can do all this with java driver.
MongoClient mongoClient = new MongoClient();
MongoDatabase mdb = mongoClient.getDatabase("TestDB");
/* run this <code snippet> in bootstrap */
BsonDocument echoFunction = new BsonDocument("value",
new BsonJavaScript("function(x1) { return x1; }"));
BsonDocument myAddFunction = new BsonDocument("value",
new BsonJavaScript("function (x, y){ return x + y; }"));
mdb.getCollection("system.js").updateOne(
new Document("_id", "echoFunction"),
new Document("$set", echoFunction),
new UpdateOptions().upsert(true));
mdb.getCollection("system.js").updateOne(
new Document("_id", "myAddFunction"),
new Document("$set", myAddFunction),
new UpdateOptions().upsert(true));
mdb.runCommand(new Document("$eval", "db.loadServerScripts()"));
/* end </code snippet> */
Document doc1 = mdb.runCommand(new Document("$eval", "echoFunction(5)"));
System.out.println(doc1);
The result is also:
Document{{retval=5.0, ok=1.0}}

You should do this instead:
return db.doEval("echoFunction(3)").toString();
If you use just function name in eval you only refer to JavaScript variable on server side storing code of function. It doesn't execute it. When you use parentheses you request to actually execute a function. If you need to send something more complex than a number I would advise to use JSON serializer.

I resolved the same issue in the following way:
I run a command in mongoShell to create my stored JavaScript functions:
db.system.js.save(
{
_id: "echoFunction" ,
value : function(x1) { return x1; }
}
)
db.system.js.save(
{
_id : "myAddFunction" ,
value : function (x, y){ return x + y; }
}
);
db.system.js.save(
{
_id: "fullFillCollumns" ,
value : function () {
for (i = 0; i < 2000; i++) {
db.numbers.save({num:i}); } }
}
);
To execute this functions from MongoDB Java Driver:
MongoClient mongoClient = new MongoClient();
MongoDatabase db = mongoClient.getDatabase("databaseName");
db.runCommand(new Document("$eval", "fullFillCollumns()"));
Document doc1 = db.runCommand(new Document("$eval", "echoFunction(5)"));
System.out.println(doc1);
Document doc2 = db.runCommand(new Document("$eval", "myAddFunction(5,8)"));
System.out.println(doc2);
I see that the collection numbers were created and filled with values. In the IntellijIdea console I see:
Document{{retval=5.0, ok=1.0}}
Document{{retval=13.0, ok=1.0}}

How to query nested mongodb fields efficiently in Java?

I'm not very experienced with running Mongo queries from Java, so I'm no expert at commands. I have a Mongo collection with ~6500 documents, each containing multiple fields (some of which have sub-fields), like below:
"_id" : NumberLong(714847),
"franchiseIds" : [
NumberLong(714848),
NumberLong(714849)
],
"profileSettings" : {
"DISCLAIMER_SETUP" : {
"settingType" : "DISCLAIMER_SETUP",
"attributeMap" : {
...
I want to have an operation which will go through the entire collection from time to time and calculate how many franchiseIds are present, since different documents could have anywhere from 1 to 4 franchiseIds.
From the Mongo shell, I did a very simple script to get this, and it calculated the result immediately:
rs_default:SECONDARY> var totalCount = 0;
rs_default:SECONDARY> db.profiles.find().forEach( function(profile) { totalCount += profile.franchiseIds.length } );
rs_default:SECONDARY> totalCount
However, when I attempted to do the same thing in Java, which is where this would run on the server from time to time, it was much less performant, taking around 15 seconds to complete:
int result = 0;
List<Profile> allProfiles = mongoTemplate.findAll(Profile.class, PROFILE_COLLECTION);
for (Profile profile : allProfiles) {
result += profile.getFranchiseIds().size();
}
return results
I realize the above isn't performant in Java as it's having to allocate memory for all of the Profiles being loaded in. In the Mongo shell script, is Mongo simply taking care of this itself?
Any ideas how I can do something similar in Java?
EDIT:
I returned only the franchiseIds field on the response from Mongo, and that helped significantly. Below is the improved code:
final Query query = new Query();
query.fields().include(FRANCHISE_IDS);
final List<Profile> allProfiles = mongoTemplate.find(query, Profile.class, PROFILE_COLLECTION);
for (Profile profile : allProfiles) {
result += profile.getFranchiseIds().size();
}
return result;

Raw javascript mongodb queries using db.eval() in java

currently I'm working on a project in JAVA, and I need to run the Javascript Mongo queries through JAVA, and I figured I can do something like that using db.eval() in java. Problem is I have the following Javascript query for Mongo, and I have no idea how can I pass the whole Script to the db.eval() method. Any idea ? please let me know.
thank you
var red = function(doc, out) {
out.count_order++;
out.sum_qty += doc.quantity;
out.sum_base_price += doc.extendedprice;
out.sum_disc_price += doc.extendedprice * (1 - doc.discount);
out.sum_charge += doc.extendedprice * (1 - doc.discount) * (1 + doc.tax);
out.avg_disc += doc.discount
};
var avg = function(out) {
out.avg_qty = out.sum_qty / out.count_order;
out.avg_price = out.sum_base_price / out.count_order;
out.avg_disc = out.avg_disc / out.count_order
};
db.deals.group( {
key : { RETURNFLAG : true, LINESTATUS : true},
cond : { "SHIPDATE" : {$lte: new Date(1998, 8, 1)}},
initial: { count_order : 0, sum_qty : 0, sum_base_price : 0, sum_disc_price : 0,
sum_charge : 0, avg_disc : 0},
reduce : red,
finalize : avg
});

Looks like you are trying to do a mapreduce and the Java driver abstracts that process and complicates things, but I think the core of your question is how to 1) store js functions and 2) run them in Java
I will address 2) first via an example (I modified you function to return "out" since I think this would otherwise need to be handled differently for mapreduce operations with the driver):
String fnc = "function(doc, out) {" +
"out.count_order++;"+
"out.sum_qty += doc.quantity;"+
"out.sum_base_price += doc.extendedprice;"+
"out.sum_disc_price += doc.extendedprice * (1 - doc.discount);"+
"out.sum_charge += doc.extendedprice * (1 - doc.discount) * (1 + doc.tax);"+
"out.avg_disc += doc.discount; "+
"return out;" +
"};";
BasicDBObject doc = BasicDBObject("_id","doc");
BasicDBObject out = BasicDBObject("_id","out");
System.out.println(
(new MongoClient()).getDB('test').eval(fnc,doc,out)
);
WRT 1) (saving functions)... you can save functions in the server in collections. But I have not yet been able to get the Java Driver to find them. In the mongoshell, you can do this (this example comes from Call stored function in mongodb):
system.js.save({
_id: "echoFunction",
value: function (x) {
return 'echo: ' + x;
}
})
And run that in the shell with
db.eval("echoFunction('test')")