Empty JSON vs null conversions to a - java

I'm trying to create an HTTP PUT endpoint in Java that takes in a delta json from the front end and I'm having some trouble figuring out how to implement "nulls".
For example if we have a database model that looks like
id : 1
firstname : Luke
lastname : Xu
age : 24
fav_color : Red
And we send over a PUT request to /person/1 with a json of {age : 25}. Currently, I have a JOOQ pojo to converts to following JSON to a java model but the problem is it is also updating my database values to be null.
There's no difference between
{age : 25}
and
{id : 1,
firstname : null,
lastname : null,
age : 25,
fav_color : null}
Once it hits my Java end point the java model just sets both cases to "null" and there's no difference between a passed in null or the value wasn't being passed in at all.
I also considered processing an input stream (type JSON) but the problem with this is that our JSON names have to be named exactly the same as the database column names which is also kind of unreasonable.
What is the standard for editing the database if we only want to send a "delta json"????


Since you're using jOOQ, I'd suggest you directly pass the JSON values to the jOOQ UpdatableRecord, which can in fact distinguish between:
null meaning not initialised (or default)
null meaning null
It does so by maintaining a changed() flag for each individual column.
For instance:
{age : 25}
... translates to this Java code:
// record.set(USER.ID, 1) I suspect this is still necessary...?
record.set(USER.AGE, 25);
record.update();
... and to this SQL statement:
UPDATE users
SET age = 25
WHERE id = 1
whereas
{id : 1,
firstname : null,
lastname : null,
age : 25,
fav_color : null}
... translates to this Java code
record.set(USER.ID, 1);
record.set(USER.FIRSTNAME, null);
record.set(USER.LASTNAME, null);
record.set(USER.AGE, 25);
record.set(USER.FAV_COLOR, null);
... and to this SQL statement
UPDATE users
SET firstname = null,
lastname = null,
age = 25,
fav_color = null
WHERE id = 1

Related

How to properly format generated routine result of JSON type by using formatJSON(JSONFormat) in jOOQ?

I'm having issue with properly formatting my JSON result when querying generated Routine by using jOOQ code generator. I'm trying to perform SELECT-clause on my get_all_orders() method defined in PL/pgSQL (mentioned in this question) which returns result of json type. This is my code for performing jOOQ-fied query:
DSLContext create = DSL.using(connection, SQLDialect.POSTGRES);
Result<Record1<String>> resultR1S = create.select(Routines.getAllOrders()).fetch();
final String strResultFinal = resultR1S.formatJSON(
new JSONFormat().header(false).recordFormat(RecordFormat.ARRAY)
);
...and this is output I get on console (bit truncated at the end because result output is waaaay too long to fit in):
[["{\"orders\" : [{\"order_id\" : 1, \"total_price\" : 29.98, \"order_date\" : \"2019-08-22T10:06:33\", \"user\" : {\"user_id\" : 1, \"username\" : \"test\"}, \"order_items\" : [{\"order_item_id\" : 1, \"amount\" : 1, \"book\" : {\"book_id\" : 1, \"title\" : \"Harry Potter and the Philosopher's Stone\", \"price\" : 29.98, \"amount\" : 400, \"is_deleted\" : false, \"authors\" : [{\"author_id\":4,\"first_name\":\"JK\",\"last_name\":\"Rowling\"}], \"categories\" : [{\"category_id\":2,\"name\":\"Lyric\",\"is_deleted\":false}]}, \"order_id\" : 1, \"total_order_item_price\" : 29.98}]}, {...}"]]
What I'm trying to achieve is to get rid off double angle brackets (at beginning and end of output) and backslash characters so it looks something like this:
{"orders" : [{"order_id" : 1, "total_price" : 29.98, "order_date" : "2019-08-22T10:06:33", "user\" : {"user_id" : 1, "username\" : "test"}, ...]}
I can't seem to find a fix for this, so is there any proper way to achieve that by using formatJSON(JSONFormat) method...or some other method?
Any help/advice is greatly appreciated.

There's a missing feature to allow for combining the use of JSON/JSONB columns with Result.formatJSON() (or of XML columns with Result.formatXML()): https://github.com/jOOQ/jOOQ/issues/10361
As a workaround, you'll have to do this work yourself, manually, and avoid the formatJSON() method.

After doing some research for appropriate JSON library to achieve what I want I've decided to do it this way (until more convenient method is available in jOOQ 3.13.1):
String strResultFinal = resultR1S.formatJSON(
new JSONFormat()
.header(false)
.recordFormat(RecordFormat.ARRAY)
);
final String fixedJSONString = strResultFinal
.substring(3, strResultFinal.length() - 3)
.replaceAll("\\\\n", "") // for some reason '\n' is being part of String (I presume for new row) and needs to be removed for proper JSON format...
.replaceAll("\\\\", ""); //...as well as escaping backslash character
Now I get desired JSON format like this (BTW, it's trimmed :) ):
{"orders" : [{"order_id" : 1, "total_price" : 29.98, "order_date" : "2019-08-22T10:06:33", "user" : {"user_id" : 1, "username" : "test"}, ..}]}

How to create and publish Index using Java Client Programatically

Is it possible to programmatically create and publish secondary indexes using Couchbases Java Client 2.2.2? I want to be able to create and publish my custom secondary indexes Running Couchbase 4.1. I know this is possible to do with Couchbase Views but I can't find the same for indexes.

couchbase-java-client-2.3.1 is needed in order to programmatically create indexes primary or secondary. Some of the usable methods can be found on the bucketManger same that is used to upsert views. Additionally the static method createIndex can be used it support DSL and String syntax
There are a few options to create your secondary indexes.
Option #1:
Statement query = createIndex(name).on(bucket.name(), x(fieldName));
N1qlQueryResult result = bucket.query(N1qlQuery.simple(query));
Option #2:
String query = "BUILD INDEX ON `" + bucket.name() + "` (" + fieldName + ")";
N1qlQueryResult result = bucket.query(N1qlQuery.simple(query));
Option #3 (Actually multiple options here since method createN1qlIndex is overloaded
bucket.bucketManager().createN1qlIndex(indexName, fields, where, true, false);

Primary index:
// Create a N1QL Primary Index (ignore if it exists)
bucket.bucketManager().createN1qlPrimaryIndex(true /* ignore if exists */, false /* defer flag */);
Secondary Index:
// Create a N1QL Index (ignore if it exists)
bucket.bucketManager().createN1qlIndex(
"my_idx_1",
true, //ignoreIfExists
false, //defer
Expression.path("field1.id"),
Expression.path("field2.id"));
or
// Create a N1QL Index (ignore if it exists)
bucket.bucketManager().createN1qlIndex(
"my_idx_2",
true, //ignoreIfExists
false, //defer
new String ("field1.id"),
new String("field2.id"));
The first secondary index (my_idx_1) is helpful if your document is something like this:
{
"field1" : {
"id" : "value"
},
"field2" : {
"id" : "value"
}
}
The second secondary index (my_idx_2) is helpful if your document is something like this:
{
"field1.id" : "value",
"field2.id" : "value"
}

You should be able to do this with any 2.x, once you have a Bucket
bucket.query(N1qlQuery.simple(queryString))
where queryString is something like
String queryString = "CREATE PRIMARY INDEX ON " + bucketName + " USING GSI;";

As of java-client 3.x+ there is a QueryIndexManager(obtained via cluster.queryIndexes()) which provides an indexing API with the below specific methods to create indexes:
createIndex(String bucketName, String indexName, Collection<String> fields)
createIndex(String bucketName, String indexName, Collection<String> fields, CreateQueryIndexOptions options)
createPrimaryIndex(String bucketName)
createPrimaryIndex(String bucketName, CreatePrimaryQueryIndexOptions options)

Cassandra Lucene Index boolean syntax

I am performing a user search system in my Cassandra database. For that purpose I installed Cassandra Lucene Index from Stratio.
I am able to lookup users by username, but the problem is as follows:
This is my Cassandra users table and the Lucene Index:
CREATE TABLE user (
username text PRIMARY KEY,
email text,
password text,
is_verified boolean,
lucene text
);
CREATE CUSTOM INDEX search_main ON user (lucene) USING 'com.stratio.cassandra.lucene.Index' WITH OPTIONS = {
'refresh_seconds': '3600',
'schema': '{
fields : {
username : {type : "string"},
is_verified : {type : "boolean"}
}
}'
};
This is a normal query performed to Lookup a user by username:
SELECT * FROM user WHERE lucene = '{filter: {type : "wildcard", field : "username", value : "*%s*"}}' LIMIT 15;
My Question is:
How could I sort the returned results to ensure that any verified users are between the first 15 results in the query? (Limit is 15).

You can use this search:
SELECT * FROM user WHERE lucene = '{filter: {type:"boolean", must:[
{type : "wildcard", field : "username", value : "*%s*"},
{type : "match", field : "is_verified", value : true}
]}}' LIMIT 15;

Spring data mongodb removes a positional operator from "$unset" update query part

I have a collection of users:
> db.users.find().pretty()
{
"_id" : ObjectId("544ab933e4b099c3cfb62e12"),
"token" : "8c9f8cf4-1689-48ab-bf53-ee071a377f60",
"categories" : [
DBRef("cue_categories", ObjectId("544ab933e4b099c3cfb62e10")),
DBRef("cue_categories", ObjectId("544ab933e4b099c3cfb62e11"))
]
}
I want to find all users who have (let's say) ObjectId("544ab933e4b099c3cfb62e10") category and remove it (because this category was deleted and I don't want users to refer to it anymore).
The valid query to do it in JSON format would be:
db.users.update({
categories:{
$in:[
DBRef("cue_categories", ObjectId("544ab933e4b099c3cfb62e10"))
]
}
},
{
$unset:{
"categories.$":true
}
})
Here's a Spring mongodb query:
Query query = new Query();
query.addCriteria(Criteria.where("categories.$id").in(categoryIds));
Update update = new Update();
update.unset("categories.$");
operations.updateMulti(query, update, User.class);
In order to make an appropriate DB reference I have to provide a list of category IDs, each category ID (in categoryIds) is an instance of org.bson.types.ObjectId.
The problem is that the result query turns out to be without a positional operator:
DEBUG o.s.data.mongodb.core.MongoTemplate - Calling update using
query: { "categories.$id" : { "$in" : [ { "$oid" :
"544ab933e4b099c3cfb62e10"}]}} and update: { "$unset" : { "categories"
: 1}} in collection: users
So the update part must be { "$unset" : { "categories.$" : 1}}
P.S.
I managed to get around by falling back to the plain Java driver use
DBObject query = new BasicDBObject("categories.$id", new BasicDBObject("$in", categoryIds));
DBObject update = new BasicDBObject("$unset", new BasicDBObject("categories.$", true));
operations.getCollection("users").updateMulti(query, update);
But my question still remains open!
P.S.S.
My case is very similar to Update Array Field Using Positional Operator ($) Does Not Work bug and looks like it was fixed for versions 1.4.1 and 1.5. That being said I use spring-data-mongodb version 1.5.1. And I'm confused. Does anybody have a clue?

You can not use positional $ operator with unset as per MongoDB documentation. It will set the value as Null. https://docs.mongodb.com/manual/reference/operator/update/positional/

Problem with filling ext grid with JSON values

I'm new in Ext and I have a problem: I'm trying to fill extjs-grid with data:
Ext.onReady(function() {
var store = new Ext.data.JsonStore({
root: 'topics',
totalProperty: 'totalCount',
idProperty: 'threadid',
remoteSort: true,
autoLoad: true, ///
fields: [
'title', 'forumtitle', 'forumid', 'author',
{name: 'replycount', type: 'int'},
{name: 'lastpost', mapping: 'lastpost', type: 'date', dateFormat: 'timestamp'},
'lastposter', 'excerpt'
],
proxy: new Ext.data.ScriptTagProxy({
url:'http://10.10.10.101:8080/myproject/statusList/getJobs/2-10/search-jobname-/sort-asdf/filterjobname-123/filterusername-davs/filterstatus-completed/filtersubmdate-today',
method : 'GET'
})
});
//
var cm = new Ext.grid.ColumnModel([
{sortable:true, id : 'id', dataIndex:'id'},
{sortable:true, id : 'title', dataIndex:'title'},
{sortable:true, id : 'forumtitle', dataIndex:'forumtitle'},
{sortable:true, id : 'forumid', dataIndex:'forumid'},
{sortable:true, id : 'author', dataIndex:'author'}
]);
var grid = new Ext.grid.GridPanel({
id: 'mainGrid',
el:'mainPageGrid',
pageSize:10,
store:store,
// stripeRows: true,
cm:cm,
stateful: false, // skipSavingSortState
viewConfig:{
forceFit:true
},
// width:1000,
// height:700,
loadMask:true,
frame:false,
bbar: new Ext.PagingToolbar({
id : 'mainGridPaginator',
store:store,
hideRefresh : true,
plugins: new Ext.ux.Andrie.pPageSize({
beforeText: 'View: ',
afterText: '',
addAfter: '-',
variations: [10, 25, 50, 100, 1000]
//comboCfg: {
//id: '${ dispview_widgetId }_bbar_pageSize'
//}
}),
displayMsg: 'Displaying items {0} - {1} of {2}',
emptyMsg:'No data found',
displayInfo:true
})
});
grid.render();
});
and the Java part:
#GET
#Path("/getJobs/{startFrom}-{startTo}/search-{searchType}-{searchName:.*}/" +
"sort-{sortType}/filterjobname-{filterJobName:.*}/filterusername-{filterUsername:.*}/" +
"filterstatus-{filterStatus:.*}/filtersubmdate-{filterSubmittedDate:.*}")
#Produces({"application/json"})
#Encoded
public String getJobs(
#PathParam("startFrom") String startFrom,
#PathParam("startTo") String startTo,
#PathParam("searchType") String searchType,
#PathParam("searchName") String searchName,
#PathParam("sortType") String sortType,
#PathParam("filterJobName") String filterJobName,
#PathParam("filterUsername") String filterUsername,
#PathParam("filterStatus") String filterStatus,
#PathParam("filterSubmittedDate") String filterSubmittedDate) {
return "{totalCount:'3',topics:[{title:'XTemplate with in EditorGridPanel',threadid:'133690',username:'kpremco',userid:'272497',dateline:'1305604761',postid:'602876',forumtitle:'Ext 3x Help',forumid:'40',replycount:'2',lastpost:'1305857807',lastposter:'kpremco',excerpt:'Hi I have an EditiorGridPanel whose one column i am using XTemplate to render and another Column is Combo Box FieldWhen i render the EditorGri'}," +
"{title:'IFrame error _flyweights is undefined',threadid:'133571',username:'Daz',userid:'52119',dateline:'1305533577',postid:'602456',forumtitle:'Ext 3x Help',forumid:'40',replycount:'1',lastpost:'1305857313',lastposter:'Daz',excerpt:'For Ext 330 using Firefox 4 Firebug, the following error is often happening when our app loads e._flyweights is undefined Yetthis '}," +
"{title:'hellllllllllllllpwhy it doesnt fire cellclick event after I change the cell value',threadid:'133827',username:'aimer311',userid:'162000',dateline:'1305700219',postid:'603309',forumtitle:'Ext 3x Help',forumid:'40',replycount:'3',lastpost:'1305856996',lastposter:'aimer311',excerpt:'okI will discribe this problem as more detail as I canI look into this problem for a whole dayI set clicksToEdit1 to a EditorGridPanelso when I'}]}";
As a result I'm getting a JavaScript error:
Syntax error at line 1 while loading:
totalCount:'3',topics:[{title:'XTemplate
---------------------^
expected ';', got ':'
Although, when I'm using Proxy's URL:
URL: 'http://extjs.com/forum/topics-browse-remote.php',
which represents same information, I don't have any problems.
Where is my failure????
P.S. Comments for the first answer:
return "{\"totalCount\":\"3\",\"topics\":[{\"title\":\"XTemplate with in EditorGridPanel\",\"threadid\":\"133690\",\"username\":\"kpremco\",\"userid\":\"272497\",\"dateline\":\"1305604761\",\"postid\":\"602876\",\"forumtitle\":\"Ext 3x Help\",\"forumid\":\"40\",\"replycount\":\"2\",\"lastpost\":\"1305857807\",\"lastposter\":\"kpremco\",\"excerpt\":\"Hi I have an EditiorGridPanel whose one column i am using XTemplate to render and another Column is Combo Box FieldWhen i render the EditorGri\"}," +
"{\"title\":\"IFrame error _flyweights is undefined\",\"threadid\":\"133571\",\"username\":\"Daz\",\"userid\":\"52119\",\"dateline\":\"1305533577\",\"postid\":\"602456\",\"forumtitle\":\"Ext 3x Help\",\"forumid\":\"40\",\"replycount\":\"1\",\"lastpost\":\"1305857313\",\"lastposter\":\"Daz\",\"excerpt\":\"For Ext 330 using Firefox 4 Firebug, the following error is often happening when our app loads e._flyweights is undefined Yet, this \"}," +
"{\"title\":\"hellllllllllllllpwhy it doesn't fire cellclick event after I change the cell value\",\"threadid\":\"133827\",\"username\":\"aimer311\",\"userid\":\"162000\",\"dateline\":\"1305700219\",\"postid\":\"603309\",\"forumtitle\":\"Ext 3x Help\",\"forumid\":\"40\",\"replycount\":\"3\",\"lastpost\":\"1305856996\",\"lastposter\":\"aimer311\",\"excerpt\":\"okI will discribe this problem as more detail as I canI look into this problem for a whole dayI set clicksToEdit1 to a EditorGridPanelso when I\"}]}";
I've got the following error:
Syntax error at line 1 while loading:
{"totalCount":"3","topics":[{"title
-------------^
expected ';', got ':'
P.S. #2. When I've added '[' to the begining of the response string and ']' to the end , erros disappered, but grid hasn't been filled with data

You're not returning (valid) JSON. Refer to the JSON site for details, but for instance, all property keys must be in double quotes. (All strings must also be in double quotes; single quotes are not valid for JSON strings.)
So for instance, this is not valid JSON:
{totalCount:'3'}
...because the key is not in quotes, and the value is using single quotes. The correct JSON would be:
{"totalCount":"3"}
...if you really want the 3 to be a string, or:
{"totalCount":3}
...if the 3 should be a number.
People frequently confuse JSON and JavaScript's object literal notation, but they are different. Specifically, JSON is a subset of object literal notation. A lot of things that are valid in object literal notation are not valid in JSON. Any time you're in doubt, you can check at jsonlint.com, which provides a proper JSON validator.

I have found the root of issue.
As I've known Ext send to my web service function with parameter 'callback=[some_callback_name]' (e.g. callback1001). It means that Extjs wanna to get results not just in JSON format, but in format 'callback1001()'. When I've return my data in this format everything became good.
Proof links:
http://www.sencha.com/forum/showthread.php?22990-Json-Invalid-label (response #6)
http://indiandeve.wordpress.com/2009/12/02/extjs-error-invalid-label-error-while-using-scripttagproxy-for-json-data-in-paging-grid-example/

Categories

Resources