I have picked up a legacy project and I am in the process of debugging something.
I have come across a custom JavaScript function (encodeJsonIntoString) which encode URI components for JavaScript object before sending over via AJAX.
The AJAX is nothing fancy:
$.ajax({
url: URL,
method: 'POST',
datatype : 'json',
data : encodeJsonIntoString(myObj),
success: ...
});
There is no custom processData or contentType set in the ajax call. What really puzzle me is why the previous developers didn't let $.ajax's data attribute to convert the JavaScript object automatically into a URI-encoded string or didn't even try using JQuery.param() to do it but to write the whole function themselves.
For a test, I have made a simple object to test the function encodeJsonIntoString:
var testDataA = {
list: [
{
lastname:"Smith",
firstname:"John"
},
{
lastname:"Black",
firstname:"Jack"
},
{
lastname:null,
firstname:"Mary"
}
]
};
After decoding URI components, the result of the function is:
list[0][lastname=Smith&list[0][firstname=John&
list[1][lastname=Black&list[1][firstname=Jack&
list[2][lastname=null&list[2][firstname=Mary
Notice there are lack of closing square brackets(]) in some places and it uses "null" for null values.
If I run JQuery.param() and decode it, I get this:
list[0][lastname]=Smith&list[0][firstname]=John&
list[1][lastname]=Black&list[1][firstname]=Jack&
list[2][lastname]=&list[2][firstname]=Mary
See the difference? But somehow the result of the function is accepted by the server(Java/Spring - #ModelAttribute) and read into the correct list structure.
I don't have access to the server side here, but I wonder if that array syntax is correctly acceptable or is it just "tolerated" by the server? Will the server see both versions of object in the same structure format?
I am tempted to just replace it with JQuery.param() to handle more robust input data in the future which may also accept special characters.
I have a NodeJS cloud function that takes a bunch of javascript objects (entries from a db) in the form of an array, and sends them to the client.
On the server I do this using:
return JSON.stringify(result);
Where "result" is the array of JS objects. Then I send data to the client.
In my Android client, I receive the String, and need to iterate through every object in the original array and process them separately. I can't! I always get errors like:
I: [Batch] com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Expected name at line 1 column 3 path $[0].
On my Android client, I have tried:
WebItemEntry [] items = new Gson().fromJson(jsonString, WebItemEntry[].class);
AND ...
ArrayList<WebItemEntry> items = new Gson().fromJson(jsonString, new TypeToken<ArrayList<WebItemEntry>>(){}.getType());
Neither seem to work. Same error as above.
The raw output I get from the database is the JSON string that consists of five individual entries that I would like to iterate through and parse. Its kind of a mess, but you can see its deliminator is the brace {. I really want to use GSON or similar in my android client to parse these entries individually and convert them to my custom Java class: WebItemEntry.
[{\"charityID\":0,\"purchaserZIP\":\"\",\"category\":\"Women\u0027s Accessories\",\"valueCents\":0}, {... same thing for a different entry here...}, {new entry ... }]
On the server ignore stringify and return plain object, those \" characters appear when js object stringify twice. so replace return JSON.stringify(result); with return result;.
I am working with a some multi stage forms in my Scala Play application at the moment, the end result of this multi step form is to send a POST request to an end point with this JSON structure,
{{ "name":"Company Name", "contact": { "firstname":"Firstname1", "surname":"Surname1", "email":"firstname1.surname1#xyz.com", "textPhone":false, "phone":"12222222222222" }, "address": { "addressLine1":"Address Line 1", "town":"Town1", "county":"County", "postcode":"LS1 3DE" }
}
For each form submission I am doing the following,
request.session + ("organisation_name" -> formData.toString())
Is there away that I can have this JSON structure in a session and push the data to the correct attributes? Or is there a way that I can take the session data and manipulate it into JSON that follows the above format?
One way to add something to a session is like this:
request.session.copy(
data = request.session.data + ("organisation_name" -> formData.toString())
)
Another way to add to a session at the return point is like this:
Redirect(routes.......).addingToSession("organisation_name" -> formData.toString())
Have tried Storing your JSON object in request session.
Or u can try caching the JSON object with the Timestamp then read it from cache map so when u go back to the previous from u can Repopululate by getting its attributes.
I need to send a request using Java to an existing REST service using GET with multiple input parameters.
If the initial url for the calculation I want is:
https://api.restservice123.com/api/calculate
I want to make a calc object e with the following parameters:
calcObj e = new calcObj();
e.token="token_ABC123";
e.country="US";
e.amount = 100;
e.price = 24;
e.customer = "bob";
the url should look something like this:
https://api.restservice123.com/api/calculate?token=token_ABC123&country=US&amount=100&price=24&customer=bob
Is there any framework that will combine the parameters from the calc object and reformat them into the url appropirate format and combine them with the api url?
I ended up making a method in the calc object that puts all the non null parameters into a list of strings and combines them using a Joiner from google common and connecting to the url using an HttpURLConnection. But this method looks bad as I'm hoping there's something out there that can already do all of this much more elegantly, but I couldn't find it.
The URIBuilder class works well for this. It has a method setParameters(java.util.List) that takes a list of name value pairs and builds a URI from them.
Background
We are building a Restful API that should return data objects as JSON. In most of the cases it fine just to return the data object, but in some cases, f.ex. pagination or validation, we need to add some metadata to the response.
What we have so far
We have wrapped all json responses like this example:
{
"metadata" :{
"status": 200|500,
"msg": "Some message here",
"next": "http://api.domain.com/users/10/20"
...
},
"data" :{
"id": 1001,
"name": "Bob"
}
}
Pros
We can add helpful metadata to the response
Cons
In most cases we don't need the metadata field, and it adds complexity to the json format
Since it's not a data object any more, but more like a enveloped response, we can not use the response right away in f.ex backbone.js without extracting the data object.
Question
What is the best practices to add metadata to a json response?
UPDATE
What I've got so far from answers below:
Remove the metadata.status an return the http response code in the
http protocol instead (200, 500 ...)
Add error msg to body of an http 500 repsonse
For pagination i natural to have some metadata telling about the pagination structure, and the data nested in that structure
Small amount of meta data can be added to http header (X-something)
You have several means to pass metadata in a RESTful API:
Http Status Code
Headers
Response Body
For the metadata.status, use the Http Status Code, that's what's for!
If metadata is refers to the whole response you could add it as header fields.
If metadata refers only to part of the response, you will have to embed the metadata as part of the object.DON'T wrap the whole response in an artifical envelope and split the wrapper in data and metadata.
And finally, be consistent across your API with the choices you make.
A good example is a GET on a whole collection with pagination. GET /items
You could return the collection size, and current page in custom headers. And pagination links in standard Link Header:
Link: <https://api.mydomain.com/v1/items?limit=25&offset=25>; rel=next
The problem with this approach is when you need to add metadata referencing specific elements in the response. In that case just embed it in the object itself. And to have a consistent approach...add always all metadata to response. So coming back to the GET /items, imagine that each item has created and updated metadata:
{
items:[
{
"id":"w67e87898dnkwu4752igd",
"message" : "some content",
"_created": "2014-02-14T10:07:39.574Z",
"_updated": "2014-02-14T10:07:39.574Z"
},
......
{
"id":"asjdfiu3748hiuqdh",
"message" : "some other content",
"_created": "2014-02-14T10:07:39.574Z",
"_updated": "2014-02-14T10:07:39.574Z"
}
],
"_total" :133,
"_links" :[
{
"next" :{
href : "https://api.mydomain.com/v1/items?limit=25&offset=25"
}
]
}
Note that a collection response is an special case. If you add metadata to a collection, the collection can no longer be returned as an array, it must be an object with an array in it. Why an object? because you want to add some metadata attributes.
Compare with the metadata in the individual items. Nothing close to wrapping the entity. You just add some attributes to the resource.
One convention is to differentiate control or metadata fields. You could prefix those fields with an underscore.
Along the lines of #Charlie's comment: for the pagination part of your question you still need to bake the metadata into the response somhow, but the status and message attributes here are somewhat redundant, since they are already covered by the HTTP protocol itself (status 200 - model found, 404 - model not found, 403 - insufficient privs, you get the idea) (see spec). Even if your server returns an error condition you can still send the message part as the response body. These two fields will cover quite much of your metadata needs.
Personally, I have tended towards (ab)using custom HTTP headers for smaller pieces of metadata (with an X- prefix), but I guess the limit where that gets unpractical is pretty low.
I've expanded a bit about this in a question with a smaller scope, but I think the points are still valid for this question.
I suggest you to read this page https://www.odata.org/ You are not forced to use OData but the way they do the work is a good example of good practice with REST.
We had the same use case, in which we needed to add pagination metadata to a JSON response. We ended up creating a collection type in Backbone that could handle this data, and a lightweight wrapper on the Rails side. This example just adds the meta data to the collection object for reference by the view.
So we created a Backbone Collection class something like this
// Example response:
// { num_pages: 4, limit_value: 25, current_page: 1, total_count: 97
// records: [{...}, {...}] }
PageableCollection = Backbone.Collection.extend({
parse: function(resp, xhr) {
this.numPages = resp.num_pages;
this.limitValue = resp.limit_value;
this.currentPage = resp.current_page;
this.totalCount = resp.total_count;
return resp.records;
}
});
And then we created this simple class on the Rails side, to emit the meta data when paginated with Kaminari
class PageableCollection
def initialize (collection)
#collection = collection
end
def as_json(opts = {})
{
:num_pages => #collection.num_pages
:limit_value => #collection.limit_value
:current_page => #collection.current_page,
:total_count => #collection.total_count
:records => #collection.to_a.as_json(opts)
}
end
end
You use it in a controller like this
class ThingsController < ApplicationController
def index
#things = Thing.all.page params[:page]
render :json => PageableCollection.new(#things)
end
end
Enjoy. Hope you find it useful.
How about returning directly the object that you want in data, like return:
{
"id": 1001,
"name": "Bob"
}
And return in headers the metadata.
Option 1 (one header for all metadata JSON):
X-METADATA = '{"status": 200|500,"msg": "Some message here","next": "http://api.domain.com/users/10/20"...}'
Option 2 (one header per each metadata field):
X-METADATA-STATUS = 200|500
X-METADATA-MSG = "Some message here",
X-METADATA-NEXT = "http://api.domain.com/users/10/20"
...
Until now I was using like you, a complex JSON with two fields, one for data and one for metadata. But I'm thinking in starting using this way that I suggested, I think it will be more easy.
Remind that some server have size limit for HTTP headers, like this example: https://www.tutorialspoint.com/What-is-the-maximum-size-of-HTTP-header-values
JSON:API solves this by defining top-level meta and data properties.