I'm looking for a good web framework for compositing multiple JSON sources fetched with HTTP requests in to static HTML. I am not looking to do this on the client-side (browser, javascript), I am looking for the best server-side solution.
So, what I need to do is:
Fetch several different JSON documents over HTTP
Format that JSON as HTML content, mostly with templates but some dynamic custom HTML
Basic login/logout/preferences customization, nothing major
Mostly stateless pages; what state there is, comes already in the JSON
User / search engine friendly / bookmarkable URLs; should be customizable accurately
How I'd like to do it:
A lean solution, perhaps just a template engine
HTML templates that have no custom syntax over HTML/XML, like Wicket and almost like Tapestry
Application server that is scalable and utilizes multiple CPUs properly (for example, a single Python process does not)
Preferably Java, but if Java doesn't have anything that fits, willing to consider others
As for the template part, if this were to be in JavaScript in the browser, something like PURE would be my tool of choice.
You might want to check out RESTx. That's an open source platform for the easy creation of RESTful resources. It allows you to write custom data access and integration logic in either Java or Python. Getting data from multiple sources and combining them is what it's made for, so this should be a pretty close fit. Data output is rendered according to what the user requested. For example, a further JSON data source, or the same data rendered in HTML.
The HTML rendering is currently according to a built-in template. However, that should be easy enough to modify. I'm one of the developers on that project, so if you need some special template functionality, let me know and I will see what I can do.
To give you an example: Assume you have two JSON resources, in your code you would write this (I'm giving a Python example here, but the Java example would look very similar):
status, data_1 = accessResource("/resource/some_resource")
status, data_2 = accessResource("/resource/some_other_resource")
# data_1 and data_2 now hold a dictionary, list, etc., depending on the JSON
# that was returned.
# ... some code that combines and processes the data and produces a dict or list
# with the result. The data object you return here is then automatically rendered
# in either HTML or JSON, depending on the client request.
return Result.ok(data)
Also take a look at the example for some simple data integration here.
I think that the only framework you need is a library that reads json. The templates can very well be standard jsp pages.
Related
I am interfacing with Shopify and they use RESTful API. When I request a resource that returns an array of items, they use RFC8288 pagination format.
For example, https://example.com/api/inventory_levels.json?limit=10 returns 10 entities along with the following response header:
Link: <https://example.com/api/inventory_levels.json?limit=10&page_info=eyJs9pZHMiO>;
rel="previous", <https://example.com/api/inventory_levels.json?limit=10&page_info=MiZHeyJs9pO>; rel="next"
Appearantly if I want to retrieve all entites from that resource I need to iterate through the 'next' URL until there's no more 'next' returning. But how am I going to parse these info using JAVA or C# code? I could use a regular expression like <(?<next_url>.*)>; rel="next" to retrieve the 'next_url' from it. But it feels like re-inventing the wheel and not robust.
If this is a well-defined feature, shouldn't there be a readily available library/infrastructure that could be used? I just don't want to be caught by surprise if one day the formatting shows up different (like having an extra space and such) and, whilst abiding to the RFC defination, breaks my hastily scrambled up RegEx solution.
Suggestion welcome for Java or C#.
Per my understanding JSP is something to serve to the client. But is it possible to use JSP simply as a template to dynamically assemble an html page, which I then serve to the client? What I mean is this
A servlet receives the call from the user
After some computation, my servlet calls the JSP to assemble the html page dynamically
The servlet gets or converts the JSP "result" (the resulting html page) to a String
The servlet can now do whatever it wants with that String. It can return it as an html webpage or it can store it in a database, or whatever. After all, the string here is a proper html page/text.
For comparison, Python has Jinja2, which does exactly what I just explained. The closest thing to Jinja2 in Java seems to be JSP.
I need a template to assemble html pages dynamically. If I can use the JSP as above then that will solve my problem in Java. Notice that I don't care for JSP per se. I just need a template similar to Jinja2 (if I could use Jinja2 in Java on App-Engine that would be ideal). Also I am very new to JSP. So if you have an answer, please format it as an example; that would be truly helpful.
I am migrating from Python App-Engine to Java App-Engine for business reasons.
This is possible, but you'll need to jump through quite a few hoops, the details of which are dependent on the specific container - in this case appengine.
A quick summary:
create a fake httpservletresponse, wrapping an output stream you access after rendering. You cannot use a httpservletresponsewrapper, even though the spec permits it this environment won't
store all request attributes in a map, you'll restore these afterwards in case they've been mutated
use requestdispatcher.include, passing in the real request and your synthetic response
restore request attributes
read string from the outputstream
Be particularly careful of side effects to your request/response, for example the constraints around only calling one of getwriter or getoutputstream, as well as finalizing the request (setting status or content length)
Or just use one of velocity, handlebars, freemarker or the various other Java templating languages. They'll all be much more straightforward.
I am using PlayFramework2 and I can't find a way to properly handle HTML escaping.
In the template system, HTML entities are filtered by default.
But when I use REST requests with Backbone.js, my JSON objects are not filtered.
I use play.libs.Json.toJson(myModel) to transform an Object into a String.
So, in my controller, I use return ok(Json.toJson(myModel)); to send the response ... but here, the attributes of my model are not secured.
I can't find a way to handle it ...
Second question :
The template engine filters HTML entities by default, this means that we have to store into our database the raw user inputs.
Is it a save behaviour ?
Third questdion :
Is there in the PlayFramework a function to manualy escape strings ? All those I can find require to add new dependencies.
Thanks !
Edit : I found a way at the Backbone.js templating level :
- Use myBackboneModel.escape('attr'); instead of myBackboneModel.get('attr');
Underscore.js templating system also includes that options : <%= attr %> renders without escaping but <%- attr %> renders with escaping !
Just be careful to the efficiency, strings are re-escaped at each rendering. That's why the Backbone .create() should be prefered.
The best practices on XSS-attacks prevention usually recommend you to reason about your output rather than your input. There's a number of reasons behind that. In my opinion the most important are:
It doesn't make any sense to reason about escaping something unless you exactly know how you are going to output/render your data. Because different ways of rendering will require different escaping strategies, e.g. properly escaped HTML string is not good enough to use it in Javascript block. Requirements and technologies change constantly, today you render your data one way - tomorrow you might be using another (let's say you will be working on a mobile client which doesn't require HTML-escaping, because it doesn't use HTML at all to render data) You can only be sure about proper escaping strategy while rendering your data. This is why modern frameworks delegate escaping to templating engines. I'd recommend reviewing the following article: XSS (Cross Site Scripting) Prevention Cheat Sheet
Escaping user's input is actually a destructive/lossy operation – if you escape user's input before persisting it to a storage you will never find out what was his original input. There's no deterministic way to 'unescape' HTML-escaped string, consider my mobile client example above.
That is why I believe that the right way to go would be to delegate escaping to your templating engines (i.e. Play and JS-templating engine you're using for Backbone). There's no need to HTML-escape string you serialize to JSON. Notice that behind the scenes JSON-serializer will JSON-escape your strings, e.g. if you have a quote in your string it will be properly escaped to ensure resulting JSON is correct, because it's a JSON serializer after all that's why it only cares about proper JSON rendering, it knows nothing about HTML (and it shouldn't). However when you rendering your JSON data in the client side you should properly HTML-escape it using the functionality provided by the JS-templating engine you're using for Backbone.
Answering another question: you can use play.api.templates.HtmlFormat to escape raw HTML-string manually:
import play.api.templates.HtmlFormat
...
HtmlFormat.escape("<b>hello</b>").toString()
// -> <b>hello</b>
If you really need to make JSON-encoder escape certain HTML strings, a good idea might be to create a wrapper for them, let's say RawString and provide custom Format[RawString] which will also HTML-escape a string in its writes method. For details see: play.api.libs.json API documentation
Just wondering which is best here. I want to output data from a table in my DB then put a lot of this data into a html table on the fly on my page. I'm working with Java on the server side. Basically I pull the results form the DB and have the raw data..just what next?
There is a chance I may want to take data from multiple tables in order to combine it into one table for my site.
I retrieve the results of the query from the DB, now do I create a text from it in the form of json which I can parse as json using jquery upon the return of the object to my browser?(kind of a sub question of this question: Is just using a stringbuilder the correct way to make a json object to output?)
Or..
Should I build the HTML as a string and output that to the browser instead?
Which is better and why?
I've built entire pages from JSON data on the client. It reduces the redundancy of repeating HTML and can lead to better performance, depending on the complexity of your HTML.
I had large a catalog that used multiple tabs for different sections. Sending it all to the client as JSON and generating the resulting HTML was way faster than downloading the equivalent HTML.
What you lose, of course, is SEO. Search engines won't be able to see the Javascript-generated output. There are ways around this, using hash URL techniques.
I used to be in favor of generating HTML on the server so that the client can be dumb and simply inject dynamic content. The pragmatic real world advantages for our small team was that we needed to be experts at fewer different technologies. We focused on the middle tier and back end and spent less time on the front end.
Lately, with tools like jQuery, it is easier and easier to do more robust client stuff without having to increase the dev bandwidth much. From a client side, I can say building dynamic HTML from JSON using jQuery isn't that hard.
From the server side, I'm sure there are tools to serialize to JSON. I wouldn't roll your own with StringBuilder. Sorry, I'm not a Java guy so don't have a recommendation.
I'd go with JSON if I knew I had anything more than just static views of the data in mind later on.
But if it's just so that you can see what the result was, and don't care too much for the data then I'd go with the straight forward HTML output.
For actually generating the JSON server-side, there are a number of libraries you can use. org.json is the canonical one, but I prefer Stringtree personally.
JSON stands for JavaScript Object Notation. But how come languages like php, java, c etc can also communication each other with json.
What I want to know is that, am i correct to say that json is not limited to js only, but served as a protocol for applications to communicate with each other over the network, which is the same purpose as XML?
JSON cannot handle complex data hierarchies like XML can (attributes, namespaces, etc.), but on the other hand you don't get the same overhead with JSON as you get with XML (if you don't need the complex data structures).
Since JSON is plain text with a special notation for JS to interpret, it's an easy protocol to adopt in other languages.
It is easy for a JS script to parse JSON, since it can be done using 'eval' in which the JS enginge can use its full power.
On the other hand, it is more complicated to generate JSON from within JS. Usually one uses the JSON package from www.json.org in which an object can easily be serialised using JSON.stringify, but it is implemented in JS so its not running with optimal performance.
So serialising JSON is about the same complexity using JS as when using Java, PHP or any other server side language.
Therefore, in my opinion, JSON is best suited when there is asymmetry between produce/consumer e.g. a web server that generates a lot of data that is consumed by the web application. Not the other way around.
But! When one choses JSON as data format it should be used in both directions, not XML<>JSON. Except for when simple get requests are used to retrieve JSON data.
yes, JSON is also wildly used as a data exchange protocol much like XML.
Typically a program (not written in JavaScript) needs a JSON library to parse and create JSON objects (although you can probably create them even without one).
Your right - it's a light weight data interchange format -- more details at: http://www.json.org
You are completely correct. JSON definition of how data should be formatted. It is more light weight than XML and therefore well suited to things like AJAX where you want to send data back and forth to the server quickly.