Develop Reference

JSON Data Limit

We have grails 2.2.4 application running on Tomcat that works with user camera and keystrokes, collects some data on the client side with Javascript and sends using POST.
In the view that collects data we have:
<g:form name="testResultsForm" id="testResultsForm" controller="customer" action="thankYou" method="post">
<h3>Dummy data!</h3>
<input type="text" style="visibility: hidden" name="testResults" id="testResults"/>
<button type="submit" class="btn btn-default">Submit dummy data</button>
</g:form>
In the JS, we assign all camera data to this html element and submit the form:
TestUtils.setValue('testResults', sendData);
$("#testResultsForm").submit();
In the grails controller we have the following line to parse the JSON:
def data = JSON.parse(params.testResults)
Everything works as expected except for when the user takes longer than normal and puts in lots of keystrokes. The errors looks something like:
2014-06-14 01:22:14,323 [http-8443-16] ERROR (org.codehaus.groovy.grails.web.errors.GrailsExceptionResolver) - JSONException occurred when processing request: [POST] /qbcheck/customer/thankYou
Expected a ',' or ']' at character 524288 of {"patkey":"","test_version":"1.4","data_version":"1.3","patientid":"","test_date":"","test_duration":0,"gender":"","dob":"","fov":62,"fps":26,"scale_factor":0,"country":46,"camera_data":{"x":[353,353,353,353,3......
It always fails at character 524288. This led us to investigate that there might be a limit on the amount of data, we looked at Tomcat and found that it allows 2MB data by default using maxpostsize property. Still we updated it to a bigger number just to be sure. Similarly, we tried looking on Grails and JS side but were not able to find any limitation.
Looking for any pointers in this regard. We are able to provide more details as required.

We found that the html "input" has a hard limit of 512 KB. Ideally, we should have received some kind of error/warning when trying to assign more data to an input value through JS. However, that does not happen
So we changed the input which was previously defined as:
<input type="text" style="visibility: hidden" name="testResults" id="testResults"/>
To a text-area:
<textarea style="visibility: hidden" name="testResults" id="testResults"/>
And this allowed us to transfer data greater than 512 KB.

xhtml: using '#' in a value attribute makes the characters after '#' being ignored

First of all, sorry for the bad title but I don't know how to describe it better.
My problem:
I want to display pictures in a <p:lightbox> element. Unfortunately the pictures contain '#' characters in their fileneame, so they look like this for example: desert_#1#.jpg.
Here is my code:
<p:panel id="showPics" closable="false" header="Fotos: ">
<p:lightBox styleClass="imagebox">
<p:dataList value="#{myBean.fotoList}" var="fl" >
<h:outputLink value="#{request.contextPath}/resources/pics/#{fl.PictureName}" title="#{fl.PictureName}" >
<h:graphicImage value="#{request.contextPath}/resources/pics/#{fl.PictureName}"/>
</h:outputLink>
</p:dataList>
</p:lightBox>
</p:panel>
the beanvalue #{fl.PictureName} returns the filename, so in our example desert_#1#.jpg
Now when I'm running my application, I get this error message:
Problem accessing /resources/pics/desert_. Reason: Not Found
So my guess is that the # characters in the picturename are recognized as references (or whatever you call them) to a beanmethod/value, which they of course aren't. Therefore the string after the first '#' in the filename isn't recognized anymore.
Unfortunately I cannot simply change the filename to get rid of the '#'s.
Could somebody tell me how to fix this? Thank you in advance!
UPDATE: I'm using JSF2.0 with Primefaces and Primefaces mobile components (since my application is a mobile web application) and Spring webflow framework. My IDE is Netbeans.

On the server side, encode the bean value using URI percent encoding. In this case, the property should use %23 where # is in the filename.

Data from database not same in java string

Here is my data when i view in SQL Developer tool
introduction
topic 1
topic end
and after i read it using a ResultSet,
ResultSet result = stmt.executeQuery();
result.getString("description")
and display in JSP page as
<bean:write name="data" property="description" />
but it will display like this
introduction topic 1 topic end
how can i keep the display same as in the SQL Developer?

Newlines aren't preserved in HTML. You need to either tell the browser it's preformatted:
<pre>
<bean:write name="data" property="description"/>
</pre>
Or replace the newlines with HTML line breaks. See this question for examples.

how can i keep the display same as in the SQL Developer?
The data presumably contains line breaks, e.g. "\r\n" or "\n". If you look at the source of your JSP, you'll probably see them there. However, HTML doesn't treat those as line breaks for display purposes - you'll need to either use the <br /> tag, or put each line in a separate paragraph, or something similar.
Basically, I don't think this is a database problem at all - I think it's an HTML problem. You can experiment with a static HTML file which you edit locally and display in your browser. Once you know the HTML you want to generate, then work on integrating it into your JSP.

Removing HTML entities while preserving line breaks with JSoup

I have been using JSoup to parse lyrics and it has been great until now, but have run into a problem.
I can use Node.html() to return the full HTML of the desired node, which retains line breaks as such:
Glóandi augu, silfurnátt
<br />Blóð alvöru, starir á
<br />Óður hundur er í vígamóð, í maga... mér
<br />
<br />Kolniður gref, kvik sem dreg hér
<br />Kolniður svart, hvergi bjart né
But has the unfortunate side-effect, as you can see, of retaining HTML entities and tags.
However, if I use Node.text(), I can get a better looking result, free of tags and entities:
Glóandi augu, silfurnátt Blóð alvöru, starir á Óður hundur er í vígamóð, í maga... mér Kolniður gref, kvik sem dreg hér Kolniður svart,
Which has another unfortunate side-effect of removing the line breaks and compressing into a single line.
Simply replacing <br /> from the node before calling Node.text() yields the same result, and it seems that that method is compressing the text onto a single line in the method itself, ignoring newlines.
Is it possible to have the best of both worlds, and have tags and entities replaced correctly which preserving the line breaks, or is there another method or way of decoding entities and removing tags without having to replace them manually?

(disclaimer) I haven't used this API ...
but a quick look at the docs suggests that you could visit each descendent node and dump out its text contents. Breaks could be inserted when special tags like <br> are encountered.
The TextNode.getWholeText() call also looks useful.

based on another answer from stackoverflow I added a few fixes and came with
String text = Jsoup.parse(html.replaceAll("(?i)<br[^>]*>", "br2nl").replaceAll("\n", "br2nl")).text();
text = text.replaceAll("br2nl ", "\n").replaceAll("br2nl", "\n").trim();
Hope this helps

Getting wrong characters in parameter

In files.jsp I am using following anchor and JSTL c:url combination -
<c:url value="downloadfile.jsp" var="dwnUrl" scope="request">
<c:param name="fileType" value="PDF"/>
<c:param name="fileId" value="${file.fileId}"/>
<c:param name="fileName" value="${file.fileName}"/>
</c:url>
Download
On downloadfile.jsp getting the file name value in JavaScript variable as -
selectedFile = <c:out value='${param.fileName}'>
Now, if file name contains some extra character e.g. XYZ 2/3" Technical then on the other page I am getting some different character as - XYZ 2/3#034; Technical
However, if I print request.getParameter("fileName"), its giving correct name. What is wrong?

The <c:out> by default escapes XML entities, such as the doublequote. This is done so to get well-formed XML and to avoid XSS.
To fix this, you should either get rid of <c:out>, since JSP 2.0, EL works perfectly fine in template text as well:
selectedFile = '${param.fileName}';
.. or, if you're still on legacy JSP 1.2 or older, set its escapeXml attribute to false:
selectedFile = '<c:out value="${param.fileName}" escapeXml="false">';
Note that I have added the singlequotes and semicolon to make JS code valid.
Needless to say, you'll need to keep XSS risks in mind if you do so.

The funky characters in your <c:param> values are being URL encoded by <c:url> as they should be. As far as downloadfile.jsp is concerned, the servlet container takes care of URL decoding incoming variables so you don't have to. This is normal behavior and shouldn't pose any problems for you.

If you simply turn escapeXml to false as #BalusC suggests, you will add an XSS vunerability to your page. Instead, you should encode the user input at the time of injection into the destination language, and escape characters that would be evaluated in the destination language. In this case, if the user input contained a single quote character (I'm assuming the string literal in your original example was supposed to be wrapped in single quotes, but the same would be true for double quotes if you were using them), any JavaScript code that followed it would be interpreted by the browser and executed. To safely do what you are trying to do, you should change the line in downloadfile.jsp to:
selectedFile = '${fn:replace(param.fileName, "'", "\'")}';
That will escape only single quotes, which would otherwise end the string literal declaration.
If you were using double quotes, then this would be appropriate:
selectedFile = "${fn:replace(param.fileName, '"', '\"')}";
It is worth noting that escapeXml could be appropriate for escaping JavaScript string literals (and it often is) when the string literal will eventually be dumped into HTML markup. However, in this case, the value should not be XML escaped as it is evaluated in the context of a file path, rather than in the context of HTML.