Develop Reference

How to post a big string/json using AJAX on Play Framework 1.4.x

I have a JSON that looks more or less like this:
{"id":"id","date":"date","csvdata":"csvdata".....}
where csvdata property is a big amount of data in JSON format too.
I was trying to POST this JSON using AJAX in Play! Framework 1.4.x so I sended just like that, but when I receive the data in the server side, the csvdata looks like [object Object] and stores it in my db.
My first thought to solve this was to send the csvdata json in string format to store it like a longtext, but when I try to do this, my request fails with the following error:
413 (Request Entity Too Large)
And Play's console show me this message:
Number of request parameters 3623 is higher than maximum of 1000, aborting. Can be configured using 'http.maxParams'
I also tried to add http.maxParams=5000 in application.conf but the only result is that Play's console says nothing and in my database this field is stored as null.
Can anyone help me, or maybe suggest another solution to my problem?
Thanks you so much in advance.

Is it possible that you sent "csvdata" as an array, not a string? Each element in the array would be a separate parameter. I have sent 100KB strings using AJAX and not run into the http.maxParams limit. You can check the contents of the request body using your browser's developer tools.
If your csvdata originates as a file on the client's machine, then the easiest way to send it is as a File. Your controller action would look like:
public static void upload(String id, Date date, File csv) {
...
}
When Play! binds a parameter to the File type, it writes the contents of the parameter to a temporary file which you can read in. (This avoids running out of memory if a large file is uploaded.) The File parameter type was designed for a normal form submit, but I have used it in AJAX when the browser supported some HTML5 features (File API and Form Data).

Java: Upload file and get back string (contents) of file

Hi I have GWT client with standard server-side Servlets.
I can upload file from GWT client-side and read it's contents at server-side
I can send it back to client as String
BUT
I have GWT FormPanel with action (myModule+"import"). FormPanel invokes POST from servlet. Browser then redirects me to myurl/import so I can see contents of uploaded file.
This is not what I wanted though. I'd simply like to have my String back. I added
submitCompleteHandler to my FormPanel, but it doesn't log any results.
I noticed that servlets have method such setContentType so I tried text/html, text/plain ... I don't know what should be there ...
To say it in one sentence, I want to send String back to client from servlet without having browser to redirect me somewhere else. Is it possible?

Since you are submitting a form you get your browser to change navigation. In order to make it work the way you want you have to send the file with ajax. For GWT there is the GWTUpload library that allows you to do that.

If the browser redirects you, it's because you gave a "target" to the FormPanel. By default, it submits within an hidden iframe (a.k.a "ajax upload").
As said in the javadoc, you have to setContentType("text/html") in your servlet if you want onSubmitComplete to be reliably called.
onSubmitComplete's results is the returned HTML's body innerHTML so you have to be very careful when sending back values with < or & in them. The only reliable way to get them back is to escape them on the server-side, and unescape them on the client-side. You can either use your own escaping mechanism, or you can use < and &. In the latter case, to unescape on the client-side, you'd either use String#replace, or create an HTML element, set it's innerHTML with the string you got back, and then get its innerText:
public String htmlUnescape(String htmlEscaped) {
Element tmp = Document.get().createDivElement();
tmp.setInnerHTML(htmlEscaped);
return tmp.getInnerText();
}
On the server-side, you'd use:
escaped = content.replace("&", "&").replace("<", "<")
(order matters here if you don't want <s to become &lt;; also, replacing < and & is enough, > and " won't cause any issue here)
In your case however, make sure first that the file's content is "text" and not "binary", as it wouldn't make sense to return it as String could cause issues depending on how you use the value on the client side.

Selenium 2: Detect content type of link destinations

I am using the Selenium 2 Java API to interact with web pages. My question is: How can i detect the content type of link destinations?
Basically, this is the background: Before clicking a link, i want to be sure that the response is an HTML file. If not, i need to handle it in another way. So, let's say there is a download link for a PDF file. The application should directly read the contents of that URL instead of opening it in the browser.
The goal is to have an application which automatically knows wheather the current location is an HTML, PDF, XML or whatever to use appropriate parsers to extract useful information out of the documents.
Update
Added bounty: Will reward it to the best solution which allows me to get the content type of a given URL.

As Jochen suggests, the way to get the Content-type without also downloading the content is HTTP HEAD, and the selenium webdrivers does not seem to offer functionality like that. You'll have to find another library to help you with fetching the content type of an url.
A Java library that can do this is Apache HttpComponents, especially HttpClient.
(The following code is untested)
HttpClient httpclient = new DefaultHttpClient();
HttpHead httphead = new HttpHead("http://foo/bar");
HttpResponse response = httpclient.execute(httphead);
BasicHeader contenttypeheader = response.getFirstHeader("Content-Type");
System.out.println(contenttypeheader);
The project publishes JavaDoc for HttpClient, the documentation for the HttpClient interface contains a nice example.

You can figure out the content type will processing the data coming in.
Not sure why you need to figure this out first.
If so, use the HEAD method and look at the Content-Type header.

You can retrieve all the URLs from the DOM, and then parse the last few characters of each URL (using a java regex) to determine the link type.
You can parse characters proceeding the last dot. For example, in the url http://yoursite.com/whatever/test.pdf, extract the pdf, and enforce your test logic accordingly.
Am I oversimplifying your problem?

how to use XML sent by html form?

i have html form with textarea in which i paste some XML, for example:
<network ip_addr="10.0.0.0/8" save_ip="true">
<subnet interf_used="200" name="lan1" />
<subnet interf_used="254" name="lan2" />
</network>
When user submit form, that data is send to Java server, so in headers i get something like that:
GET /?we=%3Cnetwork+ip_addr%3D%2210.0.0.0%2F8%22+save_ip%3D%22true%22%3E%0D%0A%3Csubnet+interf_used%3D%22200%22+name%3D%22lan1%22+%2F%3E%0D%0A%3Csubnet+interf_used%3D%22254%22+name%3D%22lan2%22+%2F%3E%0D%0A%3C%2Fnetwork%3E HTTP/1.1
how can i use that in my Java applications? I need to make some calculations on that data and re-send new generated XML.

This answer shows how to use the URLDecoder/URLEncoder classes to decode and encode url strings. It should work if you passed the 'GET' string to the URLDecoders decode method.
To answer your following question (comment)
First you need to extract this xml based response from the url string. Maybe it's enough to create a substring starting with the first < char.
The String should be fed into a XML parser to create a DOM document. The last easy task would be walking through that document and copying the values to your internal network model.
Do not think about using RegExp to extract the data. Use a parser.

Running a JavaScript command from MATLAB to fetch a PDF file

I'm currently writing some MATLAB code to interact with my company's internal reports database. So far I can access the HTML abstract page using code which looks like this:
import com.mathworks.mde.desk.*;
wb=com.mathworks.mde.webbrowser.WebBrowser.createBrowser;
wb.setCurrentLocation(ReportURL(8:end));
pause(1);
s={};
while isempty(s)
s=char(wb.getHtmlText);
pause(.1);
end
desk=MLDesktop.getInstance;
desk.removeClient(wb);
I can extract out various bits of information from the HTML text which ends up in the variable s, however the PDF of the report is accessed via what I believe is a JavaScript command (onClick="gotoFulltext('','[Report Number]')").
Any ideas as to how I execute this JavaScript command and get the contents of the PDF file into a MATLAB variable?
(MATLAB sits on top of Java, so I believe a Java solution would work...)

I think you should take a look at the JavaScript that is being called and see what the final request to the webserver looks like.
You can do this quite easily in Firefox using the FireBug plugin.
https://addons.mozilla.org/en-US/firefox/addon/1843
Once you have found the real server request then you can just request this URL or post to this URL instead of trying to run the JavaScript.

Once you have gotten the correct URL (a la the answer from pjp), your next problem is to "get the contents of the PDF file into a MATLAB variable". Whether or not this is possible may depend on what you mean by "contents"...
If you want to get the raw data in the PDF file, I don't think there is a way currently to do this in MATLAB. The URLREAD function was the first thing I thought of to read content from a URL into a string, but it has this note in the documentation:
s = urlread('url') reads the content
at a URL into the string s. If the
server returns binary data, s will
be unreadable.
Indeed, if you try to read a PDF as in the following example, s contains some text intermingled with mostly garbage:
s = urlread('http://samplepdf.com/sample.pdf');
If you want to get the text from the PDF file, you have some options. First, you can use URLWRITE to save the contents of the URL to a file:
urlwrite('http://samplepdf.com/sample.pdf','temp.pdf');
Then you should be able to use one of two submissions on The MathWorks File Exchange to extract the text from the PDF:
Extract text from a PDF document by Dimitri Shvorob
PDF Reader by Tom Gaudette
If you simply want to view the PDF, you can just open it in Adobe Acrobat with the OPEN function:
open('temp.pdf');

wb=com.mathworks.mde.webbrowser.WebBrowser.createBrowser;
wb.executeScript('javascript:alert(''Some code from a link'')');
desk=com.mathworks.mde.desk.MLDesktop.getInstance;
desk.removeClient(wb);