We have already shipped a client (.NET WinForms) application which sends customer data to Java server. While most of the data sent by client are accepted at server side, some records are truncated because of the presence of & character in it, as client sends raw & and do not URL encode it, we have fixed it by using the below code:
string dataBefore="A & B";
string dataBefore = System.Web.HttpUtility.UrlEncode(dataBefore);
It is impossible for us to update all the client applications(which are already shipped) and we are thinking of a server side fix.
With the help of Fiddler, we have made sure the data has left client in full, but when server reads as below:
//in java
String dataReceied=request.getParameter("data");
it gets truncated if data contains &
Could someone help us suggesting a server side(java) fix for this? Is it possible to access the request stream in java(instead of request.getParameter())?
You can get access to the raw query string using HttpServletRequest.getQueryString() (javadoc), which:
returns a String containing the query string or null if the URL contains no query string. The value is not decoded by the container.
You can them perform manual decoding on that string, instead of using getParameter().
#Wesley's idea of using getParameterMap() may not be useful, because you don't know which order the parameters were supplied in.
I'd suggest implementing this logic as a servlet filter, to decouple the fixing of the broken parameters from your actual servlet logic. This would involve writing a custom subclass of HttpServletRequestWrapper which overrides getParameter() and manuyally decodes the query string. Your servlet would then be able to use the HttpServletrequest API as though everything was tickety boo.
It is cut off because & signifies a new URL parameter in a request like this:
google.com?query=java&page=2. Java converts all these parameters to a Map, so that's where it goes wrong.
Have you tried iterating through request.getParameterMap()? The remaining data is most likely in the name of the next parameter. If that does not work, check out the API of HTTPServletRequest to see if there is another way to get your data.
Good luck!
PS How angry are you guys at the intern that wrote & shipped that client? That sounds messed up!
Related
I am creating a REST service in Java ,and have a doubt with regards to params for the GET method .
I have to pass the below params in a GET request
Function
"GET" File status :
Params:
Time Range:(String)
FlowId:(String)
ID_A= or ID_B= or Both (String)
IS_ADD_A= or IS_ADD_B= or both (String)
Regex=(String)
Cookie=XXXXX
So as there are 6 params,so passing it as a query string would not be an efficient way and can't but the same in body(as it is against the HTTP GET specification)
Making this as a POST call would be against the REST principle as I want to get data from the server ,
What would be an efficient way of solving this ,would passing the params as query string is out of question,passing it in body which is against the HTTP spec ,making this as headers which may also be not good ,making this as POST request which will voilate the fielding's REST principle .
Passing data in the body of an HTTP GET call is not only against the spec but causes problems with various server-side technologies which assume you don't need access to the body in a GET call. (Some client side frameworks also have some issues with GET and a query in the body) If you have queried with long parameters I'd go with POST. It's then using POST for getting data but you'd not be the only one having to go this way to support potentially large queries.
If your parameters values aren't very long, using query string is your best option here. 6 params is not a lot, as long you don't exceed the IE limit of characters in the path - 2,048 (http://www.boutell.com/newfaq/misc/urllength.html). For example Google search engine uses many more params then 6. If there is a possibility that the URL path will exceed the limit above, you should use POST instead.
I am trying to maintain some logs of a couple of Javascript objects in my webapp. The easiest way to log them would be to stringify them and put them on a jersey path as a string.
My logger works fine with regular strings but gives Error 400: The request sent by the client was syntactically incorrect when I pass a JSON stringified object. There are two things that I can't explain and are going wrong with my code:
Everything seems to be working fine on my development server but not on the server where I am deploying it. I develop on a Mac / Homebrew / Tomcat enviroment and deploy on a CentOS server.
Even on the CentOS server, logging works fine when I pass a simple one word strings as message but passing a JSON string throws up the error.
My Logger code looks like this:
#PUT
#Path("logEvent/{fn_event}/{fn_message}")
public void logEvent(#PathParam("fn_event") String event,
#PathParam("fn_message") String message)
throws Exception {
:
:
:
}
I have tried investigating catalina logs but it doesn't tell anything. Access logs give no more information than specifying "Error 400".
This may happen if you don't escape quotes in your JSON string. Try to escape it with \"
Json String may be having "Spaces" etc etc.
So when you call http://yourserver/logEvent/oneword/onewordMessage it may work fine but when you call http://yourserver/logEvent/oneword with space and with & and so many things/or one message with " and not " etc
Then in second case, your path may be incorrectly encoded. Form Encode your json stream, and then pass it as path.
Better move to Post, and pass the stream as Body of Request. Not sure why you will prefer using "entire" json file as path of your service
I have a few questions about a specific REST call I'm making in JAVA. I'm quite the novice, so I've cobbled this together from several sources. The call itself looks like this:
String src = AaRestCall.subTrackingNum(trackingNum);
The Rest call class looks like this:
public class AaRestCall {
public static String subTrackingNum (Sting trackingNum) throws IOException {
URL url = new URL("https://.../rest/" + trackingNum);
String query = "{'TRACKINGNUM': trackingNum}";
//make connection
URLConnection urlc = url.openConnection();
//use post mode
urlc.setDoOutput(true);
urlc.setAllowUserInteraction(false);
//send query
PrintStream ps = new PrintStream(urlc.getOutputStream());
ps.print(query);
ps.close();
//get result
BufferedReader br = new BufferedReader(new InputStreamReader(urlc
.getInputStream()));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line=br.readLine())!=null) {
sb.append(line);
}
br.close();
return sb.toString();
}
}
Now, I have a few questions on top of the what is wrong with this in general.
1) If this rest call is returning a JSON object, is that going to get screwed up by going to a String?
2) What's the best way to parse out the JSON that is returning?
3) I'm not really certain how to format the query field. I assume that's supposed to be documented in the REST API?
Thanks in advance.
REST is a pattern applied on top of HTTP. From your questions, it seems to me that you first need to understand how HTTP (and chatty socket protocols in general) works and what the Java API offers for deal with it.
You can use whatever Json library out there to parse the HTTP response body (provided it's a 200 OK, that you need to check for, and also watch out for HTTP redirects!), but it's not how things are usually built.
If the service exposes a real RESTful interface (opposed to a simpler HTTP+JSON) you'll need to use four HTTP verbs, and URLConnection doesn't let you do so. Plus, you'll likely want to add headers for authentication, or maybe cookies (which in fact are just HTTP headers, but are still worth to be considered separately). So my suggestion is building the client-side part of the service with the HttpClient from Apache commons, or maybe some JAX-RS library with client support (for example Apache CXF). In that way you'll have full control of the communication while also getting nicer abstractions to work with, instead of consuming the InputStream provided by your URLConnection and manually serializing/deserializing parameters/responses.
Regarding the bit about how to format the query field, again you first need to grasp the basics of HTTP. Anyway, the definite answer depends on the remote service implementation, but you'll face four options:
The query string in the service URL
A form-encoded body of your HTTP request
A multipart body of your HTTP request (similar to the former, but the different MIME type is enough to give some headache) - this is often used in HTTP+JSON services that also have a website, and the same URL can be used for uploading a form that contains a file input
A service-defined (for example application/json, or application/xml) encoding for your HTTP body (again, it's really the same as the previous two points, but the different MIME encoding means that you'll have to use a different API)
Oh my. There are a couple of areas where you can improve on this code. I'm not even going to point out the errors since I'd like you to replace the HTTP calls with a HTTP client library. I'm also unaware of the spec required by your API so getting you to use the POST or GET methods properly at this level of abstraction will take more work.
1) If this rest call is returning a JSON object, is that going to get
screwed up by going to a String?
No, but marshalling that json into an obect is your job. A library like google gson can help.
2) What's the best way to parse out the JSON that is returning?
I like to use gson like I mentioned above, but you can use another marshal/unmarhal library.
3) I'm not really certain how to format the query field. I assume
that's supposed to be documented in the REST API?
Yes. Take a look at the documentation and come up with java objects that mirror the json structure. You can then parse them with the following code.
gson.fromJson(json, MyStructure.class);
Http client
Please take a look at writing your HTTP client using a library like apache HTTP client which will make your job much easier.
Testing
Since you seem to be new to this, I'd also suggest you take a look at a tool like Postman which can help you test your API calls if you suspect that the code you've written is faulty.
I think that you should use a REST client library instead of writing your own, unless it is for educational purposes - then by all means go nuts!
The REST service will respond to your call with a HTTP response, the payload may and may not be formatted as a JSON string. If it is, I suggest that you use a JSON parsing library to convert that String into a Java representation.
And yes, you will have to resort to the particular REST API:s documentation for details.
P.S. The java URL class is broken, use URI instead.
I have been working on an android application project that uses HTTP Get to send and receive data from MySQL through a PHP file using JSON from Java.
I have lately been running into some issues in theory behind best practices using HTTP Transport and passing Parameters via a URL.
First Question:
How should I be passing my data to my PHP Webservices ?
Currently I am just passing the data through single parameters using key value pairs like so:
myurl.com/retrieveinfo.php?user_id=453&password=sha1-hash-value
Should I be moving this type of request to append a JSON object onto the URL instead? like so:
myurl.com/retrieveinfo.php?{\"users\":{\"username\":\"User1Name\" ,\"user_id\":453 , \"password\":\"sha1-hash-value\"}}
Second Question:
*How should I be handling the JSON Response from the Server ? Do I need to push this work off to a handler and make sure the UI Thread is not the one doing this work? *
Currently I am just parsing the JSON using separate methods for each Object Type such as
User.Class
private void parseUserInfo(JSONObject response){
// Do all my Parsing for a User Object
try{
JSONArray users = response.getJSONArray("users");
JSONObject user = users.getJSONObject(0);
// Get the User info etc...
}catch(JSONException ex){
ex.printStackTrace();
}
}
Notes.Class
private void parseNotes(JSONObject response){
// Do all my Parsing for a Note Object
try{
JSONArray notes = response.getJSONArray("notes");
for (int index = 0; index < notes.length() ; index++)
{
JSONObject note = notes.getJSONObject(index);
// Get all the note info etc...
}
}catch(JSONException ex){
ex.printStackTrace();
}
}
Third Question:
I would like my PHP server files to only work for my Application. So what is the best way to secure my PHP files on my server so a request to my files wont go through if its run in a browser ?
Should I be sending some temp key that only my application knows about ?
Thanks
First Question:
You don't really want to put a JSON object on the url as a query parameter. The real two debates that I see is that you either 1) use the key value pairs you were using, or 2) make this a POST and send the JSON as a payload.
Since you are not planning on exposing the API to anyone, I don't really find it important for you to follow standard nomenclatures. Do whatever you want to do. However, from a REST standpoint, anything that retrieves info should be a GET call, and the data should be key-value pairs on the query string. However, it looks like you are passing in a username and password (ok, the sha of the pass). It is considered best practice to always pass user info as the payload. So almost all login type protocols use a POST for user data. User-id's or session id's are common in the query string but usernames and passwords should almost always be in a payload.
Note: sometimes in TLS (SSL) it is considered ok to include these things in the query string.
Second Question:
Honestly, I would just use Jackson. https://github.com/FasterXML/jackson
But otherwise, it is normal to have a seperate layer for parsing. In otherwords, one class handles all the parsing. You do not want to put this code inside your models if you can avoid it. The new layer would handle parsing and would pass the Java Model objects down to the next layer.
Third Question:
The easiest way to do this would simply be to check the user-agent header on the request. Make sure that the user-agent is your application, and not a browser.
However, it would still be possible for people to "spoof" this. Using a temp key wouldn't really help either, because once people sniff the traffic they can figure out the temp key.
The standard thing here is to do some type of session based key, where the application sends some type of MAC in order to prove it is a valid client.
You could also consider using OAUTH2 to protect your api's.
I'm attempting to get started with MINA, and all of the examples seem to have data written to the session, rather than making use of a method that can write the same type of data over and over.
I'm trying to make use of org.apache.mina.filter.codec.demux.MessageEncoder / MessageDecoder to encode / decode messages, which will allow me to always perform the task in a central location instead of doing it inline in the code, like the examples do.
Let's say I have a ProtocolCodecFactory (which extends DemuxingProtocolCodecFactory) that has a LoginRequestEncoder (which implements MessageEncoder<LoginRequest>, and was added via the factory's addMessageEncoder method). Does that mean that instead of directly calling session.write() with the username/password data, I should instead do something like this?
LoginRequest request = new LoginRequest(username, password);
new ProtocolCodecFactory()
.getEncoder(session)
.encode(session, request, someProtocolEncoderOutput);
I'm not going to lie...MINA seems like it's supposed to simplify the networking process, and I'm sure it will when I get a handle on it, but I'm thoroughly confused right now.
It turns out you can simple send a request via IoSession.write(). Here is a simple example based upon my original question:
LoginRequest request = new LoginRequest(username, password);
session.write(request);