On one side I have a Java program that keeps feeding a database. On the other a web server (Apache). I need to transfer multiple entries, at once, from the program to a distant web server via a REST webservice.
While I have no trouble to send pairs of values from one to the others (using a HttpClient on the Java's side and a PHP file that writes into the server's own database on the other side), I lack knowledge about how I could build my HTTP request with data.
I've been using an Entity so far :
List<NameValuePair> list = new ArrayList<NameValuePair>();
list.add(new BasicNameValuePair("field1", "aaa"));
list.add(new BasicNameValuePair("field2", "bbb"));
httppost.setEntity(new UrlEncodedFormEntity(list));
and receiving it in my PHP file :
<?php
$field1=$_POST["field1"];
$field2=$_POST["field2"];
$con= mysql_connect("localhost","root");
if(!$con) die("Not able to connect");
mysql_select_db("mydb",$con);
mysql_query("INSERT INTO `mytable` (`field1`, `field2`) VALUES ('$field1', '$field2')");
mysql_close($con);
?>
Works great. Simple and easy. But now, as stated before, I wish to transfer several entries from my database. Each entry consists of 35 fields, which yields to some big data.
My question is : How can I insert my database entries within ONE (or as many as needed if one HTTP request capacity is full) and fetch the resulting data on the PHP file side ?
Bonus : with the previous solution, how can I compress that data?
You can use JSON for quick and simple transfer. For decoding, PHP supports $array = json_decode($json_string, TRUE);.
Not sure about compressing. Are you going to transfer lots of data this way to make it relevant?
Well, the thing you are doing will lead to SQL injection easily. Make sure you escape your data before running the insert query with mysql_real_escape_string or better - prepared statements.
Instead of using JSON, you could use Google Protocol Buffers. You need both JAVA and PHP extension to use protocol buffers but this is binary protocol that compresses the data amazingly well and allows you the flexibility to define different "repeatable" objects (entries) with same properties, so that when you open the protocol buffer on the PHP side, it will be easy to utilize the data.
Related
I'm currently trying to get fingerprint templates from java, I have already sent the templates over HTTP request but I don't know how to get it on Laravel PHP backend to save them as blob on database, I'm sending templates as ByteArrayInputStream. Thanks in advance!
My code:
ByteArrayInputStream[] templatesArray = new ByteArrayInputStream[4];
for(int i=0; i<fingers.size(); i++)
{
templatesArray[i] = new ByteArrayInputStream(fingers.get(i).getTemplate().serialize());
}
Next I send that array over http with Retrofit. I'm getting them as usual on laravel using:
$request->input('fingers')
Is it a POST? Suppose from a practical debugging standpoint I'd suggest starting with a limited script in that PHP apps public/root that simply does a var_dump($_REQUEST); If you don't get data there you will need to look at limits in the PHP environment on max_post_size.
If you do get data you should drop the java tag, add a Laravel tag, and slightly rephrase the question with what you've learned.
Assume it's essentially a file upload. You don't need to base64_encode anything on that Java side do you?
I am creating a REST service in Java ,and have a doubt with regards to params for the GET method .
I have to pass the below params in a GET request
Function
"GET" File status :
Params:
Time Range:(String)
FlowId:(String)
ID_A= or ID_B= or Both (String)
IS_ADD_A= or IS_ADD_B= or both (String)
Regex=(String)
Cookie=XXXXX
So as there are 6 params,so passing it as a query string would not be an efficient way and can't but the same in body(as it is against the HTTP GET specification)
Making this as a POST call would be against the REST principle as I want to get data from the server ,
What would be an efficient way of solving this ,would passing the params as query string is out of question,passing it in body which is against the HTTP spec ,making this as headers which may also be not good ,making this as POST request which will voilate the fielding's REST principle .
Passing data in the body of an HTTP GET call is not only against the spec but causes problems with various server-side technologies which assume you don't need access to the body in a GET call. (Some client side frameworks also have some issues with GET and a query in the body) If you have queried with long parameters I'd go with POST. It's then using POST for getting data but you'd not be the only one having to go this way to support potentially large queries.
If your parameters values aren't very long, using query string is your best option here. 6 params is not a lot, as long you don't exceed the IE limit of characters in the path - 2,048 (http://www.boutell.com/newfaq/misc/urllength.html). For example Google search engine uses many more params then 6. If there is a possibility that the URL path will exceed the limit above, you should use POST instead.
I have been working on an android application project that uses HTTP Get to send and receive data from MySQL through a PHP file using JSON from Java.
I have lately been running into some issues in theory behind best practices using HTTP Transport and passing Parameters via a URL.
First Question:
How should I be passing my data to my PHP Webservices ?
Currently I am just passing the data through single parameters using key value pairs like so:
myurl.com/retrieveinfo.php?user_id=453&password=sha1-hash-value
Should I be moving this type of request to append a JSON object onto the URL instead? like so:
myurl.com/retrieveinfo.php?{\"users\":{\"username\":\"User1Name\" ,\"user_id\":453 , \"password\":\"sha1-hash-value\"}}
Second Question:
*How should I be handling the JSON Response from the Server ? Do I need to push this work off to a handler and make sure the UI Thread is not the one doing this work? *
Currently I am just parsing the JSON using separate methods for each Object Type such as
User.Class
private void parseUserInfo(JSONObject response){
// Do all my Parsing for a User Object
try{
JSONArray users = response.getJSONArray("users");
JSONObject user = users.getJSONObject(0);
// Get the User info etc...
}catch(JSONException ex){
ex.printStackTrace();
}
}
Notes.Class
private void parseNotes(JSONObject response){
// Do all my Parsing for a Note Object
try{
JSONArray notes = response.getJSONArray("notes");
for (int index = 0; index < notes.length() ; index++)
{
JSONObject note = notes.getJSONObject(index);
// Get all the note info etc...
}
}catch(JSONException ex){
ex.printStackTrace();
}
}
Third Question:
I would like my PHP server files to only work for my Application. So what is the best way to secure my PHP files on my server so a request to my files wont go through if its run in a browser ?
Should I be sending some temp key that only my application knows about ?
Thanks
First Question:
You don't really want to put a JSON object on the url as a query parameter. The real two debates that I see is that you either 1) use the key value pairs you were using, or 2) make this a POST and send the JSON as a payload.
Since you are not planning on exposing the API to anyone, I don't really find it important for you to follow standard nomenclatures. Do whatever you want to do. However, from a REST standpoint, anything that retrieves info should be a GET call, and the data should be key-value pairs on the query string. However, it looks like you are passing in a username and password (ok, the sha of the pass). It is considered best practice to always pass user info as the payload. So almost all login type protocols use a POST for user data. User-id's or session id's are common in the query string but usernames and passwords should almost always be in a payload.
Note: sometimes in TLS (SSL) it is considered ok to include these things in the query string.
Second Question:
Honestly, I would just use Jackson. https://github.com/FasterXML/jackson
But otherwise, it is normal to have a seperate layer for parsing. In otherwords, one class handles all the parsing. You do not want to put this code inside your models if you can avoid it. The new layer would handle parsing and would pass the Java Model objects down to the next layer.
Third Question:
The easiest way to do this would simply be to check the user-agent header on the request. Make sure that the user-agent is your application, and not a browser.
However, it would still be possible for people to "spoof" this. Using a temp key wouldn't really help either, because once people sniff the traffic they can figure out the temp key.
The standard thing here is to do some type of session based key, where the application sends some type of MAC in order to prove it is a valid client.
You could also consider using OAUTH2 to protect your api's.
We have already shipped a client (.NET WinForms) application which sends customer data to Java server. While most of the data sent by client are accepted at server side, some records are truncated because of the presence of & character in it, as client sends raw & and do not URL encode it, we have fixed it by using the below code:
string dataBefore="A & B";
string dataBefore = System.Web.HttpUtility.UrlEncode(dataBefore);
It is impossible for us to update all the client applications(which are already shipped) and we are thinking of a server side fix.
With the help of Fiddler, we have made sure the data has left client in full, but when server reads as below:
//in java
String dataReceied=request.getParameter("data");
it gets truncated if data contains &
Could someone help us suggesting a server side(java) fix for this? Is it possible to access the request stream in java(instead of request.getParameter())?
You can get access to the raw query string using HttpServletRequest.getQueryString() (javadoc), which:
returns a String containing the query string or null if the URL contains no query string. The value is not decoded by the container.
You can them perform manual decoding on that string, instead of using getParameter().
#Wesley's idea of using getParameterMap() may not be useful, because you don't know which order the parameters were supplied in.
I'd suggest implementing this logic as a servlet filter, to decouple the fixing of the broken parameters from your actual servlet logic. This would involve writing a custom subclass of HttpServletRequestWrapper which overrides getParameter() and manuyally decodes the query string. Your servlet would then be able to use the HttpServletrequest API as though everything was tickety boo.
It is cut off because & signifies a new URL parameter in a request like this:
google.com?query=java&page=2. Java converts all these parameters to a Map, so that's where it goes wrong.
Have you tried iterating through request.getParameterMap()? The remaining data is most likely in the name of the next parameter. If that does not work, check out the API of HTTPServletRequest to see if there is another way to get your data.
Good luck!
PS How angry are you guys at the intern that wrote & shipped that client? That sounds messed up!
I'm planning to develop a webservice, and I like to try the RESTful architecture. The issue is that I don't know if the service is adequate for it, or it is better to use SOAP.
The service is about downloading some data from the server to a device on the local computer. The data will be split into chunks. The service will be run with an ad-hoc client at the local machine that will manage the device the file is gonna be stored in.
I was thinking on having something like:
/files/{id} --> will inform about the details of the file
/files--> list all the files
The problem is for the action. In rest only GET, POST and (PUT DELETE) are defined. But I want to have something like download. My idea, although not fully restful is to create:
/files/{id}/download
This will return something like
{ "chunk" : "base64 string with chunk data"
"next" : "http://XXX/file/id/download?chunk=1
}
When next is empty the whole set of chunks would be downloaded.
What do you think? Is it ok to do it this way or would it be better the traditional way using SOAP and defining functions like getFiles(), getFileChunk(chunkNo, file)?
Any comment is really appreciated.
See you
If using REST, you don't need to define your own "chunking" protocol as the HTTP headers Content-Length, Content-Range and Transfer-Encoding are all used for sending chunked data.
See the RFC for HTTP header fields
As John already mentioned you might want to separate between your file resources and the file resource metadata (any information about your file). Additionally a more RESTful way to access your chunks could look like this:
http://url/files/{id}/chunks
{
"complete" : false,
"chunks": [
"http://url/files/<fileid>/chunks/1",
"http://url/files/<fileid>/chunks/2",
"http://url/files/<fileid>/chunks/3",
]
}
Basically, here, you return a list of RESTFUL URIs to all your file chunks and the information if all chunks of the file are already complete. I don't see that SOAP might have any advantage there since you would define the same methods (getFile and getChunks) that are already covered by the REST verb GET.
It sounds like you really have two different resources: file-metadatas and files. What about something like:
/file/{id} // GET: Retrieve this file's data.
/file-metadata/{id} // GET: Metadata about a particular file. Contains link to file:
// {
// ...
// data: "http://.../file/156", // Where to find file's data.
// }
/file-metadata // GET: List metadata for all files.