If I'm supporting the upload of content (mostly images and video) by my REST API's users, is it safe to trust the Content-Type they declare in (multipart) uploads? Or should I, instead, run some kind of "media type detection" on the content (using, for example, Apache Tika) to ensure that the declared media type corresponds to the detected, actual one? Am I being over-zealous by introducing this media type detection step?
You certainly shouldn't blindly trust the Content-type header, or any other header. These things should be used to inform your decisions about how to process the request. So, Content-type: application/json should allow you to interpret the message body as a json object - that sort of request might then be passed to a JSON deserialiser to bind it to an object.
It would be wrong to ignore the Content-type header just because the request body contains data which looks like something else. If the request is internally inconsistent then it should be rejected. It's one thing not to send a Content-type header but quite another for the header to be wrong.
So, the only situation where you might want to use some sort of automatic detection should be where you have no reasonable information about the content - either Content-Type is very generic (such as "/") or not present at all. In that situation it's worth deciding whether some kind of autodetection is possible or valuable.
Never trust the input which you get from the user. Always run a check in your server side code be it type of file, size of file, etc. Use the REST API or Javascript to make the experience of the user smoother and faster.
You should definitely reject all the requests that are missing Content-Type header (and Content-Length as well) or have it set incorrectly.
It's definitely not about being over-zealous, rather about securing the system. If you have suspicions about the content just check it. But remember to validate the size before checking the content. If you have a proxy server (e.g. nginx) it has appropriate modules to reject requests that are too big.
Related
Is there a specific scenario where we use a POST instead of GET, to implement the functionality of get operation ?
GET is supposed to get :) and POST is used to mainly add something new or sometimes often used for updates as well (although PUT is recommended in such scenarios). There is no specific scenario where we use a POST instead of a GET, if we require this, that means we are probably doing it wrong, although nothing stops you doing this but this is bad design and you should take a step back and plan your API carefully.
There are 2 important cases for a POST i.e. POST is more secure than a GET and POST can send large amount of data but even with this I won't recommend why one will use POST to simulate a GET behaviour.
Lets understand usage of get and post :
What is GET Method?
It appends form-data to the URL in name/ value pairs. The length of the URL is limited by 2048 characters. This method must not be used if you have a password or some sensitive information to be sent to the server. It is used for submitting the form where the user can bookmark the result. It is better for data that is not secure. It cannot be used for sending binary data like images or word documents. It also provides $_GET associative array to access all the sent information using the GET method.
What is POST Method?
It appends form-data to the body of the HTTP request in such a way that data is not shown in the URL. This method does not have any restrictions on data size to be sent. Submissions by form with POST cannot be bookmarked. This method can be used to send ASCII as well as binary data like image and word documents. Data sent by the POST method goes through HTTP header so security depends on the HTTP protocol. You have to know that your information is secure by using secure HTTP. This method is a little safer than GET because the parameters are not stored in browser history or in web server logs. It also provides $_POST associative array to access all the sent information using the POST method.
Source: https://www.edureka.co/blog/get-and-post-method/
So both the methods have their specific usage.
POST method is used to send data to a server to create or update a resource.
GET method is used to request data from a specified resource.
If you want to fetch some data you can use the GET method. But if you want to update an existing resource or create any new resource you should use POST. GET will not help you to create/update resources. So exposing the api should be specific to your needs.
UPDATE
So your main question is in what scenario we can use POST to implement the functionality of GET.
To answer that, as you understand what GET and POST does, so with GET request you will only fetch the resource. But with POST request you are creating or updating the resource and also can send the response body containing the form data in the same request response scenario. So suppose you are creating a new resource and the same resource you want to see, instead of making a POST call first and making a GET call again to fetch the same resource will cost extra overhead. You can skip the GET call and see your desired response from the POST response itself. This is the scenario you can use POST instead of making an extra GET call.
Using Spring MVC, for each incoming request id like to set the statusCode and headers.
Once they are set, I need to pad the response body to make the entire response size, including all headers and content (actual data wired to the client), to be exactly X bytes (e.g. 300 bytes). The response size will vary per request, but all responses will have to padded.
There are no limitations regarding the manipulation of the response.
Using HttpServletResponse I can set the statusCose and headers and also maybe get the response size. But I couldn't find a way to set the body content/length in accordance to the required size.
If I use a ResponseEntity I can set the body but can't tell the size of the response.
How can I pad the response to the required size while setting the fields above?
First of all, as JB Nizet said, this requirement is fully outside of the HTTP protocol and you'd better fix the client side to only control the size of the body part (what the content-length header is made for).
Now assuming you really need to do that, I can imagine 2 ways to fulfil this requirement (both of them seem equally ugly...):
Use a dedicated proxy to post-process the HTTP response
Consistently add a custom header to the response that indicates the total size the response should have. Put a dedicated proxy between the clients and the server. That proxy should listen on its own port and forward everything to the server. For the response part, it should:
read the header part line by line (delimited with \r\n), store the required total size without transmitting it and forward all other headers storing the number of bytes sent
once the header part is over (empty line), read the body part and trunk or pad it to the correct size before transmitting it.
This would be a low level program that should use directly the socket interfaces, or that could directly be written in C. This really looks like plumbing, but it should be usable even for different servlet containers.
Compute the body part size and guess the header one
Control how your servlet container processes headers. The protocol requires that they are written as NAME=value\r\n and that the header part is followed by an empty line (r\n). But you should control twice whether the containers adds its own headers or whether it automatically adds some headers if you do not provide them. That should allow you to compute the header size from the headers you added to the response, but is clear coupled to a single servlet container, and when used the same way.
Alternatively, you could try to ask the response what headers it contains. Normally (at least Tomcat does it), it actually computes the header part when it commits the response (*). So you could:
set the status and the headers you need
commit the response by a call to flushBuffer()
get the generated headers through:
for (String name: resp.getHeaderNames()) {
for (String value: resp.getHeaders(name)) {
...
If you take care of the status line (should be HTTP/1.1 200 OK\r\n but here again control twice its actual size), it should be enough to compute the total header size. Provided you have enough control on the body to know what you want to write, you should be able to compute how much padding you need.
This avoids a dedicated proxy, but it can only be guaranteed to work on a (version of a) particular servlet container
(*) Beware: when you commit the response manually, the servlet container can add a Transfer-Encoding=chunked header because it cannot guess the body size. Whether it is acceptable for you use case or not, I cannot know...
Considering the comment discussion above, you could create an HttpResponse Wrapper.
Using your wrapper you can then intercept the response and override the actual size.
More info: your response wrapper writes response content to an internal byte array. It is not actually written to the "real" http output stream. In the filter you make sure 'write' and 'flush' called on the wrapper don't exceed your preferred max length.
Having said this, I agree with #jb-nijet that this is not common and looks like a flaw.
I have a situation where the client (.js) initiates a REST request and it is processed on the server side (.java). After processing, I would like to return a count to the client, which will be shown in a popup box if it is positive. How can such a construction be done? The idea I had was to set a named parameter on the HttpServletResponse object, but even this object is no where in scope in the .js code. Any ideas? I should also clarify that the primary purpose of the REST call is to download a file to the client. Thanks!
Do you want to send two things to your client - sending a file and also additional data? You haven't mentioned what framework (if any) you are using in backend to do this. You can use response header.
From your question, it seems like you don't have a good general-purpose way of responding to client requests on your server. I'd recommend you decide on a data format you'd like to use for all calls (e.g., JSON, XML, etc.) and stick with that across the board.
Once you've made that decision, you can encode your integer using whatever makes sense in your chosen format. For example, in JSON you might return: {"count":6}.
I have a REST api built for creating orders. The behavior is such that the person who creates an order received an email back saying "You created an order XXX". This email is triggered all the time.
The api appears like this
http://api.mytestevnt.com/ordering/orders - POST with request body as the order entity json.
Now i want to give a feature to the api caller to indicate if the email notification is necessary or not. What's the best way to do this?
I think it depends on whether email notification is data or metadata. If it's part of the order, then definitely add it to the request body. If it's metadata, you have two choices. If you think there will be lots of metadata, you can either edit the order to have a metadata section or you can POST the metadata separately. If there will only be a limited amount of metadata, I would suggest using a query parameter.
You should avoid using a header unless you control the entire path from the client to the server, because proxies or load balancers are allowed to strip non-standard headers.
Include in the POST body a send_email=1 or send_email=0 param. You'll extract that, and see what the user wants to do.
Search "how to get POST variables in JAVA".
Accessing post variables using Java Servlets
You can do like this:
Add a new Java attribute(like boolean emailEnabled) in your Java Request Object for your REST service.
Client side which invokes your REST service need to provide that parameter you added in your server side, you can set a default value for that too.
I have created a HTTP server which works perfectly! No problems there,
using Sockets and ServerSockets.
Now I need to make the Form entries - which are done in the web browser and when they are submitted, they are processed by Java.
I've tried making a form, then pressing sumbit, except it's given a 501 error... The first part is 'POST /'. I suppose that means a hidden variable passed... But how do I make the server recognise the hidden variable, and make it process it?
Is there any way to do this?
How you decode the parameters depends on a number of things. If the request is a POST request then the values are part of the request body (this is, the values are sent after a blank link that separates the head and the body of the request). The encoding of the request body is then either "application/x-www-form-urlencoded" or "multipart/form-data". You can read how these two encodings work here: http://www.w3.org/TR/html401/interact/forms.html#h-17.13.4
I assume that this is a toy project and that you want to write this code yourself. Otherwise there are many server implementations of the HTTP protocol for Java out there...