I have wriite a small proxy (in Java), with the intention of modifying specific data from a web server response. The data is a deflate compressed XML file: only some of the elements do I want to actually be received by the client.
I have no problems with the proxy functionality, but when I try to send my modified xml in place of the server's response, the client receives nothing (web-debugger shows content length 0) - I am sending data though, atleast there IS data in my app being written to the Socket's output stream.
See here for more details and some Code. I won't post it twice.
Related
I am writing a REST API in JAX-RS 2.0, JDK 8 for the below requirement
POST API /server/fileUpload/ (Multipart Form data) where I need to send a Big .AI (Adobe Illustrator) File in this.
The Server, takes the file and return Status 202 (Accepted), Acknowledging that file transfer happened Successfully. (From endpoint to Server)
Now at the Server, I am using Java + Imagemagik to convert .AI File (20-25 MB File) to small JPG Thumbnail, place on a Apache HTTP Server and share the location (like http://happyplace/thumbnail0987.jpg)
Now the Second Response should come from Server with Status 200 OK and Thumbnail URL
is it feasible with one REST API? (Async/similar)
or should I split it to 2 API calls, Please suggest
No. In http, one request gets one response. The client must send a second request to get a second response.
You can use WebSockets for that.
If you are calling from script the call will be async you can handle the Thumbnail URL when you get a response. When you are calling from java program i suggest to run it on a different thread, If the execution is not sequential i.e ( Remaining lines can be executed without getting URL). If url is needed for the remaining section of code you can make one call and wait for the response then execute remaining code.
You need to make different APIs for both scenarios. One for showing file upload status and another for all file conversion and manipulation.
On the client side second request must be callback of first request.
The best way to handle these kind of scenario is to use Java Reactive (Project Reactor, WebFlux).
You can return two response using custom middlewares in asp.net (however not recommended).
Return response from one middleware and subsequently you can invoke next middleware and return second response from second middleware
Consider the case where Server B makes a HTTP request to Server A (Which is an encryption server) and get an output stream (which is encrypted). So that Server B could write with the help of this encrypted stream.
Now this encrypted output stream would be opened in Server A and closed at Server B.
Is it possible to send an output stream in HTTP response?
Is it a right way to send an output stream in response? Or are there any conventions like the output stream must be closed in the same server (or same application) where it is originated?
Yes, it's possible. That's how the file download works.
First you'll have to set the Content-Type that you're going to provide. If it's simple binary file then set it as application/octet-stream. After that get outputStream of the response and dump the file content in it.
Like below
response.setContentType("application/octet-stream");
response.setContentLength(fileSizeInBytes);//new File('myfile').length(), optional step
response.getOutputStream().write(fileBytes);//do it in chunks
#Edit
Streams are endpoints of data channels. Like an HTTP address is an endpoint to the server resource.
In case of stream our program doesn't need to know where the actual resource resides. I just need to know how to interact with the stream.
So in case of HttpServletResponse stream, data lies in your server. Client's browser (or any other client) establishes a connection with your server. When we call methods on the stream like read/write, data over this connection is sent or received. These calls in case of HttpServletResponse result in HTTP packet transfer over TCP connection.
For more information on Java Stream (or any other language with similar concept) check here.
https://docs.oracle.com/javase/tutorial/essential/io/streams.html
I am working with a web framework (uPortal) that is handling errors by just throwing an exception and then hanging. The framework works by rendering XML into HTML. When there is an exception, the browser recieves rendered content up to the XML template element that is failing, and then the browser just sits and waits for a timeout. Our team's theory is that the content is sent before the error occurs, which surprised me. Other frameworks I've worked with seem to finish rendering before sending content.
My question is, is there a way to redirect the browser after content has already been sent? In this case, we are in the middle of rendering the content of a <script> tag, but the error could occur potentially anywhere in the html.
My only current thought is to inject some javascript at the top of the page, and to try to change the framework's behavior to fail quickly and close the connection and add </body> and </html> tags when an error occurs. Then the above mentioned javascript would run on pageload and detect if the entire page's content was there and do a client-side redirect if not. Maybe it could look for a special hidden div at the bottom of the page.
Are there any examples of frameworks solving this problem differently or of people using similar framework working around this issue?
You must either capture the error, or capture the output in a buffer. If you can handle the exception, you can probably print a simple script tag like
<script> window.location.href = 'some_new_url';</script>
If the browser understands the doctype to be something related to HTML, it will execute that tag.
If you can capture the output in a buffer, when you handle the error you can decide to send an HTTP redirect to the browser and destroy the output buffer up to that point.
As for other frameowrks, in PHP, you can simply enable output buffering with ob_start(), which won't start sending content until the request is fully completed.
I don't know that framework, but
In http, every response has a response-code associated with it. Since the page is already half-way transferred / rendered that status code (usually "200") was sent (and received) already.
There's no way for the browser to accept another response code (like "301" for redirect) for the same response! Also the server is not able to send another response code, because the original response code was already commited and sent to the client.
Your description of the error and knowledge of the http-protocol implies that there is probably some implementation error in the framework / server components used, OR it was done deliberatly, risking the situation that you are in now...
to redirect a page , you need to set redirect information in header. but you can write header once you start writing content ( may be header is already received by client by the time you compete writing whole document )
But, you can do it in different way as below
1.let document loading complete and record if you need to redirect the page while rendering
2. add a unique request-id identifier for each page load
3. invoke ajax call with request-id ( may be rest call) to server asking if page needs to be redirected.
4. if page needs to be redirected , do so, via javascript in browser at client end.
A HTTP response consists of headers and an optional response content.
Once you have started to write the response to the socket connection you can't revert it. In your example: If you run into an error in the middle of content generation you can't add a redirect header - the header section has already be written.
The statement above is not entirely true: in HTTP chunked transfer encoding the response is sent in separate chunks. The last chunk can have an optional trailer containing entity-header fields and theoretically a redirect header. But if you can use these mechanism is a different question. For instance a servlet container may use chunked transfer encoding but does not give you an API to set the trailer.
But writing must not start immediately: For instance HttpServletResponse maintains a buffer for the response content. If you set headers and start writing the content only the buffer is filled and you still can reset the response and start all over. But once the buffer overflows the response is written to the connection and the HttpServletResponse is now committed.
Such a mechanism gives you way to deal with errors during content generation which happen when the response is not yet committed: Just reset the response and send an error message instead. You could examine your framework if it supports such an mechanism. But obviously this is not a solution for larger responses.
A second way to avoid errors during content generation is simply to make sure that they can't happen. First gather all your data needed for the response (e.g. making unsafe database calls), then in a second step generate the response - the second now step should not fail (except if you have bugs in your code).
You already mentioned a third way to handle an error, by having the client sanitize the response and take some action it errors are detected (e.g. by including a script in the generated HTML response).
The only reliable way to do this is to create a proxy HttpServletResponse object that caches the response. You'd need to give the uPortal this proxy instead of the actual HttpServletResponse, and only send the output using the real response once the processing completes / send redirect if the processing fails.
It is HTTP protocol design limitation that you cannot send HTTP redirect once output was started.
Other possible ways rely on HTML or Javascript redirects, but since you write that the error may happen at any moment, it would be difficult to print it out in a way that the browsers would reliably interpret it as redirect.
I am using Java servlets using Apache tomcat.
I've configured a threadpool and am dealing with each request.
My page is taking in many GET requests at the same time, I'm wondering if I can respond to the server after each get request before any of the logic happens?
So server gives me a request -> I respond with either 'good send another' or 'bad send another' before I start my queueing.
Any help would be much appreciated!
EDIT
Sorry that was terribly written :(
What I'm asking for is a way to send a Header to the client (in this case it's a server which sends me lots of requests). The response would just be 200 or error based on the information I get sent.
What my program is doing:
My servlet gets sent lots of GET requests from one client. (over 100,000) Which I am using tomcat to queue and put into a threadpool. It is then assigned to a worker thread which processes it and puts it into a database.
I've been told to do is send a request back to that server saying 'ok received it'. I think I can use a header response but I don't have the URL of that client (and the client can change for different campaigns). So was wondering what the best way would be to send that response.
After doing some more research I think what I'm looking for is ServletOutputStream.
response.setContentType("text/html");
ServletOutputStream output = response.getOutputStream();
output.flush();
output.close();
Using servlet output stream where do I set the <head><body> tag? and insert the header response afterwards.
The simple answer is "sure".
If these are get requests from a web page for a web page, include a refresh timer and send back some token that can be used to identify the difference between a first-time-request and an I-requested-earlier-are-you-done request. In this case the refresh timer can be set via a meta refresh tag.
If the get requests are part of a REST API then you can define "got it and I'm working" into the protocol. For instance, return a 202 to indicate "got it but not done" and return 200 to indicate "done". As with the html page, consider sending some token back with the 202 that identifies the pending request.
I'm trying to implement the following:
Open a POST connection.
Read response code.
Write content.
Read response code.
The second step throws a following exception:
Caused by: java.io.IOException: content-length promised 345286 bytes, but received 0
I understand that 'getResponseCode' will close the writing stream, but i need to find a way of how to read the response code before actually writing anything. I would like to skip the process of writing content to the request body (100 MB) due to the (for an example) 401 code.
HTTP Doesn't work like that. You need to send the full request and only then to read the response code. In your case, your response had promised a request body of 345286 bytes, but you have sent nothing.
The correct way is to do the following:
Open a POST connection.
Write content.
Read response code.
In case you want to verify that the server permits to do it (to avoid 401s) you can add a small GET request first to verify the authentication.