I have a simple Netty test server that I would like to query a mongo database and return the result. I've setup the simple hello world tutorial from the Netty repository here: https://github.com/netty/netty/tree/4.0/example/src/main/java/io/netty/example/http/helloworld
I've modified the simple tutorial to add an asynchronous MongoDB call, which returns the same "hello world" string as the example, but after my modification the HTTP call never completes.
Original Method:
public void channelRead(ChannelHandlerContext ctx, Object msg) {
if (msg instanceof HttpRequest) {
HttpRequest req = (HttpRequest) msg;
boolean keepAlive = HttpHeaders.isKeepAlive(req);
FullHttpResponse response = new DefaultFullHttpResponse(HTTP_1_1, OK, Unpooled.wrappedBuffer(CONTENT));
response.headers().set(CONTENT_TYPE, "text/plain");
response.headers().set(CONTENT_LENGTH, response.content().readableBytes());
if (!keepAlive) {
ctx.write(response).addListener(ChannelFutureListener.CLOSE);
} else {
response.headers().set(CONNECTION, Values.KEEP_ALIVE);
ctx.write(response);
}
}
}
After My Change:
private final MongoCollection<Document> collection = ...
public void channelRead(final ChannelHandlerContext ctx, Object msg) {
if (msg instanceof HttpRequest) {
final HttpRequest req = (HttpRequest) msg;
collection.find(Filters.eq("_id", new ObjectId("..."))).first(new SingleResultCallback<Document>() {
public void onResult(Document document, Throwable throwable) {
boolean keepAlive = HttpUtil.isKeepAlive(req);
FullHttpResponse response = ...
(SAME CODE AS ABOVE)
});
}
}
I can see it's hitting my code, but the response never gets sent to the client. How do I make an async call in the ServerHandler method?
You will need to also call flush() or change write(...) to writeAndFlush(...) to ensure the content is really flushed to the socket.
Related
I am using apache async http client to stream objects from azure storage.
I only need to return the HttpResponse object which has the stream associated. My clients will actually have to read from that stream to store the file locally.
So Apache Async clients use a BasicAsyncResponseConsumer which actually buffers the entire file in local memory before calling the completed callback.
I am trying to create my own implementation of AbstractAsyncResponseConsumer so that I can stream the response body instead of actually storing it first but have been unsuccessful to do so till now.
Here is the bare bones cosumer class for reference ->
public class MyConsumer extends` AbstractAsyncResponseConsumer<HttpResponse> {
#Override
protected void onResponseReceived(HttpResponse response) throws HttpException, IOException {
}
#Override
protected void onContentReceived(ContentDecoder decoder, IOControl ioctrl) throws IOException {
}
#Override
protected void onEntityEnclosed(HttpEntity entity, ContentType contentType) throws IOException {
}
#Override
protected HttpResponse buildResult(HttpContext context) throws Exception {
return null;
}
#Override
protected void releaseResources() {
}
}
And here is the code to send the request and return the response ->
public void getFile(HttpRequestBase request) {
MyConsumer myConsumer = new MyConsumer();
HttpAsyncRequestProducer producer =
HttpAsyncMethods.create(request);
CompletableFuture<HttpResponse> future = new CompletableFuture<>();
return Future<HttpResponse> responseFuture =
httpclient.execute(producer,myConsumer,
new FutureCallback<HttpResponse>() {
#Override
public void completed(HttpResponse result) {
//This is called only when all the response body has been read
//future.complete(Result)
}
#Override
public void failed(Exception ex) {
}
#Override
public void cancelled() {
}
});
return future;
}
I will be returning a CompletableFuture of the HttpResponse object to my clients.
They shouldnt be waiting for my http client to read all the response body first in local buffer.
They ideally should start copying directly from the stream provided in the response object.
What should I add inmy implementation of the consumer to get the desired result ?
I don't know if you still have this problem, but if what you want is an InputStream that actually streams data, then you'll want to use the blocking version of Apache HttpClient.
Java's built-in InputStream and OutputStream are inherently blocking, so returning a CompletableFuture of InputStream essentially defeats the purpose. BasicAsyncResponseConsumer buffering the entire response in memory is actually the right thing to do, because that's the only way of making it truly non-blocking.
Another option you can take a look at is HttpAsyncMethods.createZeroCopyConsumer. What it does is that it stores the content to a file in a completely non-blocking way.
Here's an example:
try (CloseableHttpAsyncClient client = HttpAsyncClients.createDefault()) {
client.start();
final CompletableFuture<HttpResponse> cf = new CompletableFuture<>();
client.execute(
HttpAsyncMethods.createGet("https://example.com"),
HttpAsyncMethods.createZeroCopyConsumer(new File("foo.html")),
new FutureCallback<HttpResponse>() {
#Override
public void completed(HttpResponse result) {
cf.complete(result);
}
#Override
public void failed(Exception ex) {
cf.completeExceptionally(ex);
}
#Override
public void cancelled() {
cf.cancel(true);
}
});
// When cf completes, the file will be ready.
// The InputStream inside the HttpResponse will be the FileInputStream of the created file.
}
I'm unable to figure out how to return a String value from RequestBuilder's sendRequest() method after receiving a response. I referred to a similar question where the suggestion was to use Callback<String, String> callback but I can't figure out how to implement this. The GWT documentation for Callback does not have any examples.
What I have is a class Requester with the method generateRequest() that should make a request with RequestBuilder and return a String when called. The processResponse() method takes the response, parses it and returns a String which I'm storing in output. How can I return this output String when generateRequest() is called from another class?
public String generateRequest() {
RequestBuilder builder = new RequestBuilder(RequestBuilder.GET, URL.encode(url.getUrl()));
builder.setHeader("Authorization", authHeader);
String title = null;
try {
builder.sendRequest(null, new RequestCallback() {
public void onError(Request request, Throwable exception) {
GWT.log(exception.getMessage());
}
public void onResponseReceived(Request request, Response response) {
String output = processResponse(response);
}
});
} catch (RequestException e) {
GWT.log(e.getMessage());
}
return title;
}
I think you might be misunderstanding something.
You cannot simply return a String because the call is async (i.e. you can't return the String because the String is simply not available yet, at the time you wish to return it).
You could simply wait there until the call result is ready, but this is really terrible practice; that's why you're seeing people suggesting callbacks.
Imagine, this is the code:
statement01;
statement02;
String result = generateRequest(...);
statement04UsingResult;
statement05;
then this will not work, because result will not be available before statement04UsingResult is executed. The Request has not finished yet. (as Andrej already mentioned)
To solve this, split your code:
statement01;
statement02;
generateRequest(...);
create an new void method, wich accepts the result as parameter:
public void newMethod(String result) {
statement04UsingResult;
statement05;
}
and call from inside the onResponseRevieve-method:
builder.sendRequest(null, new RequestCallback() {
public void onError(Request request, Throwable exception) {
GWT.log(exception.getMessage());
}
public void onResponseReceived(Request request, Response response) {
newMethod(processResponse(response));
}
});
Hope that helps.
I use this to config my retrofit:
RestAdapter restAdapter = new RestAdapter.Builder()
//add headers to requests
.setRequestInterceptor(getAuthenticatedRequestInterceptor())
.setEndpoint(BASE_URL)
.setConverter(new GsonConverter(getGson()))
.build();
and The getAuthenticatedRequestInterceptor() method adds headers to request:
public AccountRequestInterceptor getAuthenticatedRequestInterceptor() {
AccountRequestInterceptor interceptor = new AccountRequestInterceptor();
Map<String, String> headers = new HashMap<>();
String accessToken = null;
try {
accessToken = TokenProvider.getInstance(mContext).getToken();
} catch (InterruptedException e) {
}
headers.put(HeadersContract.HEADER_AUTHONRIZATION, O_AUTH_AUTHENTICATION + accessToken);
interceptor.setHeader(headers);
return interceptor;
}
getToken() method is:
private synchronized string getToken() throws InterruptedException {
if (!isRefreshing()) {
//This is very important to call notify() on the same object that we call wait();
final TokenProvider myInstance = this;
setRefreshing(true);
MyApplication.getRestClient().getAccountService().getRefreshedToken(mLoginData.getRefreshToken())
.subscribe(new Observer<LoginResponse>() {
#Override
public void onCompleted() {
synchronized (myInstance) {
setRefreshing(false);
myInstance.notifyAll();
}
}
#Override
public void onError(Throwable e) {
synchronized (myInstance) {
setRefreshing(false);
myInstance.notifyAll();
}
}
#Override
public void onNext(LoginResponse loginResponse) {
synchronized (myInstance) {
mLoginData = loginResponse;
mAccountProvider.saveLoginData(loginResponse);
myInstance.notifyAll();
}
}
});
}
this.wait();
return mLoginData.getToken();
}
The TokenProvider.getInstance(mContext).getToken() has a wait() on main thread to get the response from an async method and i know that is a bad thing to do but i need this here to wait for the response to take the token from it and then return the token.how can i do this in a separate thread to avoid waiting on the main thread?
Note:
1 - that this is called before any request with retrofit.
2 - I read this and i know i can refresh token after a fail request, but for business reasons i want to avoid having an invalid token.
3 - I call MyApplication.getRestClient().getAccountService().login(loginRequest,callback...) in my Activity and before adding token everything happened in background thread. so I want to use my token and do not block the main thread.
UPDATE: I added the following Interceptor to my new OkHttp:
public class RequestTokenInterceptor implements Interceptor {
#Override
public Response intercept(Interceptor.Chain chain) throws IOException {
Request request = chain.request();
Request newRequest;
try {
Log.d("addHeader", "Before");
String token = TokenProvider.getInstance(mContext).getToken();
if (token != null) {
newRequest = request.newBuilder()
.addHeader("Bearer", token)
.build();
} else {
// I want to cancel the request or raise an exception to catch it in onError method
// of retrofit callback.
}
} catch (InterruptedException e) {
Log.d("addHeader", "Error");
e.printStackTrace();
return chain.proceed(request);
}
Log.d("addHeader", "after");
return chain.proceed(newRequest);
}
}
Now how can i cancel the request or raise an exception to catch it in onError method of retrofit callback, if token is null?
It's a little bit strange issue but let me try to help you. :)
As you know you can refresh token after a failed request with retrofit using response interceptor.
Let's try to use interceptor before request.
public class RequestTokenInterceptor implements Interceptor {
#Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
// Here where we'll try to refresh token.
// with an retrofit call
// After we succeed we'll proceed our request
Response response = chain.proceed(request);
return response;
}
}
And when you're creating your api create a new HttpClient:
OkHttpClient client = new OkHttpClient();
client.interceptors().add(new RequestTokenInterceptor());
And add your http client to your adapter like below:
.setClient(new OkClient(client))
If this works, before every request you'll try to refresh token first and then will proceed your api request. So in ui there'll be no difference with your normal api calls.
Edit:
I'm editing my answer too. If you want to return an error in else case if token null, in else case you can create your custom response:
private Response(Builder builder) {
this.request = builder.request;
this.protocol = builder.protocol;
this.code = builder.code;
this.message = builder.message;
this.handshake = builder.handshake;
this.headers = builder.headers.build();
this.body = builder.body;
this.networkResponse = builder.networkResponse;
this.cacheResponse = builder.cacheResponse;
this.priorResponse = builder.priorResponse;
}
or simply you can return a null response. if you build your custom response and set your code not to 200 such as 401 or 400+ you'll receive that response in Retrofit's callbacks failure method. Than you can do what ever you want.
If you return null you'll get a RuntimeException i think and still you can catch response in your callback's failure method.
After you create your own response in else you can create your custom callback and catch your null response and transform your custom error how ever you want like below:
public abstract class DefaultRequestCallback<T> implements Callback<T> {
public abstract void failure(YourCustomException ex);
public abstract void success(T responseBean);
#Override
public void success(T baseResponseBean, Response response) {
if (response == null) {
// Here we catch null response and transform it to our custom Exception
failure(new YourCustomException());
}
} else {
success(baseResponseBean);
}
}
#Override
public void failure(RetrofitError error) {
// Here's your failure method.
// Also you can transform default retrofit errors to your customerrors
YourCustomException ex = new YourCustomException();
failure(ex);
}
}
This can help you i think.
Edit 2:
You can build a new Response like below. There's a builder pattern in Retrofit's Response class. You can check it from there.
Response response = new Response.Builder().setCode(401).setMessage("Error Message").build();
You could make all long actions in AsyncTask doInBackground method, while in onPre- and onPostExecute you could show/hide some progress bars when user is waiting
Ok, I think if you are calling your getAuthenticatedRequestInterceptor() on the main thread and which in turns call getInstance(),in which i feel you would be creating an object of Type TokenProvider hence when you create this object in the main thread your object.wait() runs on main thread hence to run this on a background thread probably modify your getAuthenticatedRequestInterceptor() method to execute the following lines in a new thread.
try {
accessToken = TokenProvider.getInstance(mContext).getToken();
} catch (InterruptedException e) {
}
headers.put(HeadersContract.HEADER_AUTHONRIZATION, O_AUTH_AUTHENTICATION + accessToken);
interceptor.setHeader(headers);
return interceptor;
but this will have problems for notifying your RestAdapter as the main thread will proceed executing, hence i would suggest
you call getAuthenticatedRequestInterceptor() method first in a new thread and then notify your main thread to build your RestAdapter.This will free your main thread but with the strategy you are employing you will have to wait until you receive the token to make any calls.
It is in AKKA documentation written that
... Actors should not block (i.e. passively wait while occupying a Thread) on some external entity, which might be a lock, a network socket, etc. The blocking operations should be done in some special-cased thread which sends messages to the actors which shall act on them.
source http://doc.akka.io/docs/akka/2.0/general/actor-systems.html#Actor_Best_Practices
I have found the following information at the moment :
I read Sending outbound HTTP request from Akka / Scala and checked the example at https://github.com/dsciamma/fbgl1
I found following article http://nurkiewicz.blogspot.de/2012/11/non-blocking-io-discovering-akka.html explaining how to use https://github.com/AsyncHttpClient/async-http-client non blocking http client with akka. But is written in Scala.
How can i write an actor that make non-blocking http requests?
It must downlad a remote url page as file and than send the generated file object to the master actor. master actor then sends this request to parser actor to parse the file...
In the last response, Koray is using a wrong reference for the sender, the correct way to do it is:
public class ReduceActor extends UntypedActor {
#Override
public void onReceive(Object message) throws Exception {
if (message instanceof URI) {
URI url = (URI) message;
AsyncHttpClient asyncHttpClient = new AsyncHttpClient();
final ActorRef sender = getSender();
asyncHttpClient.prepareGet(url.toURL().toString()).execute(new AsyncCompletionHandler<Response>() {
#Override
public Response onCompleted(Response response) throws Exception {
File f = new File("e:/tmp/crawler/" + UUID.randomUUID().toString() + ".html");
// Do something with the Response
// ...
// System.out.println(response1.getStatusLine());
FileOutputStream fao = new FileOutputStream(f);
IOUtils.copy(response.getResponseBodyAsStream(), fao);
System.out.println("File downloaded " + f);
sender.tell(new WordCount(f));
return response;
}
#Override
public void onThrowable(Throwable t) {
// Something wrong happened.
}
});
} else
unhandled(message);
}
Checkout this other thread of akka: https://stackoverflow.com/a/11899690/575746
I have implemented this in this way.
public class ReduceActor extends UntypedActor {
#Override
public void onReceive(Object message) throws Exception {
if (message instanceof URI) {
URI url = (URI) message;
AsyncHttpClient asyncHttpClient = new AsyncHttpClient();
asyncHttpClient.prepareGet(url.toURL().toString()).execute(new AsyncCompletionHandler<Response>() {
#Override
public Response onCompleted(Response response) throws Exception {
File f = new File("e:/tmp/crawler/" + UUID.randomUUID().toString() + ".html");
// Do something with the Response
// ...
// System.out.println(response1.getStatusLine());
FileOutputStream fao = new FileOutputStream(f);
IOUtils.copy(response.getResponseBodyAsStream(), fao);
System.out.println("File downloaded " + f);
getSender().tell(new WordCount(f));
return response;
}
#Override
public void onThrowable(Throwable t) {
// Something wrong happened.
}
});
} else
unhandled(message);
}
I'm trying create a simple audio stream server like a concept proof, but I'm having some dificulties.
I'm streaming a single file to start, I searched but didn't found enought information to create a audio stream server, so I just created a simple server based on my little knowledge about servers. I've created it with netty passing the stream to ChunkedStream object and wrote it on channel:
public class CastServerHandler extends SimpleChannelHandler {
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageEvent e)
throws Exception {
HttpRequest request = (HttpRequest) e.getMessage();
if (request.getMethod() != GET) {
sendError(ctx, METHOD_NOT_ALLOWED);
return;
}
HttpResponse response = new DefaultHttpResponse(HTTP_1_1, OK);
System.out.println(response.toString());
Channel channel = e.getChannel();
channel.write(response);
ChannelFuture writeFuture;
StreamSource source = StreamSource.getInstance();
ChunkedStream stream = new ChunkedStream(source.getLiveStream());
writeFuture = channel.write(stream);
writeFuture.addListener(new ChannelFutureProgressListener() {
public void operationComplete(ChannelFuture future) {
System.out.println("terminou");
future.getChannel().close();
}
public void operationProgressed(ChannelFuture future, long amount,
long current, long total) {
System.out.println("Transferido: " + current + " de " + total);
}
});
if (!isKeepAlive(request)) {
writeFuture.addListener(ChannelFutureListener.CLOSE);
}
}
private void sendError(ChannelHandlerContext ctx, HttpResponseStatus status) {
HttpResponse response = new DefaultHttpResponse(HTTP_1_1, status);
response.setHeader(CONTENT_TYPE, "text/plain; charset=UTF-8");
response.setContent(ChannelBuffers.copiedBuffer(
"Failure: " + status.toString() + "\r\n", CharsetUtil.UTF_8));
// Close the connection as soon as the error message is sent.
ctx.getChannel().write(response)
.addListener(ChannelFutureListener.CLOSE);
}
private void writeLiveStream(Channel channel) {
StreamSource source = StreamSource.getInstance();
ChunkedStream stream = new ChunkedStream(source.getLiveStream());
channel.write(stream);
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, ExceptionEvent e)
throws Exception {
e.getCause().printStackTrace();
e.getChannel().close();
}
}
Ufortunately, I didn't successfully streamed the audio directly to web browser, so I tryied to figure out what icecast returns as response to web browser, and it return these properties in header:
Cache-Control:no-cache
Content-Type:application/ogg
Server:Icecast 2.3.2
ice-audio-info:samplerate=44100;channels=2;quality=3%2e00
icy-description:Stream de teste
icy-genre:Rock
icy-name:Radio teste Brevleq
icy-pub:0
Is there a simple way netty use to put these content in HttpResponse header (specially Content-type:applicatio/ogg)?? I hope this is the problem...
See the API of HttpResponse.
It has setHeader method.
I'd consider going with a straight binary protocol, and creating an HTTP interface only for a proxy. There's no reason to deal with a text based protocol for something like this.