I have tried a bunch of libraries to make a REST POST call using HttpConnection in Java asynchronously. I have tried many open source libraries and none of them seems to serve my purpose. Is there any way to do this in core Java.
Without knowing your requirements or expectations:
Here is a simple example without proper error handling which shows how an async HTTP call can be done using Java 8
public static void main(String ... args) throws InterruptedException, ExecutionException, TimeoutException {
Future<Object> futureResult = getObjectAsync();
Object value = futureResult.get(500, TimeUnit.MILLISECONDS);
}
public static Future<Object> getObjectAsync() {
return CompletableFuture.supplyAsync(() -> doHttpCall());
}
static Object doHttpCall() {
try {
HttpURLConnection urlConnection =
(HttpURLConnection) new URL("http://example.net/something").openConnection();
urlConnection.setRequestMethod("POST");
try (OutputStreamWriter out = new OutputStreamWriter(urlConnection.getOutputStream())) {
out.write("params as json");
}
try (InputStreamReader in = new InputStreamReader(urlConnection.getInputStream())) {
// convert to Object
return new Object();
}
} catch (IOException e ) {
throw new RuntimeException(e);
}
}
Related
I'm using a CompletableFuture for an async operation to make a download of a file and to save its contents via an OutputStream. The code below works, but the compiler gives me a warning to either use try-with-resources or to close the OutputStream in a finally-clause, although it is closed after the Future completes in whenComplete.
Code:
final OutputStream outputStream = Files.newOutputStream(file.toPath());
final String url = "https://example.com/some-download.zip";
final CompletionStage<WSResponse> futureResponse = this.client
.url(url)
.setMethod("GET")
.stream();
futureResponse.thenCompose(res -> {
downloadTask.setTotalBytes(res);
Source<ByteString, ?> responseBody = res.getBodyAsSource();
Sink<ByteString, CompletionStage<akka.Done>> outputWriter =
Sink.foreach(bytes -> {
downloadTask.addReceivedBytes(bytes.size());
System.out.println(downloadTask.getProgressAsString());
outputStream.write(bytes.toArray());
});
return responseBody.runWith(outputWriter, this.materializer);
}).whenComplete((res, error) -> {
try {
outputStream.close();
} catch (final IOException e) {
e.printStackTrace();
}
});
Warning:
Problem:
When I use try-with-resources then the program closes the OutPutStream before anything is written to the file, because of the async nature of CompletableFuture which doesn't block.
So is there a way to declare the OutputStream inside the CompletionStage and pass it down the line?
The suggestion of #AndyTurner in his comment was correct. I just had to tweak the code a little bit to make it work.
I didn't realize that responseBody.runWith() returns another CompletionStage, so that using try-with-resources or finally closed the OutputStream in the thenComponse block, so the returned CompletionStage (from responseBody.runWith()) couldn't write to it anymore. This caused the error. So we simply need to process responseBody.runWith() 'synchronously' in the same block by using .toCompletableFuture().get(). This is no problem, because the block itself runs in another thread, i.e. it stays asynchronous. Since we don't return anything anymore, we also need to use thenAccept, which takes a Consumer instead of thenCompose, which takes a Function.
final String url = "https://example.com/some-download.zip";
final CompletionStage<WSResponse> futureResponse = this.client
.url(url)
.setMethod("GET")
.stream();
futureResponse
.thenAccept(res -> {
try (OutputStream outputStream = Files.newOutputStream(file.toPath())) {
downloadTask.setTotalBytes(res);
Source<ByteString, ?> responseBody = res.getBodyAsSource();
Sink<ByteString, CompletionStage<akka.Done>> outputWriter =
Sink.foreach(bytes -> {
downloadTask.addReceivedBytes(bytes.size());
System.out.println(downloadTask.getProgressAsString());
outputStream.write(bytes.toArray());
});
responseBody.runWith(outputWriter, this.materializer).toCompletableFuture().get();
} catch (IOException | InterruptedException | ExecutionException e) {
e.printStackTrace();
}
});
I am trying to throw a timeout exception in the code below. I tried a simple condition but it's not the proper way.
My question is how can I distinct the timeout exception from SOAPException?
URL endpoint = new URL(null,
urlStr,
new URLStreamHandler() {
// The url is the parent of this stream handler, so must create clone
protected URLConnection openConnection(URL url) throws IOException {
URL cloneURL = new URL(url.toString());
HttpURLConnection cloneURLConnection = (HttpURLConnection) cloneURL.openConnection();
// TimeOut settings
cloneURLConnection.setConnectTimeout(10000);
cloneURLConnection.setReadTimeout(10000);
return cloneURLConnection;
}
});
try {
response = connection.call(request, endpoint);
} catch (SOAPException soapEx) {
if(soapEx.getMessage().contains("Message send failed")) {
throw new TimeoutExpirationException();
} else {
throw soapEx;
}
}
The following lines are from open jdk source code of call method. In the code they are only catching with Exception (also with chaining? comment). I don't think there is other way unless Oracle jdk handles this differently.
You can still try something like if(soapEx.getCause() instanceof SomeTimeoutException) (not sure if this will work)
try {
SOAPMessage response = post(message, (URL)endPoint);
return response;
} catch (Exception ex) {
// TBD -- chaining?
throw new SOAPExceptionImpl(ex);
}
If you want to check the source code HttpSoapConnection
After some hours of testing I found the proper way to distict the SOAPException from Timeout related exceptions. So the solution is to take the parent cause field of the exception and check if it's an instance of SocketTimeoutException.
try {
response = connection.call(request, endpoint);
} catch (SOAPException soapEx) {
if(soapEx.getCause().getCause() instanceof SocketTimeoutException) {
throw new TimeoutExpirationException(); //custom exception
} else {
throw soapEx;
}
}
I am creating a very basic webserver using netty and java. I will have basic functionality. It's main responsibilities would be to serve responses for API calls done from a client (e.g a browser, or a console app I am building) in JSON form or send a zip file. For that reason I have created the HttpServerHanddler class which is responsible for getting the request, parsing it to find the command and call the appropriate api call.It extends SimpleChannelInboundHandler
and overrides the following functions;
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
LOG.debug("channelActive");
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx) {
LOG.debug("In channelComplete()");
ctx.flush();
}
#Override
public void channelRead0(ChannelHandlerContext ctx, Object msg)
throws IOException {
ctx = processMessage(ctx, msg);
if (!HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
ctx.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(
ChannelFutureListener.CLOSE);
}
}
private ChannelHandlerContext processMessage(ChannelHandlerContext ctx, Object msg){
if (msg instanceof HttpRequest) {
HttpRequest request = this.request = (HttpRequest) msg;
if (HttpHeaders.is100ContinueExpected(request)) {
send100Continue(ctx);
}
//parse message to find command, parameters and cookies
ctx = executeCommand(command, parameters, cookies)
}
if (msg instanceof LastHttpContent) {
LOG.debug("msg is of LastHttpContent");
if (!HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
ctx.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(
ChannelFutureListener.CLOSE);
}
}
return ctx;
}
private ChanndelHandlerContext executeCommand(String command, HashMap<String, List<String>>> parameters, Set<Cookie> cookies>){
//switch case to see which command has to be invoked
switch(command){
//many cases
case "/report":
ctx = myApi.getReport(parameters, cookies); //This is a member var of ServerHandler
break;
//many more cases
}
return ctx;
}
In my Api class that has the getReport function.
getReport
public ChannelHandlerContext getReportFile(Map<String, List<String>> parameters,
Set<Cookie> cookies) {
//some initiliazations. Actual file handing happens bellow
File file = new File(fixedReportPath);
RandomAccessFile raf = null;
long fileLength = 0L;
try {
raf = new RandomAccessFile(file, "r");
fileLength = raf.length();
LOG.debug("creating response for file");
this.response = Response.createFileResponse(fileLength);
this.ctx.write(response);
this.ctx.write(new HttpChunkedInput(new ChunkedFile(raf, 0,
fileLength,
8192)),
this.ctx.newProgressivePromise());
} catch (FileNotFoundException fnfe) {
LOG.debug("File was not found", fnfe);
this.response = Response.createStringResponse("failure");
this.ctx.write(response);
} catch (IOException ioe) {
LOG.debug("Error getting file size", ioe);
this.response = Response.createStringResponse("failure");
this.ctx.write(response);
} finally {
try {
raf.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
return this.ctx;
}
Response class is responsible for handling various types of response creations (JsonString JsonArray JsonInteger File, etc)
public static FullHttpResponse createFileResponse(long fileLength) {
FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK);
HttpHeaders.setContentLength(response, fileLength);
response.headers().set(HttpHeaders.Names.CONTENT_TYPE, "application/octet-stream");
return response;
}
My Api works great for my Json responses(easier to achieve) but It won't work well with my json responses, but not with my file response. When making a request from e.g chrome it only hangs and does not download the file. Should I do something else when downloading a file using netty? I know its not the best wittern code, I still think I have some bits and pieces missing from totally understanding the code, but I would like your advice on how to handle download on my code. For my code I took under consideration this and this
First, some remarks on your code...
Instead of returning ctx, I would prefer to return the last Future for the last command, such that your last event (no keep alive on) could use it directly.
public void channelRead0(ChannelHandlerContext ctx, Object msg)
throws IOException {
ChannelFuture future = processMessage(ctx, msg);
if (future != null && !HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
future.addListener(ChannelFutureListener.CLOSE);
}
}
Doing this way will allow to directly close without having any "pseudo" send, even empty.
Important: Note that in Http, the response is managed such that there are chunk send for all data after the first HttpResponse item, until the last one which is empty (LastHttpContent). Sending another empty one (Empty chunk but not LastHttpContent) could break the internal logic.
Moreover, you're doing the work twice (once in read0, once in processMessage), which could lead to some issues perhaps.
Also, since you check for KeepAlive, you should ensure to set it back in the response:
if (HttpHeaders.isKeepAlive(request)) {
response.headers().set(CONNECTION, HttpHeaders.Values.KEEP_ALIVE);
}
On your send, you have 2 choices (depending on the usage of SSL or not): you've selected only the second one, which is more general, so of course valid in all cases but less efficient.
// Write the content.
ChannelFuture sendFileFuture;
ChannelFuture lastContentFuture;
if (ctx.pipeline().get(SslHandler.class) == null) {
sendFileFuture =
ctx.write(new DefaultFileRegion(raf.getChannel(), 0, fileLength), ctx.newProgressivePromise());
// Write the end marker.
lastContentFuture = ctx.writeAndFlush(LastHttpContent.EMPTY_LAST_CONTENT); // <= last writeAndFlush
} else {
sendFileFuture =
ctx.writeAndFlush(new HttpChunkedInput(new ChunkedFile(raf, 0, fileLength, 8192)),
ctx.newProgressivePromise()); // <= last writeAndFlush
// HttpChunkedInput will write the end marker (LastHttpContent) for us.
lastContentFuture = sendFileFuture;
}
This is this lastContentFuture that you can get back to the caller to check the KeepAlive.
Note however that you didn't include a single flush there (except with your EMPTY_BUFFER but which can be the main reason of your issue there!), contrary to the example (from which I copied the source).
Note that both use a writeAndFlush for the last call (or the unique one).
After reading: Getting the 'external' IP address in Java
code:
public static void main(String[] args) throws IOException
{
URL whatismyip = new URL("http://automation.whatismyip.com/n09230945.asp");
BufferedReader in = new BufferedReader(new InputStreamReader(whatismyip.openStream()));
String ip = in.readLine(); //you get the IP as a String
System.out.println(ip);
}
I thought I was a winner but I get the following error
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: http://automation.whatismyip.com/n09230945.asp
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
at getIP.main(getIP.java:12)
I think this is because the server isnt responding quick enough, is there anyway to ensure that it will get the external ip?
EDIT: okay so its getting rejected, anyone else know of another site that can do the same function
public static void main(String[] args) throws IOException
{
URL connection = new URL("http://checkip.amazonaws.com/");
URLConnection con = connection.openConnection();
String str = null;
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
str = reader.readLine();
System.out.println(str);
}
Before you run the following code take a look at this: http://www.whatismyip.com/faq/automation.asp
public static void main(String[] args) throws Exception {
URL whatismyip = new URL("http://automation.whatismyip.com/n09230945.asp");
URLConnection connection = whatismyip.openConnection();
connection.addRequestProperty("Protocol", "Http/1.1");
connection.addRequestProperty("Connection", "keep-alive");
connection.addRequestProperty("Keep-Alive", "1000");
connection.addRequestProperty("User-Agent", "Web-Agent");
BufferedReader in =
new BufferedReader(new InputStreamReader(connection.getInputStream()));
String ip = in.readLine(); //you get the IP as a String
System.out.println(ip);
}
While playing with Go I saw your question. I made a quick App on Google App Engine using Go:
Hit this URL:
http://agentgatech.appspot.com/
Java code:
new BufferedReader(new InputStreamReader(new URL('http://agentgatech.appspot.com').openStream())).readLine()
Go code for the app which you can copy and make your own app:
package hello
import (
"fmt"
"net/http"
)
func init() {
http.HandleFunc("/", handler)
}
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, r.RemoteAddr)
}
Some servers has triggers that blocks access from "non-browsers". They understand that you are some kind of automatic app that can do a DOS attack. To avoid this, you can try to use a lib to access the resource and set the "browser" header.
wget works in this way:
wget -r -p -U Mozilla http://www.site.com/resource.html
Using Java, you can use the HttpClient lib and set the "User-Agent" header.
Look the topic 5 of "Things To Try" section.
Hope this can help you.
A 403 response indicates that the server is explicitly rejecting your request for some reason. Contact the operator of WhatIsMyIP for details.
We've set up CloudFlare and as designed they're challenging unfamiliar useragents. If you can set your UA to something common, you should be able to gain access.
You can use another web service like this; http://freegeoip.net/static/index.html
Using the Check IP address link on AWS worked for me.Please note that MalformedURLException,IOException are to be added as well
public String getPublicIpAddress() throws MalformedURLException,IOException {
URL connection = new URL("http://checkip.amazonaws.com/");
URLConnection con = connection.openConnection();
String str = null;
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
str = reader.readLine();
return str;
}
This is how I do it with rxJava2 and Butterknife. You'll want to run the networking code in another thread because you'll get an exception for running network code on the main thread!
I use rxJava instead of AsyncTask because the rxJava cleans up nicely when the user moves on to the next UI before the thread is finished. (this is super useful for very busy UI's)
public class ConfigurationActivity extends AppCompatActivity {
// VIEWS
#BindView(R.id.externalip) TextInputEditText externalIp;//this could be TextView, etc.
// rxJava - note: I have this line in the base class - for demo purposes it's here
private CompositeDisposable compositeSubscription = new CompositeDisposable();
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.my_wonderful_layout);
ButterKnife.bind(this);
getExternalIpAsync();
}
// note: I have this code in the base class - for demo purposes it's here
#Override
protected void onStop() {
super.onStop();
clearRxSubscriptions();
}
// note: I have this code in the base class - for demo purposes it's here
protected void addRxSubscription(Disposable subscription) {
if (compositeSubscription != null) compositeSubscription.add(subscription);
}
// note: I have this code in the base class - for demo purposes it's here
private void clearRxSubscriptions() {
if (compositeSubscription != null) compositeSubscription.clear();
}
private void getExternalIpAsync() {
addRxSubscription(
Observable.just("")
.map(s -> getExternalIp())
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe((String ip) -> {
if (ip != null) {
externalIp.setText(ip);
}
})
);
}
private String getExternalIp() {
String externIp = null;
try {
URL connection = new URL("http://checkip.amazonaws.com/");
URLConnection con = connection.openConnection(Proxy.NO_PROXY);
con.setConnectTimeout(1000);//low value for quicker result (otherwise takes about 20secs)
con.setReadTimeout(5000);
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
externIp = reader.readLine();
} catch (Exception e) {
e.printStackTrace();
}
return externIp;
}
}
UPDATE - I've found that URLConnection is really quite shit; it'll take a long time to get a result, not really time out very well, etc. The code below improves the situation with OKhttp
private String getExternalIp() {
String externIp = "no connection";
OkHttpClient client = new OkHttpClient();//should have this as a member variable
try {
String url = "http://checkip.amazonaws.com/";
Request request = new Request.Builder().url(url).build();
Response response = client.newCall(request).execute();
ResponseBody responseBody = response.body();
if (responseBody != null) externIp = responseBody.string();
} catch (IOException e) {
e.printStackTrace();
}
return externIp;
}
I have programmed a JAX-RS web service with Jersey that queries prices from different websites and gives the result back as XML through JAXB annotated classes. Unfortunately some websites take up to 15 seconds to respond so I am using multiple threads to inquire those prices.
I would like to write a client to this webservice now and my web users will not want to wait for 30 seconds after they hit 'search' for the result to come so my idea is dynamically updating the result table as the results from my JAX-RS webservice come back.
After 30 seconds my webservice should time out and close the <result>-Element or after all threads completed.
Right now my webservice runs all threads and gives back the result after all trheads are completed, I would like to dynamically add results to the XML output as they come, how can I accomplish that?
The structure of the XML response is:
<result>
<articles>
<article>
content of article
</article>
</articles>
As the webservice gets results from websites it adds new articles to the XML
</result>
RequestController.java
#Path("/request")
public class RequestController {
#GET
#Produces("application/xml")
public Response getRequest(#QueryParam("part") String part) {
response = new Response();
driverController = new DriverController(this.response, this.part);
this.response = driverController.query();
return this.response;
}
}
DriverController.java
public class DriverController {
public Response query() {
CompletionService<Deque<Article>> completionService = new ExecutorCompletionService<Deque<Article>>(
Worker.getThreadPool());
final Deque<Article> articleQueue = new LinkedList<Article>();
int submittedTasks = 0;
// This threadwill take about 4 seconds to finish
Driver driverA = new DriverA(this.part,
this.currency, this.language);
// This thread will take about 15 seconds to finish
Driver driverN = new DriverN(this.part,
this.currency, this.language);
completionService.submit(driverA);
submittedTasks++;
completionService.submit(driverN);
submittedTasks++;
for (int i = 0; i < submittedTasks; i++) {
log.info("Tasks: " + submittedTasks);
try {
Future<Deque<Article>> completedFuture = completionService.take();
try {
Deque<Article> articleQueueFromThread = completedFuture.get();
if (articleQueueFromThread != null) {
articleQueue.addAll(articleQueueFromThread);
response.setStatus("OK");
}
} catch (ExecutionException e) {
log.error(e.getMessage());
e.printStackTrace();
}
} catch (InterruptedException e) {
log.error(e.getMessage());
e.printStackTrace();
}
}
for (Article article : articleQueue) {
this.response.addArticle(article);
}
return this.response;
}
}
Response.java
#XmlRootElement
public class Response {
Queue<Article> queue = new ConcurrentLinkedQueue<Article>();
private String status;
private String code;
private String message;
private List<Article> articles = new ArrayList<Article>();
public Response(){
}
public void setMessage(String message) {
this.message = message;
}
#XmlAttribute
public String getMessage() {
return message;
}
public void setStatus(String status) {
this.status = status;
}
#XmlAttribute
public String getStatus() {
return status;
}
public void setCode(String code) {
this.code = code;
}
#XmlAttribute
public String getCode() {
return code;
}
public void addArticle(Article article) {
this.articles.add(article);
System.out.println("Response: ADDED ARTICLE TO RESPONSE");
}
#XmlElement(name = "article")
#XmlElementWrapper(name = "articles")
public List<Article> getArticles() {
return articles;
}
}
I started to adapt your code to do it, but I decided it was easier to work up an independent example. The example starts a Grizzly+Jersey server with a single resource class in it. A GET on the resource spawns three threads that delay for 2, 4, and 6 seconds before returning some objects. After the server starts, another thread makes a request to the server. When you run it, you can plainly see that the requester receives chunks of XML as the respective threads finish their work in the server. The one thing it doesn't do is wrap separately-delivered XML chunks in a single root element since that should be relatively trivial.
The entire executable source is below, and if you have maven and git, you can clone it from github and run it with:
git clone git://github.com/zzantozz/testbed.git tmp
cd tmp
mvn compile exec:java -Dexec.mainClass=rds.jersey.JaxRsResource -pl jersey-with-streaming-xml-response
Source:
import com.sun.grizzly.http.SelectorThread;
import com.sun.jersey.api.container.grizzly.GrizzlyWebContainerFactory;
import javax.ws.rs.*;
import javax.ws.rs.core.StreamingOutput;
import javax.xml.bind.*;
import javax.xml.bind.annotation.*;
import java.io.*;
import java.net.*;
import java.util.*;
import java.util.concurrent.*;
#Path("/streaming")
public class JaxRsResource {
private static ExecutorService executorService = Executors.newFixedThreadPool(4);
private static int fooCounter;
private Marshaller marshaller;
public JaxRsResource() throws JAXBException {
marshaller = JAXBContext.newInstance(Foo.class).createMarshaller();
marshaller.setProperty("jaxb.fragment", Boolean.TRUE);
}
#GET
#Produces("application/xml")
public StreamingOutput streamStuff() {
System.out.println("Got request for streaming resource; starting delayed response threads");
final List<Future<List<Foo>>> futureFoos = new ArrayList<Future<List<Foo>>>();
futureFoos.add(executorService.submit(new DelayedFoos(2)));
futureFoos.add(executorService.submit(new DelayedFoos(4)));
futureFoos.add(executorService.submit(new DelayedFoos(6)));
return new StreamingOutput() {
public void write(OutputStream output) throws IOException {
for (Future<List<Foo>> futureFoo : futureFoos) {
writePartialOutput(futureFoo, output);
output.write("\n".getBytes());
output.flush();
}
}
};
}
private void writePartialOutput(Future<List<Foo>> futureFoo, OutputStream output) {
try {
List<Foo> foos = futureFoo.get();
System.out.println("Server sending a chunk of XML");
for (Foo foo : foos) {
marshaller.marshal(foo, output);
}
} catch (JAXBException e) {
throw new IllegalStateException("JAXB couldn't marshal. Handle it.", e);
} catch (InterruptedException e) {
throw new IllegalStateException("Task was interrupted. Handle it.", e);
} catch (ExecutionException e) {
throw new IllegalStateException("Task failed to execute. Handle it.", e);
}
}
class DelayedFoos implements Callable<List<Foo>> {
private int delaySeconds;
public DelayedFoos(int delaySeconds) {
this.delaySeconds = delaySeconds;
}
public List<Foo> call() throws Exception {
Thread.sleep(delaySeconds * 1000);
return Arrays.asList(new Foo(fooCounter++), new Foo(fooCounter++), new Foo(fooCounter++));
}
}
public static void main(String[] args) throws IOException {
System.out.println("Starting Grizzly with the JAX-RS resource");
final String baseUri = "http://localhost:9998/";
final Map<String, String> initParams = new HashMap<String, String>();
initParams.put("com.sun.jersey.config.property.packages", "rds.jersey");
SelectorThread threadSelector = GrizzlyWebContainerFactory.create(baseUri, initParams);
System.out.println("Grizzly started");
System.out.println("Starting a thread to request the streamed XML");
executorService.submit(new HttpRequester(baseUri + "streaming"));
}
}
#XmlRootElement
class Foo {
#XmlElement
private int id;
Foo() {}
public Foo(int id) {
this.id = id;
}
}
class HttpRequester implements Runnable {
private String url;
public HttpRequester(String url) {
this.url = url;
}
public void run() {
try {
System.out.println("Doing HTTP GET on " + url);
HttpURLConnection urlConnection = (HttpURLConnection) new URL(url).openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
System.out.println("Client got: " + line);
}
System.exit(0);
} catch (IOException e) {
throw new IllegalStateException("Some bad I/O happened. Handle it.", e);
}
}
}
Important points/differences to take note of:
Returning a Response from your resource method indicates that the entire response is contained in that object and doesn't allow for incremental updates to the response. Return a StreamingOutput instead. That tells Jersey that you'll be sending back a stream of data, which you can append to at will until you're done. The StreamingOutput gives you access to an OutputStream, which is what you use to send incremental updates and is the key to this whole thing. Of course, that means you have to handle the marshaling yourself. Jersey can only do the marshaling if you're returning the entire response at once.
Since the OutputStream is how you send back the data a little at a time, you either have to do the threading in your JAX-RS resource or pass the OutputStream down to your DriverController and write to it there.
Be sure to invoke flush() on the OutputStream if you want to force it to send out data immediately. Otherwise, nothing will be sent to the client until whatever internal buffer is filled up. Note that invoking flush() yourself circumvents the purpose of the buffer and makes your app more chatty.
All in all, to apply this to your project, the primary thing to do is change your resource method to return a StreamingOutput implementation and invoke your DriverController from inside that implementation, passing the OutputStream to the DriverController. Then in the DriverController, when you get some Articles back from a thread, instead of adding it to a queue for later, write it to the OutputStream immediately.
#Ryan Stewart: how would we resolve same issue in axis2.x SOAP based web service kind of environment and HTML page as web client.
What I think is DriverController can keep Future objects in session and returns very first available response(article) with a unique session identifier to client....then client can make another webservice call (preferably thru Ajax+jquery) passing saved session identifier which would trigger DriverController to search more results and send back....is it a viable solution? Would it applicable for above environment too.