Develop Reference

FileWriter closes websocket connection?

I am new to server programming and websockets and I've learnt a little bit of Java 8 this year. In school we had a project where a client webpage opens your webcam, takes a photo of a barcode and then shows a photo and the nutritional value of said product. You can also just send a raw barcode number and that is what is done in this example
My side of the project was to implement a java websocket server (the backend) using the glassfish tyrus library, then receiving the number of the barcode in string format and making a request to openfoodfacts.org using their api. Finally I parsed the json file and sent it back as string format so the client app can read the string and show the correct information (product name, image url, etc)
My code is organized into two files, Serveur.java establishes a websocket server for the client to connect to and ProduitApi.java gets the information from openfoodfacts.org with the given barcode from the client.
public class Serveur {
#javax.websocket.server.ServerEndpoint(value = "/websocket")
public static class EndPoint {
#javax.websocket.OnClose
public void onClose(javax.websocket.Session session, javax.websocket.CloseReason close_reason) {
System.out.println("onClose: " + close_reason.getReasonPhrase());
}
#javax.websocket.OnError
public void onError(javax.websocket.Session session, Throwable throwable) {
System.out.println("onError: " + throwable.getMessage());
}
#javax.websocket.OnMessage
public void onMessage(javax.websocket.Session session, String message) {
System.out.println("Message from client: " + message);
//Creation du produit avec le message du client
try {
ProduitApi produit = new ProduitApi(message);
session.getBasicRemote().sendText(produit.print());
} catch (Exception e) {
e.printStackTrace();
}
}
#javax.websocket.OnOpen
public void onOpen(javax.websocket.Session session, javax.websocket.EndpointConfig ec) throws java.io.IOException {
System.out.println("OnOpen... " + ec.getUserProperties().get("Author"));
session.getBasicRemote().sendText("{\"Handshaking\": \"Yes\"}");
}
}
public static void main(String[] args) {
Server server;
server = new Server ("localhost", 8025, "/BetterFood", null, EndPoint.class);
try {
server.start();
System.out.println("--- server is running");
System.out.println(java.nio.file.FileSystems.getDefault().getPath("client") );
System.out.print("Please press a key to stop the server.");
java.io.BufferedReader reader = new java.io.BufferedReader(new java.io.InputStreamReader(System.in));
reader.readLine();
} catch (Exception e) {
e.printStackTrace();
} finally {
server.stop();
}
}
}
as you can see, when I receive the 'barcode' message, onMessage() gets called. There I instantiate an object of class ProduitApi to use the barcode to then return the information
This is my ProduitApi file without some unnecessary details
package com.gabi.serveur;
/**
*
* #author gabriel
*/
[imports]
public class ProduitApi {
private java.lang.String barcode;
final private java.net.URL url;
private java.net.URLConnection connection;
JsonObjectBuilder constructeur_objet = Json.createObjectBuilder();
String string_json;
ProduitApi(java.lang.String barcode)throws MalformedURLException, IOException {
this.barcode = barcode;
this.url = new java.net.URL("http://world.openfoodfacts.org/api/v0/product/" + this.barcode + ".json");
connection = url.openConnection();
stream();
}
public void stream() throws IOException{
if (connection != null) {
java.io.InputStreamReader response = new java.io.InputStreamReader(connection.getInputStream());
javax.json.stream.JsonParser parser=javax.json.Json.createParser(response);
while (parser.hasNext()) {
[parsing inputStream into JsonObject]
}
public String print()throws IOException{
string_json = constructeur_objet.build().toString();
System.out.print(string_json);
//FileWriter file = new FileWriter("serveur/src/main/java/com/gabi/serveur/json/final.json");
//file.write(string_json);
//file.close();
return string_json;
}
}
My problem comes from the last function ProduitApi.print() , it is supposed to return the parsed json in string form so I can send it via the sendText() as well as printing the result into my console so I can see if everything went right. As you can see there are somme commented lines; The FileWriter object that I had created was used with the purpose of writing said string to a file in my pc and let me check inside.
HOWEVER
and here is what I don't understand, If I uncomment those lines so that the print function can also write the file to my drive, The Connection Closes and then Opens again
It can be seen in the console where after printing the json string, it prints OnClose, followed by OnOpen Signaling the connection was reset for some reason.
If I remove the FileWriter section, the connection works normally, the client's connection stays open and he can make another request
End of Console Message after request:
...cuits x22 biscuits fourrés - 304g","qte":"304 g","img":"https://images.openfoodfacts.org/images/products/800/050/031/0427/front_fr.177.400.jpg"}onClose: OnOpen... null...
Finally, my question is just why writing to a file makes my program behave this way (resetting the connection). Does it have anything to do with how streams work?
I accidentally commented the filewriter portion when another Ide said it didn't find the file because I had opened the project from a different directory.

How to stream out logs via API on dropwizard?

I would like to stream out logs via api endpoint /logs for dropwizard.
It is harder than what I thought. Dropwizard would read in the configuration via config.yml and keep those information as private. And I have no idea where would I be able to find the logger that logs everything?
Am I missing something?
Is there another way to do this?

This is a streaming example, you can also read up on this here:
calling flush() on Jersey StreamingOutput has no effect
And
Example of using StreamingOutput as Response entity in Jersey
The code:
public class StreamingTest extends io.dropwizard.Application<Configuration> {
#Override
public void run(Configuration configuration, Environment environment) throws Exception {
environment.jersey().property(ServerProperties.OUTBOUND_CONTENT_LENGTH_BUFFER, 0);
environment.jersey().register(Streamer.class);
}
public static void main(String[] args) throws Exception {
new StreamingTest().run("server", "/home/artur/dev/repo/sandbox/src/main/resources/config/test.yaml");
}
#Path("/log")
public static class Streamer {
#GET
#Produces("application/octet-stream")
public Response test() {
return Response.ok(new StreamingOutput() {
#Override
public void write(OutputStream output) throws IOException, WebApplicationException {
while (true) {
output.write("Test \n".getBytes());
output.flush();
try {
Thread.sleep(1000); // simulate waiting for stream
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}).build();
}
}
}
Which does what you want.
A few points The line evironment.jersey().property(ServerProperties.OUTBOUND_CONTENT_LENGTH_BUFFER, 0); is important. It tells the server not to buffer the output before returning it to the caller.
Additionally, you need to explicitly flush out the output by calling output.flush();
However, this seems to be the wrong way to ship logs. Have you ever looked into i.e. logstash? Or network based appenders that stream the logs directly to where you need them to be?
Hope that helps,
--artur

Download files with netty

I am creating a very basic webserver using netty and java. I will have basic functionality. It's main responsibilities would be to serve responses for API calls done from a client (e.g a browser, or a console app I am building) in JSON form or send a zip file. For that reason I have created the HttpServerHanddler class which is responsible for getting the request, parsing it to find the command and call the appropriate api call.It extends SimpleChannelInboundHandler
and overrides the following functions;
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
LOG.debug("channelActive");
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx) {
LOG.debug("In channelComplete()");
ctx.flush();
}
#Override
public void channelRead0(ChannelHandlerContext ctx, Object msg)
throws IOException {
ctx = processMessage(ctx, msg);
if (!HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
ctx.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(
ChannelFutureListener.CLOSE);
}
}
private ChannelHandlerContext processMessage(ChannelHandlerContext ctx, Object msg){
if (msg instanceof HttpRequest) {
HttpRequest request = this.request = (HttpRequest) msg;
if (HttpHeaders.is100ContinueExpected(request)) {
send100Continue(ctx);
}
//parse message to find command, parameters and cookies
ctx = executeCommand(command, parameters, cookies)
}
if (msg instanceof LastHttpContent) {
LOG.debug("msg is of LastHttpContent");
if (!HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
ctx.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(
ChannelFutureListener.CLOSE);
}
}
return ctx;
}
private ChanndelHandlerContext executeCommand(String command, HashMap<String, List<String>>> parameters, Set<Cookie> cookies>){
//switch case to see which command has to be invoked
switch(command){
//many cases
case "/report":
ctx = myApi.getReport(parameters, cookies); //This is a member var of ServerHandler
break;
//many more cases
}
return ctx;
}
In my Api class that has the getReport function.
getReport
public ChannelHandlerContext getReportFile(Map<String, List<String>> parameters,
Set<Cookie> cookies) {
//some initiliazations. Actual file handing happens bellow
File file = new File(fixedReportPath);
RandomAccessFile raf = null;
long fileLength = 0L;
try {
raf = new RandomAccessFile(file, "r");
fileLength = raf.length();
LOG.debug("creating response for file");
this.response = Response.createFileResponse(fileLength);
this.ctx.write(response);
this.ctx.write(new HttpChunkedInput(new ChunkedFile(raf, 0,
fileLength,
8192)),
this.ctx.newProgressivePromise());
} catch (FileNotFoundException fnfe) {
LOG.debug("File was not found", fnfe);
this.response = Response.createStringResponse("failure");
this.ctx.write(response);
} catch (IOException ioe) {
LOG.debug("Error getting file size", ioe);
this.response = Response.createStringResponse("failure");
this.ctx.write(response);
} finally {
try {
raf.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
return this.ctx;
}
Response class is responsible for handling various types of response creations (JsonString JsonArray JsonInteger File, etc)
public static FullHttpResponse createFileResponse(long fileLength) {
FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK);
HttpHeaders.setContentLength(response, fileLength);
response.headers().set(HttpHeaders.Names.CONTENT_TYPE, "application/octet-stream");
return response;
}
My Api works great for my Json responses(easier to achieve) but It won't work well with my json responses, but not with my file response. When making a request from e.g chrome it only hangs and does not download the file. Should I do something else when downloading a file using netty? I know its not the best wittern code, I still think I have some bits and pieces missing from totally understanding the code, but I would like your advice on how to handle download on my code. For my code I took under consideration this and this

First, some remarks on your code...
Instead of returning ctx, I would prefer to return the last Future for the last command, such that your last event (no keep alive on) could use it directly.
public void channelRead0(ChannelHandlerContext ctx, Object msg)
throws IOException {
ChannelFuture future = processMessage(ctx, msg);
if (future != null && !HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
future.addListener(ChannelFutureListener.CLOSE);
}
}
Doing this way will allow to directly close without having any "pseudo" send, even empty.
Important: Note that in Http, the response is managed such that there are chunk send for all data after the first HttpResponse item, until the last one which is empty (LastHttpContent). Sending another empty one (Empty chunk but not LastHttpContent) could break the internal logic.
Moreover, you're doing the work twice (once in read0, once in processMessage), which could lead to some issues perhaps.
Also, since you check for KeepAlive, you should ensure to set it back in the response:
if (HttpHeaders.isKeepAlive(request)) {
response.headers().set(CONNECTION, HttpHeaders.Values.KEEP_ALIVE);
}
On your send, you have 2 choices (depending on the usage of SSL or not): you've selected only the second one, which is more general, so of course valid in all cases but less efficient.
// Write the content.
ChannelFuture sendFileFuture;
ChannelFuture lastContentFuture;
if (ctx.pipeline().get(SslHandler.class) == null) {
sendFileFuture =
ctx.write(new DefaultFileRegion(raf.getChannel(), 0, fileLength), ctx.newProgressivePromise());
// Write the end marker.
lastContentFuture = ctx.writeAndFlush(LastHttpContent.EMPTY_LAST_CONTENT); // <= last writeAndFlush
} else {
sendFileFuture =
ctx.writeAndFlush(new HttpChunkedInput(new ChunkedFile(raf, 0, fileLength, 8192)),
ctx.newProgressivePromise()); // <= last writeAndFlush
// HttpChunkedInput will write the end marker (LastHttpContent) for us.
lastContentFuture = sendFileFuture;
}
This is this lastContentFuture that you can get back to the caller to check the KeepAlive.
Note however that you didn't include a single flush there (except with your EMPTY_BUFFER but which can be the main reason of your issue there!), contrary to the example (from which I copied the source).
Note that both use a writeAndFlush for the last call (or the unique one).

Check client is alive with HttpServletRequest object

I'm writing a Spring web application and I'm mapping the "/do" URL path to the following Controller's method
#Controller
public class MyController
{
#RequestMapping(value="/do", method=RequestMethod.GET)
public String do()
{
File f = new File("otherMethodEnded.tmp");
while (!f.exists())
{
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
}
}
// ok, let's continue
}
}
The otherMethodEnded.tmp file is written by one another Controller's method, so when the client calls the second URL I expect the first method to exit the while loop.
Everything works, except when the client calls the "/do" URL and then closes the connection before the response was received. The problem is that the server remains in the while (!f.exists()) loop even though the client is down and cannot call the second URL to unlock the while loop.
I would try to retrieve the connection status of the "/do" URL and exit the loop when the connection is closed by the client, but I cannot find any way to do so.
I tried with the HttpServletRequest.getSession(false) method but the returned HttpSession object is always not null, so the HttpServletRequest object is not updated in case of connection close of the client.
How can I check whether the client is still waiting for the risponse or not?

The simplest way to verify something is not right is to define a timeout value and then during your loop test if your time spent waiting has exceeded the timeout.
something like:
#Controller
public class MyController
{
private static final long MAX_LOOP_TIME = 1000 * 60 * 5; // 5 minutes? choose a value
#RequestMapping(value="/do", method=RequestMethod.GET)
public String do()
{
File f = new File("otherMethodEnded.tmp");
long startedAt = System.currentTimeMillis()
boolean forcedExit = false;
while (!forcedExit && !f.exists())
{
try {
Thread.sleep(5000);
if (System.currentTimeMillis() - startedAt > MAX_LOOP_TIME) {
forcedExit = true;
}
} catch (InterruptedException e) {
forcedExit = true;
}
}
// ok, let's continue
// if forcedExit , handle error scenario?
}
}
Additionally: InterruptedException is not something to blindly catch and ignore. see this discussion
In your case I would really exit the while loop if you're interrupted.
You only know if the client is no longer waiting on your connection when you notice the output stream you write to (response.outputstream) is closed. But there isn't a way to detect it.
(see this question for details)
Seeing as you've indicated your client does occasional callbacks, you could on the clientside poll if the other call has been completed. If this other call has completed, do the operation, otherwise return directly and have the client do the call again. (assuming you are sending json, but adapt as you require)
something like
public class MyController
{
#RequestMapping(value="/do", method=RequestMethod.GET)
public String do()
{
File f = new File("otherMethodEnded.tmp");
if (f.exists()) {
// do what you set out to do
// ok, let's continue
// and return with a response that indicates the call did what it did
// view that returns json { "result" : "success" }
return "viewThatSIgnalsToClientThatOperationSucceeded";
} else {
// view that returns json: { "result" : "retry" }
return "viewThatSignalsToClientToRetryIn5Seconds";
}
}
}
Then the clientside would run something like: (pseudojavascript as it's been a while)
val callinterval = setInterval(function() { checkServer() }, 5000);
function checkServer() {
$.ajax({
// ...
success: successFunction
});
}
function successFunction(response) {
// connection succeeded
var json = $.parseJSON(response);
if (json.result === "retry") {
// interval will call this again
} else {
clearInterval(callinterval);
if (json.result === "success") {
// do the good stuff
} else if (json.result === "failure") {
// report that the server reported an error
}
}
}
Ofcourse this is just semi-serious code but it's roughly how i'd try it if I were to have the dependency. If this is regarding afile upload, keep in mind that this file may not contain all of the bytes yet. file exists != file = completely uploaded, unless you use move it. cp / scp / etc. is not atomic.

Reusing ObjectOutputStreams in Java

I am reusing ObjectOutputStream to send updates between the two clients, this is the server code,
public void run() {
try {
toPlayer1.writeBoolean(true);
toPlayer1.flush();
while (true) {
try {
found = (boolean[][]) fromPlayer1.readObject();
player1Int = fromPlayer1.readInt();
} catch (Exception ex) {
// Handle exception here...
}
if (isWon(player1Int)) {
toPlayer1.writeInt(P1_WON);
toPlayer1.flush();
toPlayer2.writeInt(P1_WON);
toPlayer2.flush();
sendMove(toPlayer2, found, player1Int);
break;
} else {
toPlayer2.writeInt(CONTINUE);
toPlayer2.flush();
sendMove(toPlayer2, found, player1Int);
}
try {
found = (boolean[][]) fromPlayer2.readObject();
player2Int = fromPlayer2.readInt();
} catch (Exception ex) {
// Handle exception here...
}
if (isWon(player2Int)) {
toPlayer1.writeInt(P2_WIN);
toPlayer1.flush();
toPlayer2.writeInt(P2_WIN);
toPlayer2.flush();
sendMove(toPlayer1, found, player2Int);
break;
} else {
toPlayer1.writeInt(CONTINUE);
toPlayer1.flush();
sendMove(toPlayer1, found, player2Int);
}
}
} catch (IOException ex) {
System.err.println(ex);
}
}
private void sendMove(ObjectOutputStream out, boolean[][] found, int score) throws IOException {
try {
out.reset();
out.writeObject(found);
out.writeInt(score);
out.flush();
} catch (Exception ex) {
// Handle exception here...
}
out.writeInt(score);
}
the problem seems to be that some messages are not being delivered correctly, any suggestions? Am I using the flush() correctly? I have added reset(); it is still not working
update, these are the streams:
public void run() {
try {
toPlayer1 = new ObjectOutputStream(player1.getOutputStream());
fromPlayer1 = new ObjectInputStream(player1.getInputStream());
toPlayer2 = new ObjectOutputStream(player2.getOutputStream());
fromPlayer2 = new ObjectInputStream(player2.getInputStream());
regards, c

If you want an object or objects to be sent again, you need to call reset() on the ObjectOutputStream object.
The problem that reset() solves is that when you send an object in a object stream, the protocol attempts to preserve object identity. The first time you send it, the stream sends the object state. Subsequent times, it just sends a marker that says (in effect) "use this object that I sent you previously".
The reset() method says (in effect) to the ObjectOutputStream ... "forget about all objects that I sent previously".
So if you want to send the same object twice, you need to do something like this:
out.writeObject(found);
// change the state of 'found'
out.reset();
out.writeObject(found);
Note that this doesn't affect primitive values sent using their corresponding write methods. Primitive values don't have "identity" and are sent literally each time.
I should also point out that the following is very bad practice.
} catch (Exception ex) {
}
You are silently ignoring all exceptions. This is lazy and dangerous, and you are likely to come to regret it. (Don't do it even in sample code in SO Questions ... 'cos someone might copy your bad code or some Java beginner might emulate your bad habits.)