Develop Reference

FileWriter closes websocket connection?

I am new to server programming and websockets and I've learnt a little bit of Java 8 this year. In school we had a project where a client webpage opens your webcam, takes a photo of a barcode and then shows a photo and the nutritional value of said product. You can also just send a raw barcode number and that is what is done in this example
My side of the project was to implement a java websocket server (the backend) using the glassfish tyrus library, then receiving the number of the barcode in string format and making a request to openfoodfacts.org using their api. Finally I parsed the json file and sent it back as string format so the client app can read the string and show the correct information (product name, image url, etc)
My code is organized into two files, Serveur.java establishes a websocket server for the client to connect to and ProduitApi.java gets the information from openfoodfacts.org with the given barcode from the client.
public class Serveur {
#javax.websocket.server.ServerEndpoint(value = "/websocket")
public static class EndPoint {
#javax.websocket.OnClose
public void onClose(javax.websocket.Session session, javax.websocket.CloseReason close_reason) {
System.out.println("onClose: " + close_reason.getReasonPhrase());
}
#javax.websocket.OnError
public void onError(javax.websocket.Session session, Throwable throwable) {
System.out.println("onError: " + throwable.getMessage());
}
#javax.websocket.OnMessage
public void onMessage(javax.websocket.Session session, String message) {
System.out.println("Message from client: " + message);
//Creation du produit avec le message du client
try {
ProduitApi produit = new ProduitApi(message);
session.getBasicRemote().sendText(produit.print());
} catch (Exception e) {
e.printStackTrace();
}
}
#javax.websocket.OnOpen
public void onOpen(javax.websocket.Session session, javax.websocket.EndpointConfig ec) throws java.io.IOException {
System.out.println("OnOpen... " + ec.getUserProperties().get("Author"));
session.getBasicRemote().sendText("{\"Handshaking\": \"Yes\"}");
}
}
public static void main(String[] args) {
Server server;
server = new Server ("localhost", 8025, "/BetterFood", null, EndPoint.class);
try {
server.start();
System.out.println("--- server is running");
System.out.println(java.nio.file.FileSystems.getDefault().getPath("client") );
System.out.print("Please press a key to stop the server.");
java.io.BufferedReader reader = new java.io.BufferedReader(new java.io.InputStreamReader(System.in));
reader.readLine();
} catch (Exception e) {
e.printStackTrace();
} finally {
server.stop();
}
}
}
as you can see, when I receive the 'barcode' message, onMessage() gets called. There I instantiate an object of class ProduitApi to use the barcode to then return the information
This is my ProduitApi file without some unnecessary details
package com.gabi.serveur;
/**
*
* #author gabriel
*/
[imports]
public class ProduitApi {
private java.lang.String barcode;
final private java.net.URL url;
private java.net.URLConnection connection;
JsonObjectBuilder constructeur_objet = Json.createObjectBuilder();
String string_json;
ProduitApi(java.lang.String barcode)throws MalformedURLException, IOException {
this.barcode = barcode;
this.url = new java.net.URL("http://world.openfoodfacts.org/api/v0/product/" + this.barcode + ".json");
connection = url.openConnection();
stream();
}
public void stream() throws IOException{
if (connection != null) {
java.io.InputStreamReader response = new java.io.InputStreamReader(connection.getInputStream());
javax.json.stream.JsonParser parser=javax.json.Json.createParser(response);
while (parser.hasNext()) {
[parsing inputStream into JsonObject]
}
public String print()throws IOException{
string_json = constructeur_objet.build().toString();
System.out.print(string_json);
//FileWriter file = new FileWriter("serveur/src/main/java/com/gabi/serveur/json/final.json");
//file.write(string_json);
//file.close();
return string_json;
}
}
My problem comes from the last function ProduitApi.print() , it is supposed to return the parsed json in string form so I can send it via the sendText() as well as printing the result into my console so I can see if everything went right. As you can see there are somme commented lines; The FileWriter object that I had created was used with the purpose of writing said string to a file in my pc and let me check inside.
HOWEVER
and here is what I don't understand, If I uncomment those lines so that the print function can also write the file to my drive, The Connection Closes and then Opens again
It can be seen in the console where after printing the json string, it prints OnClose, followed by OnOpen Signaling the connection was reset for some reason.
If I remove the FileWriter section, the connection works normally, the client's connection stays open and he can make another request
End of Console Message after request:
...cuits x22 biscuits fourrés - 304g","qte":"304 g","img":"https://images.openfoodfacts.org/images/products/800/050/031/0427/front_fr.177.400.jpg"}onClose: OnOpen... null...
Finally, my question is just why writing to a file makes my program behave this way (resetting the connection). Does it have anything to do with how streams work?
I accidentally commented the filewriter portion when another Ide said it didn't find the file because I had opened the project from a different directory.

Download files with netty

I am creating a very basic webserver using netty and java. I will have basic functionality. It's main responsibilities would be to serve responses for API calls done from a client (e.g a browser, or a console app I am building) in JSON form or send a zip file. For that reason I have created the HttpServerHanddler class which is responsible for getting the request, parsing it to find the command and call the appropriate api call.It extends SimpleChannelInboundHandler
and overrides the following functions;
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
LOG.debug("channelActive");
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx) {
LOG.debug("In channelComplete()");
ctx.flush();
}
#Override
public void channelRead0(ChannelHandlerContext ctx, Object msg)
throws IOException {
ctx = processMessage(ctx, msg);
if (!HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
ctx.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(
ChannelFutureListener.CLOSE);
}
}
private ChannelHandlerContext processMessage(ChannelHandlerContext ctx, Object msg){
if (msg instanceof HttpRequest) {
HttpRequest request = this.request = (HttpRequest) msg;
if (HttpHeaders.is100ContinueExpected(request)) {
send100Continue(ctx);
}
//parse message to find command, parameters and cookies
ctx = executeCommand(command, parameters, cookies)
}
if (msg instanceof LastHttpContent) {
LOG.debug("msg is of LastHttpContent");
if (!HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
ctx.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(
ChannelFutureListener.CLOSE);
}
}
return ctx;
}
private ChanndelHandlerContext executeCommand(String command, HashMap<String, List<String>>> parameters, Set<Cookie> cookies>){
//switch case to see which command has to be invoked
switch(command){
//many cases
case "/report":
ctx = myApi.getReport(parameters, cookies); //This is a member var of ServerHandler
break;
//many more cases
}
return ctx;
}
In my Api class that has the getReport function.
getReport
public ChannelHandlerContext getReportFile(Map<String, List<String>> parameters,
Set<Cookie> cookies) {
//some initiliazations. Actual file handing happens bellow
File file = new File(fixedReportPath);
RandomAccessFile raf = null;
long fileLength = 0L;
try {
raf = new RandomAccessFile(file, "r");
fileLength = raf.length();
LOG.debug("creating response for file");
this.response = Response.createFileResponse(fileLength);
this.ctx.write(response);
this.ctx.write(new HttpChunkedInput(new ChunkedFile(raf, 0,
fileLength,
8192)),
this.ctx.newProgressivePromise());
} catch (FileNotFoundException fnfe) {
LOG.debug("File was not found", fnfe);
this.response = Response.createStringResponse("failure");
this.ctx.write(response);
} catch (IOException ioe) {
LOG.debug("Error getting file size", ioe);
this.response = Response.createStringResponse("failure");
this.ctx.write(response);
} finally {
try {
raf.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
return this.ctx;
}
Response class is responsible for handling various types of response creations (JsonString JsonArray JsonInteger File, etc)
public static FullHttpResponse createFileResponse(long fileLength) {
FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK);
HttpHeaders.setContentLength(response, fileLength);
response.headers().set(HttpHeaders.Names.CONTENT_TYPE, "application/octet-stream");
return response;
}
My Api works great for my Json responses(easier to achieve) but It won't work well with my json responses, but not with my file response. When making a request from e.g chrome it only hangs and does not download the file. Should I do something else when downloading a file using netty? I know its not the best wittern code, I still think I have some bits and pieces missing from totally understanding the code, but I would like your advice on how to handle download on my code. For my code I took under consideration this and this

First, some remarks on your code...
Instead of returning ctx, I would prefer to return the last Future for the last command, such that your last event (no keep alive on) could use it directly.
public void channelRead0(ChannelHandlerContext ctx, Object msg)
throws IOException {
ChannelFuture future = processMessage(ctx, msg);
if (future != null && !HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
future.addListener(ChannelFutureListener.CLOSE);
}
}
Doing this way will allow to directly close without having any "pseudo" send, even empty.
Important: Note that in Http, the response is managed such that there are chunk send for all data after the first HttpResponse item, until the last one which is empty (LastHttpContent). Sending another empty one (Empty chunk but not LastHttpContent) could break the internal logic.
Moreover, you're doing the work twice (once in read0, once in processMessage), which could lead to some issues perhaps.
Also, since you check for KeepAlive, you should ensure to set it back in the response:
if (HttpHeaders.isKeepAlive(request)) {
response.headers().set(CONNECTION, HttpHeaders.Values.KEEP_ALIVE);
}
On your send, you have 2 choices (depending on the usage of SSL or not): you've selected only the second one, which is more general, so of course valid in all cases but less efficient.
// Write the content.
ChannelFuture sendFileFuture;
ChannelFuture lastContentFuture;
if (ctx.pipeline().get(SslHandler.class) == null) {
sendFileFuture =
ctx.write(new DefaultFileRegion(raf.getChannel(), 0, fileLength), ctx.newProgressivePromise());
// Write the end marker.
lastContentFuture = ctx.writeAndFlush(LastHttpContent.EMPTY_LAST_CONTENT); // <= last writeAndFlush
} else {
sendFileFuture =
ctx.writeAndFlush(new HttpChunkedInput(new ChunkedFile(raf, 0, fileLength, 8192)),
ctx.newProgressivePromise()); // <= last writeAndFlush
// HttpChunkedInput will write the end marker (LastHttpContent) for us.
lastContentFuture = sendFileFuture;
}
This is this lastContentFuture that you can get back to the caller to check the KeepAlive.
Note however that you didn't include a single flush there (except with your EMPTY_BUFFER but which can be the main reason of your issue there!), contrary to the example (from which I copied the source).
Note that both use a writeAndFlush for the last call (or the unique one).

Check client is alive with HttpServletRequest object

I'm writing a Spring web application and I'm mapping the "/do" URL path to the following Controller's method
#Controller
public class MyController
{
#RequestMapping(value="/do", method=RequestMethod.GET)
public String do()
{
File f = new File("otherMethodEnded.tmp");
while (!f.exists())
{
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
}
}
// ok, let's continue
}
}
The otherMethodEnded.tmp file is written by one another Controller's method, so when the client calls the second URL I expect the first method to exit the while loop.
Everything works, except when the client calls the "/do" URL and then closes the connection before the response was received. The problem is that the server remains in the while (!f.exists()) loop even though the client is down and cannot call the second URL to unlock the while loop.
I would try to retrieve the connection status of the "/do" URL and exit the loop when the connection is closed by the client, but I cannot find any way to do so.
I tried with the HttpServletRequest.getSession(false) method but the returned HttpSession object is always not null, so the HttpServletRequest object is not updated in case of connection close of the client.
How can I check whether the client is still waiting for the risponse or not?

The simplest way to verify something is not right is to define a timeout value and then during your loop test if your time spent waiting has exceeded the timeout.
something like:
#Controller
public class MyController
{
private static final long MAX_LOOP_TIME = 1000 * 60 * 5; // 5 minutes? choose a value
#RequestMapping(value="/do", method=RequestMethod.GET)
public String do()
{
File f = new File("otherMethodEnded.tmp");
long startedAt = System.currentTimeMillis()
boolean forcedExit = false;
while (!forcedExit && !f.exists())
{
try {
Thread.sleep(5000);
if (System.currentTimeMillis() - startedAt > MAX_LOOP_TIME) {
forcedExit = true;
}
} catch (InterruptedException e) {
forcedExit = true;
}
}
// ok, let's continue
// if forcedExit , handle error scenario?
}
}
Additionally: InterruptedException is not something to blindly catch and ignore. see this discussion
In your case I would really exit the while loop if you're interrupted.
You only know if the client is no longer waiting on your connection when you notice the output stream you write to (response.outputstream) is closed. But there isn't a way to detect it.
(see this question for details)
Seeing as you've indicated your client does occasional callbacks, you could on the clientside poll if the other call has been completed. If this other call has completed, do the operation, otherwise return directly and have the client do the call again. (assuming you are sending json, but adapt as you require)
something like
public class MyController
{
#RequestMapping(value="/do", method=RequestMethod.GET)
public String do()
{
File f = new File("otherMethodEnded.tmp");
if (f.exists()) {
// do what you set out to do
// ok, let's continue
// and return with a response that indicates the call did what it did
// view that returns json { "result" : "success" }
return "viewThatSIgnalsToClientThatOperationSucceeded";
} else {
// view that returns json: { "result" : "retry" }
return "viewThatSignalsToClientToRetryIn5Seconds";
}
}
}
Then the clientside would run something like: (pseudojavascript as it's been a while)
val callinterval = setInterval(function() { checkServer() }, 5000);
function checkServer() {
$.ajax({
// ...
success: successFunction
});
}
function successFunction(response) {
// connection succeeded
var json = $.parseJSON(response);
if (json.result === "retry") {
// interval will call this again
} else {
clearInterval(callinterval);
if (json.result === "success") {
// do the good stuff
} else if (json.result === "failure") {
// report that the server reported an error
}
}
}
Ofcourse this is just semi-serious code but it's roughly how i'd try it if I were to have the dependency. If this is regarding afile upload, keep in mind that this file may not contain all of the bytes yet. file exists != file = completely uploaded, unless you use move it. cp / scp / etc. is not atomic.

why run and start call simultaneously in java

I am learning a tutorial about how to consume json web service.
But I have two doubts can you please help me in understanding.
I am learning from this link
http://codeoncloud.blogspot.in/2013/05/blackberry-java-json-tutorial.html
Here is one class extend by thread
public class ConnectJson extends Thread {
private String url;
public String response;
private String myinterface = ";interface=wifi";
public void run() {
HttpConnection conn = null;
InputStream in = null;
int code;
try {
conn = (HttpConnection) Connector.open(this.url + this.myinterface, Connector.READ);
conn.setRequestMethod(HttpConnection.GET);
code = conn.getResponseCode();
if (code == HttpConnection.HTTP_OK) {
in = conn.openInputStream();
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buffer = new byte[in.available()];
int len = 0;
while (-1 != (len = in.read(buffer))) {
out.write(buffer);
}
out.flush();
this.response = new String(out.toByteArray());
if (out != null){
out.close();
}
if (in != null){
in.close();
}
if (conn != null){
conn.close();
}
}
} catch (Exception e) {
Dialog.inform(e.toString());
}
}
public String jsonResult(String url){
this.url = url;
this.start();
this.run();
return response;
}
}
It is making one object of that class and call method of that class .In that method it call start as well as run method why ?
this.start();
this.run();?

In that method it call start as well as run method why ?
You'd have to ask the author of the code; looking at that class's code, it looks incorrect. It's also fairly unusual.
In the normal course of things, you don't call run directly; you start the thread (with start) and the JVM is then responsible for creating a new thread and calling run on it.
You can call run yourself if you really want that code to run right away on the current thread, but it's unusual and that class doesn't immediately look like it's designed to do that correctly. What that code actually does is start a new thread (which means run will eventually get called on that new thread), but then as you observed it also calls run directly. So run will run twice, and may well run twice simultaneously. Since the code in run uses instances variables that will be used by both threads but doesn't do anything to coordinate access to those instance variables...well, again, it looks incorrect.
I don't think I'd keep following that tutorial. You may find the Concurrency trail in the Java tutorials from Oracle might be useful. (Threads are part of it.)

How to get external IP successfully

After reading: Getting the 'external' IP address in Java
code:
public static void main(String[] args) throws IOException
{
URL whatismyip = new URL("http://automation.whatismyip.com/n09230945.asp");
BufferedReader in = new BufferedReader(new InputStreamReader(whatismyip.openStream()));
String ip = in.readLine(); //you get the IP as a String
System.out.println(ip);
}
I thought I was a winner but I get the following error
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: http://automation.whatismyip.com/n09230945.asp
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
at getIP.main(getIP.java:12)
I think this is because the server isnt responding quick enough, is there anyway to ensure that it will get the external ip?
EDIT: okay so its getting rejected, anyone else know of another site that can do the same function

public static void main(String[] args) throws IOException
{
URL connection = new URL("http://checkip.amazonaws.com/");
URLConnection con = connection.openConnection();
String str = null;
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
str = reader.readLine();
System.out.println(str);
}

Before you run the following code take a look at this: http://www.whatismyip.com/faq/automation.asp
public static void main(String[] args) throws Exception {
URL whatismyip = new URL("http://automation.whatismyip.com/n09230945.asp");
URLConnection connection = whatismyip.openConnection();
connection.addRequestProperty("Protocol", "Http/1.1");
connection.addRequestProperty("Connection", "keep-alive");
connection.addRequestProperty("Keep-Alive", "1000");
connection.addRequestProperty("User-Agent", "Web-Agent");
BufferedReader in =
new BufferedReader(new InputStreamReader(connection.getInputStream()));
String ip = in.readLine(); //you get the IP as a String
System.out.println(ip);
}

While playing with Go I saw your question. I made a quick App on Google App Engine using Go:
Hit this URL:
http://agentgatech.appspot.com/
Java code:
new BufferedReader(new InputStreamReader(new URL('http://agentgatech.appspot.com').openStream())).readLine()
Go code for the app which you can copy and make your own app:
package hello
import (
"fmt"
"net/http"
)
func init() {
http.HandleFunc("/", handler)
}
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, r.RemoteAddr)
}

Some servers has triggers that blocks access from "non-browsers". They understand that you are some kind of automatic app that can do a DOS attack. To avoid this, you can try to use a lib to access the resource and set the "browser" header.
wget works in this way:
wget -r -p -U Mozilla http://www.site.com/resource.html
Using Java, you can use the HttpClient lib and set the "User-Agent" header.
Look the topic 5 of "Things To Try" section.
Hope this can help you.

A 403 response indicates that the server is explicitly rejecting your request for some reason. Contact the operator of WhatIsMyIP for details.

We've set up CloudFlare and as designed they're challenging unfamiliar useragents. If you can set your UA to something common, you should be able to gain access.

You can use another web service like this; http://freegeoip.net/static/index.html

Using the Check IP address link on AWS worked for me.Please note that MalformedURLException,IOException are to be added as well
public String getPublicIpAddress() throws MalformedURLException,IOException {
URL connection = new URL("http://checkip.amazonaws.com/");
URLConnection con = connection.openConnection();
String str = null;
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
str = reader.readLine();
return str;
}

This is how I do it with rxJava2 and Butterknife. You'll want to run the networking code in another thread because you'll get an exception for running network code on the main thread!
I use rxJava instead of AsyncTask because the rxJava cleans up nicely when the user moves on to the next UI before the thread is finished. (this is super useful for very busy UI's)
public class ConfigurationActivity extends AppCompatActivity {
// VIEWS
#BindView(R.id.externalip) TextInputEditText externalIp;//this could be TextView, etc.
// rxJava - note: I have this line in the base class - for demo purposes it's here
private CompositeDisposable compositeSubscription = new CompositeDisposable();
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.my_wonderful_layout);
ButterKnife.bind(this);
getExternalIpAsync();
}
// note: I have this code in the base class - for demo purposes it's here
#Override
protected void onStop() {
super.onStop();
clearRxSubscriptions();
}
// note: I have this code in the base class - for demo purposes it's here
protected void addRxSubscription(Disposable subscription) {
if (compositeSubscription != null) compositeSubscription.add(subscription);
}
// note: I have this code in the base class - for demo purposes it's here
private void clearRxSubscriptions() {
if (compositeSubscription != null) compositeSubscription.clear();
}
private void getExternalIpAsync() {
addRxSubscription(
Observable.just("")
.map(s -> getExternalIp())
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe((String ip) -> {
if (ip != null) {
externalIp.setText(ip);
}
})
);
}
private String getExternalIp() {
String externIp = null;
try {
URL connection = new URL("http://checkip.amazonaws.com/");
URLConnection con = connection.openConnection(Proxy.NO_PROXY);
con.setConnectTimeout(1000);//low value for quicker result (otherwise takes about 20secs)
con.setReadTimeout(5000);
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
externIp = reader.readLine();
} catch (Exception e) {
e.printStackTrace();
}
return externIp;
}
}
UPDATE - I've found that URLConnection is really quite shit; it'll take a long time to get a result, not really time out very well, etc. The code below improves the situation with OKhttp
private String getExternalIp() {
String externIp = "no connection";
OkHttpClient client = new OkHttpClient();//should have this as a member variable
try {
String url = "http://checkip.amazonaws.com/";
Request request = new Request.Builder().url(url).build();
Response response = client.newCall(request).execute();
ResponseBody responseBody = response.body();
if (responseBody != null) externIp = responseBody.string();
} catch (IOException e) {
e.printStackTrace();
}
return externIp;
}