Develop Reference

Reading from a URL in java: when is a request actually sent?

I have an assignment for school that involves writing a simple web crawler that crawls Wikipedia. The assignment stipulates that I can't use any external libraries so I've been playing around with the java.net.URL class. Based on the official tutorial and some code given by my professor I have:
public static void main(String[] args) {
System.setProperty("sun.net.client.defaultConnectTimeout", "500");
System.setProperty("sun.net.client.defaultReadTimeout", "1000");
try {
URL url = new URL(BASE_URL + "/wiki/Physics");
InputStream is = url.openStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
String inputLine;
int lineNum = 0;
while ((inputLine = br.readLine()) != null && lineNum < 10) {
System.out.println(inputLine);
lineNum++;
}
is.close();
}
catch (MalformedURLException e) {
System.out.println(e.getMessage());
}
catch (IOException e) {
System.out.println(e.getMessage());
}
}
In addition, the assignment requires that:
Your program should not continuously send requests to wiki. Your program
must wait for at least 1 second after every 10 requests
So my question is, where exactly in the above code is the "request" being sent? And how does this connection work? Is the entire webpage being loaded in one go? or is it being downloaded line by line?
I honestly don't really understand much about networking at all so apologies if I'm misunderstanding something fundamental. Any help would be much appreciated.

InputStream is = url.openStream();
at the above line you will be sending request
BufferedReader br = new BufferedReader(new InputStreamReader(is));
at this line getting the input stream and reading.
Calling url.openStream() initiates a new TCP connection to the server that the URL resolves to. An HTTP GET request is then sent over the connection. If all goes right (i.e., 200 OK), the server sends back the HTTP response message that carries the data payload that is served up at the specified URL. You then need to read the bytes from the InputStream that the openStream() method returns in order to retrieve the data payload into your program.

Check if my server is up via java

I am starting a tomcat server in my local for a web application and it takes around 20 minutes to be up and running. I want to check if the web app is up and running and taking any requests via java. Any help?
My server is say at localhost:8001/myapp
Thanks in advance.

You can check it through many ways. Like... Set a servlet as start up on-load and inside it keep some loggers which files log messages along with exact time.

You can add something like localhost:8001/myapp/status to the app that would return information about current status. Then you can just sent http request from java and check the response

public String execute(String uri) throws Exception {
URL url = new URL(uri);
URLConnection connection = url.openConnection();
connection.setReadTimeout(1000);
BufferedReader in = new BufferedReader(
new InputStreamReader(
connection.getInputStream()));
String inputLine;
StringBuffer outputLine = new StringBuffer();
while ((inputLine = in.readLine()) != null)
outputLine.append(inputLine);
in.close();
return outputLine.toString();
}
I guess I will call this method after a certain time period to see if I'm getting a timeout exception of the raw html.

Java socket client: waiting for the response patiently

I have created a server via sockets with the help of quickserver.org. The server runs solidly.
Now I had to write the client that sends a request (just a string value) to the server for an instruction and waits for its response (xml as string). This works fine when the triggered process by the request on server is not very time consuming. Unfortunately the client connection breaks as far as the server needs a long time for the process and that leads for a connection break and the client doesn't get anything back.
Here is the client code:
public String sendAndReceive(String message) throws IOException {
PrintWriter printWriter = new PrintWriter(new OutputStreamWriter(this.socket.getOutputStream()));
BufferedReader reader = new BufferedReader(new InputStreamReader(this.socket.getInputStream()));
printWriter.print(message);
printWriter.flush();
this.socket.shutdownOutput();
String line = null;
StringBuilder xmlResponse = new StringBuilder();
while ((line = reader.readLine()) != null)
{
xmlResponse.append(line);
}
printWriter.close();
reader.close();
this.socket.close();
return xmlResponse.toString();
}
This method sends the request and waits for the response afterwards. I am not sure about the while loop but all examples I have found on web are praising this construction. On my point of view reader.readline() can be null because the server needs more time for the response and therefore the method ends without getting the response.
How is the best practice for socket clients waiting for the response patiently? What I am doing wrong?
Kind regards,
Hilderich

You are probably getting timeout.
You can use Socket.setSoTimeout(int timeout) to change timeout (in milliseconds).

POST Requests from a Servlet

I am writing a servlet using eclipse that receives POST request from a client that should do some splitting on the received text, access google geolocation api to get some data and display to the user.
On a localhost, this works perfectly fine. On an actual server (tried with Openshift and CloudBees), this doesn't work. I can see the splitting reply but not the reply from google geolocation service. There is always an error logged into the console from google service. However, the same code works perfectly fine on localhost.
After I receive the POST request in the doPost method of the servlet, I am doing the following to access the Google GeoLocation service:
//Attempting to send data to Google Geolocation Service
URL url;
HttpURLConnection connection = null;
try {
//Create connection
url = new URL("https://www.googleapis.com/geolocation/v1/geolocate?key=MyAPI");
connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/json");
connection.setUseCaches (false);
connection.setDoInput(true);
connection.setDoOutput(true);
//Send request with data (output variable has the JSON data)
DataOutputStream wr = new DataOutputStream (
connection.getOutputStream ());
wr.writeBytes (output);
wr.flush ();
wr.close ();
//Get Response
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
StringBuffer response2 = new StringBuffer();
while((line = rd.readLine()) != null) {
response2.append(line);
response2.append('\r');
}
rd.close();
//Write to Screen using out=response.getWriter();
out.println("Access Point's Location = " + response2.toString());
} catch (Exception e) {
e.printStackTrace();
} finally {
if(connection != null) {
connection.disconnect();
}
Could you tell me why this is happening and how can I make this work? Should I resort to something like AJAX or is there someother work around? I am relatively new to coding and hence, trying to refrain from learning AJAX at this stage. Please let me if there's any other way of getting this to work

Your localhost has your localhost IP as a sending IP. Openshift et al has the Openshift et al IP as a sending IP. So the Google API says "I have only seen that localhost IP twice before, that's fine!", whereas it says "I have seen this Openshift IP millions of times before! NO REPLY FOR YOU!"

Java - communication between servlet and client

On Servlet side:
for (GameParticipant activePlayer : connector.activePlayers) {
activePlayer.out.println(response);
activePlayer.out.flush();
System.out.println("Server sending board state to all game participants:" + response);
(activePlayer.out is a PrintWriter saved in the server from the HttpResponse object obtained when that client connected the first time)
On clinet side:
private void receiveMessageFromServer() {
try {
BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuilder sb = new StringBuilder();
String input = null;
while ((input = br.readLine()) != null){
sb.append(input).append(" ");
}
}
For some reason, this communication works only the first time, when the client requests connection and waits for response in the same method, while the server uses the PrintWriter obtained directly fron the available HttpRespnse in the doPost method. After that, when the servlet tries to reuse the PrintWriter to talk to the clinet outside of a doPost method, nothing happens, the message never gets to the client. Any ideas?
P.S. In client constructor:
try {
url = new URL("http://localhost:8182/stream");
conn = (HttpURLConnection) url.openConnection();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException ioE) {
ioE.printStackTrace();
}

The response output stream isn't valid outside the doPost() method, or more properly speaking the service() method. It can only be used to send one response. However PrintWriter swallows exceptions, as you will find when you check its error status, so you didn't see the problem.
In other words your entire server-side design is flawed. You can't misuse the Servlet Specification in that way.