I'm trying to write a httpserver using sockets and I meet this problem.
As everyone knows , a Http request is like this.
GET /index.html HTTP/1.1
Cache-Control: max-age=0
Host: 127.0.0.1
Accept:xxxxx
User-Agent: xxxx
Connection: keep-alive
CRLF
This is message body！
The question is how can I get full Http request including message body.
I tried to write like this.
ServerSocket serverSocket = new ServerSocket(8000);
while (true) {
Socket socket = serverSocket.accept();
new Thread() {
{
InputStream is = socket.getInputStream();
BufferedReader input = new BufferedReader(new InputStreamReader(is));
String line = null;
while ((line = input.readLine()) != null) {
System.out.println(line);
}
System.out.print("finish");
}
}.start();
}
And the console would never print "finish".Then I changed like this
ServerSocket serverSocket = new ServerSocket(8000);
while (true) {
Socket socket = serverSocket.accept();
new Thread() {
{
InputStream is = socket.getInputStream();
BufferedReader input = new BufferedReader(new InputStreamReader(is));
String line = null;
while (input.ready()) {
line = input.readLine();
System.out.println(line);
}
System.out.println("finish");
}
}.start();
}
Things go to be better, We can see "finish"! But if I refresh the page a little bit faster.The bufferdreader will not be ready and don't get in the while{} !
I want to print all the rerquest and "finish"
Please help me.
Thanks a lot!!
Both your approaches are incorrect.
In the first one, input.readLine() will return null only when the end of the stream has been reached, not when the request ended. That means that you'll loop there as long as the browser maintains the TCP connection open. That might take a while. Plus, multiple requests might be sent on the same connection, so you might end up printing all of them (I don't know if that's what you want to do).
In the second one, you have timing problem. input.ready() checks whether the receive buffer has any content to read, instead of checking whether the request ended. So you might end up printing only a part of the request instead of waiting for the whole thing. With this approach and the right timings, you might end up printing a part of a request, multiple requests, or anything in between (like a request and a half).
Also note that HTTP GET messages almost never carry any payalod, and no browser will requests like the one in your example.
Related
Hello I'm making an HTTP client. I'm trying to fetch google.com's html code. I have a problem the the BufferedReader.readLine() function is blocking endlessly because the remote server apparently doesn't send a blank line? Or could it be that my request is wrong?
Appreciate any help!
public static void main(String[] args) {
String uri = "www.google.com";
int port = 80;
Socket socket = new Socket(uri, port);
PrintWriter toServer = new PrintWriter(socket.getOutputStream(), true);
InputStream inputStream = socket.getInputStream();
get(uri, port, language, socket, toServer, inputStream);
}
public static void get(String uri, int port, String language, Socket socket, PrintWriter toServer, InputStream inputStream) {
try {
toServer.println("GET / HTTP/1.1");
toServer.println("Host: " + uri + ":" + port);
toServer.println();
// Parse header
StringBuilder stringBuilder = new StringBuilder();
BufferedReader fromServer = new BufferedReader(new InputStreamReader(inputStream));
String line;
while ((line = fromServer.readLine()) != null) {
stringBuilder.append(line);
}
System.out.println("done");
} catch (IOException e) {
e.printStackTrace();
}
}
You are sending a HTTP/1.1 request which by default enables HTTP keep-alive. This means that the server might keep the TCP connection open after the response was sent in order to accept more requests from the client. Your code instead assumes that the server will close the connection after the response was finished by explicitly expecting readline to return null. But since the server will not close the connection (or only after some long timeout) the readline will just block.
To fix this either use HTTP/1.0 (which has keep-alive off by default) instead of HTTP/1.1 or explicitly tell the server that no more requests will be send by adding a Connection: close header.
Please note that in general HTTP is way more complex than you might think if you've just seen a few examples. The problem you face in your question is only a glimpse into more problems which you will face when continuing this path. If you really want to implement your own HTTP handling instead of using established libraries please study the actual standard instead of just assuming a specific behavior.
This question already has answers here:
Linux C Socket: Blocked on recv call
(2 answers)
Closed 7 years ago.
Warning: This is a self answered question. I faced this problem recently and found it very tricky, though it sounds very simple. I decided to post a complete question and answer with code examples for future visitors.
I have a ServerSocket, that listens connection on port 12005. It is implemented in the simpliest way:
ServerSocket ss = new ServerSocket(12005);
while(true) {
executorService.submit(new SocketProcessor(ss.accept());
}
Here SocketProcessor is just a Runnable, that handles incoming connections:
#Override
public void run() {
try {
String msg = IOUtils.toString(s.getInputStream());
// process msg here
} finally {
s.close();
}
}
The problem is that execution hangs on IOUtils.toString(InputStream is) method. I thought that standard solution will work better, so I replaced it with
BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
The output was something like:
POST / HTTP/1.1
Host: http://localhost:12005
Content-Type: application/json
//other headers
<--- empty line
<--- here it hangs
If I abort the connection on the caller side, I can see a new line in output - actually it is http body:
{"name":"alex", "hobby":"football"}
I tried two different request sender implementations and they both hung when I tried to read http body. That means the problem is exactly in my server socket. What is wrong with this implementation from tutorials?
I was lucky to find the explanation on reddit forum.
In short, the structure of http post message is the following
start-line CRLF
header-field-1 CRLF
header-field-2 CRLF
header-field-N CRLF
CRLF
message-body
As you can see there is no new-line symbol after message body. Because of this readLine() method does not work as we expected. I guess Apache IOUtils.toString() method faces the same problem. To fix the problem, the following solution is suggested:
find Content-Length header and read its value N
read exact N bytes after header section is over
Here's the sample code:
BufferedReader br = new BufferedReader(new InputStreamReader(is));
boolean headersFinished = false;
int contentLength = -1;
while (!headersFinished) {
String line = br.readLine();
headersFinished = line.isEmpty();
if (line.startsWith("Content-Length:")) {
String cl = line.substring("Content-Length:".length()).trim();
contentLength = Integer.parseInt(cl);
}
}
// validate contentLength value
char[] buf = new char[contentLength]; //<-- http body is here
br.read(buf);
I have created a server via sockets with the help of quickserver.org. The server runs solidly.
Now I had to write the client that sends a request (just a string value) to the server for an instruction and waits for its response (xml as string). This works fine when the triggered process by the request on server is not very time consuming. Unfortunately the client connection breaks as far as the server needs a long time for the process and that leads for a connection break and the client doesn't get anything back.
Here is the client code:
public String sendAndReceive(String message) throws IOException {
PrintWriter printWriter = new PrintWriter(new OutputStreamWriter(this.socket.getOutputStream()));
BufferedReader reader = new BufferedReader(new InputStreamReader(this.socket.getInputStream()));
printWriter.print(message);
printWriter.flush();
this.socket.shutdownOutput();
String line = null;
StringBuilder xmlResponse = new StringBuilder();
while ((line = reader.readLine()) != null)
{
xmlResponse.append(line);
}
printWriter.close();
reader.close();
this.socket.close();
return xmlResponse.toString();
}
This method sends the request and waits for the response afterwards. I am not sure about the while loop but all examples I have found on web are praising this construction. On my point of view reader.readline() can be null because the server needs more time for the response and therefore the method ends without getting the response.
How is the best practice for socket clients waiting for the response patiently? What I am doing wrong?
Kind regards,
Hilderich
You are probably getting timeout.
You can use Socket.setSoTimeout(int timeout) to change timeout (in milliseconds).
I'm a bit of a beginner programmer so it's possible this is quite obvious and I'm overlooking the answer. But on to the question.
I have a two-part program (its a little more complicated than this example, but the situation is the same). The program has multiple messages fired between the client and the server. I have a PrintWriter on the server-side to send messages to the client, and on the client, I have a BufferedReader to read the messages sent.
When this example is run, I'm given two lines as output. The first message is both messages, and the second is NULL. What I am wondering is if there is a way to basically halt the server until I am ready for the second message, so that I can do something on the client-side before the second message is sent.
I am hoping to not use Thread.Sleep, as I would rather the Server wait around until the Client says it is ready.
This is the client:
public class Client{
public void run(){
Socket socket = null;
InputStream in = null;
BufferedReader reader = null;
try{
socket = new socket("LocalHost",1234);
in = socket.getInputStream();
reader = new BufferedReader(new InputStreamReader(in));
}
String messageFromServer = "";
try{
messageFromServer=reader.readLine();
}
System.out.println(messageFromServer);
String messageFromServer = "";
try{
messageFromServer=reader.readLine();
}
System.out.println(messagefromServer);
//close everything
}
}
This is the server:
public class Server{
public void run(){
ServerSocket server = null;
Socket client = null;
try{
server = new ServerSocket(1234);
client = server.accept();
}
PrintWriter writer = null;
OutputStream out = null;
try{
out = client.getOutputStream();
writer = new PrintWriter(out, true);
}
writer.write("Hi I'm a server");
//do some stuff that takes some time, user input, etc. etc.
writer.write("I'm still a server");
//close everything
}
Thanks :)
The problem with the way you currently have you code is the fact that you are using a BufferedReader, but the server is not terminating it's messages with a new line.
When you close the writer, the client is reaching the EOF or EOS and unblocking the read so it appears that both strings are being sent at once...
If you do something like...
writer.write("Hi I'm a server\n");
// This will force the message to be written to the client and picked up ;)
writer.flush();
writer.write("I'm still a server\n");
writer.flush();
Then you will get the messages seperatly...
You can use ObjectInputStream to read Objects instead of Strings.
This way you will read only one Message(String in your case) every call to ObjectInputStream.readObject();
BTW you can read the first message, "do something" and then read the second message. you don't have to read all of the sent messages at once.
If there are no other messages, then your thread will be blocked when trying to read an object from the ObjectInputStream.
Use it like:
ObjectInputStream inputStream = new ObjectInputStream( socket.getInputStream() )
I'm writing a proxy and have the following code:
Socket conUser;
Socket conDest;
try{
ServerSocket ss = new ServerSocket(Integer.parseInt(p.getProperty("proxy.port")));
while(true){
//Connect to user
conUser = ss.accept();
BufferedReader inputFromUser = new BufferedReader(new InputStreamReader(conUser.getInputStream()));
BufferedWriter outputToUser = new BufferedWriter(new OutputStreamWriter(conUser.getOutputStream(), "UTF8"));
//Get user request
StringBuffer req = new StringBuffer();
getUserRequest(inputFromUser, req);
System.out.println("User requested the following:");
System.out.println(req);
//Connect to server
InetAddress a = InetAddress.getByName(determineHost(req));
conDest = new Socket(a,80);
//Send request to server
BufferedWriter outputToServer = new BufferedWriter(new OutputStreamWriter(conDest.getOutputStream(), "UTF8"));
InputStreamReader inputFromServer = new InputStreamReader(conDest.getInputStream(), "UTF8");
outputToServer.write(req.toString());
outputToServer.flush();
System.out.println("==============================");
System.out.println("Server replied with the following:");
//Read reply from the server
//=========================================
int chars;
while ((chars = inputFromServer.read()) != -1) {
System.out.print((char)chars);
outputToUser.write(chars);
outputToUser.flush();
//serverReply.append(chars);
}
//Relay reply to user
//outputToUser.write(serverReply.toString());
//System.out.println(serverReply);
//outputToUser.flush();
conUser.close();
conDest.close();
}
}
catch (Exception e) {
System.err.println(e);
}
What happens is: I make a connection and it succeeds. I also send the request, and that succeeds too. I also get a reply, and am able to load the entire page's HTML, except that the read doesn't seem to terminate when it reaches the end of the content.
Specifically, I was attempting to load Google's homepage and the chunked transfer reached 0 (that is- end of chanked transfer), and thus there should've been no more input to read, but this did not cause the loop to stop reading. What's also strange to me is that pretty much all code examples of proxies do use this loop, and assuming they work, I don't see much differences between their code and mine.
How do I make the loop terminate correctly?
EDIT: for the record, yes- I know that the TCP connection should be kept open to handle further connections. This is not relevant to the problem I'm having. I need to get this loop to terminate per response.
In general the connection is not closed at the end of each response. Creating TCP connections is relatively time-consuming so the connection is left open, ready for you to send your next request.
Here are a couple of explanatory links:
http://en.wikipedia.org/wiki/HTTP_persistent_connection
http://en.wikipedia.org/wiki/HTTP_pipelining
If you want to terminate connection correctly after receiving HTTP response, your simple loop is not enough. You have to determine the end of message as described in section 4.4 Message Length of RFC 2616 and then close the connection.
However, it would be better to use existing libraries, such as built-in URLConnection.