I want to recognize end of data stream in Java Sockets. When I run the code below, it just stuck and keeps running (it stucks at value 10).
I also want the program to download binary files, but the last byte is always distinct, so I don't know how to stop the while (pragmatically).
String host = "example.com";
String path = "/";
Socket connection = new Socket(host, 80);
PrintWriter out = new PrintWriter(connection.getOutputStream());
out.write("GET "+ path +" HTTP/1.1\r\nHost: "+ host +"\r\n\r\n");
out.flush();
int dataBuffer;
while ((dataBuffer = connection.getInputStream().read()) != -1)
System.out.println(dataBuffer);
out.close();
Thanks for any hints.
Actually your code is not correct.
In HTTP 1.0 each connection is closed and as a result the client could detect when an input has ended.
In HTTP 1.1 with persistent connections, the underlying TCP connection remains open, so a client can detect when an input ends with 1 of the following 2 ways:
1) The HTTP Server puts a Content-Length header indicating the size of the response. This can be used by the client to understand when the reponse has been fully read.
2)The response is send in Chunked-Encoding meaning that it comes in chunks prefixed with the size of each chunk. The client using this information can construct the response from the chunks received by the server.
You should be using an HTTP Client library since implementing a generic HTTP client is not trivial (at all I may say).
To be specific in your code posted you should have followed one of the above approaches.
Additionally you should read in lines, since HTTP is a line terminated protocol.
I.e. something like:
BufferedReader in =new BufferedReader(new InputStreamReader( Connection.getInputStream() ) );
String s=null;
while ( (s=in.readLine()) != null) {
//Read HTTP header
if (s.isEmpty()) break;//No more headers
}
}
By sending a Connection: close as suggested by khachik, gets the job done (since the closing of the connection helps detect the end of input) but the performance gets worse because for each request you start a new connection.
It depends of course on what you are trying to do (if you care or not)
You should use existing libraries for HTTP. See here.
Your code works as expected. The server doesn't close the connection, and dataBuffer never becomes -1. This happens because connections are kept alive in HTTP 1.1 by default. Use HTTP 1.0, or put Connection: close header in your request.
For example:
out.write("GET "+ path +" HTTP/1.1\r\nHost: "+ host +"\r\nConnection: close\r\n\r\n");
out.flush();
int dataBuffer;
while ((dataBuffer = connection.getInputStream().read()) != -1)
System.out.print((char)dataBuffer);
out.close();
Related
I am trying to write a simple echo server using SSL. The first line that goes to the server is echoed exactly. When I send a second line, only the first character is echoed. The client works off of a buffered reader's read line from stdin. If I hit CR again the rest of the message comes through. The server seems to be sending all of the data. Here are output from client and server:
CLIENT:
Sending to server at 192.168.0.161
on port 9999
4 seasoNS
echo:4 seasoNS
are really good
echo:a
echo:re really good
SERVER:
server listening on 9999
has cr/lf
4 seasoNS
size to send: 10
has cr/lf
are really good
size to send: 16
exiting...
Here is the client loop:
try {
BufferedReader consoleBufferedReader = getConsoleReader();
sslsocket = getSecSocket(strAddress, port);
BufferedWriter sslBufferedWriter = getSslBufferedWriter(sslsocket);
InputStream srvrStream = sslsocket.getInputStream();
String outMsg;
while ((outMsg = consoleBufferedReader.readLine()) != null) {
byte[] srvrData = new byte[1024];
sslBufferedWriter.write(outMsg);
sslBufferedWriter.newLine();
sslBufferedWriter.flush();
int sz = srvrStream.read(srvrData);
String echoStr = new String(srvrData, 0, sz);
System.out.println("echo:" + echoStr);
}
} catch (Exception exception) {
exception.printStackTrace();
}
This problem seemed so odd that I was hoping there was something obvious that I was missing.
What you're seeing is perfectly normal.
The assumption you're making that you're going to read the whole buffer in one go is wrong:
int sz = srvrStream.read(srvrData);
Instead, you need to keep looping until you get the delimiter of your choice (possibly a new line in your case).
This applies to plain TCP connections as well as SSL/TLS connections in general. This is why application protocols must have delimiters or content length (for example, HTTP has a double new line to end its headers and uses Content-Length or chunked transfer encoding to tell the other party when the entity ends).
In practice, you might not see when your assumption doesn't work for such a small example.
However, the JSSE splits the records it sends into 1/n-1 on purpose to mitigate the BEAST attack. (OpenSSL would send 0/n.)
Hence, the problem is more immediately noticeable in this case.
Again, this is not an SSL/TLS or Java problem, the way to fix this is to treat the input you read as a stream and not to assume the size of buffers you read on one end will match the size of the buffers used to send that data from the other end.
I am writing simple Server through TCP/IP for my android app.
I am facing a problem where server only receives messages after the timeout on the app.
My server side:
System.out.println("Connection accepted");
DataOutputStream outToClient = new DataOutputStream(connectionSocket.getOutputStream());
capitalizedSentence = "READY";
writeToUser(outToClient, "READY".getBytes());
String response = readFromUser(connectionSocket);
// Server hangs on readFromUser();
if(response.contains("IL"))
{
byte[] bytes = DatatypeConverter.parseHexBinary(foo);
writeToUser(outToClient, bytes);
}
App side:
if (ack.equals("READY")) {
ack = "";
dataOutputStream.writeBytes(command);
dataOutputStream.flush();
buf = new byte[556];
dataInputStream.read(buf);
// app hangs on read() and gets timeout
}
The problem is that, server receives command only after application gets read timeout.
My current scenario is this:
Connection accepted: Application receives "READY"
Application sends commands and tries to read response from server
Server is hanging on readFromUser() and only receives message after the app gets read timeout.
Any ideas what am I doing wrong ?
Update:
The code works if I use readLine, instead of read(buf) ( if I understand clearly server hangs on read(buf) and is in waiting mode as socket is not closed yet even though no more data is beeing sent. )
However such way I have to add "\n" after each command, and server is only emulator, app works with different device who does not understand "\n" at the end and will crash.
Is there any way I could make this work without using readLine() ?
Update 2
public void writeToUser(DataOutputStream outToClient, byte[] bytes) throws IOException
{
outToClient.write(bytes);
outToClient.flush();
String s = new String(bytes);
writeLog("Sent to client: " + s);
}
public String readFromUser(Socket socket) throws IOException, InterruptedException
{
writeLog("Reading...");
BufferedReader inFromClient = new BufferedReader(new InputStreamReader(socket.getInputStream()));
String clientSentence = inFromClient.readLine();
writeLog("Received: " + clientSentence);
return clientSentence;
}
Above readFromUser() method works just as expected. But, this requires adding "\n" on each command sent from client app. And real device I have to communicate with does not understand "\n" and will treat the command as unrecognized... I am writing this server simply for testing purposes and I want it to be as close to real one as possible.
Previously I was just trying to get it done without "\n" on each command using inFromClient.read(cbuf); instead of readLine() and server always got hung on read, and received command only after socket was closed on client side. Even though I know that client sent the command and is waiting for response.
The question is how can I receive command without using readLine(); and "\n" on client side.
'how can I receive command without using readLine(); and "\n" on client side'. If you do not, or can not, want to use "\n" as a command terminator, then you must use another protocol to identify the start and end of the 'command' within the TCP octet/byte stream.
What protocol does your 'real device' server use to identify the start and end of commands? If it uses a byte-by-byte state-machine to identify a valid command then you are going to have to duplicate that in your test server.
You're reading lines but you aren't writing lines. So readLine() blocks until a line terminator arrives or the peer closes the connection.
Append a \n.
Why not you increase timeout from APPs side. Or you can write a thread that will hit server in some interval for some time and if no response then show time out. This problem is in Apps side.
I have the following code in Java that sends an HTTP request to a web server and read the response:
StringBuilder response = new StringBuilder(50000);
URL url2 = new URL(ServiceURL);
connection = (HttpURLConnection)url2.openConnection();
connection.setRequestMethod("POST");
//... (some more connection settings) ...
BufferedWriter wr = new BufferedWriter(new OutputStreamWriter(connection.getOutputStream(), "UTF-8"));
wr.write(Request);
wr.flush ();
wr.close ();
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
int i = 0;
while ((i = rd.read()) > 0) {
response.append((char)i);
}
It works for most cases, but I have a problem with one server that returns a rather large XML (something like 500KB; I guess this is pretty large for just a bunch of text..), where I keep getting a read timeout exception.
I believe it's not a network problem, because I've tried making the same request using curl and the response just arrived all right and pretty quick, something like two seconds.
When I look what's going on in the network (using wireshark to capture the packets), I noticed that the TCP receive window in my computer gets full at some point. The TCP stack sometimes survives this; I can see the server sending TCP keep-alive to keep the connection up, but in the end the TCP connection just breaks down.
Could it be that the reading part of the code (appending the received response character-by-character) is slowing my code down? Is there a more efficient way to read an HTTP response?
Reading character by character is quite slow, yes. Try reading chunks at a time into a buffer:
char[] buf = new char[2048];
int charsRead;
while((charsRead = rd.read(buf, 0, 2048)) > 0) {
response.append(buf, 0, charsRead);
}
As Phil already said reading the stream byte by byte is kinda slow. I prefer using the readLine() method of BufferedReader :
StringBuilder response = new StringBuilder();
String line = "";
while((line = rd.readLine()) != null) {
response.append(line + System.getProperty("line.separator");
}
If possible, I would consider using the Apache HTTP Client library. It is easy to use and very powerful in handling HTTP stuff.
http://hc.apache.org/httpcomponents-client-ga/
You should also keep in mind to set the socket and connection timeouts. This way you can control how long a connection is kept open (alt least on you side of the connection).
And last but not least always close your HTTP connections in a finally block after you received the response, otherwise you may run into a too many open files problem.
Hope this heps ;)
I am currently working on a simple proxy server, which receives http request from browser, process it, then forward it to the desire web server.
I try to get the request from the input stream of the socket connected by the browser, everything is fine except that the stream get stuck after receiving the last block of data.
My code is in fact very simple, as shown below:
ServerSocket servSocket = new ServerSocket(8282);
Socket workSocket = servSocket.accept();
InputStream inStream = workSocket.getInputStream();
byte[] buffer = new byte[1024];
int numberRead = 0;
while ((numberRead = inStream.read(buffer, 0, 1024)) != -1){
System.out.println(new String(buffer));
}
The loop simply cannot exit, even the request reception is finished.
Is there any method to workaround this problem?
Thanks in advance for any advice.
As in InputStream javadoc the method will block until the data is available or the EOF is encountered. So, the other side of Socket needs to close it - then the inStream.read() call will return.
Another method is to send the size of message you want to read first, so you know ahead how many bytes you have to read. Or you can use BufferedReader to read from socket in line-wise way. BufferedReader has a method readLine() which returns every time a line is read, which should work for you as HTTP protocol packages are nice divided into lines.
It will cycle until the connection is closed, and the client is probably waiting for HTTP response from you and doesn't close it.
The browser is waiting for a response before it closes the connection.
Your read-method on the other hand will block until the stream/connection is closed or new data is received.
Not a direct solution according to your current code.
As HTTP is a line based protocol, you might want to use a Buffered Reader and call readLine() on it.
The when a http request comes in it will always be concluded with a blank line, for example:
GET /someFile.html HTTP/1.1
Host: www.asdf.com
After sending that request the client connection will then wait for a response from the server before closing the connection. So if you want to parse the request from the user you are probably better off using a BufferedReader and reading full lines until you reach a lines of text that is blank line.
I have the following Java socket client app, that sends same string to socket server:
import java.net.*;
import java.io.*;
public class ServerClient {
public static void main(String[] args) throws IOException {
System.out.println("Starting a socket server client...");
Socket client = new Socket("XXX.X.XXX.XX", 12001);
BufferedOutputStream stream = new BufferedOutputStream(client.getOutputStream());
String message = "ABC";
BufferedReader inputReader = new BufferedReader(new InputStreamReader(System.in));
String input = null;
while ( true ) {
System.out.print("Would you like to send a message to Server? ");
input = inputReader.readLine();
if ( !input.equals("Y") ) break;
System.out.println("Message to send: " + message);
System.out.println("Message length is: " + message.length());
byte[] messageBytes = message.getBytes("US-ASCII");
stream.write(messageBytes, 0, messageBytes.length);
stream.flush();
}
System.out.println("Shutting down socket server client...");
stream.close();
client.close();
inputReader.close();
}
}
The first time message is sent, server receives the message; however, every subsequent time I'm trying to send this message, server is not receiving anything. Message simply disappears. I am writing to the socket successfully (no exceptions) but nothing is coming on the other side of the pipe (or so I'm told).
I do not have access to the server app, logs or code, so I'm wondering if there is any approach you can recommend to figure out why server is not receiving subsequent messages. Any ideas would be greatly appreciated!
Clarification:
New lines are not expected by the server; otherwise, how would it even receive message the first time? As a trial and error, I did try sending '\n' and "\r\n" and 0x00 characters at the end of the string - all without any luck.
I thought flushing was an issue, so I tried various outputstream classes (PrintStream, PrintWriter, FilterOutputStream), but was still running into same exact issues. Then, if "flushing" is an issue, how is it working the first time?
Other tests:
1 - use a network sniffer to see what is realy hapening on the network
2 - use some program like TCP Test Tool to send data to the server and simulate your program. (netcat can also be used, but it sends a newline after each line)
Remember:
TCP is stream oriented. not message oriented.
One write on the client could take several reads on the server to .. read
Multiple writes on the client could get read by the server in one read
You'll hardly see the above scenarios in a test application on a local network, you will see them very quick in a production environemnt, or when you start to really speed up the sending/receiving.
Following this, if you are sending messages you need a delimiter, or some other way of indicating 'here's one message', e.g. defining the protocol to be 'the first byte is the length of the following message'.
And you'd need to check the receiving end wether it read a partial message, a whole message, and any combination thereof (e.e.g one read might have read 3 and a half message..).
A quick solution for your test app, write lines. That is, a string followed by a newline character. A bufferedreader's ReadLine() could then take care of the reassembly for you on the receiving end.
It works correctly here... but I am missing a carriage return or some other end of message after sending the message.
Hard to write more without knowing what the server expects (protocol)...
Maybe you should try something like
String message = "ABC\n";