This is a a Java method that tries to crawl a designated web page. I am using writeUTF and readUTF for socket communications to a server.
static void get_html(String host, String page, int port) throws IOException {
Socket sock = new Socket(host, port);
String msg = MessageFormat.format("GET {0} HTTP/1.1\r\nHost: {1}\r\n\r\n", page, host);
DataOutputStream outToServer = new DataOutputStream(sock.getOutputStream());
DataInputStream inFromServer = new DataInputStream(sock.getInputStream());
InputStream stream = new ByteArrayInputStream(msg.getBytes(StandardCharsets.UTF_8));
BufferedReader buf = new BufferedReader(new InputStreamReader(stream));
String outMsg;
while ((outMsg = buf.readLine()) != null) {
System.out.println("Sending message: " + outMsg);
outToServer.writeUTF(outMsg);
String inMsg;
try {
inMsg = inFromServer.readUTF();
} catch (EOFException eof) {
break;
}
System.out.println(inMsg);
}
sock.close();
}
The reason I am writing it this way was to mimic the c code, where you have a while loop of send() making all deliveries from a buffer, and another while loop of recv() from a buffer untill it hits 'null'. When execute my code, it just hangs there, I suspect that is due to a call of readUTF before I finished sending all my messages. If this is the case, is there any way to fix it?
You can't do this. HTTP is defined as text lines. writeUTF() does not write text, it writes a special format starting with a 16-bit binary length word. Similarly the HTTP server won't reply with that format into your readUTF() call. See the Javadoc.
You have to use binary streams and the write() method, with \r\n as the line terminator. Depending on the output format you may or may not be able to use readLine(). Best not, then you don't have to write two pieces of code: use binary streams again.
In fact you should throw it all away and use HttpURLConnection. Implementing HTTP is not as simple as may hastily be supposed.
Related
I have a problem with socket programming in Java.
There is a server which has been written in python like this which I shouldn't not change.
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
s.send('from server\nnewline\0')
data = s.recv(BUFFER_SIZE)
s.close()
Now I want to write a code in Java which read the string from a server. Something like this:
public static String readStr(Socket client) throws IOException {
InputStreamReader inStream = new InputStreamReader(
client.getInputStream());
BufferedReader inBuff = new BufferedReader(inStream);
String answer = new String();
String str = inBuff.readLine();
while (str!=null) {
answer = answer.concat(str + "\n");
str = inBuff.readLine();
}
answer = answer.substring(0, answer.length() - 1);
System.out.println("answer:\n "+answer);
return answer;
}
But it seems that it blocks at line str = inBuff.readLine(); at the last line of the message. I tried the read() method but it was blocked too.
When designing a protocol over tcp, the best way is to include some kind of framer. This is done in the current protocol by the usage of a NUL byte.
When reading the data from the socket, you should first divide it into blocks/frames by some operations, before parsing the individual blocks.
A crude way to divide the packets into blocks is reading until you find a NUL byte, then returning that block as a byte array. (This is not the most efficient implementation)
public byte[] readPacket(InputStream in) throws IOException {
ByteArrayOutputStream tempStr = new ByteArrayOutputStream();
int read;
read=in.read();
while(read > 0){
tempStr.write(read);
read=in.read();
}
if(read == -1)
throw new EOFException();
return tempStr.toByteArray();
}
Because you now have proper frames for your data, you can now easily read the data:
byte[] frame = readPacket(in);
String[] lines = new String(frame, StandardCharsets.UTF8).split("\n");
// Do something with the lines
This is probably because the last line sent by the server does not end and readLine() method only returns when it reaches end of the line. Since you change the server's code. I recommend you use another method for reading from the stream. You may also use InputStreamReader class.
Apart from already mentioned inconsistent message/line ending - once with \n, second with \0, at the server there is no detection of end of the message. So the server will loop as long as the socket is not closed (or shut down for writing) at the client side. And as you have this line before closing the socket:
data = s.recv(BUFFER_SIZE)
in other words the client is waiting for some response from the server. But the server is stuck forever reading the message from the client in a loop.
So you need to either close the socket prior to that recv call or send some message (like empty line) and detect in on the server and eventually exit the loop.
I'm trying to send strings to a server from a client but it doesn't seem to be reading from the input stream.
Client
Scanner scanner = new Scanner(System.in);
Socket connection = new Socket("localhost", 13)
OutputStream out = connection.getOutputStream();
while(true) {
String message = scanner.nextLine();
IOUtils.write(message, out, "UTF-8");
out.flush();
}
Server
ServerSocket server = new ServerSocket(localhost,13);~
Socket connection = server.accept();
InputStream in = connection.getInputStream();
StringWriter writer = new StringWriter();
while(true) {
try {
IOUtils.copy(in, writer);
System.out.println(writer.toString());
} catch(IOException io) {}
}
It reads if I close the stream from the client's outputstream but I am trying to send multiple messages from the client to the server. Could someone please help
You seem to think that each time you call flush() at client-side, the server will know it and be able to know that this is the end of a message. That's not the case. IOUtils.copy() reads everything from a stream of bytes. While the stream end hasn't been reached, copy() won't return.
You can see streams as two sides of a long tube. If you pour 10 buckets of water in the at the end of the tube, all you'll get at the other side is a continuous flow of water.
If you need multiple separate messages, then you need to design a protocol allowing to separate messages, and read until the end of a message has been reached. It could be based on separators for example. Or you could send the length of the message followed by the message itself, to let the server know how many bytes it must read to get the next message.
I'm a bit of a beginner programmer so it's possible this is quite obvious and I'm overlooking the answer. But on to the question.
I have a two-part program (its a little more complicated than this example, but the situation is the same). The program has multiple messages fired between the client and the server. I have a PrintWriter on the server-side to send messages to the client, and on the client, I have a BufferedReader to read the messages sent.
When this example is run, I'm given two lines as output. The first message is both messages, and the second is NULL. What I am wondering is if there is a way to basically halt the server until I am ready for the second message, so that I can do something on the client-side before the second message is sent.
I am hoping to not use Thread.Sleep, as I would rather the Server wait around until the Client says it is ready.
This is the client:
public class Client{
public void run(){
Socket socket = null;
InputStream in = null;
BufferedReader reader = null;
try{
socket = new socket("LocalHost",1234);
in = socket.getInputStream();
reader = new BufferedReader(new InputStreamReader(in));
}
String messageFromServer = "";
try{
messageFromServer=reader.readLine();
}
System.out.println(messageFromServer);
String messageFromServer = "";
try{
messageFromServer=reader.readLine();
}
System.out.println(messagefromServer);
//close everything
}
}
This is the server:
public class Server{
public void run(){
ServerSocket server = null;
Socket client = null;
try{
server = new ServerSocket(1234);
client = server.accept();
}
PrintWriter writer = null;
OutputStream out = null;
try{
out = client.getOutputStream();
writer = new PrintWriter(out, true);
}
writer.write("Hi I'm a server");
//do some stuff that takes some time, user input, etc. etc.
writer.write("I'm still a server");
//close everything
}
Thanks :)
The problem with the way you currently have you code is the fact that you are using a BufferedReader, but the server is not terminating it's messages with a new line.
When you close the writer, the client is reaching the EOF or EOS and unblocking the read so it appears that both strings are being sent at once...
If you do something like...
writer.write("Hi I'm a server\n");
// This will force the message to be written to the client and picked up ;)
writer.flush();
writer.write("I'm still a server\n");
writer.flush();
Then you will get the messages seperatly...
You can use ObjectInputStream to read Objects instead of Strings.
This way you will read only one Message(String in your case) every call to ObjectInputStream.readObject();
BTW you can read the first message, "do something" and then read the second message. you don't have to read all of the sent messages at once.
If there are no other messages, then your thread will be blocked when trying to read an object from the ObjectInputStream.
Use it like:
ObjectInputStream inputStream = new ObjectInputStream( socket.getInputStream() )
I am trying to have my client connect to my server, and depending on the command send some string back to the client. Currently the app connects and can send strings to the server very nicely. However when I send the command which instructs the server to send something back it hangs. I found that the problem occurs when the client attempts to read the line send from the server.
Server
PrintWriter out = new PrintWriter(new OutputStreamWriter(clientSocket.getOutputStream()));
out.println("GETDATA" + "\n");
out.flush();
out.close();
Client
BufferedReader fromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
incomingLine = fromServer.readLine();
Log.d("HERE", "NOT " + incomingLine);
fromServer.close();
Thanks!
I made effectively this same mistake when I was first doing sockets as well.
Don't use PrintWriter with BufferedReader. They're incompatible. By comments, PrintWriter actually hides critical exceptions, so they shouldn't be used in networking. Instead, use a DataInputStream and DataOutputStream for communications.
client = new Socket(hostname, port);
inStr = new DataInputStream(client.getInputStream());
outStr = new DataOutputStream(client.getOutputStream());
Then, send and receive using writeUTF and readUTF, like so:
public void send(String data) throws IOException {
outStr.writeUTF(data); outStr.flush();
}
public String recv() throws IOException {return inStr.readUTF();}
The reason has to do with the UTF encoding; a BufferedReader expects a certain string encoding, which PrintWriter does not give. Thus, the read/write hangs.
The method readLine() expects an end of line character "\n" maybe that's your problem
I'm trying to learn sockets using Java and I sucessfully sent data to a ServerSocket running in my own machine. When I try to read from this socket using readline (so I can just echo my own sent message) my program hangs and won't return.
Here's the code:
public static void main(String[] args) throws UnknownHostException, IOException {
TCPClient cli = new TCPClient("127.0.0.1", "15000");
try {
cli.ostream.writeUTF("Teste");
String echo = cli.istream.readLine(); //it hangs in this line
System.out.println(echo);
}
TCPClient is a class I defined so I can test my program on a simpler interface before using swing on my homwework. here's the code:
public class TCPClient {
public DataOutputStream ostream = null;
public BufferedReader istream = null;
public TCPClient(String host, String port) throws UnknownHostException {
InetAddress ip = InetAddress.getByName(host);
try {
Socket socket = new Socket(host, Integer.parseInt(port));
ostream = new DataOutputStream(socket.getOutputStream());
istream = new BufferedReader(new InputStreamReader(socket.getInputStream()));
} catch (IOException ex) {
Logger.getLogger(TCPClient.class.getName()).log(Level.SEVERE, null, ex);
}
}
My server is pretty simple. After the connection is estabilished, it enters in this loop and stays here until I close the client (because of the infinite loop). Afterwards, some exception handling returns it to the point before the connection started.
while(true){
String msg = istream.readLine();
System.out.println("Arrived on server: " + msg); //just works on debug
ostream.writeUTF("ACK: " + msg);
ostream.flush();
}
I don't see what am I missing.
PS: the wierd stuff is that if I debug the server, I can see the message arriving there (I can print it, for example), but this isn't possible if I just run this code. Does this have some concurrency relation I'm overlooking?
thx
The problem is that readLine tries reading a line. It will not return the line until it's sure that the end of line has been reached. This means that it expects either a newline character, or the end of the communication. Since the server doesn't send any newline char and doesn't close the stream, the client waits indefinitely.
cli.ostream.writeUTF("Teste");
Shouldn't this be containing a new line? Otherwise read method will be waiting for new line I think.
Also as suggested you can try flushing the ostream after writing to it.
writeUTF() doesn't write a line. See the Javadoc. writeUTF() is for use with readUTF(). And Readers are for use with Writers. So change the DataOutputStream to a BufferedWriter and call write() and then newline().