Develop Reference

Accessing byte array values from socket inputStream

I am attempting to retrieve the byte values from an InputStream which is being sent to the socket. I have used many ways but it always prints me the address of the byte array instead of its contents.
Below is my code for Client and Server. When a packet is sent from the client to the server, the server instantiates a new Thread to handle the connection. So slaveSocket is the socket I want to use for this.
public class TCPClient {
public static void main(String[] args) throws IOException{
Socket socket;
String address;
int port;
String userInput;
String serverResponse;
PrintWriter out;
BufferedReader in;
//read characters from user
BufferedReader stdIn;
if (args.length != 2) {
System.err.println("Usage: java EchoClient <address> <port>");
System.exit(1);
}
byte[] mode = "octet".getBytes(Charset.forName("UTF-8"));
address = args[0];
port = Integer.parseInt(args[1]);
try{
//connect socket to server
socket = new Socket(address, port);
//Construct PrintWriter to write objects to the socket
out = new PrintWriter(socket.getOutputStream(), true);
//Construct BufferedReader to read input from the socket
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
//Another reader to read characters typed by the user
stdIn = new BufferedReader(new InputStreamReader(System.in));
//scanner for menu option
Scanner scanner = new Scanner(System.in);
int menuOption;
System.out.println("Press 1 to read from file or 2 to write to file");
menuOption = scanner.nextInt();
if (menuOption == 1){
String filename = "";
String text = "";
System.out.println("Enter file name");
filename = scanner.next();
byte[] packet = new byte[512];
//Constructing the RRQ Packet
//Ading the OPCODE
packet[0] = 1;
//adding the filename
filename.getBytes(Charset.forName("UTF-8"));
byte[] filenameB = filename.getBytes(Charset.forName("UTF-8"));
System.arraycopy(filenameB,0,packet,1, filenameB.length);
//adding a 0
packet[filenameB.length +1] = 0;
//adding the mode
System.arraycopy(mode,0,packet,1+filenameB.length+1,mode.length);
//adding the last 0
packet[1+filenameB.length+1+mode.length+1] = 0;
out.println(packet);
}else if(menuOption == 2){
}
socket.close();
}catch(UnknownHostException e){
System.err.println("Dont know about host" + address);
System.exit(1);
}catch(IOException e){
System.err.println("Couldnt get I/O for the connection to " + address);
System.exit(1);
}
}
}
public class TCPServer {
public static void main(String[] args) throws IOException{
//port of the server
int port = 10000;
//Socket objects
ServerSocket masterSocket;
Socket slaveSocket;
//instantiate the server socket
masterSocket = new ServerSocket(port);
System.out.println("Server Started");
boolean flag1 = true;
while(true){
slaveSocket = masterSocket.accept();
System.out.println("Accepted TCP connection from: " +
slaveSocket.getInetAddress() + ", " + slaveSocket.getPort() + "...");
System.out.println("Initialising new Thread...");
new TCPServerThread(slaveSocket).start();
}
}
}
public class TCPServerThread extends Thread{
private Socket slaveSocket = null;
public TCPServerThread(Socket socket){
super("TCPServerThread");
this.slaveSocket = socket;
}
public void run(){
byte[] ClientPacket = new byte[512];
PrintWriter socketOutput;
InputStream socketInput;
try{
//send packet to client
socketOutput = new PrintWriter((slaveSocket.getOutputStream()), true);
//read packet from client
socketInput = new DataInputStream(slaveSocket.getInputStream());
ClientPacket = socketInput.readAllBytes();
System.out.println(new String(ClientPacket, StandardCharsets.UTF_8));
}catch (IOException e){
System.err.println(e);
}
}
}

You've hopelessly overengineered this.
Writer and Reader do character input and output. InputStream and OutputStream do byte input and output.
You turn byte-based stuff (and in the end, network ports are byte based, not character based) into character based stuff in dangerous ways and then are attempting to read and write bytes into and out of the char-based things.
The solution is simple. Just stop doing that. You have byte-based stuff, there is absolutely no need to involve Reader and Writer.
A bunch of lines that cause problems:
out.println(packet);
PrintStreams are debug aids. You can't use them for any of this. For example, this line will print newlines (definitely not something you'd want in a byte based stream system!), and will print 'objects' - it does that by invoking the .toString() method, and the toString method of arrays are mostly useless. That explains why you see what you see. This is not how you send bytes. You cannot send bytes to a PrintStream (which is a confused mess, as it tries to let you send characters to a byte based system. As I said, you use it for debugging and nothing else. You should not be using it here at all).
new InputStreamReader(socket.getInputStream())
This is dangerous. You're turning a byte based system (InputStream) into a char-based one (Reader) and this always means somebody is making an explicit, 'out of band' (not based on the data in that stream) decision about charset encoding. In this case, as per the docs of InputStreamReader, you get the 'platform default'. Starting with JDK18, it's guaranteed to be UTF-8 fortunately, but before that, who knows what it is. You never want to call this constructor to avoid the confusion. new InputStreamReader(socket.getInputStream, StandardCharsets.UTF_8).
Mostly, though, don't make a reader in the first place. You have no interest whatsoever in reading streams of characters, you just want bytes.
If you have smallish strings and the information about where they 'end' is done 'out of band' (example: The size in bytes (not characters) is sent first, then X bytes that are the string, UTF_8 encoded), you can just read that in as bytes, and then make a string off of that, bypassing any need for Readers and Writers. Reader and Writer is useful only if the entire stream is all character based, or if you have huge strings (hundreds of megabytes) where their end can only be surmised by interpreting the data as characters first. (Mostly, those are horrible protocols that shouldn't be used).
//Construct PrintWriter to write objects to the socket
No, you can't write objects to sockets. Objects aren't bytes. You can write bytes to a socket; some objects will let themselves be turned into bytestreams but this is decidedly not a trivial job, and PrintWriter can't do it at all.
catch (IOException e) { System.err.println(e);
Most code has no reasonable route to 'deal' with them, but the solution to that is to throw them onwards. Not to catch the exception, print a note of despair, and just keep going on like nothing happened. Doing it right is also less code, so, win-win.
stdIn = new BufferedReader(new InputStreamReader(System.in));
//scanner for menu option
Scanner scanner = new Scanner(System.in);
You're making 2 different ways to read standard input. That makes no sense. Pick one.
I tried to fix it for you:
public class TCPClient {
public static void main(String[] args) throws Exception { // always throw Exception from `main`.
if (args.length != 2) {
System.err.println("Usage: java EchoClient <address> <port>");
System.exit(1);
return; // Always return after System.exit.
}
byte[] mode = "octet".getBytes(Charset.forName("UTF-8"));
String address = args[0];
int port = Integer.parseInt(args[1]);
Scanner scanner = new Scanner(System.in);
scanner.useDelimiter("\\R"); // split on newlines, not spaces. So much more logical.
// resources need to be safe-closed - use try-with!
try (var socket = new Socket(address, port);
var out = new BufferedOutputStream(socket.getOutputStream());
var in = socket.getInputStream()) {
System.out.println("Press 1 to read from file or 2 to write to file");
int menuOption = scanner.nextInt();
if (menuOption == 1) {
System.out.println("Enter file name");
String filename = scanner.next();
//Constructing the RRQ Packet
//Adding the OPCODE
out.write(1);
out.write(filename.getBytes(StandardCharsets.UTF_8));
out.write(0);
// The above is dangerous; NUL (0) is actually a valid char.
// A proper way to send strings is to send length in bytes
// first. I'll leave it to you to fix your protocol.
// If it can't be fixed, scan for `\0` chars and get rid of em.
//adding the mode
out.write(mode);
out.write(0);
}else if (menuOption == 2) {
}
}
}
Sending bytes one at a time can be slow (as it ends up sending an entire packet) but can also be useful - the data is just sent, instead of waiting perhaps for a long time for more data. In your case, you send it all in one go, so sending it all off very quickly is not a good idea. Hence, why the outputstream is wrapped in a BufferedOutputStream, which fixes that. You can always use flush() to force sending now, in case you want to keep the connection open (close(), naturally, also flushes).
It's fine if you want to use a byte[] packet instead, but it seems convoluted and unneccessary here. out.write(someByteArray), where out is an OutputStream of some sort, works fine. out.println(byteArray), where out is a Writer of some sort, or a PrintStream - doesn't work at all. (It would take the array, call toString() on it which isn't useful, then convert those bytes using some unknown charset and send that, and none of that is what you want).
You'll need to similarly eliminate PrintStream and the like from your server code.

Java TCP File Transfer Only Complete On First Attempt

Despite hours of researching this problem, I have made very little progress. According to my professor, the code should be working as written...
I have a server that stays open, and a client that requests a file. Once the client receives the file, the client closes.
When I open the server, I am able to transfer a complete .jpg image file. The client then closes while the server remains open. I start up another client and try to transfer the same image, and only a portion of the bytes are transferred/written to the disk. The file transfer is only completely successful for the first file transferred by the server!
Additionally strange, a simple .txt text file never successfully transfers. I believe the cause is on the server side because it remains open as opposed to the client, which starts over each time.
Server Code:
import java.io.*;
import java.net.*;
import java.util.Arrays;
class ft_server {
public static void main(String args[]) throws Exception {
/*
* Asks user for port number and listens on that port
*/
BufferedReader portFromUser = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Enter the port you'd like to use: ");
int portNumber = Integer.valueOf(portFromUser.readLine());
if (portNumber < 1 || portNumber > 65535) {
System.out.println("Please choose a port number between 1 and 65535.");
return;
}
portFromUser.close();
ServerSocket listenSocket = new ServerSocket(portNumber);
/*
* Finished with user input
*/
/*
* Continuously listens for clients:
*/
while (true) {
Socket clientSocket = listenSocket.accept();
BufferedReader inFromClient = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
DataOutputStream outToClient = new DataOutputStream(clientSocket.getOutputStream());
String clientIP = clientSocket.getRemoteSocketAddress().toString();
System.out.println("The client " + clientIP + " connected!");
String clientMessage = inFromClient.readLine();
System.out.println("The client requested file: " + clientMessage);
// Get file. If doesn't exist, let's client know.
// Otherwise informs client of file size.
File myFile = new File(clientMessage);
if (!myFile.exists()) {
outToClient.writeBytes("File does not exist!\n");
return;
} else {
outToClient.writeBytes(String.valueOf((int)myFile.length()) + "\n");
}
// Create array for storage of file bytes:
byte[] byteArray = new byte[(int)myFile.length()];
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(myFile));
// Read file into array:
bis.read(byteArray, 0, byteArray.length);
// Send the file:
outToClient.write(byteArray, 0, byteArray.length);
outToClient.close();
clientSocket.close();
}
}
}
Client Code:
import java.io.*;
import java.net.*;
class ft_client {
public static void main(String args[]) throws Exception {
int byteSize = 2022386;
int bytesRead;
/*
* Asks user for IP and port:
*/
BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Enter an IP address: ");
String ipAddress = inFromUser.readLine();
System.out.println("Enter a port: ");
String port = inFromUser.readLine();
Socket clientSocket;
try {
// Makes socket, port, and calls connect. Assumes it's TCP:
clientSocket = new Socket(ipAddress, Integer.valueOf(port));
} catch (Exception e) {
System.out.println(e.getMessage());
return;
}
// Creates InputStream from server to get file size and other messages:
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
// Anything written to this will be sent to the server:
DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
// Asks for a file name to download from the server:
System.out.println("What file do you want?: ");
String message = inFromUser.readLine();
outToServer.writeBytes(message + "\n");
inFromUser.close();
// Listens for confirmation from server.
// If the file exists, the file size is delivered here:
String response = inFromServer.readLine();
System.out.println("File size: " + response);
if (response.equals("File does not exist!")) {
return;
}
// Receives file from server:
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[byteSize];
InputStream is = clientSocket.getInputStream(); // calling clientSocket.getInputStream() twice???
FileOutputStream fos = new FileOutputStream(message);
BufferedOutputStream bos = new BufferedOutputStream(fos);
// Continuously writes the file to the disk until complete:
int total = 0;
while ((bytesRead = is.read(byteArray)) != -1) {
bos.write(byteArray, 0, bytesRead);
total += bytesRead;
}
bos.close();
System.out.println("File downloaded (" + total + " bytes read)");
clientSocket.close();
}
}
Are buffered readers interfering with output streams? Is there a better way to transfer files?

It's worth checking, in your server code, what value comes back from the file read() call, so:
int bytesRead = bis.read(byteArray, 0, byteArray.length);
System.out.println("File bytes read: " + bytesRead + " from file size: " + myFile.length());
The read() method is under no obligation to fill the byteArray - only to return something and to tell you how many bytes it read. From the docs, it:
Reads up to len bytes of data from this input stream into an array of
bytes. If len is not zero, the method blocks until some input is
available; otherwise, no bytes are read and 0 is returned.
You need to keep reading in a loop. I'd do this (actually, same as your client!):
int n;
while ((n = bis.read(byteArray, 0, byteArray.length)) != -1) {
// Send the chunk of n bytes
outToClient.write(byteArray, 0, n);
}
bis.close();
outToClient.close();
or something similar. I've closed the file too: it'd close on GC/finalize, but that could be a while, and meanwhile you're holding the file open.
EDIT
The specific problem with your image-read in this case is in your client code. You read the file size near the top of the code:
// Creates InputStream from server to get file size and other messages:
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
and then you access the client again:
InputStream is = clientSocket.getInputStream(); // calling clientSocket.getInputStream() twice???
and as your comment suggests, this is bad! Thank you to #EJP for highlighting this!
This causes a problem of buffer over-ingestion: the BufferedReader consumes more bytes into its belly than you extract from it, so when you visit the clientSocket inputstream the second time, the read-pointer has moved on. You never look again at what the BufferedReader consumed.
As a general rule, once you plug buffering code onto something, you must be careful to read only from that buffer. In this case, it's difficult, because you can't read image (raw binary) data from a Reader, because it will busily interpret the binary values as characters and read them as UTF-8 or something.
Even without buffers, it's a minor sin to mix Readers (text oriented) and binary data (DataStreams) on the same stream. HTTP and email does this, so you are in good company, but they get away with it by being very tightly specified. Problem is, you can easily get snarled with questions of local/default character encoding at each end, whether you're reading Unix "LF" vs Windows "CR/LF" line endings etc.
In this case, try not using BufferedReaders at all, and try using DataInput/Output streams all the way. Try writeUTF(s) and readUTF() for transferring the String data. Ideally, create them like this:
DataInputStream inFromServer = new DataInputStream (new BufferedInputStream(clientSocket.getInputStream()));
so you still get the benefits of buffering.
EDIT 2
So seeing the new client code:
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[byteSize];
FileOutputStream fos = new FileOutputStream(message);
int readBytes = inFromServer.read(byteArray);
// Continuously writes the file to the disk until complete:
int total = 0;
for (int i=0; i<byteArray.length; i++) {
fos.write(byteArray[i]);
total++;
}
fos.close();
Here, we're assuming that because the byteArray array is set to the right size, that the inFromServer.read(byteArray) will populate it - it won't. It's good to assume that any and all read operations will return you just as much data as the system has to hand: in this case, it's probably going to return as soon as it gets the first packet or two, with an underfilled array. This is same as C and Unix read behaviour too.
Try this - I'm repeatedly reading and writing a 4K buffer, until the byte count is reached (as determined by summing the return values of the reads):
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[4096];
FileOutputStream fos = new FileOutputStream(message);
int total = 0;
// Continuously writes the file to the disk until complete:
while (total < byteSize && (readBytes = inFromServer.read(byteArray)) != -1) {
fos.write(byteArray, 0, readBytes);
total += readBytes;
}
fos.close();
A variant is this - same thing, but byte at a time. Might be a bit clearer. It's going to be slow - all those reads and writes are hitting the OS, but if you put a BufferedInputStream/BufferedOutputStream around the socket/file streams, it'll iron that out. I've added them:
DataInputStream inFromServer =
new DataInputStream(new BufferedInputStream(clientSocket.getInputStream()));
...
byteSize = (int) Integer.valueOf(response);
OutputStream fos = new BufferedOutputStream(FileOutputStream(message));
int total = 0;
int ch;
// Continuously writes the file to the disk until complete:
while (total < byteSize && (ch = inFromServer.read()) != -1) {
fos.write(ch);
total ++;
}
fos.close();
And finally! the simplest answer is this. Your code, but changed to:
int readBytes = inFromServer.readFully(byteArray);
Yes! Those nice people in 1990's Javasoft added a DataInput.readFully method, which does what you want! - basically wraps the code above. It's the simplest solution, and arguably most correct approach: "use existing libraries where possible". OTOH, it's the least educational, and the time you spend getting used to read/writes like this is not deducted from your life-expectancy!
And in fact, the readFully approach has severe limitations. Try pointing it at a 1GB file and see what happens (after you've fixed up the array size at the top): you'll a) run out memory, and b) wish that while you were ingesting a huge blob, you could at least be spooling it out to disk. If you try a 2.5G file, you'll notice that some of those ints should become longs to cope with numbers >= 2^31.
If it was me, I'd do the 4K buffer one. (BTW I'm writing this on a laptop with no Java compiler installed, so I haven't actually run the above! DO respond if there are any difficulties.)

Java file not working when sent over a network

So I was implementing client and socket for java. I wanted to send huge files on tcp through sockets and I was able to send files too but the only problem was the files on the other end were either not complete or not working. I have checked the bits are being transfered then what is the error.
Client side:
Socket sock = new Socket("127.0.0.1", 1056);
byte[] mybytearray = new byte[1024];
InputStream is = sock.getInputStream();
FileOutputStream fos = new FileOutputStream("abc.mp3");
BufferedOutputStream bos = new BufferedOutputStream(fos);
int bytesRead = is.read(mybytearray, 0, mybytearray.length);
int len = 0;
while((len = is.read(mybytearray)) != -1)
{
bos.write(mybytearray, 0, len);
System.out.println("sending");
}
bos.close();
sock.close();
Server side:
ServerSocket ss = new ServerSocket(1056);
while (true) {
Socket s = ss.accept();
PrintStream out = new PrintStream(s.getOutputStream());
BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()));
String info = null;
String request = null;
System.out.println("sending");
String filename = "abc.mp3";
File fi = new File(filename);
InputStream fs = new FileInputStream(fi);
int n = fs.available();
byte buf[] = new byte[1024];
out.println("Content_Length:" + n);
out.println("");
while ((n = fs.read(buf)) >= 0) {
out.write(buf, 0, n);
System.out.println("sending");
}
out.close();
s.close();
in.close();
}

When you are connected via TCP you create a network stream which you can read and write in, similar to all other streams you worked with. Writing a large amount of data to the stream is not a good idea, so I suggest you break the selected file into smaller packets in which each packet length is 1024 bytes (1KB) and then send all the packets to the server. The SendTCP function is as follows:(I have used Windows Forms to make things more obvious)
public void SendTCP(string M, string IPA, Int32 PortN)
{
byte[] SendingBuffer = null
TcpClient client = null;
lblStatus.Text = "";
NetworkStream netstream = null;
try
{
client = new TcpClient(IPA, PortN);
lblStatus.Text = "Connected to the Server...\n";
netstream = client.GetStream();
FileStream Fs = new FileStream(M, FileMode.Open, FileAccess.Read);
int NoOfPackets = Convert.ToInt32
(Math.Ceiling(Convert.ToDouble(Fs.Length) / Convert.ToDouble(BufferSize)));
progressBar1.Maximum = NoOfPackets;
int TotalLength = (int)Fs.Length, CurrentPacketLength, counter = 0;
for (int i = 0; i < NoOfPackets; i++)
{
if (TotalLength > BufferSize)
{
CurrentPacketLength = BufferSize;
TotalLength = TotalLength - CurrentPacketLength;
}
else
CurrentPacketLength = TotalLength;
SendingBuffer = new byte[CurrentPacketLength];
Fs.Read(SendingBuffer, 0, CurrentPacketLength);
netstream.Write(SendingBuffer, 0, (int)SendingBuffer.Length);
if (progressBar1.Value >= progressBar1.Maximum)
progressBar1.Value = progressBar1.Minimum;
progressBar1.PerformStep();
}
lblStatus.Text=lblStatus.Text+"Sent "+Fs.Length.ToString()+"
bytes to the server";
Fs.Close();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
netstream.Close();
client.Close();
}
}
As you can see, a TCP client and a network stream are being constructed and a network connection is initiated. After opening the selected file according to the buffer size which is 1024 bytes, the number of packets that are going to be sent is calculated. There are two other variables CurrentPacketLength and TotalLength. If the total length of the selected file is more than the buffer size the CurrentPacketLength is set to the buffer size, otherwise why send some empty bytes, so CurrentPacketLength is set to the total length of the file. After that, I subtract the current from the total length, so actually we can say total length is showing the total amount of data that has not been sent yet. The rest is pretty much straight forward, reading the data from the file stream and writing it to the SendingBuffer according to the CurrentPacketLength and writing the buffer to the network stream.
At the server side, the application is listening for an incoming connection:
public void ReceiveTCP(int portN)
{
TcpListener Listener = null;
try
{
Listener = new TcpListener(IPAddress.Any, portN);
Listener.Start();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
byte[] RecData = new byte[BufferSize];
int RecBytes;
for (; ; )
{
TcpClient client = null;
NetworkStream netstream = null;
Status = string.Empty;
try
{
string message = "Accept the Incoming File ";
string caption = "Incoming Connection";
MessageBoxButtons buttons = MessageBoxButtons.YesNo;
DialogResult result;
if (Listener.Pending())
{
client = Listener.AcceptTcpClient();
netstream = client.GetStream();
Status = "Connected to a client\n";
result = MessageBox.Show(message, caption, buttons);
if (result == System.Windows.Forms.DialogResult.Yes)
{
string SaveFileName=string.Empty;
SaveFileDialog DialogSave = new SaveFileDialog();
DialogSave.Filter = "All files (*.*)|*.*";
DialogSave.RestoreDirectory = true;
DialogSave.Title = "Where do you want to save the file?";
DialogSave.InitialDirectory = #"C:/";
if (DialogSave.ShowDialog() == DialogResult.OK)
SaveFileName = DialogSave.FileName;
if (SaveFileName != string.Empty)
{
int totalrecbytes = 0;
FileStream Fs = new FileStream
(SaveFileName, FileMode.OpenOrCreate, FileAccess.Write);
while ((RecBytes = netstream.Read
(RecData, 0, RecData.Length)) > 0)
{
Fs.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
Fs.Close();
}
netstream.Close();
client.Close();
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
//netstream.Close();
}
}
}
A TCP listener is created and starts listening to the specified port. Again the buffer size is set to 1024 bytes. A TCP listener can pre check to see if there are any connections pending before calling the AcceptTcpClient method. It returns true if there are any pending connections. This method is a good way of avoiding the socket being blocked. Before reading anything from the network stream, a message box asks you if you want to accept the incoming connection, then a SaveFileDialog will be opened, and when you enter the file name plus extension, a file stream will be constructed and you start reading from the network stream and writing to the file stream. Create a thread in your code and run the receiving method in the created thread. I have sent more than 100 MB files in a LAN with the application.
For more details, check this article.

So, first you do this
int bytesRead = is.read(mybytearray, 0, mybytearray.length);
That reads up to 1024 bytes into mybytearray.
You don't do anything with that and I don't understand why you are doing it. You never write those bytes so they get overwritten if the while loop reads anything.
Just delete that. The while loop should cover all of this.

performance issue. The Speed is too slow while reading bytes from socket using java

these days I'm confused about the Tcp performance while using java socket. In fact the java code is very simple. details as below:
server open a port and begin to listen.
client request and after connect to server, client begin to write to socket.
after server got the request, it will open a new thread to handle this connection. (this connection is a long connection which will not time out).
the server will keep reading until it got the end separator, then give a response to the client and continue to keep reading again.
after client get the response, it will send another request again.
I find if the client write the whole message (including the end separator) one time, the communication speed is good satisfactorily, the speed can reach to 50000 messages per minute. How ever, if the client write the bytes to socket in separated times, the speed cut down quickly, just almost 1400 messages per minute, it is 1/40 times compared with the original speed. I'm quite confused about it. Any one could give me a hand? Any comments is appreciated!
my simulated server side is as below:
public class ServerForHelp {
final static int BUFSIZE = 10240;
Socket socket;
String delimiter = "" + (char) 28 + (char) 13;
public static void main(String[] args) throws IOException {
ServerSocket ss = new ServerSocket(9200);
System.out.println("begin to accept...");
while (true) {
Socket s = ss.accept();
Thread t = new Thread(new SocketThread1(s));
t.start();
}
}
public String readUntilDelimiter() throws Exception {
StringBuffer stringBuf = new StringBuffer();
InputStream stream = socket.getInputStream();
InputStreamReader reader = null;
reader = new InputStreamReader(stream);
char[] buf = new char[BUFSIZE];
while (true) {
int n = -1;
n = reader.read(buf, 0, BUFSIZE);
if (n == -1) {
return null; // it means the client has closed the connection, so return null.
} else if (n == 0) {
continue; // continue to read the data until got the delimiter from the socket.
}
stringBuf.append(buf, 0, n);
String s = stringBuf.toString();
int delimPos = s.indexOf(delimiter);
if (delimPos >= 0) {
// found the delimiter; return prefix of s up to separator and
// To make the thing simple, I have discarded the content after the delimiter.
String result = s.substring(0, delimPos);
sendTheResponse(socket);
return result;
}
}
}
private void sendTheResponse(Socket socket) throws IOException {
Writer writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
writer.write("Hi, From server response");
writer.flush();
}
}
class SocketThread1 implements Runnable {
Socket socket;
public SocketThread1(Socket socket) {
this.socket = socket;
}
#Override
public void run() {
ServerForHelp server = new ServerForHelp();
server.socket = socket;
while (true) {
try {
if (server.readUntilDelimiter() == null) // it means that the client has closed the connection, exist
break;
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
It is a normal socket programming.
and the following is my client side:
public void execute() throws Exception{
int msgCnt = 0;
Socket socket = null;
byte[] bufBytes = new byte[512];
long start = 0;
final char START_MESSAGE = 0x0B;
final char END_MESSAGE = 0x1C;
final char END_OF_RECORD = 0x0D;//\r
String MESSAGE = "HELLO, TEST";
socket = new Socket("192.168.81.39", 9200);
OutputStream os = socket.getOutputStream();
InputStream is = socket.getInputStream();
while (System.currentTimeMillis() - start < 60000)
{
// If you send the total message at one time, the speed will be improved significantly
// FORMAT 1
StringBuffer buf = new StringBuffer();
buf.append(START_MESSAGE);
buf.append(MESSAGE);
buf.append(END_MESSAGE);
buf.append(END_OF_RECORD);
os.write(buf.toString().getBytes());
// FORMAT 1 END
//FORMAT 2
// os.write(START_MESSAGE);
// os.write(MESSAGES[port].getBytes());
// os.write(END_MESSAGE);
// os.write(END_OF_RECORD);
//FORMAT 2 END
os.flush();
is.read(bufBytes);
msgCnt++;
System.out.println(msgCnt);
}
System.out.println( msgCnt + " messages per minute");
}
If I use the "FORMAT 1", to send the message, the speed could reach to 50000 messages per minute, but If use "FORMAT 2", the speed is down to 1400 messages per minute. Who is clear about the reason?
I'm trying to describe as detail as I can and any help will be appreciated very much.

Multiple very short writes to a socket in rapid succession followed by a read can trigger a bad interaction between Nagle's algorithm and TCP delayed acknowledgment; even if you disable Nagle's algorithm, you'll cause an entire packet to be sent per individual write call (with 40+ bytes of overhead, whether the write is one byte or a thousand).
Wrapping a BufferedOutputStream around the socket's output stream should give you performance similar to "FORMAT 1" (precisely because it holds things in a byte array until it fills or is flushed).
As John Nagle explained on Slashdot:
The user-level solution is to avoid write-write-read sequences on sockets. write-read-write-read is fine. write-write-write is fine. But write-write-read is a killer. So, if you can, buffer up your little writes to TCP and send them all at once.

InputStream skips bytes

For the second time I have this extremely anoying problem with an InputStream.
This InputStream belongs to a Socket that is supposed to receive an image. The code for reading this image is as below:
InputStream input = socket.getInputStream();
InputStreamReader reader = new InputStreamReader(input);
BufferedReader bufferedReader = new BufferedReader(reader);
int total = Integer.parseInt(bufferedReader.readLine());
int bytesRead = 0;
byte[] buffer = new byte[total]; // total is the total size of the image
while (bytesRead < total) {
int next = input.read(buffer, bytesRead, total-bytesRead);
if (next > 0) {
bytesRead += next;
System.out.println("Read: " + bytesRead);
}
}
Now the strange thing is that this code skips the first 1182 bytes of the image, and then reads the remaining part. So when the total size is 15000 bytes, it reads byte 1182-15000.
I checked Wireshark and the whole image is transmitted. The code throws no exceptions. input.read() returns -1 as usual.
Pervious data has been readed from the stream using a BufferedReader. This data is only 5 characters long so it can't contain the missing 1K, but my guess is that the BufferedReader.readLine() method reads (buffers) more bytes from the InputStream than needed. Could this be correct?
I've had the same problem a few months ago but I absolutely have no clue on how I solved it.
Hope anyone can help.
Thanks in advance.
EDIT: I can solve the problem by adding a 100 ms sleep between sending the image size and the image data. It solves the problem but I would still realy like to know a more appropriate solution

As the name **Buffererd**Reader indicates it will snarf more bytes than just the first line from the underlying reader and hence also from the stream. Otherwise if would not be called "buffered".
Unfortunately I'm not aware of any non-deprecated class in Java which allows mixing of binary and textual data in the way you want.
I suggest, that you modify your protocol and transfer the length of the image also in some binary encoding. Then you can stick to InputStream.

My guess is the BufferedReader will assume that whatever reading operations you're performing afterwards will go through it, so it will happily consume input in increments of its buffer size.
One thing you could do is use a BufferedInputStream on top of input, instantiate a DataInputStream on top of that to do the readLine(), and then use the BufferedInputStream in your loop. The documentation says readLine is deprecated because it doesn't convert bytes to characters properly, but I'm hoping that with your first line containing only decimal digits, it shouldn't run into that problem.
I've written a short test, I hope it covers your use case:
import java.net.*;
import java.io.*;
class test{
static byte[] b;
static {
b = new byte[123456];
java.util.Random r = new java.util.Random();
for (int i=0;i<b.length;i++)
b[i] = (byte)(r.nextInt());
}
static void client() throws Exception{
Socket socket = new Socket("127.0.0.1", 9000);
InputStream input = socket.getInputStream();
BufferedInputStream bis = new BufferedInputStream(input);
//read in length
DataInputStream dais = new DataInputStream(bis);
int total = Integer.parseInt(dais.readLine());
System.out.println("Total: "+total);
int bytesRead = 0;
byte[] buffer = new byte[total];
while (bytesRead < total) {
int next = bis.read(buffer, bytesRead, total-bytesRead);
if (next > 0) {
bytesRead += next;
System.out.println("Read: " + bytesRead);
}
}
for (int i=0;i<buffer.length;i++)
if (buffer[i]!=b[i]){
System.err.println("Error");
System.exit(1);
}
System.out.println("OK");
bis.close();
socket.close();
}
static void server() throws Exception{
ServerSocket srv = new ServerSocket(9000);
Socket sock = srv.accept();
OutputStream os = sock.getOutputStream();
BufferedOutputStream bos =new BufferedOutputStream(os);
DataOutputStream daos = new DataOutputStream(bos);
//we're sending the b buffer
//send the length in plain text, followed by newline
byte[]num = String.valueOf(b.length).getBytes();
daos.write(num,0,num.length);
daos.write(10);
//send actual buffer contents
bos.write(b, 0, b.length);
os.close();
srv.close();
sock.close();
}
public static void main(String[]args){
new Thread(new Runnable(){
public void run(){
try{server();}catch(Exception e){e.printStackTrace();}
}
}).start();
new Thread(new Runnable(){
public void run(){
try{client();}catch(Exception e){e.printStackTrace();}
}}).start();
}
}

Have you considered just sending the image data (without the preceding image size) and using an external library such as Apache Commons IO to handle this for you? In particular, I think that you will find IOUtils.toByteArray(InputStream input) interesting, e.g.
InputStream input = socket.getInputStream();
byte[] buffer = IOUtils.toByteArray(input);