I have been searching the web for a while now looking for an answer to this.
The python code for sending a file:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
with open(path, mode='rb') as f:
s.sendall(f.read())
The java code for receiving the data:
private BufferedInputStream bis = new BufferedInputStream(socket.getInputStream());
try (BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(path))) {
while (true) {
int size = bis.read(by);
if (size > 0) {
bos.write(by, 0, size);
bos.flush();
total += size;
System.out.println(size + "(" + total + ")");
if (total == length) {
break;
}
}
}
}.....
I see that the data is being sent, and i know the data is in the stream at the java end. However the bis.read(by) call hangs, and refuses to read the data until the data the connection dies.
I suspect it has something to do with the "flush" stuff in java, but i can not find any way to do a "flush" using python.
Any clues why this might happen?
I figured it out, for some reason a race condition is occuring, putting a sleep(0,2) in the python code made the java server able to responde.
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sleep(0.2)
with open(path, mode='rb') as f:
s.sendall(f.read())
Related
Despite hours of researching this problem, I have made very little progress. According to my professor, the code should be working as written...
I have a server that stays open, and a client that requests a file. Once the client receives the file, the client closes.
When I open the server, I am able to transfer a complete .jpg image file. The client then closes while the server remains open. I start up another client and try to transfer the same image, and only a portion of the bytes are transferred/written to the disk. The file transfer is only completely successful for the first file transferred by the server!
Additionally strange, a simple .txt text file never successfully transfers. I believe the cause is on the server side because it remains open as opposed to the client, which starts over each time.
Server Code:
import java.io.*;
import java.net.*;
import java.util.Arrays;
class ft_server {
public static void main(String args[]) throws Exception {
/*
* Asks user for port number and listens on that port
*/
BufferedReader portFromUser = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Enter the port you'd like to use: ");
int portNumber = Integer.valueOf(portFromUser.readLine());
if (portNumber < 1 || portNumber > 65535) {
System.out.println("Please choose a port number between 1 and 65535.");
return;
}
portFromUser.close();
ServerSocket listenSocket = new ServerSocket(portNumber);
/*
* Finished with user input
*/
/*
* Continuously listens for clients:
*/
while (true) {
Socket clientSocket = listenSocket.accept();
BufferedReader inFromClient = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
DataOutputStream outToClient = new DataOutputStream(clientSocket.getOutputStream());
String clientIP = clientSocket.getRemoteSocketAddress().toString();
System.out.println("The client " + clientIP + " connected!");
String clientMessage = inFromClient.readLine();
System.out.println("The client requested file: " + clientMessage);
// Get file. If doesn't exist, let's client know.
// Otherwise informs client of file size.
File myFile = new File(clientMessage);
if (!myFile.exists()) {
outToClient.writeBytes("File does not exist!\n");
return;
} else {
outToClient.writeBytes(String.valueOf((int)myFile.length()) + "\n");
}
// Create array for storage of file bytes:
byte[] byteArray = new byte[(int)myFile.length()];
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(myFile));
// Read file into array:
bis.read(byteArray, 0, byteArray.length);
// Send the file:
outToClient.write(byteArray, 0, byteArray.length);
outToClient.close();
clientSocket.close();
}
}
}
Client Code:
import java.io.*;
import java.net.*;
class ft_client {
public static void main(String args[]) throws Exception {
int byteSize = 2022386;
int bytesRead;
/*
* Asks user for IP and port:
*/
BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Enter an IP address: ");
String ipAddress = inFromUser.readLine();
System.out.println("Enter a port: ");
String port = inFromUser.readLine();
Socket clientSocket;
try {
// Makes socket, port, and calls connect. Assumes it's TCP:
clientSocket = new Socket(ipAddress, Integer.valueOf(port));
} catch (Exception e) {
System.out.println(e.getMessage());
return;
}
// Creates InputStream from server to get file size and other messages:
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
// Anything written to this will be sent to the server:
DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
// Asks for a file name to download from the server:
System.out.println("What file do you want?: ");
String message = inFromUser.readLine();
outToServer.writeBytes(message + "\n");
inFromUser.close();
// Listens for confirmation from server.
// If the file exists, the file size is delivered here:
String response = inFromServer.readLine();
System.out.println("File size: " + response);
if (response.equals("File does not exist!")) {
return;
}
// Receives file from server:
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[byteSize];
InputStream is = clientSocket.getInputStream(); // calling clientSocket.getInputStream() twice???
FileOutputStream fos = new FileOutputStream(message);
BufferedOutputStream bos = new BufferedOutputStream(fos);
// Continuously writes the file to the disk until complete:
int total = 0;
while ((bytesRead = is.read(byteArray)) != -1) {
bos.write(byteArray, 0, bytesRead);
total += bytesRead;
}
bos.close();
System.out.println("File downloaded (" + total + " bytes read)");
clientSocket.close();
}
}
Are buffered readers interfering with output streams? Is there a better way to transfer files?
It's worth checking, in your server code, what value comes back from the file read() call, so:
int bytesRead = bis.read(byteArray, 0, byteArray.length);
System.out.println("File bytes read: " + bytesRead + " from file size: " + myFile.length());
The read() method is under no obligation to fill the byteArray - only to return something and to tell you how many bytes it read. From the docs, it:
Reads up to len bytes of data from this input stream into an array of
bytes. If len is not zero, the method blocks until some input is
available; otherwise, no bytes are read and 0 is returned.
You need to keep reading in a loop. I'd do this (actually, same as your client!):
int n;
while ((n = bis.read(byteArray, 0, byteArray.length)) != -1) {
// Send the chunk of n bytes
outToClient.write(byteArray, 0, n);
}
bis.close();
outToClient.close();
or something similar. I've closed the file too: it'd close on GC/finalize, but that could be a while, and meanwhile you're holding the file open.
EDIT
The specific problem with your image-read in this case is in your client code. You read the file size near the top of the code:
// Creates InputStream from server to get file size and other messages:
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
and then you access the client again:
InputStream is = clientSocket.getInputStream(); // calling clientSocket.getInputStream() twice???
and as your comment suggests, this is bad! Thank you to #EJP for highlighting this!
This causes a problem of buffer over-ingestion: the BufferedReader consumes more bytes into its belly than you extract from it, so when you visit the clientSocket inputstream the second time, the read-pointer has moved on. You never look again at what the BufferedReader consumed.
As a general rule, once you plug buffering code onto something, you must be careful to read only from that buffer. In this case, it's difficult, because you can't read image (raw binary) data from a Reader, because it will busily interpret the binary values as characters and read them as UTF-8 or something.
Even without buffers, it's a minor sin to mix Readers (text oriented) and binary data (DataStreams) on the same stream. HTTP and email does this, so you are in good company, but they get away with it by being very tightly specified. Problem is, you can easily get snarled with questions of local/default character encoding at each end, whether you're reading Unix "LF" vs Windows "CR/LF" line endings etc.
In this case, try not using BufferedReaders at all, and try using DataInput/Output streams all the way. Try writeUTF(s) and readUTF() for transferring the String data. Ideally, create them like this:
DataInputStream inFromServer = new DataInputStream (new BufferedInputStream(clientSocket.getInputStream()));
so you still get the benefits of buffering.
EDIT 2
So seeing the new client code:
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[byteSize];
FileOutputStream fos = new FileOutputStream(message);
int readBytes = inFromServer.read(byteArray);
// Continuously writes the file to the disk until complete:
int total = 0;
for (int i=0; i<byteArray.length; i++) {
fos.write(byteArray[i]);
total++;
}
fos.close();
Here, we're assuming that because the byteArray array is set to the right size, that the inFromServer.read(byteArray) will populate it - it won't. It's good to assume that any and all read operations will return you just as much data as the system has to hand: in this case, it's probably going to return as soon as it gets the first packet or two, with an underfilled array. This is same as C and Unix read behaviour too.
Try this - I'm repeatedly reading and writing a 4K buffer, until the byte count is reached (as determined by summing the return values of the reads):
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[4096];
FileOutputStream fos = new FileOutputStream(message);
int total = 0;
// Continuously writes the file to the disk until complete:
while (total < byteSize && (readBytes = inFromServer.read(byteArray)) != -1) {
fos.write(byteArray, 0, readBytes);
total += readBytes;
}
fos.close();
A variant is this - same thing, but byte at a time. Might be a bit clearer. It's going to be slow - all those reads and writes are hitting the OS, but if you put a BufferedInputStream/BufferedOutputStream around the socket/file streams, it'll iron that out. I've added them:
DataInputStream inFromServer =
new DataInputStream(new BufferedInputStream(clientSocket.getInputStream()));
...
byteSize = (int) Integer.valueOf(response);
OutputStream fos = new BufferedOutputStream(FileOutputStream(message));
int total = 0;
int ch;
// Continuously writes the file to the disk until complete:
while (total < byteSize && (ch = inFromServer.read()) != -1) {
fos.write(ch);
total ++;
}
fos.close();
And finally! the simplest answer is this. Your code, but changed to:
int readBytes = inFromServer.readFully(byteArray);
Yes! Those nice people in 1990's Javasoft added a DataInput.readFully method, which does what you want! - basically wraps the code above. It's the simplest solution, and arguably most correct approach: "use existing libraries where possible". OTOH, it's the least educational, and the time you spend getting used to read/writes like this is not deducted from your life-expectancy!
And in fact, the readFully approach has severe limitations. Try pointing it at a 1GB file and see what happens (after you've fixed up the array size at the top): you'll a) run out memory, and b) wish that while you were ingesting a huge blob, you could at least be spooling it out to disk. If you try a 2.5G file, you'll notice that some of those ints should become longs to cope with numbers >= 2^31.
If it was me, I'd do the 4K buffer one. (BTW I'm writing this on a laptop with no Java compiler installed, so I haven't actually run the above! DO respond if there are any difficulties.)
I have a problem with sending directory names over socket from my C++ client, to my Java server.
Sending ordinary messages like "hello world", works great , but the following doesn't and I can not figure out what the problem is:
char const * files = ffd.cFileName; // get directory name
string str(files, 0, strlen(files)); // convert pointer to string, right?
char mess[str.size()];
strcpy(mess, str.c_str()); // make char array :)
cout << "Send file: " << mess << " with strlen: " << strlen(mess) << " and sizeof: " << sizeof(mess) << endl;
int sent = 0;
if ((sent = send(connectSocket, mess, sizeof(mess), 0)) == SOCKET_ERROR)
{
closesocket(connectSocket);
WSACleanup();
connectToServer();
}
The java server just receives the directory names like this:
wam
p
Win
dow
s
Win
dow
s.o
ld
wxW
idg
ets
I can not understand what I'm missing because I have tried every possible way to do this and the C++ client prints like:
"Send file: windows with strlen: 7 and sizeof: 7"
I do not think that the java server is the problem since I can receive normal strings and messages perfectly, but anyway here is the JAVA code:
is = socket.getInputStream();
byteArray = new byteArray[1024];
while (true) {
c = is.read(byteArray, 0, byteArray.length);
String recv = new String(byteArray, 0, c);
System.out.println(recv);
if (recv.equals("<EOF>")){
break;
}
list.add(recv);
}
If you request something else or anything just leave a comment and I will fix it.
Question: are you sending via TCP or UDP? I'm guessing TCP, and if that is the case, you need to treat the socket as more of a stream. That stream may get broken up into a bunch of packets - you don't really control that. What I might do is to prefix the string length of each directory (ex, 3foo, 4barz, etc), read from the socket and determine what constitutes as a logical block or string, and then assemble / print the strings based on that. If you go with that route, you need to track how much you read each time until you think you are done.
I solved it, Just added byteArray = new byte[1024]; and now it works:
while (true) {
byteArray = new byte[1024]; // I ADDED THIS AND NOW THE JAVA SERVER RECEIVES IT CORRECTLY!
c = is.read(byteArray, 0, byteArray.length);
recv = new String(byteArray, 0, c);
System.out.println(recv);
if (recv.equals("<EOF>")){
break;
}
list.add(recv);
}
I'm more of a Java developer than a C# developer, but I work with both languages; Unfortunately I'm nowhere near the level of most people in either language, but that's why I'm constantly asking questions and reading to expand my knowledge.
Currently I've been working on a Server/Client in Java which works wonderfully, I've written a test client in Java for a game that I've been working on in Unity3D. Honestly, I would just use Java for the entire game if the community was there to easily get level designers etc.
In this code, I'm using a BufferedInputStream in java, and the setup looks like this:
DataInputStream dataIn = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
Throughout this code, I perform the following logic to check to make sure all of the data for a specified packet as arrived:
if(dataIn.available() < 4) {
continue;
}
dataIn.mark(4);
int length = dataIn.readInt();
System.out.println("Packet length is" + length);
if(dataIn.available() < length) {
System.out.println("Only read " + dataIn.available() + "/ " + length + " bytes.");
dataIn.reset();
continue;
}
and I've been struggling to find an equivalent to this in C#. -- Other issues I've noticed is that a byte being sent by java's DataOutputStream, and the byte being read by C#'s BinaryReader is not always the same, but that's another problem.
Something like this reads all the expected data in a memorystream. Further processing of the received data is possible by using the memorystream as a stream, or by getting the bytes in it with memoryStream.ToArray().
using (var ns = new NetworkStream(socket))
{
int dataLength = 0;
// reading the datalength
for (int i = 0; i < 4; i++)
{
while (!ns.DataAvailable)
{
System.Threading.Thread.Sleep(20);
}
dataLength = (dataLength << 8) + ns.ReadByte();
}
// reading the data
byte[] buffer = new byte[1024];
int bytesRead;
using (var memoryStream = new MemoryStream())
{
while (memoryStream.Length < dataLength)
{
while (!ns.DataAvailable)
{
System.Threading.Thread.Sleep(20);
}
bytesRead = ns.Read(buffer, 0, buffer.Length);
memoryStream.Write(buffer, 0, bytesRead);
}
}
}
Edit: minimalistic approach
Beware of the socket.ReceiveBufferSize when using this approach! If it's smaller than data size you're in for a long sleep.
Socket socket = listener.AcceptSocket();
while (socket.Available < 4)
{
System.Threading.Thread.Sleep(20);
}
byte[] lengthBuffer = new byte[4];
socket.Receive(lengthBuffer);
if (BitConverter.IsLittleEndian) Array.Reverse(lengthBuffer);
int dataLength = BitConverter.ToInt32(lengthBuffer, 0);
while (socket.Available < dataLength)
{
System.Threading.Thread.Sleep(20);
}
byte[] dataBuffer = new byte[dataLength];
socket.Receive(dataBuffer);
Local on Linux. It's about 10 seconds for a 20k message. My guess is my Java is bad and Python is fine.
py client:
def scan(self, msg):
try:
print 'begin scan'
HOST = 'localhost'
PORT = 33000
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT));
s.sendall(msg)
data = s.recv(1024)
s.close()
print 'Received', repr(data)
except Exception, e:
print "error: " + str(e)
Java server:
ServerSocket service = new ServerSocket(33000);
while(true) {
debug("Begin waiting for connection");
//this spins
Socket connection = service.accept();
debug("Connection received from " + connection.getInetAddress().getHostName());
OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream());
BufferedInputStream in = new BufferedInputStream(connection.getInputStream());
ScanResultsHeader results = new ScanResultsHeader();
Scanner scanner = new Scanner();
results = scanner.scan("scannerfake#gmail.com", "123", in);
and
public ScanResultsHeader scan (String userEmail,
String imapRetrievalId,
BufferedInputStream mimeEmail)
throws IOException, FileNotFoundException, MimeException, ScannerException {
//how fast would it be to just slurp up stream?
debug("slurp!");
String slurp = IOUtils.toString(mimeEmail);
debug("slurped " + slurp.length() + " characters");
slurp = slurp.toLowerCase();
debug("lc'ed it");
//...
My guess is I'm juggling the input streams wrong. One catch is the "BufferedInputStream mimeEmail" signature is required by the library API scan is using, so I'll need to get to that form eventually. But I noticed the simple act of slurping up a string takes ludicrously long so I'm already doing something incorrect.
Revising my answer....
If you are reading efficiently, and it appears you are, it will only be taking a lot time because either
You are creating a new connection every time you send a message which can be very expensive.
You are not sending the data as fast as you think.
The message is very large (unlikely but it could be)
There are plenty of examples on how to do this and a good library you can use is IOUtils which makes it simpler.
You should be able to send about 200K/s messages over a single socket in Java.
If you have a sends X bytes protocol using Big Endian you can do this.
DataInputStream dis = new DataInputStream( ...
int len = dis.readInt();
byte[] bytes = new byte[len];
dis.readFully(bytes);
String text = new String(bytes, "UTF-8");
Original problem was that the client isn't sending an end-of-input so the "slurp" operation keeps waiting for more stuff to cross the connection.
Solution was to implement an application-layer protocol to send the size of the message in advance, then stop listening for more message after that many bytes. I would have preferred a standard library -- something like, FiniteInputStream extends BufferedInputStream and takes a size as an argument, but wrote my own.
I set up a server with a ServerSocket, connect to it with a client machine. They're directly networked through a switch and the ping time is <1ms.
Now, I try to push a "lot" of data from the client to the server through the socket's output stream. It takes 23 minutes to transfer 0.6Gb. I can push a much larger file in seconds via scp.
Any idea what I might be doing wrong? I'm basically just looping and calling writeInt on the socket. The speed issue doesn't matter where the data is coming from, even if I'm just sending a constant integer and not reading from disk.
I tried setting the send and receive buffer on both sides to 4Mb, no dice. I use a buffered stream for the reader and writer, no dice.
Am I missing something?
EDIT: code
Here's where I make the socket
System.out.println("Connecting to " + hostname);
serverAddr = InetAddress.getByName(hostname);
// connect and wait for port assignment
Socket initialSock = new Socket();
initialSock.connect(new InetSocketAddress(serverAddr, LDAMaster.LDA_MASTER_PORT));
int newPort = LDAHelper.readConnectionForwardPacket(new DataInputStream(initialSock.getInputStream()));
initialSock.close();
initialSock = null;
System.out.println("Forwarded to " + newPort);
// got my new port, connect to it
sock = new Socket();
sock.setReceiveBufferSize(RECEIVE_BUFFER_SIZE);
sock.setSendBufferSize(SEND_BUFFER_SIZE);
sock.connect(new InetSocketAddress(serverAddr, newPort));
System.out.println("Connected to " + hostname + ":" + newPort + " with buffers snd=" + sock.getSendBufferSize() + " rcv=" + sock.getReceiveBufferSize());
// get the MD5s
try {
byte[] dataMd5 = LDAHelper.md5File(dataFile),
indexMd5 = LDAHelper.md5File(indexFile);
long freeSpace = 90210; // ** TODO: actually set this **
output = new DataOutputStream(new BufferedOutputStream(sock.getOutputStream()));
input = new DataInputStream(new BufferedInputStream(sock.getInputStream()));
Here's where I do the server-side connection:
ServerSocket servSock = new ServerSocket();
servSock.setSoTimeout(SO_TIMEOUT);
servSock.setReuseAddress(true);
servSock.bind(new InetSocketAddress(LDA_MASTER_PORT));
int currPort = LDA_START_PORT;
while (true) {
try {
Socket conn = servSock.accept();
System.out.println("Got a connection. Sending them to port " + currPort);
clients.add(new MasterClientCommunicator(this, currPort));
clients.get(clients.size()-1).start();
Thread.sleep(500);
LDAHelper.sendConnectionForwardPacket(new DataOutputStream(conn.getOutputStream()), currPort);
currPort++;
} catch (SocketTimeoutException e) {
System.out.println("Done listening. Dispatching instructions.");
break;
}
catch (IOException e) {
e.printStackTrace();
}
catch (Exception e) {
e.printStackTrace();
}
}
Alright, here's where I'm shipping over ~0.6Gb of data.
public static void sendTermDeltaPacket(DataOutputStream out, TIntIntHashMap[] termDelta) throws IOException {
long bytesTransferred = 0, numZeros = 0;
long start = System.currentTimeMillis();
out.write(PACKET_TERM_DELTA); // header
out.flush();
for (int z=0; z < termDelta.length; z++) {
out.writeInt(termDelta[z].size()); // # of elements for each term
bytesTransferred += 4;
}
for (int z=0; z < termDelta.length; z++) {
for (int i=0; i < termDelta[z].size(); i++) {
out.writeInt(1);
out.writeInt(1);
}
}
It seems pretty straightforward so far...
You do not want to write single bytes when you are transferring large amounts of data.
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;
public class Transfer {
public static void main(String[] args) {
final String largeFile = "/home/dr/test.dat"; // REPLACE
final int BUFFER_SIZE = 65536;
new Thread(new Runnable() {
public void run() {
try {
ServerSocket serverSocket = new ServerSocket(12345);
Socket clientSocket = serverSocket.accept();
long startTime = System.currentTimeMillis();
byte[] buffer = new byte[BUFFER_SIZE];
int read;
int totalRead = 0;
InputStream clientInputStream = clientSocket.getInputStream();
while ((read = clientInputStream.read(buffer)) != -1) {
totalRead += read;
}
long endTime = System.currentTimeMillis();
System.out.println(totalRead + " bytes read in " + (endTime - startTime) + " ms.");
} catch (IOException e) {
}
}
}).start();
new Thread(new Runnable() {
public void run() {
try {
Thread.sleep(1000);
Socket socket = new Socket("localhost", 12345);
FileInputStream fileInputStream = new FileInputStream(largeFile);
OutputStream socketOutputStream = socket.getOutputStream();
long startTime = System.currentTimeMillis();
byte[] buffer = new byte[BUFFER_SIZE];
int read;
int readTotal = 0;
while ((read = fileInputStream.read(buffer)) != -1) {
socketOutputStream.write(buffer, 0, read);
readTotal += read;
}
socketOutputStream.close();
fileInputStream.close();
socket.close();
long endTime = System.currentTimeMillis();
System.out.println(readTotal + " bytes written in " + (endTime - startTime) + " ms.");
} catch (Exception e) {
}
}
}).start();
}
}
This copies 1 GiB of data in short over 19 seconds on my machine. The key here is using the InputStream.read and OutputStream.write methods that accept a byte array as parameter. The size of the buffer is not really important, it just should be a bit larger than, say, 5. Experiment with BUFFER_SIZE above to see how it effects the speed but also keep in mind that it probably is different for every machine you are running this program on. 64 KiB seem to be a good compromise.
Hey, I figured I'd follow up for anyone that was interested.
Here's the bizarre moral of the story:
NEVER USE DataInputStream/DataOutputStream and sockets!!
If I wrap the socket in a BufferedOutputStream/BufferedInputStream, life is great. Writing to it raw is just fine.
But wrap the socket in a DataInputStream/DataOutputStream, or even have DataOutputStream(BufferedOutputStream(sock.getOutputStream())) is EXTREMELY SLOW.
An explanation for that would be really interesting to me. But after swapping everything in and out, this is what's up. Try it yourself if you don't believe me.
Thanks for all the quick help, though.
Maybe you should try sending ur data in chunks(frames) instead of writing each byte seperately. And align your frames with the TCP packet size for best performance.
Can you try doing this over loopback, it should then transfer the data in second.
If it takes minutes, there is something wrong with your application. If is only slow sending data over the internet it could be you network link which is slow.
My guess is that you have a 10 Mb/s network between your client and your server and this is why your transfer is going slowly. If this is the case, try using a DeflatoutOutputStream and an InflatorInputStream for your connection.
How are you implementing the receiving end? Please post your receiving code as well.
Since TCP is a reliable protocol, it will take steps to make sure the client is able to receive all of the data sent by the sender. This means that if your client cannot get the data out of the data receive buffer in time, then the sending side will simply stop sending more data until the client has a chance to read all the bytes in the receiving buffer.
If your receiving side is reading data one byte at a time, then your sender probably will spend a lot of time waiting for the receiving buffer to clear, hence the long transfer times. I'll suggest changing your receiving code to reading as many bytes as possible in each read operation . See if that will solve your problem.
Since I cannot yet comment on this site, I must write answer to #Erik here.
The problem is that DataOutputStream doesn't buffer. The whole Stream-thing in Java is based on decorators design pattern. So you could write
DataOutputStream out = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream()));
It will wrap the original stream in a BufferedOutputStream which is more efficient, which is then wrapped into a DataOutputStream which offers additional nice features like writeInt(), writeLong() and so on.
#Erik: using DataXxxputStream is not the problem here. Problem is you were sending data in too small chunks. Using a buffer solved your problem because even you would write bit by bit the buffer would solve the problem.
Bombe's solution is much nicer, generic and faster.
You should download a good packet sniffer. I'm a huge fan of WireShark personally and I end up using it every time I do some socket programming. Just keep in mind you've got to have the client and server running on different systems in order to pick up any packets.
Things to try:
Is the CPU at 100% while the data is being sent? If so, use visualvm and do a CPU profiling to see where the time is spent
Use a SocketChannel from java.nio - these are generally faster since they can use native IO more easily - of course this only helps if your operation is CPU bound
If it's not CPU bound, there's something going wrong at the network level. Use a packet sniffer to analyze this.
I was using PrintWriter to send data. I removed that and sent data with BufferedOutputStream.send(String.getBytes()) and got about 10x faster sending.
How is your heap size set? I had a similar problem recently with the socket transfer of large amounts of data and just by looking at JConsole I realized that the application was spending most of its time doing full GCs.
Try -Xmx1g
USe Byte buffer for sending the data