Develop Reference

Force Java Android Socket to Send Data Immediately

As a hobby project, I'm writing an android voip client. When writing voice data to the socket (Vars.mediaSocket), many times, the data isn't immediately sent out over the wifi but just stalls and then all at once it will send 20 seconds worth of voice. Then it will stall again and wait for 30 seconds and then send 30 seconds of voice. The wait is not consistent but after a while it will continuously send voice data immediately. I've tried everything from using DataOutputStream to setting the socket output buffer size, setting the sendbuffer size huge, small, and lastly, buffering the voice data from its 32 byte chunks to anything from 128bytes to 32kb.
Utils.logcat(Const.LOGD, encTag, "MediaCodec encoder thread has started");
isEncoding = true;
byte[] amrbuffer = new byte[32];
short[] wavbuffer = new short[160];
int outputCounter = 0;
//setup the wave audio recorder. since it is released and restarted, it needs to be setup here and not onCreate
wavRecorder = null; //remove pointer to the old recorder for safety
wavRecorder = new AudioRecord(MediaRecorder.AudioSource.MIC, SAMPLESWAV, AudioFormat.CHANNEL_IN_MONO, FORMAT, 160);
wavRecorder.startRecording();
AmrEncoder.init(0);
while(!micMute)
{
int totalRead = 0, dataRead;
while(totalRead < 160)
{//although unlikely to be necessary, buffer the mic input
dataRead = wavRecorder.read(wavbuffer, totalRead, 160 - totalRead);
totalRead = totalRead + dataRead;
}
int encodeLength = AmrEncoder.encode(AmrEncoder.Mode.MR122.ordinal(), wavbuffer, amrbuffer);
try
{
Vars.mediaSocket.getOutputStream().write(amrbuffer);
Vars.mediaSocket.getOutputStream().flush();
}
catch (IOException i)
{
Utils.logcat(Const.LOGE, encTag, "Cannot send amr out the media socket");
Utils.dumpException(tag, i);
}
Is there something I'm missing? To simulate a second cell phone, I have another client which just simply reads the voice data, throws it away, and reads again in a loop. I can confirm in the simulated second cell phone when the real cell phone stops sending voice, the simulated one's socket.read hangs until the real one starts sending voice again.
I'm really hoping not to have to write a jni for the socket as I don't know anything about that and was hoping I could write the app as a standard java app.
CASE CLOSED: turned out to be a server side bug but the simplifying back to basics suggestions is still a good idea.

You are adding most of the latency yourself by reading large amounts of data before writing any of it. You should just use the standard Java copy loop:
byte[] buffer = new byte[8192];
int count;
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
You need to adapt this to incorporate your codec step. Note that you don't need a buffer the size of the entire input. You can tune its size to suit yourself but 8192 is a good starting point. You can increase it to say 32k but don't decrease it. If your codec needs the data in fixed-size chunks, use a buffer of that size and DataInputStream.readFully(). But the larger the buffer the more the latency.
EDIT Specific issues with your code:
byte[] amrbuffer = new byte[AMRBUFFERSIZE];
byte[] outputbuffer = new byte [outputBufferSize];
Remove (see below).
short[] wavbuffer = new short[WAVBUFFERSIZE];
int outputCounter = 0;
Remove outputCounter.
//setup the wave audio recorder. since it is released and restarted, it needs to be setup here and not onCreate
wavRecorder = null; //remove pointer to the old recorder for safety
Pointless. Remove.
wavRecorder = new AudioRecord(MediaRecorder.AudioSource.MIC, SAMPLESWAV, AudioFormat.CHANNEL_IN_MONO, FORMAT, WAVBUFFERSIZE);
wavRecorder.startRecording();
AmrEncoder.init(0);
OK.
try
{
Vars.mediaSocket.setSendBufferSize(outputBufferSize);
}
catch (SocketException e)
{
e.printStackTrace();
}
Pointless. Remove. The socket send buffer should be as large as possible. Unless you know that its default size is < outputBufferSize there is no benefit to this. In any case we are getting rid of outputBuffer altogether.
while(!micMute)
{
int totalRead = 0, dataRead;
while(totalRead < WAVBUFFERSIZE)
{//although unlikely to be necessary, buffer the mic input
dataRead = wavRecorder.read(wavbuffer, totalRead, WAVBUFFERSIZE - totalRead);
totalRead = totalRead + dataRead;
}
int encodeLength = AmrEncoder.encode(AmrEncoder.Mode.MR122.ordinal(), wavbuffer, amrbuffer);
OK.
if(outputCounter == outputBufferSize)
{
Utils.logcat(Const.LOGD, encTag, "Sending output buffer");
try
{
Vars.mediaSocket.getOutputStream().write(outputbuffer);
Vars.mediaSocket.getOutputStream().flush();
}
catch (IOException i)
{
Utils.logcat(Const.LOGE, encTag, "Cannot send amr out the media socket");
Utils.dumpException(tag, i);
}
outputCounter = 0;
}
System.arraycopy(amrbuffer, 0, outputbuffer, outputCounter, encodeLength);
outputCounter = outputCounter + encodeLength;
Utils.logcat(Const.LOGD, encTag, "Output buffer fill: " + outputCounter);
Remove all the above and substitute
Vars.mediaSocket.getOutputStream().write(amrbuffer, 0, encodeLength);
This also means you can get rid of 'outputBuffer' as promised.
NB Don't flush inside loops. As a matter of fact flushing a socket output stream does nothing, but the general principle still holds.

faster wifi file transfer between android devices?

I am working with a android application in which I am wanting to share the file from one device to another through Wi-Fi. I am getting the speed of around 1.5 Mega Byte/s. Is there any way by which I can transfer the file by much higher data rate?
Can you please tell why we are getting this less data rate even the devices and router is capable of handling more than 150Mbps (18.75MBps) data rate...
Is it possible to use UFTP and will it be solving the purpose??
here is the code :
byte[] buf = new byte[2048];
try {
int bytesRead = 0;
while ((bytesRead = dis.read(buf, 0, buf.length)) != -1) {
fLength = fLength - bytesRead;
dos.write(buf, 0, bytesRead);
Log.i("File Tranfer Thread", String.valueOf(fLength) + Thread.currentThread().getName());
}
}
}
Thanks

Your code is fast.
There one thing you can try that worth the cost is playing with packet size. Try to modify the pack size in order to see the faster solution. Sometimes bigger packet is faster to send.
packet size bigger byte[] buf = new byte[2048*10];
packet size smaller byte[] buf = new byte[512];
packet size 3 byte[] buf = new byte[2048*5];

Best way to download remote images without third party libraries?

Welcome
I need to download sycnronously (one at time) a lot of small remote images (between 50kb and 100kb) from a server and to store them as PNG in the device. I need to achieve this without third party libraries and I'm using this code but it is too munch slow:
URL javaUrl = new URL(URLParser.parse(this.url));
URLConnection connection = javaUrl.openConnection();
InputStream input = new BufferedInputStream(javaUrl.openStream());
ByteArrayOutputStream output = new ByteArrayOutputStream();
byte data[] = new byte[1024];
long total = 0;
int count;
while ((count = input.read(data)) != -1) {
total += count;
output.write(data, 0, count);
}
// conversion to bitmap
InputStream in = new ByteArrayInputStream(output.toByteArray());
Bitmap original = BitmapFactory.decodeStream(in);
// storing bitmap as PNG file
FileOutputStream out = new FileOutputStream(filename);
original.compress(Bitmap.CompressFormat.PNG, 90, out);
output.flush();
output.close();
input.close();
in.close();
original.recycle();
The problem is that the download is very slow. With very fast WIFI internet in the device (13MB, download speed of 1.4mbytes/s), it is taking 3-4 seconds to download the image in the device, but only 100-200ms to download the image in my PC using google chrome for example.
It is something wrong in my download algorithm? can be improved?
Thanks

You have a totally unnecessary byte array in the middle.
BitmapFactory.decodeStream() accepts an InputStream and you get an InputStream from URL.openStream().
It might not give you the speed boost you're looking for, but it'll at least get rid of a completely useless step in your code.

api level 8 HttpURLConnection

this is my first post - so I hope did everything right concerning code-formatting.
Maybe you can help me with the following problem:
With API-Level 10 and above, all my HttpURLConnections are working fine, but with API-Level 8 some (or sometimes most) of my attempts to download the content of a file (txt, html…) fails.
I worked around this by just repeating the download whenever the server returned the response code “-1”. This is not very satisfying but it works (more or less) – but I guess it only works because of the fact that the content I receive is very short (max 100 chars) – and sometimes I think some chars at the end were omitted, too.
But now I tried to implement an update routine (downloading and installing an apk-File) which most of the times fails, because the apk file is fragmented (I guess), so I get a “Parsing error” when trying to install the file.
What am I doing wrong?
Thank you for any help (and staying friendly if you are better than me g)!
Here’s the code of my update routine (which – as you can see – has no repeating mechanism as described above):
String download_url = "http://url/";
String download_file = "name.apk";
//Local
String update_file = download_file;
File update_file_stream = context.getFileStreamPath(update_file);
//Connect
URL url = new URL(download_url + download_file);
HttpURLConnection c = (HttpURLConnection) url.openConnection();
c.setRequestMethod("GET");
c.setDoOutput(true);
c.connect();
//Delete local File?
if(update_file_stream.exists()){
update_file_stream.delete();
}
//Download File
FileOutputStream fos = context.openFileOutput(update_file, Context.MODE_WORLD_READABLE);
InputStream is = c.getInputStream();
byte[] buffer = new byte[4096];
int len1 = 0;
while ((len1 = is.read(buffer)) != -1) {
fos.write(buffer, 0, len1);
}
fos.close();
is.close();
//Install
Intent intent = new Intent(Intent.ACTION_VIEW);
intent.setDataAndType(Uri.fromFile(update_file_stream), "application/vnd.android.package-archive");
intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
context.startActivity(intent);

Look like Network on Main Thread issue.
In the previous android sdk version (maybe 8) you can do some network on Main Thread
but after ICS or Honeycomb you can't do any network stuff on Main Thread.

Java TCP socket: data transfer is slow

I set up a server with a ServerSocket, connect to it with a client machine. They're directly networked through a switch and the ping time is <1ms.
Now, I try to push a "lot" of data from the client to the server through the socket's output stream. It takes 23 minutes to transfer 0.6Gb. I can push a much larger file in seconds via scp.
Any idea what I might be doing wrong? I'm basically just looping and calling writeInt on the socket. The speed issue doesn't matter where the data is coming from, even if I'm just sending a constant integer and not reading from disk.
I tried setting the send and receive buffer on both sides to 4Mb, no dice. I use a buffered stream for the reader and writer, no dice.
Am I missing something?
EDIT: code
Here's where I make the socket
System.out.println("Connecting to " + hostname);
serverAddr = InetAddress.getByName(hostname);
// connect and wait for port assignment
Socket initialSock = new Socket();
initialSock.connect(new InetSocketAddress(serverAddr, LDAMaster.LDA_MASTER_PORT));
int newPort = LDAHelper.readConnectionForwardPacket(new DataInputStream(initialSock.getInputStream()));
initialSock.close();
initialSock = null;
System.out.println("Forwarded to " + newPort);
// got my new port, connect to it
sock = new Socket();
sock.setReceiveBufferSize(RECEIVE_BUFFER_SIZE);
sock.setSendBufferSize(SEND_BUFFER_SIZE);
sock.connect(new InetSocketAddress(serverAddr, newPort));
System.out.println("Connected to " + hostname + ":" + newPort + " with buffers snd=" + sock.getSendBufferSize() + " rcv=" + sock.getReceiveBufferSize());
// get the MD5s
try {
byte[] dataMd5 = LDAHelper.md5File(dataFile),
indexMd5 = LDAHelper.md5File(indexFile);
long freeSpace = 90210; // ** TODO: actually set this **
output = new DataOutputStream(new BufferedOutputStream(sock.getOutputStream()));
input = new DataInputStream(new BufferedInputStream(sock.getInputStream()));
Here's where I do the server-side connection:
ServerSocket servSock = new ServerSocket();
servSock.setSoTimeout(SO_TIMEOUT);
servSock.setReuseAddress(true);
servSock.bind(new InetSocketAddress(LDA_MASTER_PORT));
int currPort = LDA_START_PORT;
while (true) {
try {
Socket conn = servSock.accept();
System.out.println("Got a connection. Sending them to port " + currPort);
clients.add(new MasterClientCommunicator(this, currPort));
clients.get(clients.size()-1).start();
Thread.sleep(500);
LDAHelper.sendConnectionForwardPacket(new DataOutputStream(conn.getOutputStream()), currPort);
currPort++;
} catch (SocketTimeoutException e) {
System.out.println("Done listening. Dispatching instructions.");
break;
}
catch (IOException e) {
e.printStackTrace();
}
catch (Exception e) {
e.printStackTrace();
}
}
Alright, here's where I'm shipping over ~0.6Gb of data.
public static void sendTermDeltaPacket(DataOutputStream out, TIntIntHashMap[] termDelta) throws IOException {
long bytesTransferred = 0, numZeros = 0;
long start = System.currentTimeMillis();
out.write(PACKET_TERM_DELTA); // header
out.flush();
for (int z=0; z < termDelta.length; z++) {
out.writeInt(termDelta[z].size()); // # of elements for each term
bytesTransferred += 4;
}
for (int z=0; z < termDelta.length; z++) {
for (int i=0; i < termDelta[z].size(); i++) {
out.writeInt(1);
out.writeInt(1);
}
}
It seems pretty straightforward so far...

You do not want to write single bytes when you are transferring large amounts of data.
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;
public class Transfer {
public static void main(String[] args) {
final String largeFile = "/home/dr/test.dat"; // REPLACE
final int BUFFER_SIZE = 65536;
new Thread(new Runnable() {
public void run() {
try {
ServerSocket serverSocket = new ServerSocket(12345);
Socket clientSocket = serverSocket.accept();
long startTime = System.currentTimeMillis();
byte[] buffer = new byte[BUFFER_SIZE];
int read;
int totalRead = 0;
InputStream clientInputStream = clientSocket.getInputStream();
while ((read = clientInputStream.read(buffer)) != -1) {
totalRead += read;
}
long endTime = System.currentTimeMillis();
System.out.println(totalRead + " bytes read in " + (endTime - startTime) + " ms.");
} catch (IOException e) {
}
}
}).start();
new Thread(new Runnable() {
public void run() {
try {
Thread.sleep(1000);
Socket socket = new Socket("localhost", 12345);
FileInputStream fileInputStream = new FileInputStream(largeFile);
OutputStream socketOutputStream = socket.getOutputStream();
long startTime = System.currentTimeMillis();
byte[] buffer = new byte[BUFFER_SIZE];
int read;
int readTotal = 0;
while ((read = fileInputStream.read(buffer)) != -1) {
socketOutputStream.write(buffer, 0, read);
readTotal += read;
}
socketOutputStream.close();
fileInputStream.close();
socket.close();
long endTime = System.currentTimeMillis();
System.out.println(readTotal + " bytes written in " + (endTime - startTime) + " ms.");
} catch (Exception e) {
}
}
}).start();
}
}
This copies 1 GiB of data in short over 19 seconds on my machine. The key here is using the InputStream.read and OutputStream.write methods that accept a byte array as parameter. The size of the buffer is not really important, it just should be a bit larger than, say, 5. Experiment with BUFFER_SIZE above to see how it effects the speed but also keep in mind that it probably is different for every machine you are running this program on. 64 KiB seem to be a good compromise.

Hey, I figured I'd follow up for anyone that was interested.
Here's the bizarre moral of the story:
NEVER USE DataInputStream/DataOutputStream and sockets!!
If I wrap the socket in a BufferedOutputStream/BufferedInputStream, life is great. Writing to it raw is just fine.
But wrap the socket in a DataInputStream/DataOutputStream, or even have DataOutputStream(BufferedOutputStream(sock.getOutputStream())) is EXTREMELY SLOW.
An explanation for that would be really interesting to me. But after swapping everything in and out, this is what's up. Try it yourself if you don't believe me.
Thanks for all the quick help, though.

Maybe you should try sending ur data in chunks(frames) instead of writing each byte seperately. And align your frames with the TCP packet size for best performance.

Can you try doing this over loopback, it should then transfer the data in second.
If it takes minutes, there is something wrong with your application. If is only slow sending data over the internet it could be you network link which is slow.
My guess is that you have a 10 Mb/s network between your client and your server and this is why your transfer is going slowly. If this is the case, try using a DeflatoutOutputStream and an InflatorInputStream for your connection.

How are you implementing the receiving end? Please post your receiving code as well.
Since TCP is a reliable protocol, it will take steps to make sure the client is able to receive all of the data sent by the sender. This means that if your client cannot get the data out of the data receive buffer in time, then the sending side will simply stop sending more data until the client has a chance to read all the bytes in the receiving buffer.
If your receiving side is reading data one byte at a time, then your sender probably will spend a lot of time waiting for the receiving buffer to clear, hence the long transfer times. I'll suggest changing your receiving code to reading as many bytes as possible in each read operation . See if that will solve your problem.

Since I cannot yet comment on this site, I must write answer to #Erik here.
The problem is that DataOutputStream doesn't buffer. The whole Stream-thing in Java is based on decorators design pattern. So you could write
DataOutputStream out = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream()));
It will wrap the original stream in a BufferedOutputStream which is more efficient, which is then wrapped into a DataOutputStream which offers additional nice features like writeInt(), writeLong() and so on.

#Erik: using DataXxxputStream is not the problem here. Problem is you were sending data in too small chunks. Using a buffer solved your problem because even you would write bit by bit the buffer would solve the problem.
Bombe's solution is much nicer, generic and faster.

You should download a good packet sniffer. I'm a huge fan of WireShark personally and I end up using it every time I do some socket programming. Just keep in mind you've got to have the client and server running on different systems in order to pick up any packets.

Things to try:
Is the CPU at 100% while the data is being sent? If so, use visualvm and do a CPU profiling to see where the time is spent
Use a SocketChannel from java.nio - these are generally faster since they can use native IO more easily - of course this only helps if your operation is CPU bound
If it's not CPU bound, there's something going wrong at the network level. Use a packet sniffer to analyze this.

I was using PrintWriter to send data. I removed that and sent data with BufferedOutputStream.send(String.getBytes()) and got about 10x faster sending.

How is your heap size set? I had a similar problem recently with the socket transfer of large amounts of data and just by looking at JConsole I realized that the application was spending most of its time doing full GCs.
Try -Xmx1g

USe Byte buffer for sending the data