Develop Reference

faster wifi file transfer between android devices?

I am working with a android application in which I am wanting to share the file from one device to another through Wi-Fi. I am getting the speed of around 1.5 Mega Byte/s. Is there any way by which I can transfer the file by much higher data rate?
Can you please tell why we are getting this less data rate even the devices and router is capable of handling more than 150Mbps (18.75MBps) data rate...
Is it possible to use UFTP and will it be solving the purpose??
here is the code :
byte[] buf = new byte[2048];
try {
int bytesRead = 0;
while ((bytesRead = dis.read(buf, 0, buf.length)) != -1) {
fLength = fLength - bytesRead;
dos.write(buf, 0, bytesRead);
Log.i("File Tranfer Thread", String.valueOf(fLength) + Thread.currentThread().getName());
}
}
}
Thanks

Your code is fast.
There one thing you can try that worth the cost is playing with packet size. Try to modify the pack size in order to see the faster solution. Sometimes bigger packet is faster to send.
packet size bigger byte[] buf = new byte[2048*10];
packet size smaller byte[] buf = new byte[512];
packet size 3 byte[] buf = new byte[2048*5];

How can I make sure I received whole file through socket stream?

Ok, So I'm making a Java program that has a server and client and I'm sending a Zip file from server to client. I have sending the file down, almost. But recieving I've found some inconsistency. My code isn't always getting the full archive. I'm guessing it's terminating before the BufferedReader has the full thing. Here's the code for the client:
public void run(String[] args) {
try {
clientSocket = new Socket("jacob-custom-pc", 4444);
out = new PrintWriter(clientSocket.getOutputStream(), true);
in = new BufferedInputStream(clientSocket.getInputStream());
BufferedReader inRead = new BufferedReader(new InputStreamReader(in));
int size = 0;
while(true) {
if(in.available() > 0) {
byte[] array = new byte[in.available()];
in.read(array);
System.out.println(array.length);
System.out.println("recieved file!");
FileOutputStream fileOut = new FileOutputStream("out.zip");
fileOut.write(array);
fileOut.close();
break;
}
}
}
} catch(IOException e) {
e.printStackTrace();
System.exit(-1);
}
}
So how can I be sure the full archive is there before it writes the file?

On the sending side write the file size before you start writing the file. On the reading side Read the file size so you know how many bytes to expect. Then call read until you have gotten everything you expect. With network sockets it may take more than one call to read to get everything that was sent. This is especially true as your data gets larger.

HTTP sends a content-length: x+\n in bytes. This is elegant, it might throw a TimeoutException if the conn is broken.

You are using a TCP socket. The ZIP file is probably larger than the network MTU, so it will be split up into multiple packets and reassembled at the other side. Still, something like this might happen:
client connects
server starts sending. The ZIP file is bigger than the MTU and therefore split up into multiple packets.
client busy-waits in the while (true) until it gets the first packets.
client notices that data has arrived (in.available() > 0)
client reads all available data, writes it to the file and exits
the last packets arrive
So as you can see: Unless the client machine is crazily slow and the network is crazily fast and has a huge MTU, your code simply won't receive the entire file by design. That's how you built it.
A different approach: Prefix the data with the length.
Socket clientSocket = new Socket("jacob-custom-pc", 4444);
DataInputStream dataReader = new DataInputStream(clientSocket.getInputStream());
FileOutputStream out = new FileOutputStream("out.zip");
long size = dataReader.readLong();
long chunks = size / 1024;
int lastChunk = (int)(size - (chunks * 1024));
byte[] buf = new byte[1024];
for (long i = 0; i < chunks; i++) {
dataReader.read(buf);
out.write(buf);
}
dataReader.read(buf, 0, lastChunk);
out.write(buf, 0, lastChunk);
And the server uses DataOutputStream to send the size of the file before the actual file. I didn't test this, but it should work.

How can I make sure I received whole file through socket stream?
By fixing your code. You are using InputStream.available() as a test for end of stream. That's not what it's for. Change your copy loop to this, which is also a whole lot simpler:
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
Use with any buffer size greater than zero, typically 8192.

In.available() just tells you that there is no data to be consumed by in.read() without blocking (waiting) at the moment but it does not mean the end of stream. But, they may arrive into your PC at any time, with TCP/IP packet. Normally, you never use in.available(). In.read() suffices everything for the reading the stream entirely. The pattern for reading the input streams is
byte[] buf;
int size;
while ((size = in.read(buf)) != -1)
process(buf, size);
// end of stream has reached
This way you will read the stream entirely, until its end.
update If you want to read multiple files, then chunk you stream into "packets" and prefix every one with an integer size. You then read until size bytes is received instead of in.read = -1.
update2 Anyway, never use in.available for demarking between the chunks of data. If you do that, you imply that there is a time delay between incoming data pieces. You can do this only in the real-time systems. But Windows, Java and TCP/IP are all these layers incompatible with real-time.

damaged large file transfer

When I try to send a large file from server by splitting it, some of the packages don't arrive at the client... as you can see in the console output
http://s7.postimg.org/94yjfame3/error.png
the client receive only 19799.. bytes , and the server sent 62800.. bytes.
the code is too long to past here... but here are the basics:
// server side -> send data
BufferedOutputStream out = new BufferedOutputStream(socket.getOutputStream());
byte[] somePackageInfo= new byte[500];
byte[] streamOut = new byte[20000];
while(getDataFromLargeFile(somePackageInfo,streamOut) != 0) {
out.write(somePackageInfo,0,500);
out.write(streamOut);
out.flush();
}
out.write(0);
out.flush();
// client side -> get data
BufferedInputStream in = new BufferedInputStream(socket.getInputStream());
byte[] somePackageInfo= new byte[500];
byte[] streamIn= new byte[20000];
while(true) {
if(in.read(somePackageInfo,0,500) == 0) break;
in.read(streamIn);
saveDataToLargeFile(somePackageInfo,streamIn);
}
I tried to slow down the transfer (sleep(500)) but only most of the packages arrived.
tried to remove the flush() but still only most of the packages arrived.
what causes this problem and how can i fix it?

Your copy code is wrong. You are ignoring the count returned by read, and assuming that it fills the buffer. It isn't required to do that. See the Javadoc.
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
Use with any buffer size greater than zero, typically 8192. Use at both ends.
Adding sleeps is literally a waste of time.

Java -- How to read an unknown number of bytes from an inputStream (socket/socketServer)?

Looking to read in some bytes over a socket using an inputStream. The bytes sent by the server may be of variable quantity, and the client doesn't know in advance the length of the byte array. How may this be accomplished?
byte b[];
sock.getInputStream().read(b);
This causes a 'might not be initialized error' from the Net BzEAnSZ. Help.

You need to expand the buffer as needed, by reading in chunks of bytes, 1024 at a time as in this example code I wrote some time ago
byte[] resultBuff = new byte[0];
byte[] buff = new byte[1024];
int k = -1;
while((k = sock.getInputStream().read(buff, 0, buff.length)) > -1) {
byte[] tbuff = new byte[resultBuff.length + k]; // temp buffer size = bytes already read + bytes last read
System.arraycopy(resultBuff, 0, tbuff, 0, resultBuff.length); // copy previous bytes
System.arraycopy(buff, 0, tbuff, resultBuff.length, k); // copy current lot
resultBuff = tbuff; // call the temp buffer as your result buff
}
System.out.println(resultBuff.length + " bytes read.");
return resultBuff;

Assuming the sender closes the stream at the end of the data:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buf = new byte[4096];
while(true) {
int n = is.read(buf);
if( n < 0 ) break;
baos.write(buf,0,n);
}
byte data[] = baos.toByteArray();

Read an int, which is the size of the next segment of data being received. Create a buffer with that size, or use a roomy pre-existing buffer. Read into the buffer, making sure it is limited to the aforeread size. Rinse and repeat :)
If you really don't know the size in advance as you said, read into an expanding ByteArrayOutputStream as the other answers have mentioned. However, the size method really is the most reliable.

Without re-inventing the wheel, using Apache Commons:
IOUtils.toByteArray(inputStream);
For example, complete code with error handling:
public static byte[] readInputStreamToByteArray(InputStream inputStream) {
if (inputStream == null) {
// normally, the caller should check for null after getting the InputStream object from a resource
throw new FileProcessingException("Cannot read from InputStream that is NULL. The resource requested by the caller may not exist or was not looked up correctly.");
}
try {
return IOUtils.toByteArray(inputStream);
} catch (IOException e) {
throw new FileProcessingException("Error reading input stream.", e);
} finally {
closeStream(inputStream);
}
}
private static void closeStream(Closeable closeable) {
try {
if (closeable != null) {
closeable.close();
}
} catch (Exception e) {
throw new FileProcessingException("IO Error closing a stream.", e);
}
}
Where FileProcessingException is your app-specific meaningful RT exception that will travel uninterrupted to your proper handler w/o polluting the code in between.

The simple answer is:
byte b[] = new byte[BIG_ENOUGH];
int nosRead = sock.getInputStream().read(b);
where BIG_ENOUGH is big enough.
But in general there is a big problem with this. A single read call is not guaranteed to return all that the other end has written.
If the nosRead value is BIG_ENOUGH, your application has no way of knowing for sure if there are more bytes to come; the other end may have sent exactly BIG_ENOUGH bytes ... or more than BIG_ENOUGH bytes. In the former case, you application will block (for ever) if you try to read. In the latter case, your application has to do (at least) another read to get the rest of the data.
If the nosRead value is less than BIG_ENOUGH, your application still doesn't know. It might have received everything there is, part of the data may have been delayed (due to network packet fragmentation, network packet loss, network partition, etc), or the other end might have blocked or crashed part way through sending the data.
The best answer is that EITHER your application needs to know beforehand how many bytes to expect, OR the application protocol needs to somehow tell the application how many bytes to expect or when all bytes have been sent.
Possible approaches are:
the application protocol uses fixed message sizes (not applicable to your example)
the application protocol message sizes are specified in message headers
the application protocol uses end-of-message markers
the application protocol is not message based, and the other end closes the connection to say that is the end.
Without one of these strategies, your application is left to guess, and is liable to get it wrong occasionally.
Then you use multiple read calls and (maybe) multiple buffers.

Stream all Input data into Output stream. Here is working example:
InputStream inputStream = null;
byte[] tempStorage = new byte[1024];//try to read 1Kb at time
int bLength;
try{
ByteArrayOutputStream outputByteArrayStream = new ByteArrayOutputStream();
if (fileName.startsWith("http"))
inputStream = new URL(fileName).openStream();
else
inputStream = new FileInputStream(fileName);
while ((bLength = inputStream.read(tempStorage)) != -1) {
outputByteArrayStream.write(tempStorage, 0, bLength);
}
outputByteArrayStream.flush();
//Here is the byte array at the end
byte[] finalByteArray = outputByteArrayStream.toByteArray();
outputByteArrayStream.close();
inputStream.close();
}catch(Exception e){
e.printStackTrace();
if (inputStream != null) inputStream.close();
}

Either:
Have the sender close the socket after transferring the bytes. Then at the receiver just keep reading until EOS.
Have the sender prefix a length word as per Chris's suggestion, then read that many bytes.
Use a self-describing protocol such as XML, Serialization, ...

Use BufferedInputStream, and use the available() method which returns the size of bytes available for reading, and then construct a byte[] with that size. Problem solved. :)
BufferedInputStream buf = new BufferedInputStream(is);
int size = buf.available();

Here is a simpler example using ByteArrayOutputStream...
socketInputStream = socket.getInputStream();
int expectedDataLength = 128; //todo - set accordingly/experiment. Does not have to be precise value.
ByteArrayOutputStream baos = new ByteArrayOutputStream(expectedDataLength);
byte[] chunk = new byte[expectedDataLength];
int numBytesJustRead;
while((numBytesJustRead = socketInputStream.read(chunk)) != -1) {
baos.write(chunk, 0, numBytesJustRead);
}
return baos.toString("UTF-8");
However, if the server does not return a -1, you will need to detect the end of the data some other way - e.g., maybe the returned content always ends with a certain marker (e.g., ""), or you could possibly solve using socket.setSoTimeout(). (Mentioning this as it is seems to be a common problem.)

This is both a late answer and self-advertising, but anyone checking out this question may want to take a look here:
https://github.com/GregoryConrad/SmartSocket

This question is 7 years old, but i had a similiar problem, while making a NIO and OIO compatible system (Client and Server might be whatever they want, OIO or NIO).
This was quit the challenge, because of the blocking InputStreams.
I found a way, which makes it possible and i want to post it, to help people with similiar problems.
Reading a byte array of dynamic sice is done here with the DataInputStream, which kann be simply wrapped around the socketInputStream. Also, i do not want to introduce a specific communication protocoll (like first sending the size of bytes, that will be send), because i want to make this as vanilla as possible. First of, i have a simple utility Buffer class, which looks like this:
import java.util.ArrayList;
import java.util.List;
public class Buffer {
private byte[] core;
private int capacity;
public Buffer(int size){
this.capacity = size;
clear();
}
public List<Byte> list() {
final List<Byte> result = new ArrayList<>();
for(byte b : core) {
result.add(b);
}
return result;
}
public void reallocate(int capacity) {
this.capacity = capacity;
}
public void teardown() {
this.core = null;
}
public void clear() {
core = new byte[capacity];
}
public byte[] array() {
return core;
}
}
This class only exists, because of the dumb way, byte <=> Byte autoboxing in Java works with this List. This is not realy needed at all in this example, but i did not want to leave something out of this explanation.
Next up, the 2 simple, core methods. In those, a StringBuilder is used as a "callback". It will be filled with the result which has been read and the amount of bytes read will be returned. This might be done different of course.
private int readNext(StringBuilder stringBuilder, Buffer buffer) throws IOException {
// Attempt to read up to the buffers size
int read = in.read(buffer.array());
// If EOF is reached (-1 read)
// we disconnect, because the
// other end disconnected.
if(read == -1) {
disconnect();
return -1;
}
// Add the read byte[] as
// a String to the stringBuilder.
stringBuilder.append(new String(buffer.array()).trim());
buffer.clear();
return read;
}
private Optional<String> readBlocking() throws IOException {
final Buffer buffer = new Buffer(256);
final StringBuilder stringBuilder = new StringBuilder();
// This call blocks. Therefor
// if we continue past this point
// we WILL have some sort of
// result. This might be -1, which
// means, EOF (disconnect.)
if(readNext(stringBuilder, buffer) == -1) {
return Optional.empty();
}
while(in.available() > 0) {
buffer.reallocate(in.available());
if(readNext(stringBuilder, buffer) == -1) {
return Optional.empty();
}
}
buffer.teardown();
return Optional.of(stringBuilder.toString());
}
The first method readNext will fill the buffer, with byte[] from the DataInputStream and return the amount bytes read this way.
In the secon method, readBlocking, i utilized the blocking nature, not to worry about consumer-producer-problems. Simply readBlocking will block, untill a new byte-array is received. Before we call this blocking method, we allocate a Buffer-size. Note, i called reallocate after the first read (inside the while loop). This is not needed. You can safely delete this line and the code will still work. I did it, because of the uniqueness of my problem.
The 2 things, i did not explain in more detail are:
1. in (the DataInputStream and the only short varaible here, sorry for that)
2. disconnect (your disconnect routine)
All in all, you can now use it, this way:
// The in has to be an attribute, or an parameter to the readBlocking method
DataInputStream in = new DataInputStream(socket.getInputStream());
final Optional<String> rawDataOptional = readBlocking();
rawDataOptional.ifPresent(string -> threadPool.execute(() -> handle(string)));
This will provide you with a way of reading byte arrays of any shape or form over a socket (or any InputStream realy). Hope this helps!

Help with creating a Speex Voip server and client

Im trying to create a Speex Voip client and server. I have the basics down and its working OK on the local machine over UDP. I am using JSpeex for portability. Im looking for tips on creating the client and server. What are your thoughts?
The JSpeex library can only encode 320 bytes per call so the packets sent to the server are tiny (in my case ~244 bytes). Would it be better for the client to wait until about 1 or 2 KB of encoded data is ready before sending or let the server handle buffering the packets?
Also, any help on how to implement buffering the data would be nice.
Some of what I have that works on the local machine.
Client:
public void run() {
int nBytesToRead = (m_inputAudioFormat.getFrameSize() * 160);
int nAvailable = 0;
byte[] abPCMData = new byte[nBytesToRead];
byte[] abSpeexData = null;
UserSpeexPacket userSpeexPacket = new UserSpeexPacket("Xiphias3", "TheLounge", null, 0);
while (m_captureThread != null) {
nAvailable = m_line.available();
if (nAvailable >= nBytesToRead) {
int nBytesRead = m_line.read(abPCMData, 0, nBytesToRead);
if (nBytesRead == -1) break;
if (nBytesRead < nBytesToRead)
Arrays.fill(abPCMData, nBytesRead, abPCMData.length, (byte) 0);
abSpeexData = createSpeexPacketFromPCM(abPCMData, 0, abPCMData.length);
//DatagramPacket packet = new DatagramPacket(abSpeexData, 0, abSpeexData.length, m_connection.getInetAddress(), m_nServerPort);
userSpeexPacket.setSpeexData(abSpeexData);
userSpeexPacket.incrementPacketNumber();
DatagramPacket packet = UserSpeexPacket.userSpeexPacketToDatagramPacket(m_connection.getInetAddress(), m_connection.getPort(), userSpeexPacket);
try {
m_connection.send(packet);
}
catch(IOException iox) {
System.out.println("Connection to server lost: " + iox.getMessage());
break;
}
}
}
closeLine();
disconnect();
}
public byte[] createSpeexPacketFromPCM(byte[] abPCMData, int nOffset, int nLength)
{
byte[] abEncodedData = null;
m_speexEncoder.processData(abPCMData, nOffset, nLength);
abEncodedData = new byte[m_speexEncoder.getProcessedDataByteSize()];
m_speexEncoder.getProcessedData(abEncodedData, 0);
return abEncodedData;
}
Server:
DatagramPacket packet = new DatagramPacket(new byte[2048], 0, 2048);
byte[] abPCMData = null;
long lPrevVolPrintTime = 0;
while (m_bServerRunning) {
try {
m_serverSocket.receive(packet);
//System.out.println("Packet size is " + packet.getData().length);
//System.out.println("Got packet from " + packet.getAddress().getHostAddress());
//abPCMData = decodeSpeexPacket(packet.getData(), 0, packet.getLength());
UserSpeexPacket usp = UserSpeexPacket.datagramPacketToUserSpeexPacket(packet);
abPCMData = decodeSpeexPacket(usp.getSpeexData(), 0, usp.getSpeexData().length);
m_srcDataLine.write(abPCMData, 0, abPCMData.length);
if (System.currentTimeMillis() >= (lPrevVolPrintTime + 500)) {
//System.out.println("Current volume: " + AudioUtil.getVolumeLevelForPCM22050Hz16Bit1Channel(abPCMData, 0, abPCMData.length));
lPrevVolPrintTime = System.currentTimeMillis();
}
}
catch (IOException iox) {
if (m_bServerRunning) {
System.out.println("Server socket broke: " + iox.getMessage());
stopServer();
}
}
}

I am working on a similar project. From everything I've read, and personal experience, your best option is to work with small bits of data and send them as soon as you can. You want any jitter buffering to be done on the side of the receiver.
It is typical for a VoIP application to send 50-100 packets per second. For uLaw encoding at 8000Hz, this would result in a packet size of 80-160 bytes. The reasoning for this is that some packets will inevitably be dropped, and you want the impact to the receiver to be as small as possible. So with 10ms or 20ms of audio data per packet, a dropped packet may result in a small hiccup, but not nearly as bad as losing 2k of audio data (~250ms).
Additionally, with a large packet size, you must accumulate all of the data at the sender before sending it. So given a typical network latency of 50ms, with 20ms of audio data per packet, the receiver is not going to hear what the sender says for a minimum of 70ms. Now imagine what happens when 250ms of audio is being sent at once. 270ms will elapse between the sender speaking and the receiver playing that audio.
Users seem to be more forgiving of packet loss here and there, which results in sub-par audio quality, because the audio quality of most telephones isn't that great to begin with. However, users are also used to very low latency on modern telephone circuits, so introducing a round-trip delay of even 250ms can be extremely frustrating.
Now, as far as implementing buffering, I have found a good strategy to use a Queue (whoops, using .NET here :)), and then wrap that in a class that tracks the desired minimum and maximum number of packets in the Queue. Use rigorous locking since you will most likely be accessing it from multiple threads. If the Queue "bottoms out" and has zero packets in it (buffer underrun), set a flag and return null until the packet count reaches your desired minimum. Your consumer will have to check for null being returned and not queue anything into the output buffer, however. Alternatively your consumer could keep track of the last packet and repeatedly enqueue it, which may cause looping audio, but in some cases that may "sound" better than silence. You will have to do this until the producer puts enough packets into the queue to reach the minimum. This will result in a longer period of silence for the user, but that is generally better accepted than short, frequent periods of silence (choppiness). If you get a burst of packets and the producer fills up the queue (reaching the desired maximum), you can either start ignoring the new packets, or drop enough packets out of the front of the queue to return to the minimum.
Picking those min/max values is tough though. You are trying to balance smooth audio (no underruns) with minimum latency between sender and receiver. VoIP is fun but it sure can be frustrating! Good luck!