Develop Reference

copy bytes from one array to another without iteration in Java

is there a quicker way to copy bytes from one array to another without iteration ?
I'm reading bytes via Bluetooth from the input stream
public void run() {
byte[] buffer = new byte[100]; // buffer store for the stream
int bytes; // bytes returned from read()
// Keep listening to the InputStream until an exception occurs
while (true) {
try {
// Read from the InputStream
bytes = mmInStream.read(buffer); // Get number of bytes and message in "buffer"
h.obtainMessage(RECIEVE_MESSAGE, bytes, -1, buffer).sendToTarget(); // Send to message queue Handler
} catch (IOException e) {
break;
}
}
}
This sends a random number of bytes to the handler.
I then read these and put them into an array to then process the data when the download is complete. The data being sent from a PIC micro is 6055 bytes long.
h = new Handler() {
public void handleMessage(android.os.Message msg) {
switch (msg.what) {
case SUCCESS_CONNECT:
// Connected so start connected thread
btSocket = (BluetoothSocket) msg.obj;
byteCount = 0;
arrayCount = 0;
mConnectedThread = new FullDataActivity.ConnectedThread(btSocket);
mConnectedThread.start();
mConnectedThread.write(getFullDataConByte); // Send 255 to start
break;
case RECIEVE_MESSAGE: // if receive massage
byte[] readBuf = (byte[]) msg.obj;
// iterate through obj and copy bytes to fullDataChunk array
for (int a = 0; a < msg.arg1; a++) {
fullDataChunk[byteCount] = readBuf[a];
Log.d("readBuf[a] = ",Integer.toString(readBuf[a] & 0xFF));
byteCount++;
}
// if all bytes done process
if (byteCount == 6055) {// process data when complete.
My handler is missing bytes somewhere and corrupting the data when its copying them in the for loop. I don't know if the run method is sending new bytes to the handler before the rest are processed in the for loop.
I've done a Log of the bytes sent in the Run and they are right. It's when the handler is processing that they go wrong.
I either need to ensure the data sent is processed before new data is sent, or copy the data to the array quicker ?
Anyone any ideas.

The fastest way to copy an array should be by using System.arraycopy()
.

Did this in the run, then the handler receives full array without errors.
public void run() {
byte[] fullBuffer = new byte[6055];
byte[] buffer = new byte[100]; // buffer store for the stream
int bytes; // bytes returned from read()
int bytesCount = 0;
// Keep listening to the InputStream until an exception occurs
while (true) {
try {
// Read from the InputStream
bytes = mmInStream.read(buffer); // Get number of bytes and message in "buffer"
System.arraycopy(buffer,0,fullBuffer,bytesCount,bytes);
bytesCount = bytesCount + bytes;
Log.d("FD Read - ", Integer.toString(bytesCount));
if(bytesCount >= 6055){
h.obtainMessage(RECIEVE_MESSAGE, bytesCount, -1, fullBuffer).sendToTarget(); // Send to message queue Handler
Log.d("FD Read - ", "Message sent");
bytesCount = 0;
Log.d("FD Read - ", "bytesCount re-set");
}
//h.obtainMessage(RECIEVE_MESSAGE, bytes, -1, buffer).sendToTarget(); // Send to message queue Handler
} catch (IOException e) {
break;
}
}
}

Java file not working when sent over a network

So I was implementing client and socket for java. I wanted to send huge files on tcp through sockets and I was able to send files too but the only problem was the files on the other end were either not complete or not working. I have checked the bits are being transfered then what is the error.
Client side:
Socket sock = new Socket("127.0.0.1", 1056);
byte[] mybytearray = new byte[1024];
InputStream is = sock.getInputStream();
FileOutputStream fos = new FileOutputStream("abc.mp3");
BufferedOutputStream bos = new BufferedOutputStream(fos);
int bytesRead = is.read(mybytearray, 0, mybytearray.length);
int len = 0;
while((len = is.read(mybytearray)) != -1)
{
bos.write(mybytearray, 0, len);
System.out.println("sending");
}
bos.close();
sock.close();
Server side:
ServerSocket ss = new ServerSocket(1056);
while (true) {
Socket s = ss.accept();
PrintStream out = new PrintStream(s.getOutputStream());
BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()));
String info = null;
String request = null;
System.out.println("sending");
String filename = "abc.mp3";
File fi = new File(filename);
InputStream fs = new FileInputStream(fi);
int n = fs.available();
byte buf[] = new byte[1024];
out.println("Content_Length:" + n);
out.println("");
while ((n = fs.read(buf)) >= 0) {
out.write(buf, 0, n);
System.out.println("sending");
}
out.close();
s.close();
in.close();
}

When you are connected via TCP you create a network stream which you can read and write in, similar to all other streams you worked with. Writing a large amount of data to the stream is not a good idea, so I suggest you break the selected file into smaller packets in which each packet length is 1024 bytes (1KB) and then send all the packets to the server. The SendTCP function is as follows:(I have used Windows Forms to make things more obvious)
public void SendTCP(string M, string IPA, Int32 PortN)
{
byte[] SendingBuffer = null
TcpClient client = null;
lblStatus.Text = "";
NetworkStream netstream = null;
try
{
client = new TcpClient(IPA, PortN);
lblStatus.Text = "Connected to the Server...\n";
netstream = client.GetStream();
FileStream Fs = new FileStream(M, FileMode.Open, FileAccess.Read);
int NoOfPackets = Convert.ToInt32
(Math.Ceiling(Convert.ToDouble(Fs.Length) / Convert.ToDouble(BufferSize)));
progressBar1.Maximum = NoOfPackets;
int TotalLength = (int)Fs.Length, CurrentPacketLength, counter = 0;
for (int i = 0; i < NoOfPackets; i++)
{
if (TotalLength > BufferSize)
{
CurrentPacketLength = BufferSize;
TotalLength = TotalLength - CurrentPacketLength;
}
else
CurrentPacketLength = TotalLength;
SendingBuffer = new byte[CurrentPacketLength];
Fs.Read(SendingBuffer, 0, CurrentPacketLength);
netstream.Write(SendingBuffer, 0, (int)SendingBuffer.Length);
if (progressBar1.Value >= progressBar1.Maximum)
progressBar1.Value = progressBar1.Minimum;
progressBar1.PerformStep();
}
lblStatus.Text=lblStatus.Text+"Sent "+Fs.Length.ToString()+"
bytes to the server";
Fs.Close();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
netstream.Close();
client.Close();
}
}
As you can see, a TCP client and a network stream are being constructed and a network connection is initiated. After opening the selected file according to the buffer size which is 1024 bytes, the number of packets that are going to be sent is calculated. There are two other variables CurrentPacketLength and TotalLength. If the total length of the selected file is more than the buffer size the CurrentPacketLength is set to the buffer size, otherwise why send some empty bytes, so CurrentPacketLength is set to the total length of the file. After that, I subtract the current from the total length, so actually we can say total length is showing the total amount of data that has not been sent yet. The rest is pretty much straight forward, reading the data from the file stream and writing it to the SendingBuffer according to the CurrentPacketLength and writing the buffer to the network stream.
At the server side, the application is listening for an incoming connection:
public void ReceiveTCP(int portN)
{
TcpListener Listener = null;
try
{
Listener = new TcpListener(IPAddress.Any, portN);
Listener.Start();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
byte[] RecData = new byte[BufferSize];
int RecBytes;
for (; ; )
{
TcpClient client = null;
NetworkStream netstream = null;
Status = string.Empty;
try
{
string message = "Accept the Incoming File ";
string caption = "Incoming Connection";
MessageBoxButtons buttons = MessageBoxButtons.YesNo;
DialogResult result;
if (Listener.Pending())
{
client = Listener.AcceptTcpClient();
netstream = client.GetStream();
Status = "Connected to a client\n";
result = MessageBox.Show(message, caption, buttons);
if (result == System.Windows.Forms.DialogResult.Yes)
{
string SaveFileName=string.Empty;
SaveFileDialog DialogSave = new SaveFileDialog();
DialogSave.Filter = "All files (*.*)|*.*";
DialogSave.RestoreDirectory = true;
DialogSave.Title = "Where do you want to save the file?";
DialogSave.InitialDirectory = #"C:/";
if (DialogSave.ShowDialog() == DialogResult.OK)
SaveFileName = DialogSave.FileName;
if (SaveFileName != string.Empty)
{
int totalrecbytes = 0;
FileStream Fs = new FileStream
(SaveFileName, FileMode.OpenOrCreate, FileAccess.Write);
while ((RecBytes = netstream.Read
(RecData, 0, RecData.Length)) > 0)
{
Fs.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
Fs.Close();
}
netstream.Close();
client.Close();
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
//netstream.Close();
}
}
}
A TCP listener is created and starts listening to the specified port. Again the buffer size is set to 1024 bytes. A TCP listener can pre check to see if there are any connections pending before calling the AcceptTcpClient method. It returns true if there are any pending connections. This method is a good way of avoiding the socket being blocked. Before reading anything from the network stream, a message box asks you if you want to accept the incoming connection, then a SaveFileDialog will be opened, and when you enter the file name plus extension, a file stream will be constructed and you start reading from the network stream and writing to the file stream. Create a thread in your code and run the receiving method in the created thread. I have sent more than 100 MB files in a LAN with the application.
For more details, check this article.

So, first you do this
int bytesRead = is.read(mybytearray, 0, mybytearray.length);
That reads up to 1024 bytes into mybytearray.
You don't do anything with that and I don't understand why you are doing it. You never write those bytes so they get overwritten if the while loop reads anything.
Just delete that. The while loop should cover all of this.

send large files over socket java

I am trying to transfer larger files over socket.I will be transferring the file in chunks.As shown in the code.link
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
before sending this file,i want to send an object which holds the details of the file.Using which stream should i send Object + File.
I am new to streams,can i get any sample code.
Can i send the byte length of the object first to read the object ,save it and send file data.is it possible,any sample code ?
Thanks
421

I'm not sure whether using Java's serialization mechanism is the best way to do a simple file transfer. As your question suggest, you try to avoid keeping the whole file in memory at any time. This can be done with objects using the Proxy pattern but if all you want to do is transfer the file, this might not be the most straight-forward solution. (Also, it will effectively tie your peer to be implemented in Java too.)
Instead, why not take a look at an extremely successful protocol that does exactly what you need: HTTP.
Content-Type: application/octet-stream
Content-Length: 542183
542183 bytes of data follow...
It should not be too hard for you to write a parser for the meta-data header.

You need to ensure the order of writing/reading. If write an object -> write raw bytes on client, then read an object -> read raw bytes on server. When reading, ObjectInputStream should be able to find the boundary of the serialized object data.
If you want to keep a socket connection long-live and use its streams multiple times, wrapping socket's Output/InputStream in a ObjectOutput/InputStream is not a good idea IMO. When you close a object stream, it closes the underlying stream as well.
So you may want to write the length of serialized object data first (file length is contained in the object so you don't need to write it explictly), e.g. 4 bytes of BigEndian encoded int. Then serialize the object into a ByteArrayOutputStream, and write the bytes in its buffer. On the server, read 4 bytes first, decode the bytes back to an int, and read that many bytes into a byte[], wrap the byte array with a ByteArrayInputStream and deserialize the object from it.
Write like this:
......
OutputStream out = socket.getOutputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(fileInfoObject);
oos.close();
byte[] header = encodeInteger(baos.size());
out.write(header, 0, 4);
baos.writeTo(out);
// write the file to out just as your question shows
On the receiving side:
......
InputStream in = socket.getInputStream();
// read the int
byte[] header = new byte[4];
in.read(header, 0, 4);
int size = decodeInteger(header);
// read the object
byte[] objectbuf = new byte[size];
int count;
while((count += in.read(objectbuf)) < size); // not sure if this works...
ObjectInputStram ois = new ObjectInputStream(new ByteArrayInputStream(objectbuf));
Object fileInfoObject = ois.readObject();
ois.close();
// read the file
FileOutputStream fos = new FileOutputStream(new File("somefile"));
byte[] buffer = new byte[8192];
count = 0;
long left = castedFileInfoObject.fileSize;
// also not sure if this works, not tested.
int maxRead = buffer.length;
while (true) {
count = in.read(buffer, 0, maxRead);
left -= count;
if (left < 8192) {
maxRead = (int)left;
}
fos.write(buffer, 0, count);
if (left == 0) {
break;
}
}
I haven't tested the sample code in my answer.. just to show the idea.

Class MyClass should implement the Serializable interface. Then, an object of this class can be written to an ObjectOutputStream and read back from ObjectInputStream using writeObject and readObject methods (See below).
On Client:
Socket socket = new Socket(url, port);
OutputStream os = socket.getOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(os);
MyClass obj = new Myclass();
oos.writeObject(obj);
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0) {
out.write(buffer, 0, count);
}
On server:
ServerSocket sSocket = new ServerSocket(port);
Socket socket = sSocket.accept();
InputStream is = socket.getInputStream();
ObjectInputStream ois = new ObjectInputStream(is);
MyClass obj = (MyClass)ois.readObject();
byte arr[];
try {
while(arr = (byte[])ois.readObject()) {
//do something with arr
}
} catch(java.io.EOFException) {
// End of data
}
If you need to send more data after the file is finished, you need a way to figure out the number of bytes the file consists of. Then, you can send the number of bytes beforehand over the socket to the server. On the server, read only that many bytes of information for the file and then do the same for the rest of the data you are going to send. This strategy of pre-sending the file size is recommended and is mostly used while doing any data transfer. If you can do that, you don't have to rely on catching java.io.EOFException to detect end of data.

Send file length with outputstream and receive length and byte[] with inputstream for streaming frames from one device to the other Android/Java

I have searched and searched and everything I have found has been helpful but I keep getting an out of memory error. The images I send are .06 MB so I know the problem isn't from decoding the byte[] into a bitmap. When I remove the while loops this works like a charm for one frame but I want multiple frames. I am getting a byte[] and sending it to a different device using sockets but I am at a loss how to do this. My problem is that I don't send and receive the correct byte[] length. This is what i am doing currently.
while (count != -1) {
//first send the byte[] length
dataOutputStream.writeInt(sendPackage.length);
//pass a byte array
publishProgress("sending file to client");
showMyToastOnUiThread(String.valueOf(sendPackage.length));
outputStream.write(sendPackage, 0, sendPackage.length);
outputStream.flush();
}
Receive byte[] on different device:
int count = inputStream.read();
while (count != -1) {
int byteArrayLength = dataInputStream.readInt();
Log.i(MainActivity.TAG, "Starting convert to byte array");
byte[] receivedBytes = convertInputStreamToByteArray(inputStream, byteArrayLength);
Bitmap bitmap = BitmapFactory.decodeByteArray(receivedBytes, 0, receivedBytes.length);
publishProgress(bitmap);
}
//convert inputstream to byte[]
public byte[] convertInputStreamToByteArray(InputStream inputStream, int readLength) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] data = new byte[readLength];
try {
Log.i(MainActivity.TAG, "Starting convert to byte array while loop");
int readTotal = 0;
int count = 0;
while (count >= 0 && readTotal < readLength) {
count = inputStream.read(data, readTotal, readLength - readTotal);
if (readLength > 0) {
readTotal += count;
}
}
Log.i(MainActivity.TAG, "Finished convert to byte array while loop");
} catch (IOException e) {
Log.e(MainActivity.TAG, "error: " + e.getMessage());
e.printStackTrace();
}
return data;
}

This is the problem:
int count = inputStream.read();
while (count != -1) {
You're consuming a byte and then ignoring it. That means the next value you read (the size) will be incorrect. You need a different way of telling whether you're at the end of the stream. Some options:
Send a -1 when you're finished; that way you can stop as soon as readInt returns -1
If you know it, send the number of images you're going to send before you start sending them
Use mark(1), then read(), then reset() - if your stream supports marking. I don't know whether it will or not. You could always wrap it in BufferedInputStream if not.
Reimplement DataInputStream.readInt yourself in a way which detects the end of the stream as being an expected possibility instead of throwing an exception
Just catch an exception in readInt (not nice - getting to the end of the stream isn't really exceptional)

Transfer a byte[] via TCP Socket in android

I need to transfer a byte[] via TCP Socket in android -client side.
i tried to use the following code :
public static int readBytes(byte[] myByteArray) throws IOException {
InputStream in = socket.getInputStream();
DataInputStream dis = new DataInputStream(in);
int len = dis.readInt(); //<-here i get the error
byte[] data = new byte[len];
if (len > 0) {
dis.readFully(data,0,len);
}
myByteArray=data;
return data.length;
}
but I get the following error:
12-23 17:30:49.814: E/AndroidRuntime(11717):
java.lang.OutOfMemoryError: array size too large(Heap Size=5699KB,
Allocated=3403KB, Bitmap Size=78KB)

You probably get the OutOfMemoryError on the line below. (The line that allocates new memory using new.)
Print len before doing the allocation and you'll probably see that it's a huge number (a lot larger than what you expect).
Log.d("YOU_TAG", "len = " + len);
Presumably you don't send the size of the byte array on the sending side. (Or if you do, your reading / writing has come out of sync.)

you must read in a closed loop like this:
while(true){
int len = dis.readInt();
}