Develop Reference

How to read bytes on receiver in socket stream (java)

I have set up a socket connection between a server and i client. Right now i'm trying to send data from my client to the server. Actually the data is a byte array which contain numbers in the index 14 to 27. An example of the array is here:
{27, 14, 16, 18, 20, 22, 14, 17, 15, 17} and so on.
Have made it as an byte array because the data have to be in bytes.
The difficulty is that when i sent a line from the array to the server, i don't know how to read it other than a string. And if it is a string, it return some weird numbers like the one you see in the picture.
Some code how i do it:
Sender
for (int i = 0; i < data.length; i++) {
writer.write(data[i]);
}
writer.flush();
writer.close();
Receiver
public void readResponse(Socket client) throws IOException{
String userInput;
BufferedReader stdIn = new BufferedReader(new InputStreamReader(client.getInputStream()));
System.out.println("Response from client:");
while ((userInput = stdIn.readLine()) != null) {
System.out.println(userInput);
}
}
My byte array is made like this:
private byte data[] = new byte[12];
if i change it to Byte with uppercase, i can read it with my code, but i'm not sure if it in bytes then? Have to use some math to calculate an average.
private Byte data[] = new Byte[12];
So, how do i read it?
Update:
So i understand that i'm going to use a different input/output stream. Right now i have changed it too a Datainput and output stream.
Code looks like this:
Server
public void readResponse(Socket client) throws IOException{
DataInputStream input = null;
byte data;
try {
input = new DataInputStream(client.getInputStream());
}
catch (IOException e) {
System.out.println(e);
}
System.out.println("Response from client:");
while ((data = input.readByte()) != -1) {
System.out.println(data);
}
}
Client
public void sentData(Socket client) throws IOException{
DataOutputStream output = null;
try {
output = new DataOutputStream(client.getOutputStream());
}
catch (IOException e) {
System.out.println(e);
}
for (int i = 0; i < data.length; i++) {
output.write(data[i]);
}
output.flush();
output.close();
}
As you can see in my client, i want to sent a byte at a time to the server, but it still shows weird numbers like [?][?][?].

All the *Reader classes are meant for text data only. When working with binary data, just use the streams directly. In your case, just use BufferedInputStream instead of BufferedInputStreamReader.

There is also TCP protocol your program has to conform to, with it own buffering and flushing mechanism. This existence of this layer is non-obvious when you first code your Java program with raw byte stream.
I would suggest you either construct a deterministic protocol, e.g. using a marker byte like "000" to indicate the start of your transmission, and encoded payload that excludes the "000", and finally, "000" to terminate your transmission. (This still doesn't deal with loss in transmission well).
Alternatively, Google's Protocol Buffer, or Msgpack to help along with some of the intermediary process.

BinaryWriter to MemoryStream in Java

I have C# program that makes a tcp connection with another c# program. In the c# program to send an message I did this :
private TcpClient client;
private void SendPulse()
{
byte[] send_Buffer;
port = 11000;
while (true)
{
lock (locked)
{
try
{
BlockID = 1003;
using (MemoryStream ms = new MemoryStream())
{
using (BinaryWriter w = new BinaryWriter(ms))
{
NetworkStream stream = client.GetStream();
BlockID = 1003;
LengthMessage = 84;
// Header :
w.Write(BeginMessage);
w.Write(BlockID);
w.Write(LengthMessage);
w.Write(RadarID);
w.Write(Time);
w.Write(ModeSystem);
w.Write(Icd_primary_var);
w.Write(Icd_secondary_ver);
// Data :
w.Write(StatusSystem);
send_Buffer = ms.ToArray();
stream.Write(send_Buffer, 0, send_Buffer.Length);
Thread.Sleep(3000); // Send pulse every 3 seconds.
}
}
}
catch
{
}
}
}
}
The idea is to write in binarywriter and than convert the memory we wrote on to byte array and to send it.
Now I have Java programming. I want to do it too, I have connection with C# but I dont know how to send it. I did DataOutputStream but it send every parameter alone, I want all in 1 array of bytes exactly like in the c# code.
Thanks for helpers.

If you want to use DataOutputStream, you can wrap it around a BufferedOutputStream and flush() it when you are done.
Or you can use an NIO ByteBuffer and write it to the socket.
To make the message easier to decode I would add the length to the start, unless you know it every message will be that length.

I keep getting a UTF error when transferring image data as bytes over a socket

I'm trying to send images from my computer to my phone by using a Java program connect to an Android app.
The code I use to send the image:
byte[] data = imageToByteArray(img);
sendMessage(Main.imageCheck + data.length);
out.write(data);
out.flush();
This code is run from the Java program on my computer. The sendMessage() method is:
public void sendMessage(String s) {
if (out != null && server.isConnected()) {
try {
out.writeUTF(s);
out.flush();
} catch (IOException e) {
e.printStackTrace();
}
}
}
The code I use to receive the image and other data:
while ((input = in.readUTF()) != null) {
if (!input.equalsIgnoreCase(pulseCheckMessage)) {
if (input.contains(imageCheck)) {
//byte[] data = Base64.decode(input.replace(imageCheck, ""), Base64.DEFAULT);
byte[] data = new byte[Integer.parseInt(input.replace(imageCheck, ""))];
int length = in.read(data);
updateScreenImage = BitmapFactory.decodeByteArray(data, 0, length);
MainActivity.this.runOnUiThread(updateUiRunnable);
} else {
inLine = "Received: " + input;
MainActivity.this.runOnUiThread(updateUiRunnable);
}
}
pulseTakenTime = System.currentTimeMillis();
pulseCheckReceived = true;
isAlive = true;
}
The commented out code (including the base64 class) was when I was sending the image data as a string by using Base64.encode to convert the byte[] array to a string to send over, which worked without a problem except that it was too slow. So I decided to send the bytes over which I thought would be faster.
imageCheck and Main.imageCheck are strings that are the same. The app receives that string and recognizes it and expects a byte[] array so it tries to do in.read(data) however I get an error:
ERROR: java.nio.charset.ModifiedUtf8.decode(ModifiedUtf8.java:57)
ERROR: java.io.DataInputStream.decodeUTF(DataInputStream.java:444)
ERROR: java.io.DataInputStream.decodeUTF(DataInputStream.java:438)
ERROR: java.io.DataInputStream.readUTF(DataInputStream.java:433)
ERROR: com.mangopearapples.pccontroller.MainActivity.run(MainActivity.java:238)
Thanks.

My guess is that int length = in.read(data); doesn't read all the bytes that have been sent, as documented:
Reads some number of bytes from the contained input stream and stores them into the buffer array b. The number of bytes actually read is returned as an integer
(emphasis mine)
So, the next iteration tries to read the unread bytes of the image as UTF, and it fails. Always use a loop to read bytes from a stream, which stops once all the expected number of bytes have been read.
To confirm that this is the real problem, debug and see if the length returned matches with the actual length.

Java -- How to read an unknown number of bytes from an inputStream (socket/socketServer)?

Looking to read in some bytes over a socket using an inputStream. The bytes sent by the server may be of variable quantity, and the client doesn't know in advance the length of the byte array. How may this be accomplished?
byte b[];
sock.getInputStream().read(b);
This causes a 'might not be initialized error' from the Net BzEAnSZ. Help.

You need to expand the buffer as needed, by reading in chunks of bytes, 1024 at a time as in this example code I wrote some time ago
byte[] resultBuff = new byte[0];
byte[] buff = new byte[1024];
int k = -1;
while((k = sock.getInputStream().read(buff, 0, buff.length)) > -1) {
byte[] tbuff = new byte[resultBuff.length + k]; // temp buffer size = bytes already read + bytes last read
System.arraycopy(resultBuff, 0, tbuff, 0, resultBuff.length); // copy previous bytes
System.arraycopy(buff, 0, tbuff, resultBuff.length, k); // copy current lot
resultBuff = tbuff; // call the temp buffer as your result buff
}
System.out.println(resultBuff.length + " bytes read.");
return resultBuff;

Assuming the sender closes the stream at the end of the data:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buf = new byte[4096];
while(true) {
int n = is.read(buf);
if( n < 0 ) break;
baos.write(buf,0,n);
}
byte data[] = baos.toByteArray();

Read an int, which is the size of the next segment of data being received. Create a buffer with that size, or use a roomy pre-existing buffer. Read into the buffer, making sure it is limited to the aforeread size. Rinse and repeat :)
If you really don't know the size in advance as you said, read into an expanding ByteArrayOutputStream as the other answers have mentioned. However, the size method really is the most reliable.

Without re-inventing the wheel, using Apache Commons:
IOUtils.toByteArray(inputStream);
For example, complete code with error handling:
public static byte[] readInputStreamToByteArray(InputStream inputStream) {
if (inputStream == null) {
// normally, the caller should check for null after getting the InputStream object from a resource
throw new FileProcessingException("Cannot read from InputStream that is NULL. The resource requested by the caller may not exist or was not looked up correctly.");
}
try {
return IOUtils.toByteArray(inputStream);
} catch (IOException e) {
throw new FileProcessingException("Error reading input stream.", e);
} finally {
closeStream(inputStream);
}
}
private static void closeStream(Closeable closeable) {
try {
if (closeable != null) {
closeable.close();
}
} catch (Exception e) {
throw new FileProcessingException("IO Error closing a stream.", e);
}
}
Where FileProcessingException is your app-specific meaningful RT exception that will travel uninterrupted to your proper handler w/o polluting the code in between.

The simple answer is:
byte b[] = new byte[BIG_ENOUGH];
int nosRead = sock.getInputStream().read(b);
where BIG_ENOUGH is big enough.
But in general there is a big problem with this. A single read call is not guaranteed to return all that the other end has written.
If the nosRead value is BIG_ENOUGH, your application has no way of knowing for sure if there are more bytes to come; the other end may have sent exactly BIG_ENOUGH bytes ... or more than BIG_ENOUGH bytes. In the former case, you application will block (for ever) if you try to read. In the latter case, your application has to do (at least) another read to get the rest of the data.
If the nosRead value is less than BIG_ENOUGH, your application still doesn't know. It might have received everything there is, part of the data may have been delayed (due to network packet fragmentation, network packet loss, network partition, etc), or the other end might have blocked or crashed part way through sending the data.
The best answer is that EITHER your application needs to know beforehand how many bytes to expect, OR the application protocol needs to somehow tell the application how many bytes to expect or when all bytes have been sent.
Possible approaches are:
the application protocol uses fixed message sizes (not applicable to your example)
the application protocol message sizes are specified in message headers
the application protocol uses end-of-message markers
the application protocol is not message based, and the other end closes the connection to say that is the end.
Without one of these strategies, your application is left to guess, and is liable to get it wrong occasionally.
Then you use multiple read calls and (maybe) multiple buffers.

Stream all Input data into Output stream. Here is working example:
InputStream inputStream = null;
byte[] tempStorage = new byte[1024];//try to read 1Kb at time
int bLength;
try{
ByteArrayOutputStream outputByteArrayStream = new ByteArrayOutputStream();
if (fileName.startsWith("http"))
inputStream = new URL(fileName).openStream();
else
inputStream = new FileInputStream(fileName);
while ((bLength = inputStream.read(tempStorage)) != -1) {
outputByteArrayStream.write(tempStorage, 0, bLength);
}
outputByteArrayStream.flush();
//Here is the byte array at the end
byte[] finalByteArray = outputByteArrayStream.toByteArray();
outputByteArrayStream.close();
inputStream.close();
}catch(Exception e){
e.printStackTrace();
if (inputStream != null) inputStream.close();
}

Either:
Have the sender close the socket after transferring the bytes. Then at the receiver just keep reading until EOS.
Have the sender prefix a length word as per Chris's suggestion, then read that many bytes.
Use a self-describing protocol such as XML, Serialization, ...

Use BufferedInputStream, and use the available() method which returns the size of bytes available for reading, and then construct a byte[] with that size. Problem solved. :)
BufferedInputStream buf = new BufferedInputStream(is);
int size = buf.available();

Here is a simpler example using ByteArrayOutputStream...
socketInputStream = socket.getInputStream();
int expectedDataLength = 128; //todo - set accordingly/experiment. Does not have to be precise value.
ByteArrayOutputStream baos = new ByteArrayOutputStream(expectedDataLength);
byte[] chunk = new byte[expectedDataLength];
int numBytesJustRead;
while((numBytesJustRead = socketInputStream.read(chunk)) != -1) {
baos.write(chunk, 0, numBytesJustRead);
}
return baos.toString("UTF-8");
However, if the server does not return a -1, you will need to detect the end of the data some other way - e.g., maybe the returned content always ends with a certain marker (e.g., ""), or you could possibly solve using socket.setSoTimeout(). (Mentioning this as it is seems to be a common problem.)

This is both a late answer and self-advertising, but anyone checking out this question may want to take a look here:
https://github.com/GregoryConrad/SmartSocket

This question is 7 years old, but i had a similiar problem, while making a NIO and OIO compatible system (Client and Server might be whatever they want, OIO or NIO).
This was quit the challenge, because of the blocking InputStreams.
I found a way, which makes it possible and i want to post it, to help people with similiar problems.
Reading a byte array of dynamic sice is done here with the DataInputStream, which kann be simply wrapped around the socketInputStream. Also, i do not want to introduce a specific communication protocoll (like first sending the size of bytes, that will be send), because i want to make this as vanilla as possible. First of, i have a simple utility Buffer class, which looks like this:
import java.util.ArrayList;
import java.util.List;
public class Buffer {
private byte[] core;
private int capacity;
public Buffer(int size){
this.capacity = size;
clear();
}
public List<Byte> list() {
final List<Byte> result = new ArrayList<>();
for(byte b : core) {
result.add(b);
}
return result;
}
public void reallocate(int capacity) {
this.capacity = capacity;
}
public void teardown() {
this.core = null;
}
public void clear() {
core = new byte[capacity];
}
public byte[] array() {
return core;
}
}
This class only exists, because of the dumb way, byte <=> Byte autoboxing in Java works with this List. This is not realy needed at all in this example, but i did not want to leave something out of this explanation.
Next up, the 2 simple, core methods. In those, a StringBuilder is used as a "callback". It will be filled with the result which has been read and the amount of bytes read will be returned. This might be done different of course.
private int readNext(StringBuilder stringBuilder, Buffer buffer) throws IOException {
// Attempt to read up to the buffers size
int read = in.read(buffer.array());
// If EOF is reached (-1 read)
// we disconnect, because the
// other end disconnected.
if(read == -1) {
disconnect();
return -1;
}
// Add the read byte[] as
// a String to the stringBuilder.
stringBuilder.append(new String(buffer.array()).trim());
buffer.clear();
return read;
}
private Optional<String> readBlocking() throws IOException {
final Buffer buffer = new Buffer(256);
final StringBuilder stringBuilder = new StringBuilder();
// This call blocks. Therefor
// if we continue past this point
// we WILL have some sort of
// result. This might be -1, which
// means, EOF (disconnect.)
if(readNext(stringBuilder, buffer) == -1) {
return Optional.empty();
}
while(in.available() > 0) {
buffer.reallocate(in.available());
if(readNext(stringBuilder, buffer) == -1) {
return Optional.empty();
}
}
buffer.teardown();
return Optional.of(stringBuilder.toString());
}
The first method readNext will fill the buffer, with byte[] from the DataInputStream and return the amount bytes read this way.
In the secon method, readBlocking, i utilized the blocking nature, not to worry about consumer-producer-problems. Simply readBlocking will block, untill a new byte-array is received. Before we call this blocking method, we allocate a Buffer-size. Note, i called reallocate after the first read (inside the while loop). This is not needed. You can safely delete this line and the code will still work. I did it, because of the uniqueness of my problem.
The 2 things, i did not explain in more detail are:
1. in (the DataInputStream and the only short varaible here, sorry for that)
2. disconnect (your disconnect routine)
All in all, you can now use it, this way:
// The in has to be an attribute, or an parameter to the readBlocking method
DataInputStream in = new DataInputStream(socket.getInputStream());
final Optional<String> rawDataOptional = readBlocking();
rawDataOptional.ifPresent(string -> threadPool.execute(() -> handle(string)));
This will provide you with a way of reading byte arrays of any shape or form over a socket (or any InputStream realy). Hope this helps!

how to send an array of bytes over a TCP connection (java programming)

Can somebody demonstrate how to send an array of bytes over a TCP connection from a sender program to a receiver program in Java.
byte[] myByteArray
(I'm new to Java programming, and can't seem to find an example of how to do this that shows both ends of the connection (sender and receiver.) If you know of an existing example, maybe you could post the link. (No need to reinvent the wheel.) P.S. This is NOT homework! :-)

The InputStream and OutputStream classes in Java natively handle byte arrays. The one thing you may want to add is the length at the beginning of the message so that the receiver knows how many bytes to expect. I typically like to offer a method that allows controlling which bytes in the byte array to send, much like the standard API.
Something like this:
private Socket socket;
public void sendBytes(byte[] myByteArray) throws IOException {
sendBytes(myByteArray, 0, myByteArray.length);
}
public void sendBytes(byte[] myByteArray, int start, int len) throws IOException {
if (len < 0)
throw new IllegalArgumentException("Negative length not allowed");
if (start < 0 || start >= myByteArray.length)
throw new IndexOutOfBoundsException("Out of bounds: " + start);
// Other checks if needed.
// May be better to save the streams in the support class;
// just like the socket variable.
OutputStream out = socket.getOutputStream();
DataOutputStream dos = new DataOutputStream(out);
dos.writeInt(len);
if (len > 0) {
dos.write(myByteArray, start, len);
}
}
EDIT: To add the receiving side:
public byte[] readBytes() throws IOException {
// Again, probably better to store these objects references in the support class
InputStream in = socket.getInputStream();
DataInputStream dis = new DataInputStream(in);
int len = dis.readInt();
byte[] data = new byte[len];
if (len > 0) {
dis.readFully(data);
}
return data;
}

Just start with this example from the Really Big Index. Notice though, that it's designed to transmit and receive characters, not bytes. This isn't a big deal, though - you can just deal with the raw InputStream and OutputStream objects that the Socket class provides. See the API for more info about the different types of readers, writers and streams. Methods you'll be interested in are OutputStream.write(byte[]) and InputStream.read(byte[]).

The Oracle Socket Communications Tutorial would seem to be the appropriate launch point.
Note that it's going to extra trouble to turn characters into bytes. If you want to work at the byte level, just peel that off.

This Sun Sockets tutorial should give you a good starting point

What you need to use is the write method of an java.io.OutputStream, and the read method of an java.io.InputStream, both of which you can retrieve from the Socket you open.

I would ask you to use ObjectOutputStream and ObjectInputStream. These send everything as an object and receive as the same.
ObjectOutputStream os = new ObjectOutputStream(socket.getOutputStream());
os.flush();
ObjectInputStream is = new ObjectInputStream(socket.getInputStream());
os.writeObject(byte_array_that_you_want_to_send);
byte[] temp = (byte[]) is.readObject();
Also remember first create the output stream, flush it and then go ahead with the input stream because if something left out in the stream the input stream wont be created.

I'm guessing that the question is worded incorrectly. I found this when searching for an answer to why my use of InputStream and OutputStream seemed to be setting the entire array to 0 upon encountering a byte of value 0. Do these assume that the bytes contain valid ASCII and not binary. Since the question doesn't come right out and ask this, and nobody else seems to have caught it as a possibility, I guess I'll have to satisfy my quest elsewhere.
What I was trying to do was write a TransparentSocket class that can instantiate either a TCP (Socket/ServerSocket) or a UDP (DatagramSocket) to use the DatagramPacket transparently. It works for UDP, but not (yet) for TCP.
Follow-up: I seem to have verified that these streams are themselves useless for binary transfers, but that they can be passed to a more programmer-friendly instantiation, e.g.,
new DataOutputStream(socket.getOutputStream()).writeInt(5);
^ So much for that idea. It writes data in a "portable" way, i.e., probably ASCII, which is no help at all, especially when emulating software over which I have no control!

import java.io.*;
import java.net.*;
public class ByteSocketClient
{
public static void main(String[] args) throws UnknownHostException, IOException
{
Socket s=new Socket("",6000);
DataOutputStream dout=new DataOutputStream(new BufferedOutputStream(s.getOutputStream()));
byte[] a = {(byte)0xC0,(byte)0xA8,(byte)0x01,(byte)0x02,(byte)0x53,(byte)0x4D,(byte)0x41,(byte)0x52,(byte)0x54};
dout.write(a);
dout.close();
s.close();
}
}

Here is an example that streams 100 byte wav file frames at a time.
private Socket socket;
public void streamWav(byte[] myByteArray, int start, int len) throws IOException {
Path path = Paths.get("path/to/file.wav");
byte[] data = Files.readAllBytes(path);
OutputStream out = socket.getOutputStream();
DataOutputStream os = new DataOutputStream(out);
os.writeInt(len);
if (len > 0) {
os.write(data, start, len);
}
}
public void readWav() throws IOException {
InputStream in = socket.getInputStream();
int frameLength = 100; // use useful number of bytes
int input;
boolean active = true;
while(active) {
byte[] frame = new byte[frameLength];
for(int i=0; i<frameLength; i++) {
input = in.read();
if(input < 0) {
active = false;
break;
} else frame[i] = (byte) input;
}
// playWavPiece(frame);
// streamed frame byte array is full
// use it here ;-)
}
}