Develop Reference

How to converte Long-->byteArray->String-->byteArray-->Long over socket?

I want to send messages over socket that contain long values. I want to send string messages. To save bytes and make messages shorter, I want to send the byte value of a 64 bit long in 8-character string. Following is my code, but I received wrong value on the other side of the socket:
This the server:
public class ClockSkewServer {
public static void main (String args[]){
try {
ServerSocket s = new ServerSocket(3000);
System.out.println("Server is listening!");
Socket connection = s.accept();
connection.setKeepAlive(true);
BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
PrintWriter pw = new PrintWriter(connection.getOutputStream(), true);
while (true) {
String message = br.readLine();
System.out.println(message);
Long receivedTime = ByteUtil.stringToLong(message);
System.out.println(receivedTime);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
And this is the client:
public class ClockSkewClient {
public static void main (String args[]) throws UnknownHostException, IOException, InterruptedException{
Socket s1 = new Socket(args[0], 3000);
s1.setKeepAlive(true);
PrintWriter out1 = new PrintWriter(s1.getOutputStream(), true);
long l = new Long("1490917469228");
//System.out.println(l);
while (true){
Thread.sleep(1000);
String message = ByteUtil.longToString(l);
System.out.println("Client message: " + message);
System.out.println(ByteUtil.stringToLong(message));
out1.println(message);
}
}
}
And this my byte conversion class:
public class ByteUtil {
private static ByteBuffer longBuffer = ByteBuffer.allocate(Long.BYTES);
private static ByteBuffer shortBuffer = ByteBuffer.allocate(Short.BYTES);
private static ByteBuffer byteBuffer = ByteBuffer.allocate(Byte.BYTES);
public static byte[] longToBytes(long x) {
longBuffer.putLong(0, x);
return longBuffer.array();
}
public static long bytesToLong(byte[] bytes) {
return ByteBuffer.wrap(bytes).getLong();
}
public static String longToString (long x) throws UnsupportedEncodingException{
return new String (longToBytes(x), "ISO-8859-1");
}
public static long stringToLong (String s) throws UnsupportedEncodingException{
return bytesToLong(s.getBytes("ISO-8859-1"));
}
}
ByteUtil is working fine, as the client can retrieve the original long value correctly, but when I send over socket it get distorted.
I successfully get the long value in the client side as 1490917469228, but on the server side I get 1490911439916.
I know we can send bytes and avoid this problem, but for some reason I insist to send String over socket.
Thanks

I want to send messages over socket that contain long values. I want to send string messages. To save bytes and make messages shorter, I want to send the byte value of a 64 bit long in 8-character string.
Why? Sending 64 bits as 8 chars could take 16 bytes. 128 bits. It certainly doesn't save any bytes whatsoever, and all these conversions just waste time and code space: especially development time, especially as your code doesn't work.
Just use DataOutputStream.writeLong() and DataInputStream.readLong().

copy bytes from one array to another without iteration in Java

is there a quicker way to copy bytes from one array to another without iteration ?
I'm reading bytes via Bluetooth from the input stream
public void run() {
byte[] buffer = new byte[100]; // buffer store for the stream
int bytes; // bytes returned from read()
// Keep listening to the InputStream until an exception occurs
while (true) {
try {
// Read from the InputStream
bytes = mmInStream.read(buffer); // Get number of bytes and message in "buffer"
h.obtainMessage(RECIEVE_MESSAGE, bytes, -1, buffer).sendToTarget(); // Send to message queue Handler
} catch (IOException e) {
break;
}
}
}
This sends a random number of bytes to the handler.
I then read these and put them into an array to then process the data when the download is complete. The data being sent from a PIC micro is 6055 bytes long.
h = new Handler() {
public void handleMessage(android.os.Message msg) {
switch (msg.what) {
case SUCCESS_CONNECT:
// Connected so start connected thread
btSocket = (BluetoothSocket) msg.obj;
byteCount = 0;
arrayCount = 0;
mConnectedThread = new FullDataActivity.ConnectedThread(btSocket);
mConnectedThread.start();
mConnectedThread.write(getFullDataConByte); // Send 255 to start
break;
case RECIEVE_MESSAGE: // if receive massage
byte[] readBuf = (byte[]) msg.obj;
// iterate through obj and copy bytes to fullDataChunk array
for (int a = 0; a < msg.arg1; a++) {
fullDataChunk[byteCount] = readBuf[a];
Log.d("readBuf[a] = ",Integer.toString(readBuf[a] & 0xFF));
byteCount++;
}
// if all bytes done process
if (byteCount == 6055) {// process data when complete.
My handler is missing bytes somewhere and corrupting the data when its copying them in the for loop. I don't know if the run method is sending new bytes to the handler before the rest are processed in the for loop.
I've done a Log of the bytes sent in the Run and they are right. It's when the handler is processing that they go wrong.
I either need to ensure the data sent is processed before new data is sent, or copy the data to the array quicker ?
Anyone any ideas.

The fastest way to copy an array should be by using System.arraycopy()
.

Did this in the run, then the handler receives full array without errors.
public void run() {
byte[] fullBuffer = new byte[6055];
byte[] buffer = new byte[100]; // buffer store for the stream
int bytes; // bytes returned from read()
int bytesCount = 0;
// Keep listening to the InputStream until an exception occurs
while (true) {
try {
// Read from the InputStream
bytes = mmInStream.read(buffer); // Get number of bytes and message in "buffer"
System.arraycopy(buffer,0,fullBuffer,bytesCount,bytes);
bytesCount = bytesCount + bytes;
Log.d("FD Read - ", Integer.toString(bytesCount));
if(bytesCount >= 6055){
h.obtainMessage(RECIEVE_MESSAGE, bytesCount, -1, fullBuffer).sendToTarget(); // Send to message queue Handler
Log.d("FD Read - ", "Message sent");
bytesCount = 0;
Log.d("FD Read - ", "bytesCount re-set");
}
//h.obtainMessage(RECIEVE_MESSAGE, bytes, -1, buffer).sendToTarget(); // Send to message queue Handler
} catch (IOException e) {
break;
}
}
}

C#-Server and Java-Client: TCP Socket Communication Issues

I have wrote a server program in C# using TCPListner and a client program in Java using socket but I fail to send complex objects from Java client to C# server.
When I send a simple string from Java client to C# server by converting the string into byte array,
it always show some invalid characters at the start of message when converted back to String (using Encoding.utf8.getstring(bytesArray) ) in C# server. When I pass a String from C# to Java Client it shows invalid Header error.
Please help me if any one have any alternative or know abut any free API which can solve my problem. I have tried Java-cs-bridge to send complex objects but it always show Exception on C# server.
Here is the code:
C# Server Code - Main Function
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.IO;
namespace netSocketServer
{
class Program
{
static void Main(string[] args)
{
TcpListener server = new TcpListener(IPAddress.Any, 8888);
var IP = Dns.GetHostEntry(Dns.GetHostName()).AddressList.Where(ip =>ip.AddressFamily == AddressFamily.InterNetwork).Select(ip =>ip).FirstOrDefault();
server.Start();
Console.WriteLine("Server is Running at " + IP.ToString());
TcpClient clientSocket = server.AcceptTcpClient();
Console.WriteLine("Client Connected ... ");
Writer wr = new Writer(clientSocket);
wr.start();
Reader r = new Reader(clientSocket);
r.start();
Console.Read();
}
}
}
C# Server Reader Class
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Net.Sockets;
using System.Net;
using System.IO;
namespace netSocketServer
{
class Reader
{
TcpClient socket;
NetworkStream ns;
public Reader(TcpClient s)
{
socket = s;
ns = socket.GetStream() ;
}
public void start()
{
new Thread(
t => {
while (true)
{
try
{
int size = ns.ReadByte();
byte[] buff = new byte[size];
ns.Read(buff,0,size);
String message = Encoding.UTF8.getString(buff);
Console.WriteLine("Message from Client : {0}",message);
ns.Flush();
}
catch (Exception e)
{
Console.WriteLine("Client Disconnected : " + e.Message);
}
}
}).Start();
}
}
}
C# Server Writer Class
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net.Sockets;
using System.Text;
using System.Threading;
namespace netSocketServer
{
class Writer
{
TcpClient socket;
NetworkStream ns;
public Writer(TcpClient s)
{
socket = s;
ns = socket.GetStream();
}
public void start()
{
new Thread(
t => {
while (true)
{
try
{
Console.Write("Please Enter your Message : ");
string Message = Console.ReadLine();
byte[] buff = Encoding.UTF8.GetBytes(Message);
byte size = (byte)Message.Length;
ns.WriteByte(size);
ns.Write(buff, 0, buff.Length);
ns.Flush();
}
catch(IOException e)
{
Console.WriteLine("Client Disconnected : " + e.Message);
socket.Close();
Thread.CurrentThread.Abort();
Console.WriteLine("Press any key to Closse Server .... ");
}
}
}).Start();
}
}
}
Java Client - Main Function
package javaclient.net;
import java.io.IOException;
import java.net.Socket;
import java.util.Scanner;
/**
*
* #author Numan
*/
public class JavaClientNet {
/**
* #param args the command line arguments
*/
public static void main(String[] args)
{
Socket socket;
Read r;
Writer wr;
Scanner s = new Scanner(System.in);
try
{
// TODO code application logic here
System.out.print("Please Enter Server IP : ");
socket = new Socket(s.next(), 8888);
wr = new Writer(socket);
wr.start();
r = new Read(socket);
r.start();
}
catch (IOException ex)
{
System.out.println(ex.getMessage());
}
}
}
Java Client - Reader Class
package javaclient.net;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.net.Socket;
/**
*
* #author Numan
*/
public class Read extends Thread
{
Socket socket;
ObjectInputStream inStream;
Read(Socket s)
{
socket = s;
try {
inStream = new ObjectInputStream(socket.getInputStream());
}
catch (IOException ex)
{
System.out.println(ex.getMessage());
}
}
#Override
public void run()
{
while(true)
{
try
{
String str;
byte size = inStream.readByte();
byte[] buf = new byte[size];
inStream.read(buf);
str = new String(buf);
System.out.println("Message form Server : "+str);
}
catch(IOException e)
{
System.out.println(e.getMessage());
Thread.currentThread().stop();
}
}
}
}
Java Client - Writer Class
package javaclient.net;
import java.io.IOException;
import java.io.ObjectOutputStream;
import java.net.Socket;
import java.util.Scanner;
import javacsconverter.core.tobyte.ToByteConvertHelper;
/**
*
* #author Numan
*/
public class Writer extends Thread
{
Socket socket;
ObjectOutputStream outStream;
Scanner scanner = new Scanner(System.in);
Writer(Socket s)
{
socket =s;
try
{
outStream = new ObjectOutputStream(socket.getOutputStream());
}
catch (IOException ex)
{
System.out.println(ex.getMessage());
}
}
#Override
public void run()
{
while(true)
{
try
{
System.out.print("Please Enter Your Message : ");
String str = scanner.nextLine();
byte[] buff = str.getBytes();
outStream.write(buff);
outStream.flush();
}
catch (IOException ex)
{
System.out.println(ex.getMessage());
}
}
}
}

General notes
Please do not abort the threads (both C# and Java).
C# Server
Program class
There is a data race because the static Console class is used by multiple threads:
Main thread: the Program.Main() method calls the Console.Read() method;
Worker thread: the Writer.start() method calls the Console.ReadLine() method.
Please consider replacing the Console.Read() method call of the Program.Main() method with something different, for example, Thread.Sleep(Timeout.Infinite).
Reader class
There is a mistake — the Stream.Read() method is not guaranteed to read the array of the specified "size" at once (one call), the return value should be used to determine the actual number of bytes read. Let's see the original implementation:
int size = ns.ReadByte();
byte[] buff = new byte[size];
// The Stream.Read() method does not guarantee to read the **whole array** "at once".
// Please use the return value of the method.
ns.Read(buff, 0, size);
String message = Encoding.UTF8.GetString(buff);
Corrected version:
/// <summary>
/// Helper method to read the specified byte array (number of bytes to read is the size of the array).
/// </summary>
/// <param name="inputStream">Input stream.</param>
/// <param name="buffer">The output buffer.</param>
private static void ReadFully(Stream inputStream, byte[] buffer)
{
if (inputStream == null)
{
throw new ArgumentNullException("inputStream");
}
if (buffer == null)
{
throw new ArgumentNullException("buffer");
}
int totalBytesRead = 0;
int bytesLeft = buffer.Length;
if (bytesLeft <= 0)
{
throw new ArgumentException("There is nothing to read for the specified buffer", "buffer");
}
while (totalBytesRead < buffer.Length)
{
var bytesRead = inputStream.Read(buffer, totalBytesRead, bytesLeft);
if (bytesRead > 0)
{
totalBytesRead += bytesRead;
bytesLeft -= bytesRead;
}
else
{
throw new InvalidOperationException("Input stream reaches the end before reading all the bytes");
}
}
}
public void start()
{
...
int size = ns.ReadByte();
byte[] buff = new byte[size];
ReadFully(ns, buff);
using (var memoryStream = new MemoryStream(buff, false))
{
// The StreamReader class is used to extract the UTF-8 string which is encoded with the byte order mark (BOM).
using (var streamReader = new StreamReader(memoryStream, Encoding.UTF8))
{
string message = streamReader.ReadToEnd();
Console.WriteLine("Message from Client: {0}", message);
}
}
...
}
Writer class
First of all, to describe and determine byte the order of the text stream consider including the byte order mark (BOM) for each message (for example).
Also, there is a mistake — wrong "buffer length" value is sent. Let's see the original implementation:
string Message = Console.ReadLine();
byte[] buff = Encoding.UTF8.GetBytes(Message);
// Problem: instead of the length of the string, the size of byte array must be used
// because the UTF-8 encoding is used: generally, string length != "encoded number of bytes".
byte size = (byte)Message.Length;
ns.WriteByte(size);
ns.Write(buff, 0, buff.Length);
ns.Flush();
Corrected version:
// UTF-8 with BOM.
var encoding = new UTF8Encoding(true);
// Buffer encoded as UTF-8 with BOM.
byte[] buff = encoding.GetPreamble()
.Concat(encoding.GetBytes(message))
.ToArray();
// Size of the encoded buffer.
byte size = Convert.ToByte(buff.Length);
ns.WriteByte(size);
ns.Write(buff, 0, buff.Length);
ns.Flush();
Alternative corrected version — the StreamWriter class is used to encode the string as UTF-8 with the byte order mark (BOM):
string message = Console.ReadLine();
using (var memoryStream = new MemoryStream())
{
using (var streamWriter = new StreamWriter(memoryStream, Encoding.UTF8, 1024, true))
{
streamWriter.Write(message);
}
memoryStream.Flush();
byte size = Convert.ToByte(memoryStream.Length);
ns.WriteByte(size);
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(ns);
ns.Flush();
}
Java Client
Read class
First, please consider using DataInputStream class because the following statement is not true according to the question:
An ObjectInputStream deserializes primitive data and objects previously written using an ObjectOutputStream.
-- java.io.ObjectInputStream class, Java™ Platform
Standard Ed. 7.
The instantiation of the stream is almost the same:
inStream = new DataInputStream(socket.getInputStream());
Second, there is a mistake — reading the byte array, but ignoring the return value (actual number of bytes read):
String str;
byte size = inStream.readByte();
byte[] buf = new byte[size];
// The InputStream.read() method does not guarantee to read the **whole array** "at once".
// Please use the return value of the method.
inStream.read(buf);
str = new String(buf);
Third, as stated above, the byte order mark (BOM) is included.
Corrected version:
// Note: inStream must be an instance of DataInputStream class.
byte size = inStream.readByte();
byte[] buf = new byte[size];
// The DataInputStream.readFully() method reads the number of bytes required to fill the buffer entirely.
inStream.readFully(buf);
// Create in-memory stream for the byte array and read the UTF-8 string.
try (ByteArrayInputStream inputStream = new ByteArrayInputStream(buf);
// The BOMInputStream class belongs to Apache Commons IO library.
BOMInputStream bomInputStream = new BOMInputStream(inputStream, false)) {
String charsetName = bomInputStream.getBOMCharsetName();
// The IOUtils class belongs to Apache Commons IO library.
String message = IOUtils.toString(bomInputStream, charsetName);
System.out.println("Message form Server : " + message);
}
Writer class
There is a mistake — the encoding is not specified explicitly. Let's see the original implementation:
String str = scanner.nextLine();
byte[] buff = str.getBytes();
Corrected version:
String str = scanner.nextLine();
byte[] byteOrderMarkBytes = ByteOrderMark.UTF_8.getBytes();
byte[] stringBytes = str.getBytes(StandardCharsets.UTF_8);
// The ArrayUtils.addAll() method belongs to Apache Commons Lang library.
byte[] buff = ArrayUtils.addAll(byteOrderMarkBytes, stringBytes);
outStream.writeByte(buff.length);
outStream.write(buff);
outStream.flush();
Alternative corrected version — the ByteArrayOutputStream class is used to concatenate the arrays:
String str = scanner.nextLine();
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
byte[] byteOrderMarkBytes = ByteOrderMark.UTF_8.getBytes();
byteArrayOutputStream.write(byteOrderMarkBytes);
byte[] stringBytes = str.getBytes(StandardCharsets.UTF_8);
byteArrayOutputStream.write(stringBytes);
byteArrayOutputStream.flush();
byte[] buff = byteArrayOutputStream.toByteArray();
outStream.writeByte(buff.length);
outStream.write(buff);
outStream.flush();
Hope this helps!

PushbackInputStream: Push back buffer is full

Why I am getting the following exception:
Exception in thread "main" java.io.IOException: Push back buffer is full
at java.io.PushbackInputStream.unread(PushbackInputStream.java:232)
at java.io.PushbackInputStream.unread(PushbackInputStream.java:252)
at org.tests.io.PushBackStream_FUN.read(PushBackStream_FUN.java:32)
at org.tests.io.PushBackStream_FUN.main(PushBackStream_FUN.java:43)
In this code:
public class PushBackStream_FUN {
public int write(String outFile) throws Exception {
FileOutputStream outputStream = new FileOutputStream(new File(outFile));
String str = new String("Hello World");
byte[] data = str.getBytes();
outputStream.write(data);
outputStream.close();
return data.length;
}
public void read(String inFile, int ln) throws Exception {
PushbackInputStream inputStream = new PushbackInputStream(new FileInputStream(new File(inFile)));
byte[] data = new byte[ln];
String str;
// read
inputStream.read(data);
str = new String(data);
System.out.println("MSG_0 = "+str);
// unread
inputStream.unread(data);
// read
inputStream.read(data);
str = new String(data);
System.out.println("MSG_1 = "+str);
}
public static void main(final String[] args) throws Exception {
PushBackStream_FUN fun = new PushBackStream_FUN();
String path = "aome/path/output_PushBack_FUN";
int ln = fun.write(path);
fun.read(path, ln);
}
}
UPDATE
Think this is the solution. Java sources to the rescue. I have made some "experiments". It turns out that when I specify PushbackInputStream with a specified buffer size it works. The java sources tells me this:
public PushbackInputStream(InputStream in) {
this(in, 1);
}
public PushbackInputStream(InputStream in, int size) {
super(in);
if (size <= 0) {
throw new IllegalArgumentException("size <= 0");
}
this.buf = new byte[size];
this.pos = size;
}
I think that if you use PushbackInputStream with constrcutor that uses default buffer size, you can only unread single byte. I am unreading more than single byte, thus the exception.

By default, PushbackInputStream only allocates enough space to be able to unread() for a single character. If you want to be able to push back more than that you must specify the capacity at construction time.
In your case it'd look something like this:
final PushbackInputStream pis = new PushbackInputStream( inputStream, ln );

Netty java getting data from ByteBuf

How to get a byte array from ByteBuf efficiently in the code below? I need to get the array and then serialize it.
package testingNetty;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
public class ServerHandler extends ChannelInboundHandlerAdapter {
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
System.out.println("Message receive");
ByteBuf buff = (ByteBuf) msg;
// There is I need get bytes from buff and make serialization
byte[] bytes = BuffConvertor.GetBytes(buff);
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
// Close the connection when an exception is raised.
cause.printStackTrace();
ctx.close();
}
}

ByteBuf buf = ...
byte[] bytes = new byte[buf.readableBytes()];
buf.readBytes(bytes);
If you don't want the readerIndex to change:
ByteBuf buf = ...
byte[] bytes = new byte[buf.readableBytes()];
int readerIndex = buf.readerIndex();
buf.getBytes(readerIndex, bytes);
If you want to minimize the memory copy, you can use the backing array of the ByteBuf, if it's available:
ByteBuf buf = ...
byte[] bytes;
int offset;
int length = buf.readableBytes();
if (buf.hasArray()) {
bytes = buf.array();
offset = buf.arrayOffset();
} else {
bytes = new byte[length];
buf.getBytes(buf.readerIndex(), bytes);
offset = 0;
}
Please note that you can't simply use buf.array(), because:
Not all ByteBufs have backing array. Some are off-heap buffers (i.e. direct memory)
Even if a ByteBuf has a backing array (i.e. buf.hasArray() returns true), the following isn't necessarily true because the buffer might be a slice of other buffer or a pooled buffer:
buf.array()[0] == buf.getByte(0)
buf.array().length == buf.capacity()

Another option is ByteBufUtil.getBytes(ByteBuf buf, int start, int length, boolean copy)
See ByteBufUtil