Hi and thanks in advance,
So I'm trying to take an array of JList items and convert them to a string array (which I think I've gotten right), and then I'm trying to send that string array over to my client who will then attempt to display them back into a JList on their side.
I've tried a few different things but none are working.
Here is my latest code attempt to send the string array over:
String[] FilesList = (String[]) lClient1Files.getSelectedValues();
FilesBuffer = FilesList.getBytes();
DatagramPacket DGPFilesResponse = new DatagramPacket(FilesBuffer,FilesBuffer.length, DGP.getAddress(), DGP.getPort());
SeederSocket.send(DGPFilesResponse);
The line: FilesBuffer = FilesList.getBytes(); is causing the issue because getBytes() isn't applicable here.
So my questions are:
1) How do I send the array of JList items(they are names) over to the client (it doesn't particularly have to be a string array), and
2) How would I receive the list on the clients side, so that I can use it?
Thank you.
One must make a binary format for the string array.
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (DataOutputStream dos = new DataOutputStream(baos)) {
dos.writeInt(filesList.length);
for (String files : filesList) {
dos.writeUTF(files);
}
}
byte[] bytes = baos.toByteArray();
This internally for a String writes first the length in bytes, and uses String.getBytes("UTF-8") so any string can be written.
Reading goes with the reversed input classes.
If you think of having many clients out there, maybe with different versions,
then add in the message a version number.
On the other side
ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
try (DataInputStream dis = new DataInputStream(baos)) {
int stringsCount = dis.readInt();
String[] filesList = new String[stringsCount];
for (int i = 0; i < stringsCount; ++i) {
filesList[i] = dis.readUTF();
}
return filesList;
}
The UDP payload has to be a byte[]. You need to choose a way to encode your data into a byte[], such that it can be converted back at the receiving end.
So you need to write encode() and decode() so that unit tests like this work:
#Test
public void encodesAndDecodesStringArray() {
String[] strings = new String[] { "foo", "bar" };
byte[] encoded = Encoder.encode(strings);
String[] decoded = Encoder.decode(encoded);
assertThat(decoded, is(strings));
}
There are literally hundreds of encoding schemes you could choose from. Delimiter-separated, length-separated, JSON, XML, BSON, ASN.1 ... take a look at Wikipedia's List of data serialization formats.
A very simple option that might work for you is delimiter-separation:
public byte[] encode(String[] strings) {
return String.join(",", strings).getBytes(UTF_8);
}
public String[] decode(byte[] encodedArray) {
return new String(encodedArray, UTF_8).split(",");
}
But note that this very basic scheme fails if any of the input strings contains a "," (or whatever delimiter you choose). Pick a scheme that works for you.
Consider using JSON -- there are easy to use libraries to read and write JSON. Readable ASCII in network traces is often convenient. The space overhead is not that high. It's ready for arbitrarily complex hierarchical data structures.
Consider that if you change the structure of the data produced by your sender, the receiver must also change. If that matters, consider encoding a protocol version into what you send (it might be enough to just say "the first two bytes are the version", and always stick a 0x0001 in there to start with).
Related
I have a need to send and receive large byte array over internet(http restful service).
the simplest way I can think of is to convert the byte array into string.
I searched around and found this post Java Byte Array to String to Byte Array
I had the follow code to verify the accuracy of the transformation.
System.out.println("message");
System.out.println (message);
String message = "Die Strahlengriffelgewächse stammen...";
byte[] pack = Fbs.packExce(message);
System.out.println ("pack");
System.out.println (pack);
System.out.println ("packlenght:" + pack.length);
String toString = new String(pack);
System.out.println ("toString");
System.out.println (toString);
byte[] toBytes = toString.getBytes();
System.out.println ("toBytes");
System.out.println (toBytes);
System.out.println ("toByteslength:" +toBytes.length);
the "Fbs.packExce()" is a method of taking in large chunk of string and churning out byte array of large size.
I changed the length of the message, checked and printed out the length of byte arrays before converting to string and after converting back.
I got the following results:
...
pack
[B#5680a178
packlenght:748
...
toBytes
[B#5fdef03a
toByteslength:750
----------------------
...
pack
[B#5680a178
packlenght:1016
...
toBytes
[B#5fdef03a
toByteslength:1018
I had omitted the "message" since it is too long.
8 times out of 10, I can see that the derived byte array(the new one, saying "toBytes") is longer by 2 bytes than the original byte array ( the "pack")
I said 8 of 10, because there were also scenarios when the length are the same between the derived and the original, see below
...
pack
[B#5680a178
packlenght:824
toString
...
toBytes
[B#5fdef03a
toByteslength:824
...
I can not figure out the exact rules.
does anyone has any idea?
or are there any better ways of converting byte array to and from string?
cheers
the simplest way I can think of is to convert the byte array into string.
The simplest way is the wrong way. For most character encodings, converting an arbitrary byte sequence to a text is likely to be lossy.
A better (i.e. more robust) way is to use Base64 encoding. Read the javadoc for the Base64 class and its dependent encode and decoder classes.
If you do persist in trying to convert arbitrary bytes top characters and back using new String(byte[]) and the like:
Be sure that you chose a character encoding where a Bytes -> Characters -> Bytes conversion sequence is not lossy. (LATIN-1 will work)
Don't rely on the current execution platform's default character encoding for the encoding / decoding charset.
In a client / server system, the client and server have to use the same encoding.
I have a need to send and receive large byte array over internet(http
restful service).
the simplest way I can think of is to convert the byte array into
string.
If that's all about sending/receiving byte array with jaxrs, each jaxrs implementation is perfectly capable of transmitting byte[]. See specification, section 4.2.4.
as per suggestion by Stephen C, I turned to Base64 basic mode:
following are my current complete verification code:
String message = "Die Strahlengriffelgewächse stammen ... ...
System.out.println("message");
System.out.println (message);
byte[] pack = Fbs.packExce(message);
System.out.println ("pack");
System.out.println (pack);
System.out.println ("packlenght:" + pack.length);
String toString = Base64.getEncoder().encodeToString(pack);
System.out.println ("toString");
System.out.println (toString);
byte[] toBytes = Base64.getDecoder().decode(toString);
System.out.println ("toBytes");
System.out.println (toBytes);
System.out.println ("toByteslength:" +toBytes.length);
String toBytesExtraction = extractExce(toBytes);
System.out.println ("toBytesExtraction");
System.out.println (toBytesExtraction);
String extraction = extractExce(pack);
System.out.println ("extraction");
System.out.println (extraction);
public static byte[] packExce(String text){
FlatBufferBuilder builder = new FlatBufferBuilder(0);
int textOffset = builder.createString(text);
Exce.startExce(builder);
Exce.addText(builder, textOffset);
int exce = Exce.endExce(builder);
Bucket.startBucket(builder);
Bucket.addContentType(builder, Post.Exce);
Bucket.addContent(builder, exce);
int buck = Bucket.endBucket(builder);
builder.finish(buck);
return builder.sizedByteArray();
//ByteBuffer buf = builder.dataBuffer();
//return buf;
//return Base64.getMimeEncoder().encodeToString(buf.array());
}
private String extractExce(byte[] bucket ){
String message = null;
ByteBuffer buf = ByteBuffer.wrap(bucket);
Bucket cont = Bucket.getRootAsBucket(buf);
System.out.println (cont.contentType());
if (cont.contentType() == Post.Exce){
message = ((Exce)cont.content(new Exce())).text();
}
return message;
}
and it seems work for my purpose:
...
pack
[B#5680a178
packlenght:2020
...
toBytes
[B#5fdef03a
toByteslength:2020
'''
----------------------
...
pack
[B#5680a178
packlenght:1872
...
toBytes
[B#5fdef03a
toByteslength:1872
...
and both extraction respectively from "toBytes" and "pack" faithfully restored the original "message"
String toBytesExtraction = extractExce(toBytes);
String extraction = extractExce(pack);
as a matter of fact, what I did not mention is that my original implementation had been base64 mime. my start point had been ByteBuffer then (my current is byte[]).
following are my code snippets if you are interested in.
coder
...
ByteBuffer buf = builder.dataBuffer();
return Base64.getMimeEncoder().encodeToString(buf.array());
decoder
ByteBuffer buf = ByteBuffer.wrap(Base64.getMimeDecoder().decode(bucket));
my guess is that the problem might have come from base64.mime.
because my first step of trouble location had been removing base64.mime, and using ByteBuffer directly. and it was a success...
well, I am a bit wandering off.
Back to the topic, I am still having no idea about the "2 bytes vary" regarding byte arrays before and after converting by "new String(byte[]) and "String.getBytes()" ...
cheers
I'm trying to decompress a json object in Java that was initially compressed in PHP. Here's how it gets compressed into PHP:
function zip_json_encode(&$arr) {
$uncompressed = json_encode($arr);
return pack('L', strlen($uncompressed)).gzcompress($uncompressed);
}
and decoded (again in PHP):
function unzip_json_decode(&$data) {
$uncompressed = #gzuncompress(substr($data,4));
return json_decode($uncompressed, $array_instead_of_object);
}
That gets put into MySQL and now it must be pulled out of the db by Java. We pull it out from the ResultSet like this:
String field = rs.getString("field");
I then pass that string to a method to decompress it. This is where it falls apart.
private String decompressHistory(String historyString) throws SQLException {
StringBuffer buffer = new StringBuffer();
try {
byte[] historyBytes = historyString.substring(4).getBytes();
ByteArrayInputStream bin = new ByteArrayInputStream(historyBytes);
InflaterInputStream in = new InflaterInputStream(bin, new Inflater(true));
int len;
byte[] buf = new byte[1024];
while ((len = in.read(buf)) != -1) {
// buf should be decoded, right?
}
} catch (IOException e) {
e.getStackTrace();
}
return buffer.toString();
}
Not quite sure what's going wrong here, but any pointers would be appreciated!
You need to get rid of the true in Inflater(true). Use just Inflater(). The true makes it expect raw deflate data. Without the true, it is expecting zlib-wrapped deflate data. PHP's gzcompress() produces zlib-wrapped deflate data.
Gzipped data is binary, byte[]. Using String, Unicode text, not only needs conversion, but is faulty.
For instance this involves a conversion:
byte[] historyBytes = historyString.substring(4).getBytes();
byte[] historyBytes = historyString.substring(4).getBytes("ISO-8859-1");
The first version uses the default platform encoding, making the application non-portable.
The first to-do is to use binary data in the database as VARBINARY or BLOB.
ImputStream field = rs.getBinaryStream("field");
try (InputStream in = new GZIPInputStream(field)) {
...
}
Or so. Mind the other answer.
In the end, neither of the above solutions worked, but both have merits. When we pulled the data out of mysql and cast it to bytes we have a number of missing character bytes (67). This made it impossible to decompress on the java side. As for the answers above. Mark is correct that gzcompress() uses zlib and therefore you should use the Inflater() class in Java.
Joop is correct that the data conversion is faulty. Our table was too large to convert it to varbinary or blob. That may have solved the problem, but didn't work for us. We ended up having java make a request to our PHP app, then simply unpacked the compressed data on the PHP side. This worked well. Hopefully this is helpful to anyone else that stumbles across it.
I'm new to Java and I'm no sure how to do the following:
A Scala application somewhere converts a String into bytes:
ByteBuffer.wrap(str.getBytes)
I collect this byte array as a Java String, and I wish to do the inverse of what the Scala code above did, hence get the original String (object str above).
Getting the ByteBuffer as a String to begin with is the only option I have, as I'm reading it from an AWS Kinesis stream (or is it?). The Scala code shouldn't change either.
Example string:
String str = "AAAAAAAAAAGZ7dFR0XmV23BRuufU+eCekJe6TGGUBBu5WSLIse4ERy9............";
How can this be achieved in Java?
EDIT
Okay, so I'll try to elaborate a little more about the process:
A 3rd party Scala application produces CSV rows which I need to consume
Before storing those rows in an AWS Kinesis stream, the application does the following to each row:
ByteBuffer.wrap(output.getBytes);
I read the data from the stream as a string, and the string could look like the following one:
String str = "AAAAAAAAAAGZ7dFR0XmV23BRuufU+eCekJe6TGGUBBu5WSLIse4ERy9............";
I need to restore the contents of the string above into its original, readable, form;
I hope I've made it clearer now, sorry for puzzling you all to begin with.
If you want to go from byte[] to String, try new String(yourBytes).
Both getBytes and the String(byte[]) uses the default character encoding.
From Amazon Kinesis Service API Reference:
The data blob to put into the record, which is Base64-encoded when the blob is serialized.
You need to base64 decode the string. Using Java 8 it would look like:
byte[] bytes = Base64.getDecoder().decode("AAAAAAAAAAGZ7dFR0XmV23BR........");
str = new String(bytes, "utf-8"));
Other options: Base64 Encoding in Java
I m not sure if I understand the question exactly but do you mean this?
String decoded = new String(bytes);
public static void main(String[] args){
String decoded = new String(bytesData);
String actualString;
try{
actualString = new String(bytesData,"UTF-8");
System.out.printLn("String is" + actualString);
}catch(UnsupportedEncodingException e){
e.printstacktrace();
}
}
Sorry,wrong answer.
Again,ByteBuffer is a java class. SO they may work the same way
You need java version..
From kafka ApiUtils:
def writeShortString(buffer:ByteBuffer,string:String){
if(String == null){
buffer.putShort(-1)
}
else{
val encodedString = string.getBytes(“utf-8”)
if(encodedString.length > Short.MaxValue){
throw YourException(Your Message)
else{
buffer.putShort(encodedString.length.asInstanceOf[Short])
buffer.put(encodedString)
}
}
}
For Kinesis data blobs:
private CharsetDecoder decoder = Charset.forName("UTF-8").newDecoder();
decoder.decode(record.getData()).toString();
String request=new String("UEsDBBQACAAIAHhoEEMAAAAAAAAAAAAAAAABAAAAMe1X247bNhD9FWPfs7Ls7GYNMEwpirLZSKTCy3r9xIfEKAJkd4F4W7R/36FuFH2RW/S18IM5Zw6PRsMZUkSf/nz+Mftj//Pw/fXl4016O7+Z7V++vn77/vLbxxtrincPN58woipzin2xTBuMSF2XnBLDpXA5MQQjrh0XOX/kuSUl3qEkBqIZXBQyRhQr8KbIXC1LZ+AJh28HlBwTohmkklYYnMa0Do2Y1CrFBN1hu36K2YMH0ZIzYRyVQsO/D/8IQYRSL+1KSYQzu5rhOxA7AVFuVZ+WncbLe2DFUBxCm4qQqA5oZNXO8RwezgvOFMw7Bw6Y1paLtSPWbKTiZjfmn/qE1oUTtsoa4ciydT5yRRasDqcsDuoMJppXI2UbRWQVVPdqq9WH+XyezheyTn9FyciDDHkaBNs8gc45sCZa11KZEO8Jkiv+yFTQTo6B3q6ZqrgZEXrgUZpgBp3zsLZKkIphYUgpUdKbfYF1tK+/H95en93b/vA2VFqvUEioSZiT4h/7t9nzfvb28y9Iz4AOhAUO8CLASzwL+BKjNRM56M5R0o1QRaAIoNS0IcZqvEDJEYKgpZmThcu4MhucrpqFglaLcejx47plVV3KHVOdKSm1dbtWXavoGSwQh+cIg5KRu5nZthBEGoyg2OTxM8lIyWY5KChOTcsM7p7NwjKdILBgXOauw3O8uIMXOwahI6msmMuIyHGa+jcNNtKkJNBXRbMd+v3jPaz0MTaKrdvv8lwxrTtr4+U6aGjj0LRDnMT0LFdJYTZ+A7nssmJcopHVGk2WktG4KKVUYUpsQpYZGwke2b3ZikaWtplVmUe7gZHbpoubv379fLcNQ8XWbaf3A6BCh214DdhoXEuXyac+iJnfgsZA7+8eGFtx1rdSff5/Df77GvyrJYiTnsRdAcetIdR0Vk25qxWvfGP1h1nO4SiBw4zKnEHv3oP8NVakQxQjsfthlT7cLWOdU1YkYljJ6o0Uw65yTuOE1EhIs4Ed4XycyTVC8J8JMJn0BudpXMmU0/sqmfGSTUU9yRgRLsV92T3yXoj8otc7WUV4OdSbr6tfYCevraZE3FJ5a9etSMxrpm5ZprlheLvd3p6d0xOQvlqCyXWKnq6/5IpfT5ZeMu3W1yrzGkFPVuakV09V5pRTX63Mqww9VXqzSOBC5JdLc9Kr/2FpnvKaqZOlOSb4w2y8pSanN7fuUtdZhRW0+SJrPvgGCzHxxXKfP/gExIt5upw/pHC3ieBBCm5rTkh897753fnjJHIg/4HtZRe3/im95U8sTVXPghun/+5szq8R3LMyRQTdwJ3Ah5oOrAAjKWCxmLOadV/hyQnSAf7aAAdTHigB2erAr/k8/fCOwjhbZ7ZNw/x+tVymKzjZRkQ/ayQaW3HGk/HN+m9QSwcI78e6fhgEAACNDwAAUEsBAhQAFAAIAAgAeGgQQ+/Hun4YBAAAjQ8AAAEAAAAAAAAAAAAAAAAAAAAAADFQSwUGAAAAAAEAAQAvAAAARwQAAAAA");
byte[] resByte=new byte[11474836];
resByte=Base64.decode(request.toString().getBytes());
InputStream input = new ByteArrayInputStream(resByte);
byte[] readByte = new byte[11474836];
ZipInputStream zip = new ZipInputStream(input);
int noofbyteRead = 0;
if ( zip.getNextEntry() != null )
{
noofbyteRead = zip.read(readByte);
}
byte[] writeByte = new byte[ noofbyteRead ];
System.arraycopy( readByte,0,writeByte,0,noofbyteRead);
zip.close();
input.close();
/* String actualXmlmessage = new String(writeByte);*/
String s1 = new String(writeByte);
System.out.println(s1);
s1 displays only half of the content. Why it is not reading the full content?
Before even answering the question, let me make some comments:
There is no need to create a new String from a String constant.
String request= "UEsDBBQACAAIAHhoE....." ;
Base64.decode does not receive a byte[] to hold the result. So, it is most probably creating the one it returns. In consequence, resByte does not need to be initialized. resByte's declaration should be just
byte[] resByte;
That said, and without many more details, instruction String s1 = new String(writeByte); will use the platform default encoding (this depends on OS and OS configuration if you have not manually set it). If your encoding is such as UTF-16 (where every character corresponds to 2-bytes), you would be obtaining exactly half the characters the number of bytes in writeByte .
If s1 looks garbled (which is very different to "half the content" or even "just the content", and you should have pointed this out in the question), then this is what's happening almost for sure.
The solution is to use:
String s1= new String(writeByte,charSetName) ;
Where charSetName corresponds to the character set of the very original input (not only before it was base-64-encoded, but even before it was zipped).
I tried this code to send and receive an Integer with ASN.1 generated classes
Client sending an Integer:
ClientFirstRequest h = new ClientFirstRequest();
h.clientInt.setValue(9);
BerOutputStream bos = new BerOutputStream(_socket.getOutputStream());
h.encode(bos);
Server receiving it:
ClientFirstRequest h = new ClientFirstRequest();
BerInputStream in = new BerInputStream(socket.getInputStream());
h.decode(in);
ASN1Integer ClientNumber= h.clientInt;
int clientNumbervalue = (int)ClientNumber.getValue();
It work perfectly, but in the second sequence I have to send two argument, an Int and a String
Server sending an Integer and a String
ServerFirstResponse response1 = new ServerFirstResponse();
response1.serverInt.setValue(clientNumbervalue);
response1.serverString.setValue(randomString);
BerOutputStream bos = new BerOutputStream(socket.getOutputStream());
h.encode(bos);
Client receiving them
ServerFirstResponse response1 = new ServerFirstResponse();
BerInputStream in = new BerInputStream(_socket.getInputStream());
response1.decode(in);
But I got an error
com.chaosinmotion.asn1.AsnFatalException:
In decoding process, one of the elements of your SEQUENCE (or an element of an inner sequnce/set) is not OPTIONAL and not initialized!
(If exists)name of this element is : serverString at
com.turkcelltech.jac.Sequence.check_OptionalAndInitialized_Status(Sequence.java:259)
at
com.turkcelltech.jac.Sequence.fillSequenceVariables(Sequence.java:246)
at com.turkcelltech.jac.Sequence.decode(Sequence.java:105) at
Client.main(Client.java:54)
Please contact the vendor of the ASN.1 Tool you are using. They should be better able to how to handle errors in use of their ASN.1 Tool. Each ASN.1 vendor writes code differently even though the end result should be the same encoded stream of bytes regardless of which tool you are using. Note that you have not indicated here which ASN.1 Tool you are using.