Develop Reference

How to get two-sequence representation of UTF-8 character using JavaMail's MimeUtility or Apache Commons and quoted-printable?

I'm having a string which contains the German ü character. Its UTF value is 0xFC, but its quoted-printable sequence should actually be =C3=BC instead of =FC. However, using JavaMail's MimeUtility like below, I can only get the single-sequence representation.
String s = "Für";
ByteArrayOutputStream baos = new ByteArrayOutputStream ();
OutputStream encodedOut = MimeUtility.encode (baos, "quoted-printable");
encodedOut.write (s.getBytes (StandardCharsets.UTF_8));
String encoded = baos.toString (); // F=FCr
(Defining StandardCharsets.US_ASCII instead of UTF_8 resulted in F?r, which is - obviously - not what I want.)
I have also already taken a look into Apache Commons' QuotedPrintableCodec, which I used like this:
String s = "Für";
QuotedPrintableCodec qpc = new QuotedPrintableCodec ();
String encoded = qpc.encode (s, StandardCharsets.UTF_8);
However, this resulted in F=EF=BF=BDr, which is similar to the result Java's URLEncoder would produce (% instead of = as an escape character, F%EF%BF%BDr), and which is not understandable to me.
I'm getting the string from a JavaMail MimeMessage using a ByteArrayOutputStream like so:
ByteArrayOutputStream baos = new ByteArrayOutputStream ();
message.writeTo (baos);
String s = baos.toString ();
On the initial store procedure, I receive a string containing a literal � (whose correct quoted-printable sequence seems to be =EF=BF=BD) instead of an umlaut-u. However, on any consecutive request Thunderbird makes (e.g. copying to Sent), I receive the correct ü. Is that something I can fix?
What I would like to receive is the two-sequence representation as required by IMAP and the respective mail clients. How would I go about that?

Java UDP - Sending a string array from server to client

Hi and thanks in advance,
So I'm trying to take an array of JList items and convert them to a string array (which I think I've gotten right), and then I'm trying to send that string array over to my client who will then attempt to display them back into a JList on their side.
I've tried a few different things but none are working.
Here is my latest code attempt to send the string array over:
String[] FilesList = (String[]) lClient1Files.getSelectedValues();
FilesBuffer = FilesList.getBytes();
DatagramPacket DGPFilesResponse = new DatagramPacket(FilesBuffer,FilesBuffer.length, DGP.getAddress(), DGP.getPort());
SeederSocket.send(DGPFilesResponse);
The line: FilesBuffer = FilesList.getBytes(); is causing the issue because getBytes() isn't applicable here.
So my questions are:
1) How do I send the array of JList items(they are names) over to the client (it doesn't particularly have to be a string array), and
2) How would I receive the list on the clients side, so that I can use it?
Thank you.

One must make a binary format for the string array.
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (DataOutputStream dos = new DataOutputStream(baos)) {
dos.writeInt(filesList.length);
for (String files : filesList) {
dos.writeUTF(files);
}
}
byte[] bytes = baos.toByteArray();
This internally for a String writes first the length in bytes, and uses String.getBytes("UTF-8") so any string can be written.
Reading goes with the reversed input classes.
If you think of having many clients out there, maybe with different versions,
then add in the message a version number.
On the other side
ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
try (DataInputStream dis = new DataInputStream(baos)) {
int stringsCount = dis.readInt();
String[] filesList = new String[stringsCount];
for (int i = 0; i < stringsCount; ++i) {
filesList[i] = dis.readUTF();
}
return filesList;
}

The UDP payload has to be a byte[]. You need to choose a way to encode your data into a byte[], such that it can be converted back at the receiving end.
So you need to write encode() and decode() so that unit tests like this work:
#Test
public void encodesAndDecodesStringArray() {
String[] strings = new String[] { "foo", "bar" };
byte[] encoded = Encoder.encode(strings);
String[] decoded = Encoder.decode(encoded);
assertThat(decoded, is(strings));
}
There are literally hundreds of encoding schemes you could choose from. Delimiter-separated, length-separated, JSON, XML, BSON, ASN.1 ... take a look at Wikipedia's List of data serialization formats.
A very simple option that might work for you is delimiter-separation:
public byte[] encode(String[] strings) {
return String.join(",", strings).getBytes(UTF_8);
}
public String[] decode(byte[] encodedArray) {
return new String(encodedArray, UTF_8).split(",");
}
But note that this very basic scheme fails if any of the input strings contains a "," (or whatever delimiter you choose). Pick a scheme that works for you.
Consider using JSON -- there are easy to use libraries to read and write JSON. Readable ASCII in network traces is often convenient. The space overhead is not that high. It's ready for arbitrarily complex hierarchical data structures.
Consider that if you change the structure of the data produced by your sender, the receiver must also change. If that matters, consider encoding a protocol version into what you send (it might be enough to just say "the first two bytes are the version", and always stick a 0x0001 in there to start with).

gRPC-java - Compilation of a .proto file

I have compiled my .proto file using the protobuf compiler and received a selection of Java files. I received a proto.java file and a .java file for each item in the .proto file, including the message type and each RPC call e.g. publicKeyRequest.java and Quote.java as the RPC and request parameter type.
Is this all the files that are needed as I still cannot seem to to get any simple response back from the server?
I want to generate a request for the PublicKeyRequest RPC call. I generated the request object, but I do not know how to actually send it via the channel.
This is the full .proto file:
syntax = "proto3";
option java_multiple_files = true;
option java_package = "io.grpc.decryptiondevice";
option java_outer_classname = "DecryptionDeviceProto";
package decryptiondevice;
service DecryptionDevice {
// Decryption Request RPC
//
// Request contains ciphertext and proof
// Returns the plaintext record
rpc DecryptRecord(DecryptionRequest) returns (Record) {}
// Get Signed Root Tree Hash RPC
//
// Caller provides a nonce
// Returns a signed RTH and nonce
rpc GetRootTreeHash(RootTreeHashRequest) returns (RootTreeHash) {}
// Get Public key RPC
//
// Returns a Remote attestation report containing the public key as user data
rpc GetPublicKey(PublicKeyRequest) returns (Quote) {}
}
// Decryption Request
// - Byte array containing ciphertext
// - Proofs represented as JSON trees
message DecryptionRequest {
bytes ciphertext = 1;
string proofOfPresence = 2;
string proofOfExtension = 3;
}
// A plaintext record
message Record {
bytes plaintext = 1;
}
// RTH request contains
// - A random nonce
message RootTreeHashRequest {
bytes nonce = 1;
}
// Root Tree Hash
// Random nonce used as message ID
// Signature over rth and nonce
message RootTreeHash {
bytes rth = 1;
bytes nonce = 2;
bytes sig = 3;
}
// Public key request message
message PublicKeyRequest {
bytes nonce = 1;
}
// Attestation Quote, containing the public key
message Quote {
string quote = 1; //some format.. to be defined later
//PEM formatted key
bytes RSA_EncryptionKey = 2;
bytes RSA_VerificationKey = 3;
}
And this is the code I am attempting to run on the client side:
public static void main(String[] args) {
DeviceClient client = new DeviceClient("localhost", 50051);
MannagedChanel channel = ManagedChannelBuilder.forAddress("localhost", 50051).usePlaintext(true);
ByteString nonce = ByteString.copyFromUtf8("someRandomString");
PublicKeyRequest keyRequest = PublicKeyRequest.newBuilder().setNonce(nonce).build();
// Here I want to send this to the server
ByteString response = DecryptionDeviceProto.getKey(keyRequest, channel);//this line is not even close to being valid, but this is the sort thing I wish to achieve
Sys.out.println(response);
}
Apologies if this is very wrong, I am new to gRPC.
A few points about this system:
A client and server has already been written in Go which has been tested and works with this same .proto file.
I am attempting to rewrite the client in Java to communicate with the same server.

There are two sets of files that need to be generated: Java Protobuf and Java gRPC. To my knowledge, for all languages except Go, these are two separate generation steps (that can be combined into one protoc invocation, but they are conceptually separate).
It seems you are generating the Java Protobuf code, but not the Java gRPC code. You need to use the protoc-gen-grpc-java plugin to protoc. If you are using Maven or Gradle, read grpc-java's README. If you are running protoc manually, you can download a pre-built binary from Maven Central and see an answer to a similar question.

Sending the length of jSON array though sockets (java to python)

I am trying to connect with a python server (from my colleague), with java. The aim (for now) is to send a json array. We start by sending the length first. It works with an equivalent python client, which I am trying to translate into python.
This is an excerpt from my java code
PrintWriter out = new PrintWriter(clientSocket.getOutputStream(), true);
long length = (long) arraytosend.length();
out.print(length);
String arraytosend = new JSONArray(test2).toString();
out.println(new JSONArray(test2).toString());
The python server first reads the length like this (I just copied the relevant commands de):
N_BYTES_MSG_LEN = 8
raw_len = connection.recv(N_BYTES_MSG_LEN)
# here it output 51 as raw_len
try:
msg_len = struct.unpack(MSG_LEN_TYPE, raw_len)[0]
print msg_len
logger.debug('announced message length: {}'.format(msg_len))
except:
logger.warning('could not interpret message length')
return None
# read the incoming message
raw_data = connection.recv(msg_len)
if len(raw_data) < msg_len:
logger.info('lost connection')
return None
After the "51" it immediately goes to lost connection.
The python client code (which I am trying to translate into java), works like this:
try:
raw_data = json.dumps(dict(data=data))
except:
logger.warning('Failed to create a json representation of the data')
return False
# TODO: this could fail for *very* large objects
raw_len = struct.pack('Q', len(raw_data))
try:
connection.sendall(raw_len)
connection.sendall(raw_data)
except Exception:
logger.warning('lost connection while sending data')
raise

Your receiver is assuming the length is expressed in 8 bytes (N_BYTES_MSG_LEN). But you send the long as string. PrintWriter.write(long) is the same as PrintWriter.write(Long.valueof(long).toString). For example if the length is 356 it sends "356". You should lef pad your length first: "00000356".

I found the solution, you have to take into account that java uses. You can do this by changing the server (python) code to:
raw_len = struct.pack('!Q', len(raw_data))
And you can then send it with:
JSONObject request = new JSONObject();
request.append("data", array);
byte[] outBytes = jsonObject.toString().getBytes("UTF-8");
out.writeLong(outBytes.length);

Array of bytes: how to know exactly numbers of byte received

I'm using array of bytes to store data packet received from another computer.
receivedData = new byte[1024];
receivedPacket = new DatagramPacket(receivedData, receivedData.length);
socket.receive(receivedPacket);
receivedData = receivedPacket.getData();
String res = new String(receivedData); // PROBLEM HERE
the problem is at last line: because I declare receivedData as a byte stream with length 1024. so, always last line will create a new string from whole array, although it doesn't know exactly how many real byte I received. So, I meet frustrated error: res is not received as I wish. (because length of real bytes that I received not fix whole array)
So, my question is: how can I fix this point, how can I know how many bytes I really received to convert to string?

Try using DatagramPacket.getLength().
receivedData = new byte[1024];
receivedPacket = new DatagramPacket(receivedData, receivedData.length);
socket.receive(receivedPacket);
receivedData = receivedPacket.getData();
String charsetName = "US-ASCII"; // set to desired charset
String res = new String(receivedData, 0, receivedPacket.getLength(), charsetName);
Edited to add charset. Thanks, parsifal.

From the javadoc for DatagramSocket.receive():
The length field of the datagram packet object contains the length of
the received message
You can then construct your String using the constructor that takes a byte array and offsets.

Call DatagramPacket.getLength() to find out how many bytes were actually received.
And when you construct the String from those bytes, be sure to specify the encoding (as it is, you're using the JDK default encoding, which may differ from the server's encoding).