Develop Reference

Lengthy string compression/decompression in Java

I am looking for String length compression to avoid lengthy filename as below. The string contains UTF-8 characters as well.
"dt=20200623_isValid=valid_module_name=A&B&C_data_source=internet_part-00001-1234-9d12-1234-123d-1234567890a1.b001.json"
Tried Huffman compression from GitHub here, it reduces the size but not much on the String length.
Size before compression: 944
Size after compression: 569
Compressed string:
01011111001111100011101000111011101011001000111110001101000011011001000110001111010001010111111001010110001111010001010001101101010000101101110001110000000110101011010110100000111111001101011111100111101111110100000010101011011110011000010011001000101110010011101001000001111101001010111110000001001101010000111100001110101001100100111110001011101110111011101001001010011000111110111000101100000101100110000010100110001111101110001010011000111110101001010011000111110111011010111011001101100110110111000011100110100111000111011101110111010011100011101111001100100010101
Please advise how to achieve length compression in Java? (The decompressed file Name value is needed for further processing).

You should try ZLIB/GZ Compression. You can find GZ compression snippet here compression and decompression of string data in java
ZLIB compression implementation is also fairly easy. You can use the below code as a starter and improve upon it.
Detailed explanation on compressions How are zlib, gzip and zip related? What do they have in common and how are they different?
Read Deflator strategies before proceeding ahead: Java Deflater strategies - DEFAULT_STRATEGY, FILTERED and HUFFMAN_ONLY
public void compressFile(String originalFileName, String compressedFileName) {
try (FileInputStream fileInputStream = new FileInputStream(originalFileName);
FileOutputStream fileOutputStream = new FileOutputStream(compressedFileName);
DeflaterOutputStream deflaterOutputStream = new DeflaterOutputStream(fileOutputStream))
{
int data;
while ((data = fileInputStream.read()) != -1) {
deflaterOutputStream.write(data);
}
} catch (IOException e) {
e.printStackTrace();
}
}
You can decompress using Inflator.
public void decompressFile(String fileTobeDecomporessed, String outputfile) {
try (
FileInputStream fileInputStream = new FileInputStream(fileTobeDecomporessed);
FileOutputStream fileOutputStream = new FileOutputStream(outputfile);
InflaterInputStream inflaterInputStream = new InflaterInputStream(fileInputStream)) {
int data;
while ((data = inflaterInputStream.read()) != -1) {
fileOutputStream.write(data);
}
} catch (IOException e) {
e.printStackTrace();
}
}
Refer: http://cr.openjdk.java.net/~iris/se/11/latestSpec/api/java.base/java/util/zip/Deflater.html

Of course using one character per binary digit is going to use up a lot of space. That library is using 16 bits (the size of a char) to represent a single bit, so it is literally making its result 16 times larger than it needs to be.
A far more compact way to represent binary data is by converting it to hexadecimal.
byte[] compressedBytes = new BigInteger(compressedString, 2).toByteArray();
Formatter formatter = new Formatter();
for (byte b : compressedBytes) {
formatter.format("%02x", b);
}
String hex = formatter.toString();
Then the result is 142 bytes:
BE7C7477591F1A1B231E8AFCAC7A28DA85B8E0356B41F9AFCF7E8156F30991727483E95F026A1E1D4C9F17777494C7DC582CC14C7DC531F5298FBB5D9B36E1CD38EEEE9C779915
You could even go a step farther and Base64 encode it, reducing the result to 96 bytes:
String s = Base64.getEncoder().encodeToString(compressedBytes);
Result:
AL58dHdZHxobIx6K/Kx6KNqFuOA1a0H5r89+gVbzCZFydIPpXwJqHh1Mnxd3dJTH3FgswUx9xTH1KY+7XZs24c047u6cd5kV

Why is my binary data bigger after getting it from the webserver?

I need to serve a binary file through a web service implemented in Python/Django. The problem is, that when I compare the original file with the transferred file with vbindiff I see trailing bytes on the transferred file, sadly rendering it useless.
The Binary File is accessed saved by a client in Java with:
HttpURLConnection userdataConnection = null;
URL userdataUrl = null;
try {
userdataUrl = new URL("http://localhost:8000/app/vuforia/10");
userdataConnection = (HttpURLConnection) userdataUrl.openConnection();
userdataConnection.setRequestMethod("GET");
userdataConnection.setRequestProperty("Content-Type", "application/octet-stream");
userdataConnection.connect();
InputStream userdataStream = new BufferedInputStream(userdataConnection.getInputStream());
try (ByteArrayOutputStream fileStream = new ByteArrayOutputStream()) {
byte[] buffer = new byte[4094];
while (userdataStream.read(buffer) != -1) {
fileStream.write(buffer);
}
byte[] fileBytes = fileStream.toByteArray();
try (FileOutputStream fos = new FileOutputStream("./test.dat")) {
fos.write(fileBytes);
}
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
I think that HttpURLConnection.getInputStream only reads the body of the response, or not?
This code serves the data in the backend
in views.py:
if request.method == "GET":
all_data = VuforiaDatabase.objects.all()
data = all_data.get(id=version)
return FileResponse(data.get_dat_bytes())
in models.py:
def get_dat_bytes(self):
return self.dat_upload.open()
How do I go about transferring the binary data 1:1?

You’re ignoring the return value of InputStream.read.
From the documentation:
Returns:
the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached.
Your code is assuming that the buffer is filled with every call to userdataStream.read(buffer), instead of checking how many bytes were actually read into buffer.
You don’t need to read from an InputStream at all. Just use Files.copy:
Path file = Paths.get("./test.dat");
try (InputStream userdataStream = new BufferedInputStream(userdataConnection.getInputStream())) {
Files.copy(userdataStream, file, StandardCopyOption.REPLACE_EXISTING);
}

You always write a multiple the 4094 bytes, no matter how many bytes you actually read.
Don't do .write(buffer); write the amount you actually read. This is what userdataStream.read returns you. It can return a number smaller than the buffer size, but still positive.
If you project is using Apache Commons already, you can just use copyInputStreamToFile.
Note: 4K = 4096, not 4094, and it's a ridiculously small buffer, unless you operate something like a smartcard. On a PC, use something like a few hundred kb at least.

Java byte array compression

I'm trying to use the java DeflaterOutputStream and InflaterOutputStream classes to compress a byte array, but both appear to not be working correctly. I assume I'm incorrectly implementing them.
public static byte[] compress(byte[] in) {
try {
ByteArrayOutputStream out = new ByteArrayOutputStream();
DeflaterOutputStream defl = new DeflaterOutputStream(out);
defl.write(in);
defl.flush();
defl.close();
return out.toByteArray();
} catch (Exception e) {
e.printStackTrace();
System.exit(150);
return null;
}
}
public static byte[] decompress(byte[] in) {
try {
ByteArrayOutputStream out = new ByteArrayOutputStream();
InflaterOutputStream infl = new InflaterOutputStream(out);
infl.write(in);
infl.flush();
infl.close();
return out.toByteArray();
} catch (Exception e) {
e.printStackTrace();
System.exit(150);
return null;
}
}
Here's the two methods I'm using to compress and decompress the byte array. Most implementations I've seen online use a fixed size buffer array for the decompression portion, but I'd prefer to avoid that if possible, because I'd need to make that buffer array have a size of one if I want to have any significant compression.
If anyone can explain to me what I'm doing wrong it would be appreciated. Also, to explain why I know these methods aren't working correctly: The "compressed" byte array that it outputs is always larger than the uncompressed one, no matter what size byte array I attempt to provide it.

This will depend on the data you are compressing. For example if we take an array of 0 bytes it compresses well:
byte[] plain = new byte[10000];
byte[] compressed = compress(plain);
System.out.println(compressed.length); // 33
byte[] result = decompress(compressed);
System.out.println(result.length); // 10000

Compression always has overhead to allow for future decompression. If the compression produced no reduction in length (the data was unique or nearly unique) then the output file could be longer than the input file

Encoding a file to Base64 in Java

The Imgur API allows you to send a Base64 of an image to upload that image.
I have tested that this works when I send a Base64 that looks like this:
/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAQDAwQDAwQEAwQFBAQFBgoHBgYGBg0JCggKDw0QEA8NDw4RExgUERIXEg4PFRwVFxkZGxsbEBQdHx0aHxgaGxr/2wBDAQQFBQYFBgwHBwwaEQ8RGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhr/wgARCADIAKQDASIAAhEBAxEB/8QAHAAAAQUBAQEAAAAAAAAAAAAAAgABAwQFBgcI/8QAGgEAAwEBAQEAAAAAAAAAAAAAAgMEAQUABv/aAAwDAQACEAMQAAAB3xJsmd2cDMwdZG4lK6OC1HHQKytQCJEtwESpVG0o0pZiatCTosZEs8JMwmaTpN3EpHvn5/K2zdJy1HM73Gt38ysDPby5nU+f7GixpgxCYNA063zJ1uMzpZO7FK9qV/njHz+4HKfU/P6RySkVbN38WWrrPUPFfZ+bSTCKSKMotGZCtGRJY5kSHWd0hr8X03nnX5WW0dLt886bzRXa/OjBPT23q3kXrnPcIOwKYXbRlSW4STrqRCQlHxuz5xbLf2uLqdLn3saWibtCsqiHzRSW9zZ9k+fvf4CZk6NQkzAJOjE3EpL0TIC5nzDp8rpxScnfvUr5jpOf2+b0j5/0DilHT1MvoOhFv+mmISMgYSkYC3JUK9khC8PSeCVtHwja9Z4WpPHtpY3Y5udYq2Of0uqwppiVh2NLI97rN3ziQfe4FAbIHmiMGzKNAcwuEV7uDbkgZGc+fice/X6k+S4DPb00sk7YMXE2MQK7EwWLQ9D3fNu3Lj7FnIeZmms1Du6IhyOy+Jq1Go5XmpanWQGfNW8+vKHUx1WK82K2CpUKYrGmE7U6HU8l0LuTu2MMRk21hrC7sYF833ZeT2+YuloUekxevJnUrtYr87o+eXMt7TmN/ApgrEJnXLNHJQi1dzkUutJnWsluqJYHWDZo/O9qEZshwrmpArySrHI1ma99sO9lyti6ZWg1kclmfBoLaqblPSjLV6ayFqaUTSIsrjNbzc+DoNNXuPt+l3Jm+Ryekc60OSkqgL9EI2NcoyX2rxA1avjid2LLCSYMQJATMkkpbiSR1NJLF3bKU73hSW3PrpMXkUkujPXFI2iSXvTpIs//xAAqEAACAgEEAgICAgEFAAAAAAABAgMEAAUREhMQICEiMDEUMgYVIyRAQf/aAAgBAQABBQI/gOFgv/UOFQweRqOKyyLm3uPxT3IK2SaqhR3mgyaUyitrNrTGH29h+HVdSGnw1tMaYWrK1ceZ2jSytpZERydbqwLTsS2l9B+C3aSlBNK6yLqfPJv91hUmnjSBHWzI3Zo1RdQbbbyfAP4LNfvu6v8A8nIf615atbBfV3mkZmuXxbbR9VSlMjLIh8HwPwW7R7ZJDztNvhiV8hM8Uh1qeKCOHnlPTa1iyiLEh9B72pOC3mbuUczqBys7Y0rKHmbn2HNFkMOE+o9ZXEUer6hLUiGtuLEnVqAsjqaQgYrAhz9Jf7gEmkVjqn1HrrL8KFlXntzVVQizJXyzaM2OeYAkhDH6/vO9Y0gbY/seg9das9C2u8WJ7EVlSuA5FIKMc2pyyq42YZXhDNFoc0zj03wet7Vkm1SYu4WBe+emOHHc0qy3JX0iGOOwpOKvLKEUsGoHxvm/kZ/74nhWzDqGnTaXOkjT1KoMzTyFDZkWQ6ezJNPfO8tn+SbNdqsnYrw1dclow6drpv2fIwessSWI5f8AH5IMnPSYlByReOUjtZnrdss8TxVdSjEMAtfHNJEou1a7yzfzy8b+d8bbjfIFwHi8u/KBuuZgI7E47U1BjIi4V3yAmKSjb5DnnPA/x3ed8LBR/qUJNyUyh0M0srOCx3K/2dmsXZ3/AJmXCP4wwHFxHxJd85Z2YD53y+TKJmrUjPdknZQ0UUkhYZHG8pnbg47ZVsScmxfEB5urbYr5yztXN83w40XM3XFizJ9cctJnDxDO1YytwiktPJHtgHmEcWD74JNsMhODN838ahZ7wExo/s2N4jjSfNQ2lsSrw8gYBiHjgbfA2b4pzfN8ml+oQ5InGN3GEjG/RwFkxjku/lfA+cL8cjl5YDi/rnhfGO+KjHJuxskjZGSu0iupUEfIXJJfrM/OXxvm/gnF/atviRuQ0UKY1+jC51KBRJ/kJAk1buburtkViujm/wDLOpWMDeJxJYMBOCsxzpcmOGNMjfjhiebDNHGWuOuHUSSl6cDunczrxk4kZt9AN1PDFh7MbTbSKr74ejD1jOaAGzI+bnN9g05fC7b7nOTkYv658caXngkwToypw2jr0SKywonRJjcVOoR1rKvDJHiI7ladhgtaVhHSndlpPDNNAVxY/k/Bxf0fRGAxTkVnhi2WlwCAZzhK7xLgk45JAJmem6l0ljLuxzkc+fK5/8QAJhEAAgIBAwQBBQEAAAAAAAAAAAECEQMSITEEEBMiQRQgIzJRBf/aAAgBAwEBPwEQl2jwRlq2+RtDJMfdCQjqOq8G0RPq+r/J8Iw5Pp/U80XtY/sXaeTxx1Eox6iLyXuR+p/TEmxdN1OKX5ImCOhPahyHI1vuj/Qm6UUYVc1EhPJhnUWSyZckd2eVz/Yl9ubLoMmrMvUw4nGrPE1M3SpIrTLs2X3zTWN3W4+pcpe8rMWVY/atyeVzfsZVq2TE9jUN99VGWKyqnyJPU5SFJLHsZJR1xmRl5oKQoLQ75FIRqRqssnljHgUlwhcDfqr5MSrHRJuqY7iLLR5WWZLmtIqTIJcvkerS5RRKLyVXBdIZ1K9tiy+2bJKNKJFyTIErlGk6MFx9RjslDUShT3KNMmeD+s8DxnhnyWxRjB9mkabJ4tR9Oz2s1NfBHKrPJI9L/hoUvkcYobhHkU1Ww3fdE1RqIwixbbIeKMuTQlIpLv8A/8QAKREAAgEDAwMEAQUAAAAAAAAAAAECAxESBBAhEyIxFDJBUQUgQmGBsf/aAAgBAgEBPwEY3YZLySg4rL4I3FyU0LZjJkmaXRLULOo7L/SNHR6VdP5f9lektTyenqRWTVkRQt2O5Tp9Waiym3SkoW4JuhfKfA9RQmu1mqm5zTvdCW1t5I/G0025MrWUMidOFaOTFSpwZVji+BbW2uUKSmrsgoU+SrWUoux1k6Zw+St3JGIkW308M1Zvg6EYxtFWK9NyWJSpqKeJTniuUT7mYiW9ilU6bLpxUUTp9/cU4NRlFGPTk0ifv4Ghli20KM2+RQ+yfkV8nbwfvFThJ3JwHSudJbUXjO4o9t2YtLgbSdmU2oXy8kaeTIRsi1zEx201OL7pEWpokWtK7HaTuIX6LRQpv4RS1P8AB6uHi52+RO5kkepUT1cfoWof0dd/RihokuB2+hVJs6rXwRnx4E5P2qw4v5ZGKWzJu22JObTG8uWRqyj4KcnKN3stv//EADkQAAEDAgMECAQFAwUAAAAAAAEAAhEDIRIiMRNBUWEEECAjMDJCcVJigZEUcqGxwQUkQDRDgoPR/9oACAEBAAY/AvBzGJ/xSHXBUvmp0bjqWf8AoQdTIc06Ef4ffVADw3qabMQPxJx6Njo03elFrnVGk6xchNpdN/uKJ0M3hSPHy3rv8g/lbbpjy2bx6ioAw8ghU05ShrS6QzeN6pisAy0S391TpjHVqBgs1q2j6QpUz5byT4rq1bQbuJQ6Z08HaVPIPhCyuWKpm91jkQ30jVQG4ag0KbbDhH68VtKsRT1HxeM19e9CiyWt+birDEWFWELH0kGo/wBIjRdw18i86Qm1W0mMqDWH+ZS7o7abvlKG0BbTd5iE19MhzXCQR4tUc1ZMcysKWIZmOEoFrg+LwE/8OddRhlOpfhaTKjrh3D2Cx1DiJXRqL8WEhz44t4ff9kGUmhrGiAB4jGDzVXhgVQnQuUnRNwnRBxO9Oc0kGUZvPFRJA5J1dzC8Me2HE6Tb+fDfUdo0SqFPo9QMqk5nR910d3SO+2eLdGqlpwOdeCsKGO4QwiAvdysoGqqhzjtdqy07vDdG9zf3VXaEmHw1dxinfKw1BdB+/emrvmOa3cS1fVcFhot93HVAbyZ8Og1tPave/K3mielEea7W2UU6DWTreU8fDp1DBeq5sudw5LDixDgVbRQgagy8OKa5tP8ADs4uP8eG3Y95ToMMc3Jxq0g1wqEGCg1/lQq0icKy/qs7sO4ysYcXmm7NukLa+lxhWIXRNocu0B80+HUo1JwvEGFD/L6HjQqv0lhwRGPfdF5uUNkfJqm7MRa/ui9mjWy7mE+k8ta0C7h601gaG0+agkOadCELOxjeoezbt9Mu0WxfRbTkWgz4Rp1mh7DuKqH+m14a4Xp1Lgo0DkdGbCJKwO8uvUy0zZB/RyHMd6SYw8kMoADrnnwXRodLt/JZqbZ+IZT+iIxFh+ZUX8HeG6dIVTBmMAD7KbHksxVN/wALgVWBGRzo9l0cB8tqX9k10HAHW59YcEKbzf09d1bsS4wBvKIo964cFedkWHEAdEXQiYa0clZCeKqB4AZtJWJginQZAMalUWjj13QjwNno3UpoawVDq7Erd2zSyzceuGAn2RDcoc6ahP7KGnZ0twWEOxNbp2GjruVr2CSYtvR2Y7oWHPmhyupPXT6LSAa5zodUHFHaDFDjZO9IOUAdmfA2TDFP1c+ok9ilUpk4gZeDxQAEYjKazl2Z7ZjtywkIVJmKN/eE3F5sPbjf2rAqGU3n6I7QYTzU0ofHA3WYR11Kd5pswpx7Y6rNcs9YALB5zxlGB0dhHGpiRwODv+uFNXG4/mhHHivoZXfMe9vyvhRTfUawaAmV/t3+6xGHNbrdVHVJzAmykTh4wsha/wCqizfcqa7sfysP8r/TscD8TUI6LTbz8v8AKhpph/yjEm7N9ZzeYhHK76uVnKNq4/VHeFpog6LTCxYRrC0IXd3PBYzQfH5VhNLEQodjpv8AZZak/wDFZXOPIrNgA/KF5h9ld6AMmNFfd1AF7oHPsZldNY6QAspOt4K7ym+fdYaDHUx+XVTmZ+iO0dM8Chs21GVx63b/AHWZpUU2uceQU7M/VTEDnZNlhAO9ZwOQlGW390MWWVrPavP2W9aFWY76PKG2bUI5PXleowx+YSsmL7QskgLFhzcVElvu1XB7f//EACcQAQACAgEDBAMBAAMAAAAAAAEAESExQRBRYXGBkbEgocHR4fDx/9oACAEBAAE/Iduow9YTmcdDJgKh4uU/i3MyuoMdvUh1OgigVI8k8ez7r/0hnXtFjK4dxR0qVE6PQ8/hfUjL4H8c/AmNv0HL2g6M2vB/3zCCHmP/AAeSYbtet7h/jEA0IJKiSo9TuV1Iw6EccTs+XiKvcFfPaNXCYN0cnPGR9ZWaXh6p3P6S3qLv3iEosHWxyuIq1GPqfp0qPQYj+bydvlwBB+XXtmsfUMW9ncjKi42Qroav3AinePJGG0EVd+UEaVh7RlMAAMAS6l3FF6PP5Ed4EP0tcvQIXHIgXlitCkeWZgDApPJ7yvd9EMuMFselUSuQ7eSWNByP1LUJEwk2lzToMda6V0CMzAKexqDbi3dygG8Qnx4irC6JVy4q/NDskZeolqxtIN1LtO8s9cBFXLvlcUzENQEwhF6aznrUqGZa9Kx3nb8DDFojiOqUOYJfIEfGYD0p6RxbZY+YxlUFgs1AaHaopIW8hD3F8Shi9HppOepNrHftB4KvDxeEqrZ0tUVdHY+4nL/6B3j8WlItRBqqvEOCinSChVaqWcEa45382TdjCPRY/Ai2WRBV32SjZb0jVTRCmTsWCXTYZQHjfnzNaywWPjIEvrA4unLNWl85PMz+2v7Lo9DoR6Cc9SPjpuJ7mFuwSyhnUw57J/R2mO1vK0oUdVmBIAthtw7SzbGxu4SHmrt4jWEJPZM+E2228r22goDsV+JTn8K9AF8YsnxUC+TcjmvY5jcassMUAoNuI2z4YSs18WHHFzT7Smh481H++DFVXeWgA+Ygha0EVf6mDF9p5TKXCUj1ZeXuUzfJuNh/3xAxqywYf937RaV9TJvla8W8S2AByhOM4avTZXvK0W5L457VKJ7hbfOIJyA5xKxY+LWVEGYGh85WwqtVZxOY4MS8wYgjvpcGbvalExKI/oL/ANmPWFtlZDx5m0DKy/cv+sq3LJdWuv3U2NLlFy77Q0xyZThAgQ8HkWRcE0qn5NH4mVw0aPuRl6PJkrmZacTLcMy6htqXMJcuEMM6st7Q5bdHcpuNstUHIRt5mWtr4ZmvjF3llKgWiGjvOn6gkCg3hX/EFsCA8xeZrybPJ2h2MKwYV7GLF6HhDWpQRuM2qwH3leQENLd757RW20luKO8El1WZ0xTsqX8zPIn7DUuOPCyTaveEN1t3o3FUCpWyGxdNlPaFslzwYrXRXMuMlLYfUlATdge8aJx0McdorW02LeZWPYCaipaFtLxNlnxFp/H3D6CJx3hG++N5gdBiZgDMslRNyoWBKuM5RjeVaApeh5Yi1Lb4/wBSli+Lu8EQz+DtGKzU5G8SV58Q+HGtoK/2Z+8Yq5g6ncldoCtRxGUeirH8mt1HY+sTMZcdEWafr6THqDuHiEuco7jVaHkEeUTR27w6HF31ehLYR69IAWRITh9+mpzBCeUjnJLMKvmHKUuK6QqOWXwyrhVXoQW0oKdu366GIsQaipiaW2Bkx0H3RY5l8sZmUUeC43XjqSqpciicg2A/TcVKjWTmP0F9ujy5KP7Kp1gPYhLgYQKalm40Yj+qWVR4JQLnxmN86eAisQWNw9gnjzKH2YlPxEFPapRecXk1HPbrc+puipUzh9xp/kxMxZxxCwzr94yulycCWo4KPwwudx2yLY2sbz5gvQax+JYyNUco4MQrP94g+EN1fbcshrKNv1KzSXM1FdFyFO7MLmXk4Bi+TPIhmkr3OZa+8piqnnxXM0V93aJk+ADZLii9m7B7bhfKTmxDJR2MQurvxfGJj1wu6zwE7EEFupiDT3Etzbc7cQFYhfdYcvWFmA3HNjXJAOTCCTs1uEgbSwOd8Q2GteL39Qc6OaZ8jL8ygTRwv/KKxuTklXq1ZjwH9imN5rHzPCwmYoGPig/DmMhUat7QW7VcQ/lDdR/sc3ra0ZgUHkwiBR3lx5esDbh32mez8Sns/EKC/YI1gL8TuJ4zBOO8VoR2zV+mPDxeT/Y1Q1dqoiwQlPN+5n5jtLaat8oIyctRngww2ka9I+T4jZ0yns/Ep7PxC04d9p//2gAMAwEAAgADAAAAEHFPQc6ZbIw/crMYmGT4O7zlCwthnch91AzB56vC3beuga3fk8XEe27QmZ37MgnELGj6b84Oqg8brPkMjMT47LLSc4PjMxgrioJcI8O1YoOb36zLZ7iejp+n+gMpFebIupMFsXHXvvfn/IIgP//EACcRAQACAQIEBgMBAAAAAAAAAAEAESExQRBRYbFxgZGhwfDR4fEg/9oACAEDAQE/EA4LIFkVYQ07Wp8+HWXKuBjaQSmBmoOEc1MCpX2luBd7Qr2Y+QR1TXa7HfvLYLKX5c47jx1QohPD4VGnPxLgOPp5ENr086xzrMEzdeutle2+NtZW1/gDDHLq7/EEczu0PcqLtJo8vSHilcuV7wFStKPQx+4rY6SyVKiOjFU9/mIAreTh8Se6EK9GrimsPseL9ZZBctMXgWgcKiOWryrY01yv5YFssFYwctI0vQ++cIO/T75QbDF9fKOAVvrrAxGWxmSAcA0vSv3KtUbHTJcqGmolRmis6bLXREHeWMZz3SDqYXLiPnwLCdUEXKI3FM2w0TpBrWGmNko89MeMoAVV9/rBPIRLG4iaLeEZQufL+9YBttQ5wqvS/jkW9IkK9TTXgc4gg2jHMYLpgyXls35Mq5qxJQwZOf38wpZwd8vavU4LmW+seocJjiTN7BBmMr0gWiVlJmM7wp3198xp3nPwawR9cOcRRiYmryg3dnrOQ2eGZYCqOi/uOZn98I5qE1NbG2ETgAgFzAeIoIbaQDoiaiFIGk2kTHD/xAAlEQEAAgIBAgYDAQAAAAAAAAABABEhMUFRgRBhcZGxwaHw8dH/2gAIAQIBAT8QfBUibh1SGBlaePM9Tk3AC0xHSXt+Bq/EQExXHaNJ1X+QcKKMNpeN2fRFLUjg4p6Vx8S6pjWevl++0oPBcuyaRpVh6Rz6RtjDGIVoI9/fMoZ2dBzXYmpEwdG8/wB7TFAb8SWTihu5rHvGpceaY7QAe4moviHhYbZrAz4FrwO+a8elRymPTU2AQ7MTkchn1iOtolFGrfqClO5WMrMQdBrzva71j+EKt92/eDrY5/ekCOO41yoc8d481AdCGMplEIWjW9dzkg5vn2jPUjNUK3jdef1HhrHwMsh0YrDLzGBWoL0I3YwC27hsDz5rPPbefSWZrcFHDBu+sueIJPiez+/8jhUKrW/xClbydfX3jWOIlUUMxvKyiow8OpcOkFmImgszACm/jB8ygwlXFqblTdqGK+n5gVTQPM+/qCsi7ncYqcQ5kcar9P3MtMKKyfnG3/cwQXA/EPiPtBfh3+oRQD6k16RGy2Zih7/NxLUtBLIomkS8yjEwMTIgRaAtkWNXw//EACYQAQACAgICAgIDAQEBAAAAAAEAESExQVFhcYGRobEQwfDR8SD/2gAIAQEAAT8QsLLt5gucv3LQMv3EeX7ltqi9/cLyt+43yfuXeW6lwgCH0C8XxfqZsrKR2/czu37lt8/czysC0uPMTsnzLXt+4CG37jdu/uMvPPcz/wAcxKgQMygwxHeGDNRyKBrE2JBgbABfTOWgHTm5iSgcexgBEBaFLfiOI7NkTFfwYolIZngwWOeYM0zuJjUB4hiG0IOC4svcyLfuC8jFe/d91A5EsH1RPEbcNw281ydUzVXcrqIUCUi5y+z/AJF5rs8128te87jXb65Es/f/AMAPLHG4CmmswUq7YYTQzLD1MHGplEvITZWVZ8i5HBy10zMJTLbs35fOfUTtByrTdrn5i9rxdU+X9QtR5bPE814H9FUaoQ25Fy3SeIFHhzhLpC67YFQM95FoDgdwFxUaWw3cqZnuWt9sGJuBnMCtRpKgRKIQE5X2l+i2JZE1VWsLWEp7eYzcDBQ335i4am4W7fmEB9oCKyEwHct5XMRppH+o+FEeT4W2scVUuVIdycBXBa34wblAMQBQGgD4lOVR4IxrMGjNQKfcS17/ALmmG2VTmG7JuCriioOwwjzQq+4MoRBgO8/E3HKGW34giKVMXRpdGiXoibAQdtr+IfxVEgc7zsGnxBYABEa4uk341D7DVYjtduBx1pgxWWRRYjBdrlhx3HftG1bhW9za+4mY5RIQFS5qBZu/4Cn1+ZepgdmAwSKcGFYXcdLhloqgMCWZOaxEwjWtBN2B1KQcYOUN0C0aa8Mt2zpWvNvdxBSALICHdKHgtWQXAKAWgIoUux/GS9zlXbCev4EAFjZ3E4AAoyL0yU/1OFGj6jOchX6lDAQbcH/ZZQtKUFQiMBWUdGt4gIwF8enHsnxVlo/3mG2lbVJGeU5LF9y9ODEtl8fwNx5e44Xtg5zC7uZEy+j9XYLr51K+5qN2CwlW5x4jtaeyq2wKEL7R+HApu+Q4047jw75WBpqKS05N3mNrU0NLmQuyx/LKrUgo+JqU4FqwJZoxOzWqLk1cXLYq5ia3bM9w4mpme47+X9xC/wCNCJmaLVibHOtQSJ60mgOAOYwRggPB81Gru8JPIxNxRsyVx/1HaIso3eeJgCa7t0WkJjwGr8MO6lAJdAFXFJOiD4ejxEIbAt6B/ZXwSyXbweI9ShEJOK5gU33Ly9sHuVz/AASY89ooVy0XivsiUwtq2VFl/OJS7VGrVJUKYvljZslVlS6d/D8REFJA/wB5qLOo4bIfgFXysoBNVRLhEf8AGyWcioLtn2rVwQtRdh2wQXbsK8ejCRDZNA+f8qMzJt0W7oiWruXwcS8ZlXiBTnmaY9sIZIYFaALVaAj4TjaWpXrAPQvM0DonVB22/wCGZe7dAKa4zxOfVe7gvkbqvPEOqG8g7OM79QuUMwZlLVVWLlOowGxDCsx8g5qCAC42SwwT9RnEuBJn6inR4Y5Gn8onNdP5lI0MNbtnxK6HueUF+CCj7hpPll4hplVRopB5Hs35quYUtf8A/EA5ffJmOGu7gwAWYDVu/CY72Nza7ZQ6TXssKDWCBYuEb5Mph0HZQsb0fjE3uADFa0KSHWvUuVYxAvQ2K2H3ctKIhYTT6SzHmYIHqocyq7p3EbtQSumzBaeuMy2zfoIbWBq3HUq1ZUt2JdpcsQfmYH3Ff2l+Y01EM3Bn1V0PSdPSUkSnqvm7MFNcIs7jmgEaChYYosbdIGog009Q8lKsUV8w2FeKwCipxArLkBFLMgyuqwiRyquYIL5Vez7qHy1cxohXkz9SnD61OeVPkUaR0IBW7zqzyhCjC6a1YyMOFg0VtbHs4ZUUs9Rla44gxeo/ycscn5js4jdhhW7iXWqnKaqqlNsMLxIcurAXHvKgWVQqFKbrzGpr9VVQzirTsFT8Qc5Yql7KMiVV9LMqGT0ZByhS/EquEcLqiegb7IRYhtmE1C4poasT1CVAsinsPZxHsizeVzYynyvmIKADlIAvtguuIiYn8EDyuo34W6XuR4qNGkHBzfgxpy6jaEmEEgMtuKBdEcaUDQPJ+YbUL5alSi8R4pctWKQJTsOMB9xW57qsNtbruDQzlCq5dM5ZeEiC9kbsYuWmFDmXDKcbyRhmZ05lBXLLRoNzK9sUij4lG2nHMCOGYz0z3niGo+Guw0jPwVL/AIULlsAYC6wfKy4HewcADO9yw0ADQB1KrglBV0iA5ah5IaWoo5Y8MqRh/P3S9j3ys5oXvtV/FmI+LKuWoJdFcS70GK6lq2HqKRc6iJaO2F6t/cSNLq8fctaY7oO9zFdM9Fm/ACstRLRp5H7W/MtmNCz4L8vxG70tH0PEdt2yxRVazLEWd1pXhZCv3mXUEmGBC3J9uWXQgaFbWduAPmKF1bCApmAvAhoblowy+IQFaXhgWLwUdpe6dGMhYVaLduJdPKyq5cc3C/Dcyx2f05qAPc7rUJR9w0RCCJCxFTeLtgyenIItK7Cy4FCuHNMvnMbYtSXhRbPgCAEzZSxKWoBCLYYZL7iClxKTEscNlbKt/bKioomZlAleSrypmHMp8JTA8y4S1fxCXAzLGeY8URzVcq4RKM2Zft/DLn6iRVH2IAq3HSD7QcuoBW84IhRdmPEO0n0ksYsoVW5f7ineHgxcZEAazEKeFt4wrbFrTsP+xMdIAJ4qJMilw852PJLCsycDpnmZqrPGYlLXoISmWjugEzqtPcPYBgbAD9RcksbzLnTLTEyiqWLeeVgNOGzG4kDn6S1UNjaR1QFrYRD1xaIroLT9SxbgxO85fA3FNq6dt5BUPZGr8gADqlIvLAIWdP8A5KGYoOB3nb6gNrxph7NPtIlqcw6o7CvwgqHBSi26u09RV4sEG72vZgiDg4g+oDE4P8I/UdlZXha7dShw8hHoYwehvsl+i5uiewmvmFUSso+aIPwQtY1ahzT0dRcAZAxOiaefxHkGkvD2obL4ipoYAawRIZWC0ZghWUdZ+ec3G4iqgd1X7jcgI7AF/cF8USksDw3xHbpPer0MBLalm/WCA6hbYA7cTkMzXndGYGH2Cq+VX9orsSjK+hl+xOkWfV1El53RW+FiYFYGTtHbr8Iv5lbKA7qvAdZ6l7cyJkHkz5ItAw1WK+oOM3aPQvE2qx2rG5P/AARFrIs/MNJsZ4PMJTLxfj5iWTKuzlxrUMSsSChQDS+4FBZHvrbkfBAe8aJ0W1PcL/FjApzhTDUSlmtBi6LY2mV2TNZClYBkxYmmAEF0TTumJZueW/iVBtsi6y0wvggX7oH0A5Y5W8ZEeHUZ8MdznFoK+4/54AlqtlOdxhPWuz8QmQOBphHMf4EL3z8o7lZ4hyV9yLKD3Ig3y1b9zKVI3dER9rUPvDCJ3hBRxYU/ZGQOSVeeKaMBKb7u9cVHty1xh2VsPmUR1S36MRN2+Be0lwDkKOOyo+e4WS+JVGDwi4l0vajtP0scMxX/AHJ+3R0T/9k=
This one is really long and generated by the website http://www.base64-image.de/step-1.php
However, my application code generates a Base64 that looks like this
_9j_4AAQSkZJRgABAQEASABIAAD_2wBDAAYEBQ
It is shorter and I don't know why. Could someone please explain to me how to get the long version? Thankyou so much. Below is the code I am using to get the short version. I have obviously spent a long time Googling to figure this out and no one seems interested in this long thing.
String file1 = "C:\\Users\\J\\Desktop\\test5.jpg";
System.out.println("HERE");
FileInputStream imageInFile = new FileInputStream(file1);
byte imageData[] = new byte[(int) file1.length()];
imageInFile.read(imageData);
byte[] encodedBytes = Base64.encodeBase64(imageData);
System.out.println("encodedBytes " + new String(encodedBytes));
String convertedImageData = Base64.encodeBase64URLSafeString(imageData);
System.out.println(convertedImageData);
Below is the output of both of those:
HERE
encodedBytes /9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAYEBQ==
_9j_4AAQSkZJRgABAQEASABIAAD_2wBDAAYEBQ
Thanks for reading. I have tried MANY different things and come here now to ask.

imageInFile.read(imageData);
does not read the whole file into the array.
It reads some bytes length matching its internal buffer and returns as the result the number actually read.
You need to repeat the process in a loop till read no longer returns the data.
Check the javadoc.

Try these methods:
public static String imgToBase64String(final RenderedImage img, final String formatName) {
final ByteArrayOutputStream os = new ByteArrayOutputStream();
try {
ImageIO.write(img, formatName, Base64.getEncoder().wrap(os));
return os.toString(StandardCharsets.ISO_8859_1.name());
} catch (final IOException ioe) {
throw new UncheckedIOException(ioe);
}
}
public static BufferedImage base64StringToImg(final String base64String) {
try {
return ImageIO.read(new ByteArrayInputStream(Base64.getDecoder().decode(base64String)));
} catch (final IOException ioe) {
throw new UncheckedIOException(ioe);
}
}
Taken from: Java BufferedImage to PNG format Base64 String

The years have passed...
Nowadays, I think the solution can be fount here: https://www.geeksforgeeks.org/url-encoding-decoding-using-base64-in-java/
So this worked for me:
Base64.getUrlEncoder().encodeToString(bytes);
To get the byte[] bytes (completely) filled as another topic...
I tested adding an html inline image when sending an email...
<img src="data:image/jpeg;base64,_9j_4AAQSkZ...

Java decompressing array of bytes

On server (C++), binary data is compressed using ZLib function:
compress2()
and it's sent over to client (Java).
On client side (Java), data should be decompressed using the following code snippet:
public static String unpack(byte[] packedBuffer) {
InflaterInputStream inStream = new InflaterInputStream(new ByteArrayInputStream( packedBuffer);
ByteArrayOutputStream outStream = new ByteArrayOutputStream();
int readByte;
try {
while((readByte = inStream.read()) != -1) {
outStream.write(readByte);
}
} catch(Exception e) {
JMDCLog.logError(" unpacking buffer of size: " + packedBuffer.length);
e.printStackTrace();
// ... the rest of the code follows
}
Problem is that when it tries to read in while loop it always throws:
java.util.zip.ZipException: invalid stored block lengths
Before I check for other possible causes can someone please tell me can I compress on one side with compress2 and decompress it on the other side using above code, so I can eliminate this as a problem? Also if someone has a possible clue about what might be wrong here (I know I didn't provide too much of of the code in here but projects are rather big.
Thanks.

I think the problem is not with unpack method but in packedBuffer content. Unpack works fine
public static byte[] pack(String s) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
DeflaterOutputStream dout = new DeflaterOutputStream(out);
dout.write(s.getBytes());
dout.close();
return out.toByteArray();
}
public static void main(String[] args) throws Exception {
byte[] a = pack("123");
String s = unpack(a); // calls your unpack
System.out.println(s);
}
output
123

public static String unpack(byte[] packedBuffer) {
try (GZipInputStream inStream = new GZipInputStream(
new ByteArrayInputStream(packedBuffer));
ByteArrayOutputStream outStream = new ByteArrayOutputStream()) {
inStream.transferTo(outStream);
//...
return outStream.toString(StandardCharsets.UTF_8);
} catch(Exception e) {
JMDCLog.logError(" unpacking buffer of size: " + packedBuffer.length);
e.printStackTrace();
throw new IllegalArgumentException(e);
}
}
ZLib is the zip format, hence a GZipInputStream is fine.
A you seem to expect the bytes to represent text, hence be in some encoding, add that encoding, Charset, to the conversion to String (which always holds Unicode).
Note, UTF-8 is the encoding of the bytes. In your case it might be an other encoding.
The ugly try-with-resources syntax closes the streams even on exception or here the return.
I rethrowed a RuntimeException as it seems dangerous to do something with no result.