Develop Reference

Why do popular Java Base64 encoding libraries use OutputStreams for Encoding and InputStreams for encoding?

I have been trying to solve a memory issue in a Java program where we are loading an entire file into memory, base64 encoding it and then using it as a form parameter in a post request. This is cause OOME due to the extremely large file size.
I am working on a solution where I am able to stream the file through a base64 encoder, into the request body of an Http Post request. One of the common patterns I have noticed in all of the popular encoding libraries( Guava, java.util.Base64, android.util.Base64 and org.apache.batik.util ) is that if the library supports encoding with Streams, the Encoding is always done through an OutputStream and the Decoding is always done through an InputStream.
I am having trouble finding/determining the reasoning behind these decisions. Given that so many of these popular and well-written libraries align with this api design, I assume that there is a reason for this.
It doesn't seem very difficult to adapt one of these decoders to become an InputStream or accept an InputStream, but I am wondering if there is a valid architectural reason these encoders are designed this way.
Why do common libraries do Base64 encoding through an OuputStream and Base64 decoding through an InputStream?
Examples to back up my claims:
java.util.Base64
- Base64.Decoder.wrap(InputStream stream)
- Base64.Encoder.wrap(OutputStream stream)
android.util.Base64
- Base64InputStream // An InputStream that does Base64 decoding on the data read through it.
- Base64OutputStream // An OutputStream that does Base64 encoding
google.common.io.BaseEncoding
- decodingStream(Reader reader)
- encodingStream(Writer writer)
org.apache.batik.util
- Base64DecodeStream implements InputStream
- Base64EncodeStream implements OutputStream

Well, yes, you can reverse it, but this makes the most sense. Base64 is used to make binary data - generated or operated on by the application - compatible with a text based outside environment. So the base 64 encoded data is always required on the outside and the decoded binary data is required on the inside.
An application generally doesn't perform any operations on the base 64 encoded data itself; it is just needed to communicate binary data with another application when a text interface is required or expected.
If you want to export your binary data to the outside, naturally you would use an output stream. If that data needs to be encoded in base 64, you make sure you send the data to an output stream that encodes to base 64.
If you want to import your binary data from the outside then you would use an input stream. If that data is encoded in base 64 then you first need to decode it, so you make sure you decode it before treating it as a binary stream.
Lets create a bit of a picture. Say you have an application that operates in a textual oriented environment but operates on binary data. The important part is the direction of the arrows from the context of the application on the left.
Then you get for the input (read calls):
{APPLICATION} <- (binary data decoding) <- (base64 decoding) <- (file input stream) <- [BASE 64 ENCODED FILE]
for this you naturally use input streams.
So let's look at the output (write calls):
{APPLICATION} -> (binary data encoding) -> (base64 encoding) -> (file output stream) -> [BASE 64 ENCODED FILE]
for this you naturally use output streams.
These stream can be connected to each other by chaining them together, i.e. using one stream as parent of the other stream.
Here is an example in Java. Note that creating the binary encoder/decoder in the data class itself is a bit ugly; generally you would use another class for that - I hope it suffices for demonstration purposes.
import static java.nio.charset.StandardCharsets.UTF_8;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Base64;
public class BinaryHandlingApplication {
/**
* A data class that encodes to binary output, e.g. to interact with an application in another language.
*
* Binary format: [32 bit int element string size][UTF-8 element string][32 bit element count]
* The integers are signed, big endian values.
* The UTF-8 string should not contain a BOM.
* Note that this class doesn't know anything about files or base 64 encoding.
*/
public static class DataClass {
private String element;
private int elementCount;
public DataClass(String element) {
this.element = element;
this.elementCount = 1;
}
public String getElement() {
return element;
}
public void setElementCount(int count) {
this.elementCount = count;
}
public int getElementCount() {
return elementCount;
}
public String toString() {
return String.format("%s count is %d", element, elementCount);
}
public void save(OutputStream out) throws IOException {
DataOutputStream dataOutputStream = new DataOutputStream(out);
// so here we have a chain of:
// a dataoutputstream on a base 64 encoding stream on a fileoutputstream
byte[] utf8EncodedString = element.getBytes(UTF_8);
dataOutputStream.writeInt(utf8EncodedString.length);
dataOutputStream.write(utf8EncodedString);
dataOutputStream.writeInt(elementCount);
}
public void load(InputStream in) throws IOException {
DataInputStream dataInputStream = new DataInputStream(in);
// so here we have a chain of:
// a datainputstream on a base 64 decoding stream on a fileinputstream
int utf8EncodedStringSize = dataInputStream.readInt();
byte[] utf8EncodedString = new byte[utf8EncodedStringSize];
dataInputStream.readFully(utf8EncodedString);
this.element = new String(utf8EncodedString, UTF_8);
this.elementCount = dataInputStream.readInt();
}
}
/**
* Create the a base 64 output stream to a file; the file is the text oriented
* environment.
*/
private static OutputStream createBase64OutputStreamToFile(String filename) throws FileNotFoundException {
FileOutputStream textOutputStream = new FileOutputStream(filename);
return Base64.getUrlEncoder().wrap(textOutputStream);
}
/**
* Create the a base 64 input stream from a file; the file is the text oriented
* environment.
*/
private static InputStream createBase64InputStreamFromFile(String filename) throws FileNotFoundException {
FileInputStream textInputStream = new FileInputStream(filename);
return Base64.getUrlDecoder().wrap(textInputStream);
}
public static void main(String[] args) throws IOException {
// this text file acts as the text oriented environment for which we need to encode
String filename = "apples.txt";
// create the initial class
DataClass instance = new DataClass("them apples");
System.out.println(instance);
// perform some operation on the data
int newElementCount = instance.getElementCount() + 2;
instance.setElementCount(newElementCount);
// write it away
try (OutputStream out = createBase64OutputStreamToFile(filename)) {
instance.save(out);
}
// read it into another instance, who cares
DataClass changedInstance = new DataClass("Uh yeah, forgot no-parameter constructor");
try (InputStream in = createBase64InputStreamFromFile(filename)) {
changedInstance.load(in);
}
System.out.println(changedInstance);
}
}
Especially note the chaining of the streams and of course the absence of any buffers whatsoever. I've used URL-safe base 64 (in case you want to use HTTP GET instead).
In your case, of course, you could generate a HTTP POST request using an URL and directly encode to the retrieved OutputStream stream by wrapping it. That way no base 64 encoded data needs to be (extensively) buffered. See examples on how to get to the OutputStream here.
Remember, if you need to buffer, you're doing it wrong.
As mentioned in the comments, HTTP POST doesn't need base 64 encoding but whatever, now you know how you can encode base 64 directly to a HTTP connection.
java.util.Base64 specific note:
Although base 64 is text, the base64 stream generates / consumes bytes;
it simply assumes ASCII encoding (this can be fun for UTF-16 text).
Personally I think this is a terrible design decision; they should have wrapped a Reader and Writer instead, even if that slows down encoding slightly.
To their defense, the various base 64 standards and RFC also get this wrong.

Java converting UTF-8 Text to ASCII produces weird output

I'm trying to convert the content of a plain text file (UTF-8) into ASCII (ISO-8859-15) to write it to an output file. I've written a few lines of code (see below) which read the content of the file, write it to an array of bytes, decode it with the UTF-8 charset, encode it with the ISO-8859-15 Charset and write the result to a file. This works just fine, except for a question mark (Hx: 3F) that suddenly appears at the very beginning of the output file.
import java.io.*;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.Charset;
public class Main {
public static void main(String[] args) {
/* Read / write the file to a byte array */
File input = new File("input.txt");
byte[] bytes = new byte[(int) input.length()];
try (FileInputStream fileInput = new FileInputStream(input)){
fileInput.read(bytes);
} catch(IOException e) {
if(e instanceof FileNotFoundException) {
System.err.println("File not found.");
} else {
e.printStackTrace();
}
}
/* Getting the charsets */
Charset utf8charset = Charset.forName("UTF-8");
Charset iso885915charset = Charset.forName("ISO-8859-15");
/* Wrapping the bytes from the file into a buffer */
ByteBuffer inputBuffer = ByteBuffer.wrap(bytes);
/* Encoding the text file from UTF-8 */
CharBuffer data = utf8charset.decode(inputBuffer);
/* Decoding the text file to ISO-8859-15 and writing it to an array*/
ByteBuffer outputBuffer = iso885915charset.encode(data);
byte[] outputData = outputBuffer.array();
System.out.println(new String(outputData));
File output = new File("output.txt");
/* Writing the output to a file */
try(BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream(output))) {
out.write(outputData);
out.close();
} catch(IOException e) {
e.printStackTrace();
}
}
}
Input file:
ABC
DEF
GHI
Output file:
?ABC
DEF
GHI
If you have an idea, what might be the cause of this weird behavior, please let me know. Also, if there is anything weird about my code in general, please point it out to me, since I'm not very experienced in the use of Java yet.
Thanks :)

Consume a c# base64 encoded file in java

I want to transfer a file from C# to a java webservice which accepts base64 strings. The problem is that when I encode the file using the c# Convert class, it produces a string based on a little endian unsigned byte[].
In Java byte[] are signed / big endian. When I decode the delivered string, I get a different byte[] and therefor the file is corrupt.
How can I encode a byte[] in C# to a base64, which is equal to the byte[] that is decoded in java using the same string?
C# side:
byte[] attachment = File.ReadAllBytes(#"c:\temp\test.pdf");
String attachmentBase64 = Convert.ToBase64String(attachment, Base64FormattingOptions.None);
Java side:
#POST
#Path("/localpdf")
#Consumes("text/xml")
#Produces("text/xml")
public String getExtractedDataFromEncodedPdf(#FormParam("file") String base64String) {
if(base64String == null) return null;
byte[] data = Base64.decodeBase64(base64String.getBytes());
FileOutputStream ms;
try {
ms = new FileOutputStream(new File("C:\\Temp\\test1234.pdf"));
ms.write(data);
ms.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
File test1234.pdf is corrupt.

"Signed" and "big-endian" are very different things, and I believe you're confusing yourself.
Yes, bytes are signed in Java and unsigned in C# - but I strongly suspect you're getting the same actual bits in both cases... it's just that a bit pattern of (say) 11111111 represents 255 in C# and -1 in Java. Unless you're viewing the bytes as numbers (which is rarely useful) it won't matter - it certainly doesn't matter if you just use the bytes to write out a file on the Java side.

Encoding a file to Base64 in Java

The Imgur API allows you to send a Base64 of an image to upload that image.
I have tested that this works when I send a Base64 that looks like this:
/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAQDAwQDAwQEAwQFBAQFBgoHBgYGBg0JCggKDw0QEA8NDw4RExgUERIXEg4PFRwVFxkZGxsbEBQdHx0aHxgaGxr/2wBDAQQFBQYFBgwHBwwaEQ8RGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhr/wgARCADIAKQDASIAAhEBAxEB/8QAHAAAAQUBAQEAAAAAAAAAAAAAAgABAwQFBgcI/8QAGgEAAwEBAQEAAAAAAAAAAAAAAgMEAQUABv/aAAwDAQACEAMQAAAB3xJsmd2cDMwdZG4lK6OC1HHQKytQCJEtwESpVG0o0pZiatCTosZEs8JMwmaTpN3EpHvn5/K2zdJy1HM73Gt38ysDPby5nU+f7GixpgxCYNA063zJ1uMzpZO7FK9qV/njHz+4HKfU/P6RySkVbN38WWrrPUPFfZ+bSTCKSKMotGZCtGRJY5kSHWd0hr8X03nnX5WW0dLt886bzRXa/OjBPT23q3kXrnPcIOwKYXbRlSW4STrqRCQlHxuz5xbLf2uLqdLn3saWibtCsqiHzRSW9zZ9k+fvf4CZk6NQkzAJOjE3EpL0TIC5nzDp8rpxScnfvUr5jpOf2+b0j5/0DilHT1MvoOhFv+mmISMgYSkYC3JUK9khC8PSeCVtHwja9Z4WpPHtpY3Y5udYq2Of0uqwppiVh2NLI97rN3ziQfe4FAbIHmiMGzKNAcwuEV7uDbkgZGc+fice/X6k+S4DPb00sk7YMXE2MQK7EwWLQ9D3fNu3Lj7FnIeZmms1Du6IhyOy+Jq1Go5XmpanWQGfNW8+vKHUx1WK82K2CpUKYrGmE7U6HU8l0LuTu2MMRk21hrC7sYF833ZeT2+YuloUekxevJnUrtYr87o+eXMt7TmN/ApgrEJnXLNHJQi1dzkUutJnWsluqJYHWDZo/O9qEZshwrmpArySrHI1ma99sO9lyti6ZWg1kclmfBoLaqblPSjLV6ayFqaUTSIsrjNbzc+DoNNXuPt+l3Jm+Ryekc60OSkqgL9EI2NcoyX2rxA1avjid2LLCSYMQJATMkkpbiSR1NJLF3bKU73hSW3PrpMXkUkujPXFI2iSXvTpIs//xAAqEAACAgEEAgICAgEFAAAAAAABAgMEAAUREhMQICEiMDEUMgYVIyRAQf/aAAgBAQABBQI/gOFgv/UOFQweRqOKyyLm3uPxT3IK2SaqhR3mgyaUyitrNrTGH29h+HVdSGnw1tMaYWrK1ceZ2jSytpZERydbqwLTsS2l9B+C3aSlBNK6yLqfPJv91hUmnjSBHWzI3Zo1RdQbbbyfAP4LNfvu6v8A8nIf615atbBfV3mkZmuXxbbR9VSlMjLIh8HwPwW7R7ZJDztNvhiV8hM8Uh1qeKCOHnlPTa1iyiLEh9B72pOC3mbuUczqBys7Y0rKHmbn2HNFkMOE+o9ZXEUer6hLUiGtuLEnVqAsjqaQgYrAhz9Jf7gEmkVjqn1HrrL8KFlXntzVVQizJXyzaM2OeYAkhDH6/vO9Y0gbY/seg9das9C2u8WJ7EVlSuA5FIKMc2pyyq42YZXhDNFoc0zj03wet7Vkm1SYu4WBe+emOHHc0qy3JX0iGOOwpOKvLKEUsGoHxvm/kZ/74nhWzDqGnTaXOkjT1KoMzTyFDZkWQ6ezJNPfO8tn+SbNdqsnYrw1dclow6drpv2fIwessSWI5f8AH5IMnPSYlByReOUjtZnrdss8TxVdSjEMAtfHNJEou1a7yzfzy8b+d8bbjfIFwHi8u/KBuuZgI7E47U1BjIi4V3yAmKSjb5DnnPA/x3ed8LBR/qUJNyUyh0M0srOCx3K/2dmsXZ3/AJmXCP4wwHFxHxJd85Z2YD53y+TKJmrUjPdknZQ0UUkhYZHG8pnbg47ZVsScmxfEB5urbYr5yztXN83w40XM3XFizJ9cctJnDxDO1YytwiktPJHtgHmEcWD74JNsMhODN838ahZ7wExo/s2N4jjSfNQ2lsSrw8gYBiHjgbfA2b4pzfN8ml+oQ5InGN3GEjG/RwFkxjku/lfA+cL8cjl5YDi/rnhfGO+KjHJuxskjZGSu0iupUEfIXJJfrM/OXxvm/gnF/atviRuQ0UKY1+jC51KBRJ/kJAk1buburtkViujm/wDLOpWMDeJxJYMBOCsxzpcmOGNMjfjhiebDNHGWuOuHUSSl6cDunczrxk4kZt9AN1PDFh7MbTbSKr74ejD1jOaAGzI+bnN9g05fC7b7nOTkYv658caXngkwToypw2jr0SKywonRJjcVOoR1rKvDJHiI7ladhgtaVhHSndlpPDNNAVxY/k/Bxf0fRGAxTkVnhi2WlwCAZzhK7xLgk45JAJmem6l0ljLuxzkc+fK5/8QAJhEAAgIBAwQBBQEAAAAAAAAAAAECEQMSITEEEBMiQRQgIzJRBf/aAAgBAwEBPwEQl2jwRlq2+RtDJMfdCQjqOq8G0RPq+r/J8Iw5Pp/U80XtY/sXaeTxx1Eox6iLyXuR+p/TEmxdN1OKX5ImCOhPahyHI1vuj/Qm6UUYVc1EhPJhnUWSyZckd2eVz/Yl9ubLoMmrMvUw4nGrPE1M3SpIrTLs2X3zTWN3W4+pcpe8rMWVY/atyeVzfsZVq2TE9jUN99VGWKyqnyJPU5SFJLHsZJR1xmRl5oKQoLQ75FIRqRqssnljHgUlwhcDfqr5MSrHRJuqY7iLLR5WWZLmtIqTIJcvkerS5RRKLyVXBdIZ1K9tiy+2bJKNKJFyTIErlGk6MFx9RjslDUShT3KNMmeD+s8DxnhnyWxRjB9mkabJ4tR9Oz2s1NfBHKrPJI9L/hoUvkcYobhHkU1Ww3fdE1RqIwixbbIeKMuTQlIpLv8A/8QAKREAAgEDAwMEAQUAAAAAAAAAAAECAxESBBAhEyIxFDJBUQUgQmGBsf/aAAgBAgEBPwEY3YZLySg4rL4I3FyU0LZjJkmaXRLULOo7L/SNHR6VdP5f9lektTyenqRWTVkRQt2O5Tp9Waiym3SkoW4JuhfKfA9RQmu1mqm5zTvdCW1t5I/G0025MrWUMidOFaOTFSpwZVji+BbW2uUKSmrsgoU+SrWUoux1k6Zw+St3JGIkW308M1Zvg6EYxtFWK9NyWJSpqKeJTniuUT7mYiW9ilU6bLpxUUTp9/cU4NRlFGPTk0ifv4Ghli20KM2+RQ+yfkV8nbwfvFThJ3JwHSudJbUXjO4o9t2YtLgbSdmU2oXy8kaeTIRsi1zEx201OL7pEWpokWtK7HaTuIX6LRQpv4RS1P8AB6uHi52+RO5kkepUT1cfoWof0dd/RihokuB2+hVJs6rXwRnx4E5P2qw4v5ZGKWzJu22JObTG8uWRqyj4KcnKN3stv//EADkQAAEDAgMECAQFAwUAAAAAAAEAAhEDIRIiMRNBUWEEECAjMDJCcVJigZEUcqGxwQUkQDRDgoPR/9oACAEBAAY/AvBzGJ/xSHXBUvmp0bjqWf8AoQdTIc06Ef4ffVADw3qabMQPxJx6Njo03elFrnVGk6xchNpdN/uKJ0M3hSPHy3rv8g/lbbpjy2bx6ioAw8ghU05ShrS6QzeN6pisAy0S391TpjHVqBgs1q2j6QpUz5byT4rq1bQbuJQ6Z08HaVPIPhCyuWKpm91jkQ30jVQG4ag0KbbDhH68VtKsRT1HxeM19e9CiyWt+birDEWFWELH0kGo/wBIjRdw18i86Qm1W0mMqDWH+ZS7o7abvlKG0BbTd5iE19MhzXCQR4tUc1ZMcysKWIZmOEoFrg+LwE/8OddRhlOpfhaTKjrh3D2Cx1DiJXRqL8WEhz44t4ff9kGUmhrGiAB4jGDzVXhgVQnQuUnRNwnRBxO9Oc0kGUZvPFRJA5J1dzC8Me2HE6Tb+fDfUdo0SqFPo9QMqk5nR910d3SO+2eLdGqlpwOdeCsKGO4QwiAvdysoGqqhzjtdqy07vDdG9zf3VXaEmHw1dxinfKw1BdB+/emrvmOa3cS1fVcFhot93HVAbyZ8Og1tPave/K3mielEea7W2UU6DWTreU8fDp1DBeq5sudw5LDixDgVbRQgagy8OKa5tP8ADs4uP8eG3Y95ToMMc3Jxq0g1wqEGCg1/lQq0icKy/qs7sO4ysYcXmm7NukLa+lxhWIXRNocu0B80+HUo1JwvEGFD/L6HjQqv0lhwRGPfdF5uUNkfJqm7MRa/ui9mjWy7mE+k8ta0C7h601gaG0+agkOadCELOxjeoezbt9Mu0WxfRbTkWgz4Rp1mh7DuKqH+m14a4Xp1Lgo0DkdGbCJKwO8uvUy0zZB/RyHMd6SYw8kMoADrnnwXRodLt/JZqbZ+IZT+iIxFh+ZUX8HeG6dIVTBmMAD7KbHksxVN/wALgVWBGRzo9l0cB8tqX9k10HAHW59YcEKbzf09d1bsS4wBvKIo964cFedkWHEAdEXQiYa0clZCeKqB4AZtJWJginQZAMalUWjj13QjwNno3UpoawVDq7Erd2zSyzceuGAn2RDcoc6ahP7KGnZ0twWEOxNbp2GjruVr2CSYtvR2Y7oWHPmhyupPXT6LSAa5zodUHFHaDFDjZO9IOUAdmfA2TDFP1c+ok9ilUpk4gZeDxQAEYjKazl2Z7ZjtywkIVJmKN/eE3F5sPbjf2rAqGU3n6I7QYTzU0ofHA3WYR11Kd5pswpx7Y6rNcs9YALB5zxlGB0dhHGpiRwODv+uFNXG4/mhHHivoZXfMe9vyvhRTfUawaAmV/t3+6xGHNbrdVHVJzAmykTh4wsha/wCqizfcqa7sfysP8r/TscD8TUI6LTbz8v8AKhpph/yjEm7N9ZzeYhHK76uVnKNq4/VHeFpog6LTCxYRrC0IXd3PBYzQfH5VhNLEQodjpv8AZZak/wDFZXOPIrNgA/KF5h9ld6AMmNFfd1AF7oHPsZldNY6QAspOt4K7ym+fdYaDHUx+XVTmZ+iO0dM8Chs21GVx63b/AHWZpUU2uceQU7M/VTEDnZNlhAO9ZwOQlGW390MWWVrPavP2W9aFWY76PKG2bUI5PXleowx+YSsmL7QskgLFhzcVElvu1XB7f//EACcQAQACAgEDBAMBAAMAAAAAAAEAESExQRBRYXGBkbEgocHR4fDx/9oACAEBAAE/Iduow9YTmcdDJgKh4uU/i3MyuoMdvUh1OgigVI8k8ez7r/0hnXtFjK4dxR0qVE6PQ8/hfUjL4H8c/AmNv0HL2g6M2vB/3zCCHmP/AAeSYbtet7h/jEA0IJKiSo9TuV1Iw6EccTs+XiKvcFfPaNXCYN0cnPGR9ZWaXh6p3P6S3qLv3iEosHWxyuIq1GPqfp0qPQYj+bydvlwBB+XXtmsfUMW9ncjKi42Qroav3AinePJGG0EVd+UEaVh7RlMAAMAS6l3FF6PP5Ed4EP0tcvQIXHIgXlitCkeWZgDApPJ7yvd9EMuMFselUSuQ7eSWNByP1LUJEwk2lzToMda6V0CMzAKexqDbi3dygG8Qnx4irC6JVy4q/NDskZeolqxtIN1LtO8s9cBFXLvlcUzENQEwhF6aznrUqGZa9Kx3nb8DDFojiOqUOYJfIEfGYD0p6RxbZY+YxlUFgs1AaHaopIW8hD3F8Shi9HppOepNrHftB4KvDxeEqrZ0tUVdHY+4nL/6B3j8WlItRBqqvEOCinSChVaqWcEa45382TdjCPRY/Ai2WRBV32SjZb0jVTRCmTsWCXTYZQHjfnzNaywWPjIEvrA4unLNWl85PMz+2v7Lo9DoR6Cc9SPjpuJ7mFuwSyhnUw57J/R2mO1vK0oUdVmBIAthtw7SzbGxu4SHmrt4jWEJPZM+E2228r22goDsV+JTn8K9AF8YsnxUC+TcjmvY5jcassMUAoNuI2z4YSs18WHHFzT7Smh481H++DFVXeWgA+Ygha0EVf6mDF9p5TKXCUj1ZeXuUzfJuNh/3xAxqywYf937RaV9TJvla8W8S2AByhOM4avTZXvK0W5L457VKJ7hbfOIJyA5xKxY+LWVEGYGh85WwqtVZxOY4MS8wYgjvpcGbvalExKI/oL/ANmPWFtlZDx5m0DKy/cv+sq3LJdWuv3U2NLlFy77Q0xyZThAgQ8HkWRcE0qn5NH4mVw0aPuRl6PJkrmZacTLcMy6htqXMJcuEMM6st7Q5bdHcpuNstUHIRt5mWtr4ZmvjF3llKgWiGjvOn6gkCg3hX/EFsCA8xeZrybPJ2h2MKwYV7GLF6HhDWpQRuM2qwH3leQENLd757RW20luKO8El1WZ0xTsqX8zPIn7DUuOPCyTaveEN1t3o3FUCpWyGxdNlPaFslzwYrXRXMuMlLYfUlATdge8aJx0McdorW02LeZWPYCaipaFtLxNlnxFp/H3D6CJx3hG++N5gdBiZgDMslRNyoWBKuM5RjeVaApeh5Yi1Lb4/wBSli+Lu8EQz+DtGKzU5G8SV58Q+HGtoK/2Z+8Yq5g6ncldoCtRxGUeirH8mt1HY+sTMZcdEWafr6THqDuHiEuco7jVaHkEeUTR27w6HF31ehLYR69IAWRITh9+mpzBCeUjnJLMKvmHKUuK6QqOWXwyrhVXoQW0oKdu366GIsQaipiaW2Bkx0H3RY5l8sZmUUeC43XjqSqpciicg2A/TcVKjWTmP0F9ujy5KP7Kp1gPYhLgYQKalm40Yj+qWVR4JQLnxmN86eAisQWNw9gnjzKH2YlPxEFPapRecXk1HPbrc+puipUzh9xp/kxMxZxxCwzr94yulycCWo4KPwwudx2yLY2sbz5gvQax+JYyNUco4MQrP94g+EN1fbcshrKNv1KzSXM1FdFyFO7MLmXk4Bi+TPIhmkr3OZa+8piqnnxXM0V93aJk+ADZLii9m7B7bhfKTmxDJR2MQurvxfGJj1wu6zwE7EEFupiDT3Etzbc7cQFYhfdYcvWFmA3HNjXJAOTCCTs1uEgbSwOd8Q2GteL39Qc6OaZ8jL8ygTRwv/KKxuTklXq1ZjwH9imN5rHzPCwmYoGPig/DmMhUat7QW7VcQ/lDdR/sc3ra0ZgUHkwiBR3lx5esDbh32mez8Sns/EKC/YI1gL8TuJ4zBOO8VoR2zV+mPDxeT/Y1Q1dqoiwQlPN+5n5jtLaat8oIyctRngww2ka9I+T4jZ0yns/Ep7PxC04d9p//2gAMAwEAAgADAAAAEHFPQc6ZbIw/crMYmGT4O7zlCwthnch91AzB56vC3beuga3fk8XEe27QmZ37MgnELGj6b84Oqg8brPkMjMT47LLSc4PjMxgrioJcI8O1YoOb36zLZ7iejp+n+gMpFebIupMFsXHXvvfn/IIgP//EACcRAQACAQIEBgMBAAAAAAAAAAEAESExQRBRYbFxgZGhwfDR4fEg/9oACAEDAQE/EA4LIFkVYQ07Wp8+HWXKuBjaQSmBmoOEc1MCpX2luBd7Qr2Y+QR1TXa7HfvLYLKX5c47jx1QohPD4VGnPxLgOPp5ENr086xzrMEzdeutle2+NtZW1/gDDHLq7/EEczu0PcqLtJo8vSHilcuV7wFStKPQx+4rY6SyVKiOjFU9/mIAreTh8Se6EK9GrimsPseL9ZZBctMXgWgcKiOWryrY01yv5YFssFYwctI0vQ++cIO/T75QbDF9fKOAVvrrAxGWxmSAcA0vSv3KtUbHTJcqGmolRmis6bLXREHeWMZz3SDqYXLiPnwLCdUEXKI3FM2w0TpBrWGmNko89MeMoAVV9/rBPIRLG4iaLeEZQufL+9YBttQ5wqvS/jkW9IkK9TTXgc4gg2jHMYLpgyXls35Mq5qxJQwZOf38wpZwd8vavU4LmW+seocJjiTN7BBmMr0gWiVlJmM7wp3198xp3nPwawR9cOcRRiYmryg3dnrOQ2eGZYCqOi/uOZn98I5qE1NbG2ETgAgFzAeIoIbaQDoiaiFIGk2kTHD/xAAlEQEAAgIBAgYDAQAAAAAAAAABABEhMUFRgRBhcZGxwaHw8dH/2gAIAQIBAT8QfBUibh1SGBlaePM9Tk3AC0xHSXt+Bq/EQExXHaNJ1X+QcKKMNpeN2fRFLUjg4p6Vx8S6pjWevl++0oPBcuyaRpVh6Rz6RtjDGIVoI9/fMoZ2dBzXYmpEwdG8/wB7TFAb8SWTihu5rHvGpceaY7QAe4moviHhYbZrAz4FrwO+a8elRymPTU2AQ7MTkchn1iOtolFGrfqClO5WMrMQdBrzva71j+EKt92/eDrY5/ekCOO41yoc8d481AdCGMplEIWjW9dzkg5vn2jPUjNUK3jdef1HhrHwMsh0YrDLzGBWoL0I3YwC27hsDz5rPPbefSWZrcFHDBu+sueIJPiez+/8jhUKrW/xClbydfX3jWOIlUUMxvKyiow8OpcOkFmImgszACm/jB8ygwlXFqblTdqGK+n5gVTQPM+/qCsi7ncYqcQ5kcar9P3MtMKKyfnG3/cwQXA/EPiPtBfh3+oRQD6k16RGy2Zih7/NxLUtBLIomkS8yjEwMTIgRaAtkWNXw//EACYQAQACAgICAgIDAQEBAAAAAAEAESExQVFhcYGRobEQwfDR8SD/2gAIAQEAAT8QsLLt5gucv3LQMv3EeX7ltqi9/cLyt+43yfuXeW6lwgCH0C8XxfqZsrKR2/czu37lt8/czysC0uPMTsnzLXt+4CG37jdu/uMvPPcz/wAcxKgQMygwxHeGDNRyKBrE2JBgbABfTOWgHTm5iSgcexgBEBaFLfiOI7NkTFfwYolIZngwWOeYM0zuJjUB4hiG0IOC4svcyLfuC8jFe/d91A5EsH1RPEbcNw281ydUzVXcrqIUCUi5y+z/AJF5rs8128te87jXb65Es/f/AMAPLHG4CmmswUq7YYTQzLD1MHGplEvITZWVZ8i5HBy10zMJTLbs35fOfUTtByrTdrn5i9rxdU+X9QtR5bPE814H9FUaoQ25Fy3SeIFHhzhLpC67YFQM95FoDgdwFxUaWw3cqZnuWt9sGJuBnMCtRpKgRKIQE5X2l+i2JZE1VWsLWEp7eYzcDBQ335i4am4W7fmEB9oCKyEwHct5XMRppH+o+FEeT4W2scVUuVIdycBXBa34wblAMQBQGgD4lOVR4IxrMGjNQKfcS17/ALmmG2VTmG7JuCriioOwwjzQq+4MoRBgO8/E3HKGW34giKVMXRpdGiXoibAQdtr+IfxVEgc7zsGnxBYABEa4uk341D7DVYjtduBx1pgxWWRRYjBdrlhx3HftG1bhW9za+4mY5RIQFS5qBZu/4Cn1+ZepgdmAwSKcGFYXcdLhloqgMCWZOaxEwjWtBN2B1KQcYOUN0C0aa8Mt2zpWvNvdxBSALICHdKHgtWQXAKAWgIoUux/GS9zlXbCev4EAFjZ3E4AAoyL0yU/1OFGj6jOchX6lDAQbcH/ZZQtKUFQiMBWUdGt4gIwF8enHsnxVlo/3mG2lbVJGeU5LF9y9ODEtl8fwNx5e44Xtg5zC7uZEy+j9XYLr51K+5qN2CwlW5x4jtaeyq2wKEL7R+HApu+Q4047jw75WBpqKS05N3mNrU0NLmQuyx/LKrUgo+JqU4FqwJZoxOzWqLk1cXLYq5ia3bM9w4mpme47+X9xC/wCNCJmaLVibHOtQSJ60mgOAOYwRggPB81Gru8JPIxNxRsyVx/1HaIso3eeJgCa7t0WkJjwGr8MO6lAJdAFXFJOiD4ejxEIbAt6B/ZXwSyXbweI9ShEJOK5gU33Ly9sHuVz/AASY89ooVy0XivsiUwtq2VFl/OJS7VGrVJUKYvljZslVlS6d/D8REFJA/wB5qLOo4bIfgFXysoBNVRLhEf8AGyWcioLtn2rVwQtRdh2wQXbsK8ejCRDZNA+f8qMzJt0W7oiWruXwcS8ZlXiBTnmaY9sIZIYFaALVaAj4TjaWpXrAPQvM0DonVB22/wCGZe7dAKa4zxOfVe7gvkbqvPEOqG8g7OM79QuUMwZlLVVWLlOowGxDCsx8g5qCAC42SwwT9RnEuBJn6inR4Y5Gn8onNdP5lI0MNbtnxK6HueUF+CCj7hpPll4hplVRopB5Hs35quYUtf8A/EA5ffJmOGu7gwAWYDVu/CY72Nza7ZQ6TXssKDWCBYuEb5Mph0HZQsb0fjE3uADFa0KSHWvUuVYxAvQ2K2H3ctKIhYTT6SzHmYIHqocyq7p3EbtQSumzBaeuMy2zfoIbWBq3HUq1ZUt2JdpcsQfmYH3Ff2l+Y01EM3Bn1V0PSdPSUkSnqvm7MFNcIs7jmgEaChYYosbdIGog009Q8lKsUV8w2FeKwCipxArLkBFLMgyuqwiRyquYIL5Vez7qHy1cxohXkz9SnD61OeVPkUaR0IBW7zqzyhCjC6a1YyMOFg0VtbHs4ZUUs9Rla44gxeo/ycscn5js4jdhhW7iXWqnKaqqlNsMLxIcurAXHvKgWVQqFKbrzGpr9VVQzirTsFT8Qc5Yql7KMiVV9LMqGT0ZByhS/EquEcLqiegb7IRYhtmE1C4poasT1CVAsinsPZxHsizeVzYynyvmIKADlIAvtguuIiYn8EDyuo34W6XuR4qNGkHBzfgxpy6jaEmEEgMtuKBdEcaUDQPJ+YbUL5alSi8R4pctWKQJTsOMB9xW57qsNtbruDQzlCq5dM5ZeEiC9kbsYuWmFDmXDKcbyRhmZ05lBXLLRoNzK9sUij4lG2nHMCOGYz0z3niGo+Guw0jPwVL/AIULlsAYC6wfKy4HewcADO9yw0ADQB1KrglBV0iA5ah5IaWoo5Y8MqRh/P3S9j3ys5oXvtV/FmI+LKuWoJdFcS70GK6lq2HqKRc6iJaO2F6t/cSNLq8fctaY7oO9zFdM9Fm/ACstRLRp5H7W/MtmNCz4L8vxG70tH0PEdt2yxRVazLEWd1pXhZCv3mXUEmGBC3J9uWXQgaFbWduAPmKF1bCApmAvAhoblowy+IQFaXhgWLwUdpe6dGMhYVaLduJdPKyq5cc3C/Dcyx2f05qAPc7rUJR9w0RCCJCxFTeLtgyenIItK7Cy4FCuHNMvnMbYtSXhRbPgCAEzZSxKWoBCLYYZL7iClxKTEscNlbKt/bKioomZlAleSrypmHMp8JTA8y4S1fxCXAzLGeY8URzVcq4RKM2Zft/DLn6iRVH2IAq3HSD7QcuoBW84IhRdmPEO0n0ksYsoVW5f7ineHgxcZEAazEKeFt4wrbFrTsP+xMdIAJ4qJMilw852PJLCsycDpnmZqrPGYlLXoISmWjugEzqtPcPYBgbAD9RcksbzLnTLTEyiqWLeeVgNOGzG4kDn6S1UNjaR1QFrYRD1xaIroLT9SxbgxO85fA3FNq6dt5BUPZGr8gADqlIvLAIWdP8A5KGYoOB3nb6gNrxph7NPtIlqcw6o7CvwgqHBSi26u09RV4sEG72vZgiDg4g+oDE4P8I/UdlZXha7dShw8hHoYwehvsl+i5uiewmvmFUSso+aIPwQtY1ahzT0dRcAZAxOiaefxHkGkvD2obL4ipoYAawRIZWC0ZghWUdZ+ec3G4iqgd1X7jcgI7AF/cF8USksDw3xHbpPer0MBLalm/WCA6hbYA7cTkMzXndGYGH2Cq+VX9orsSjK+hl+xOkWfV1El53RW+FiYFYGTtHbr8Iv5lbKA7qvAdZ6l7cyJkHkz5ItAw1WK+oOM3aPQvE2qx2rG5P/AARFrIs/MNJsZ4PMJTLxfj5iWTKuzlxrUMSsSChQDS+4FBZHvrbkfBAe8aJ0W1PcL/FjApzhTDUSlmtBi6LY2mV2TNZClYBkxYmmAEF0TTumJZueW/iVBtsi6y0wvggX7oH0A5Y5W8ZEeHUZ8MdznFoK+4/54AlqtlOdxhPWuz8QmQOBphHMf4EL3z8o7lZ4hyV9yLKD3Ig3y1b9zKVI3dER9rUPvDCJ3hBRxYU/ZGQOSVeeKaMBKb7u9cVHty1xh2VsPmUR1S36MRN2+Be0lwDkKOOyo+e4WS+JVGDwi4l0vajtP0scMxX/AHJ+3R0T/9k=
This one is really long and generated by the website http://www.base64-image.de/step-1.php
However, my application code generates a Base64 that looks like this
_9j_4AAQSkZJRgABAQEASABIAAD_2wBDAAYEBQ
It is shorter and I don't know why. Could someone please explain to me how to get the long version? Thankyou so much. Below is the code I am using to get the short version. I have obviously spent a long time Googling to figure this out and no one seems interested in this long thing.
String file1 = "C:\\Users\\J\\Desktop\\test5.jpg";
System.out.println("HERE");
FileInputStream imageInFile = new FileInputStream(file1);
byte imageData[] = new byte[(int) file1.length()];
imageInFile.read(imageData);
byte[] encodedBytes = Base64.encodeBase64(imageData);
System.out.println("encodedBytes " + new String(encodedBytes));
String convertedImageData = Base64.encodeBase64URLSafeString(imageData);
System.out.println(convertedImageData);
Below is the output of both of those:
HERE
encodedBytes /9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAYEBQ==
_9j_4AAQSkZJRgABAQEASABIAAD_2wBDAAYEBQ
Thanks for reading. I have tried MANY different things and come here now to ask.

imageInFile.read(imageData);
does not read the whole file into the array.
It reads some bytes length matching its internal buffer and returns as the result the number actually read.
You need to repeat the process in a loop till read no longer returns the data.
Check the javadoc.

Try these methods:
public static String imgToBase64String(final RenderedImage img, final String formatName) {
final ByteArrayOutputStream os = new ByteArrayOutputStream();
try {
ImageIO.write(img, formatName, Base64.getEncoder().wrap(os));
return os.toString(StandardCharsets.ISO_8859_1.name());
} catch (final IOException ioe) {
throw new UncheckedIOException(ioe);
}
}
public static BufferedImage base64StringToImg(final String base64String) {
try {
return ImageIO.read(new ByteArrayInputStream(Base64.getDecoder().decode(base64String)));
} catch (final IOException ioe) {
throw new UncheckedIOException(ioe);
}
}
Taken from: Java BufferedImage to PNG format Base64 String

The years have passed...
Nowadays, I think the solution can be fount here: https://www.geeksforgeeks.org/url-encoding-decoding-using-base64-in-java/
So this worked for me:
Base64.getUrlEncoder().encodeToString(bytes);
To get the byte[] bytes (completely) filled as another topic...
I tested adding an html inline image when sending an email...
<img src="data:image/jpeg;base64,_9j_4AAQSkZ...

Java crypt base64 and compression at the same time

I need some consulting here.
I'm writing a client/server software.
One requirement is to use an XML file to transfer information between the client and the server. This decision was made to allow clients to be created on any language. I know I could use json for simpler communication, but I needed a better Object to Text Mapping, so I ended up with XML. So, to diminish the XMl size problem, I decided to compress it before sending over socket. On top of it, I want to encrypt the data so it will be harder to be broken by some interceptor. I have read a lot about encrypting and encryption types, but for testing purposes I'll just use Base64 and generic compression just to join up all the technologies, than I'll specialize the specifics. I found a lot of code and good advice here on stack exchange, and now I'm having some trouble.
What I have:
In memory Objects;
Static methods to convert Objects to XML in memory;
No use for Serialization because the clients can be written on C, .net, perl, etc;
Avoid disk write to reduce disk I/O, keeping everything in-memory;
The need to work with punctuation chars (UTF-8);
The need to work with stronger encrypt on the future;
The process I idealized is:
Convert object to XML - OK
Establish Client/server communication - OK
Encode XML String to Base64 - OK (no compression)
3.1. Compress XML String and than encode to Base64 - OK
Transfer between client/server - OK
Decode Base64 String to XML String - OK
5.1. Uncompress transferred byte array and decode Base64 - NOT OK
So, I can't find where I'm getting wrong on the "Compress String -> Encode it to Base64 -> transmit -> decode Base64 to String -> uncompress it" process... Here is the snippet from my code "inspired" on stack overflow:
I'm using this answer (with .net compatibility) as a reference to compress/uncompress:
How can I Zip and Unzip a string using GZIPOutputStream that is compatible with .Net?
public class Compressor2 {
public static byte[] compress(String string) throws IOException {
byte[] blockcopy = ByteBuffer
.allocate(4)
.order(java.nio.ByteOrder.LITTLE_ENDIAN)
.putInt(string.length())
.array();
ByteArrayOutputStream os = new ByteArrayOutputStream(string.length());
GZIPOutputStream gos = new GZIPOutputStream(os);
gos.write(string.getBytes());
gos.close();
os.close();
byte[] compressed = new byte[4 + os.toByteArray().length];
System.arraycopy(blockcopy, 0, compressed, 0, 4);
System.arraycopy(os.toByteArray(), 0, compressed, 4, os.toByteArray().length);
compressed = MyBase64.encode(new String(compressed)).getBytes();
return compressed;
}
public static String decompress(byte[] compressed) throws IOException {
compressed = MyBase64.decode(new String(compressed)).getBytes();
System.out.println(compressed);
System.out.println(new String(compressed));
final int BUFFER_SIZE = 32;
//ByteArrayInputStream is = new ByteArrayInputStream(compressed, 4, compressed.length - 4);
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
StringBuilder string = new StringBuilder();
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
while ((bytesRead = gis.read(data)) != -1) {
string.append(new String(data, 0, bytesRead));
}
gis.close();
is.close();
return string.toString();
}
from MyBase 64:
public static String encode(String text) throws UnsupportedEncodingException {
byte[] encodedBytes = Base64.encodeBase64(text.getBytes());
return new String(encodedBytes, "UTF-8");
}
public static String decode(String text) throws UnsupportedEncodingException {
byte[] decodedBytes = Base64.decodeBase64(text);
return new String(decodedBytes, "UTF-8");
}
and the test case:
/**
* Test of compress method, of class Compressor2.
*/
#Test
public void testCompress() throws Exception {
System.out.println("compress");
String string = "Hello all, how can I transfer this with punctuation? like á é í ó ú";
byte[] expResult = Compressor2.compress(string);
byte[] result = Compressor2.compress(string);
System.out.write(result);
System.out.print("\n");
System.out.println(new String(result));
assertArrayEquals(expResult, result);
for (byte character : result) {
System.out.print(String.valueOf(character));
}
System.out.print("\n");
for (byte character : result) {
System.out.print((character));
}
System.out.print("\n");
}
/**
* Test of decompress method, of class Compressor2.
*/
#Test
public void testDecompress() throws Exception {
System.out.println("decompress");
String expResult = "Hello all, how can I transfer this with punctuation? like á é í ó ú";
//String expResult = "The lazy dog ate an apple";
byte[] compressed = Compressor2.compress(expResult);
System.out.println(expResult);
System.out.write(compressed);
System.out.print("\n");
System.out.println("going to decompress");
String result = Compressor2.decompress(compressed);
System.out.println(result);
assertEquals(expResult, result);
// TODO punctuation errors?
}
System Out for checking the values:
compress
QwAAAB/vv70IAAAAAAAAAAXvv73vv70N77+9MAwE77+9VX4Adu+/ve+/vTHvv73vv70oFu+/vRMl77+9Mg8lBRUd77+9F++/vW5j77+9AlJd77+977+9RCLvv70Ob2Tvv73vv70G77+977+9Me+/vTPvv73vv71c77+977+9UDkZcSFuxIN4Ee+/vQ/nn5ENSAAAAA==
QwAAAB/vv70IAAAAAAAAAAXvv73vv70N77+9MAwE77+9VX4Adu+/ve+/vTHvv73vv70oFu+/vRMl77+9Mg8lBRUd77+9F++/vW5j77+9AlJd77+977+9RCLvv70Ob2Tvv73vv70G77+977+9Me+/vTPvv73vv71c77+977+9UDkZcSFuxIN4Ee+/vQ/nn5ENSAAAAA==
811196565656647118118554873656565656565656565658811811855511181185548785555435777651196955554357868852651001174347118101434711884721181185551118118554811170117434711882771085555435777103561086682851005555435770434347118875310655554357651087410055554357555543578267761181185548799850841181185551118118554871555543575555435777101434711884801181185551118118554999555543575555435785681079099837011712073785269101434711881471101105369788365656565656161
811196565656647118118554873656565656565656565658811811855511181185548785555435777651196955554357868852651001174347118101434711884721181185551118118554811170117434711882771085555435777103561086682851005555435770434347118875310655554357651087410055554357555543578267761181185548799850841181185551118118554871555543575555435777101434711884801181185551118118554999555543575555435785681079099837011712073785269101434711881471101105369788365656565656161
decompress
Hello all, how can I transfer this with punctuation? like á é í ó ú
QwAAAB/vv70IAAAAAAAAAAXvv73vv70N77+9MAwE77+9VX4Adu+/ve+/vTHvv73vv70oFu+/vRMl77+9Mg8lBRUd77+9F++/vW5j77+9AlJd77+977+9RCLvv70Ob2Tvv73vv70G77+977+9Me+/vTPvv73vv71c77+977+9UDkZcSFuxIN4Ee+/vQ/nn5ENSAAAAA==
going to decompress
[B#19f9bdc4
C
Sorry for the long post, I'm trying to give you something to work on to check where I'm getting this wrong. I Appreciate any help...
Edit.: Forgot to say, if you take the "MyBase64.decode or encode" line from the Compressor2 Classe, it works like a charm...