I used ostermillerutils library to create base64 string but I get OutOfMemory error if the image is heavy. If the image I try to convert is a simple image, the code is working fine.
public String createBase64String(InputStream in) {
//collect = new ByteArrayOutputStream();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for(int readNum; (readNum = in.read(buf)) != -1; ) {
bos.write(buf, 0, readNum);
}
}
catch (IOException ex) {
Logger.getInstance().debug("XML createBase64String: IOException");
return null;
}
finally {
if (in != null) {
try {
in.close();
}
catch (IOException ex) {
;
}
}
}
byte[] ba = bos.toByteArray();
String coded = Base64.encodeToString(ba);
return coded;
}
I also tried doing this but the base64 was incorrect when I tried to decode it.
public void createBase64String(InputStream in) throws IOException {
//collect = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int readNum = 0;
try {
while((readNum = in.read(buf)) != -1)
{
smtp.addBase64(Base64.encodeBase64String(buf));
}
}
catch (IOException ex) {
Logger.getInstance().debug("XML createBase64String: IOException");
}
finally {
if (in != null) {
in.close();
}
}
}
Please suggest solutions for JDK 1.4 and also for later versions of Java.
If you like to write the encoded content straight into a file then use the following code
public void encode(File file, OutputStream base64OutputStream) {
InputStream is = new FileInputStream(file);
OutputStream out = new Base64OutputStream(base64OutputStream)
IOUtils.copy(is, out);
is.close();
out.close();
}
IOUtils class from Apache Commons IO.
EDIT
Since you want to do it using BufferedWriter, use it as follows
OutputStream out = Base64OutputStream(smtpSocket.getOutputStream());
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(out));
IOUtils.copy(is, bw);
It sounds like the problem is that you're having to manipulate too much data in memory when you read the entire image. One fix would be to increase the Java heap size until you have enough memory, but that would just be avoiding the problem rather than solving it.
A better option would be to look at a streaming implementation of a Base64 encoder. This would mean you're only working on a subset of the image at any time. I believe that Base64OutputStream from Apache Commons would do the job for you.
I've fixed my problem by using javabase64-1.3.1.jar library.
OutputStream fos2 = FileUtil.getOutputStream(base64FileName, FileUtil.HDD);
InputStream in2 = FileUtil.getInputStream(fileName, FileUtil.HDD);
Base64.encode(in2, fos2);
in2.close();
fos2.close();
I stored the base64 string to a text file first.
public void createBase64String(InputStream in) throws IOException {
baos = new ByteArrayOutputStream();
byte[] buf = new byte[BUFFER_SIZE];
int readNum = 0;
smtp.addBase64("\t\t");
try {
while ((readNum = in.read(buf)) >= 0) {
baos.write(buf, 0, readNum);
smtp.addBase64(baos.toString());
baos.reset();
}
}
catch (IOException ex) {
LogUtil.error("Sending of Base64 String to SMTP: IOException: " + ex);
}
finally {
if (in != null) {
in.close();
baos.close();
}
}
baos = null;
buf = null;
}
then send each line to smtp's socket outputstream.
From Java 8 onwards, there is a simple way to implement base64 encoding in an output stream with one line of code and no external dependencies:
import java.util.Base64;
OutputStream os = ...
OutputStream base64 = Base64.getEncoder().wrap(os);
Base64 also provides other flavors of base64 encoder; see javadocs:
Base64
Base64.Encoder.wrap
Related
I have a requirement to create a zip file from input stream data, and before writing to zip I need to find the checksum for the input stream.
To do that I am using below codes:
private String writeZipFileToFS(List<ResponsePacks> attachmentList) throws IOException
{
File fileToWrite = new File(getZipPath() + "fileName.zip");
try
{
FileUtils.copyInputStreamToFile(compress(attachmentList), fileToWrite);
}
catch (IOException e)
{
throw e;
}
return fileName;
}
private InputStream compress(List<ResponsePacks> attachmentList)
{
byte buffer[] = new byte[2048];
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
ZipOutputStream zipFileToSend = new ZipOutputStream(byteStream);
try
{
for (ResponsePacks info : attachmentList)
{
// only for successful requests files would need to be added
zipFileToSend.putNextEntry(new ZipEntry(info.getFileName()));
InputStream in = info.getFileContentStream();
getCheckSum(in, info.getFileName());
int length;
while ((length = in.read(buffer)) >= 0)
{
zipFileToSend.write(buffer, 0, length);
}
zipFileToSend.closeEntry();
}
zipFileToSend.close();
}
catch (IOException e)
{
throw e;
}
return new ByteArrayInputStream(byteStream.toByteArray());
}
private static void getCheckSum(InputStream is, String fileName)
{
byte[] dataCopy = null;
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
try
{
IOUtils.copy(is, outputStream);
dataCopy = outputStream.toByteArray();
printLog("Byte Array Size {}", dataCopy.length);
String checkSum = calculateChecksum(dataCopy);
printLog("Checksum for file {} {}", fileName, checkSum);
outputStream.flush();
outputStream.close();
}
catch (Exception e)
{
printLog("Error on calculationg checksum {}", e.getMessage());
}
}
private static String calculateChecksum(byte[] dataCopy)
{
try (ZipInputStream zipInputStream = new ZipInputStream(new ByteArrayInputStream(dataCopy)))
{
ZipEntry zipEntry;
MessageDigest digest = DigestUtils.getSha256Digest();
DWriter writer = new DWriter(digest);
while ((zipEntry = zipInputStream.getNextEntry()) != null)
{
byte[] entityData = IOUtils.toByteArray(zipInputStream);
if (!zipEntry.isDirectory())
{
writer.write(entityData);
}
}
if (writer.getChecksum() != null)
{
return writer.getChecksum();
}
}
catch (Exception e)
{
throw e;
}
return "";
}
static class DWriter
{
private final MessageDigest myDigest;
DWriter(MessageDigest digest)
{
myDigest = digest;
}
public void write(byte[] data)
{
myDigest.update(data);
}
public String getChecksum()
{
return new String(Hex.encodeHex(myDigest.digest()));
}
}
But problem is if I am adding code to calculate the checksum then zip file creating with empty content and if I am removing the checksum calculation code then zip file creating with proper contents.
And also when I check the log I found InputStream contents different contents but still I am getting the same checkSum (empty string) always as below:
Byte Array Size 20854
Checksum for file 20200910173919142.json e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Byte Array Size 14383
Checksum for file 1599752440405.zip e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
I am unable to find where I am doing wrong, due to which zip file is creating with empty content, and checkSum also creating same always.
Requesting to help me to find where I am doing wrong.
You consume twice the same inputstream: first you read it to get the checksum and the you read it again to write the zip entry.
getCheckSum(in, info.getFileName());
int length;
while ((length = in.read(buffer)) >= 0)
{
zipFileToSend.write(buffer, 0, length);
}
The second time you're trying to read, there's nothing to read anymore, so nothing gets written into the zip entry.
Some input streams can be reset and read multiple times, if that's not the case here you would need to save the data into a ByteArrayOutputStream (as you're already doing inside the getCheckSum() method), and then you could read that data multiple times.
I have following method for coping jpg photos from one folder to another:
public static void copyImage(String from, String to) {
try {
File sourceimage = new File(from);
BufferedImage image = ImageIO.read(sourceimage);
ImageIO.write(image, "jpg", new File(to));
} catch (IOException ex) {
Logger.getLogger(ImgLib.class.getName()).log(Level.SEVERE, null, ex);
} catch (NullPointerException ex){
Logger.getLogger(ImgLib.class.getName()).log(Level.SEVERE, null, ex);
}
}
It works, but little bit loosing quality of photo.
How I can achieve "perfect" cloning without loosing quality?
Yes, you are right. In this line:
ImageIO.write(image, "jpg", new File(to));
Your method is still re-encoding the image data which, with a lossy format like JPEG, will inevitably cause a loss of fidelity in the image.
I think, you may try to copy the image file using this code:
InputStream is = null;
OutputStream os = null;
try {
is = new FileInputStream(new File("path/to/img/src"));
os = new FileOutputStream(new File("path/to/img/dest"));
byte[] buffer = new byte[8192];
int length;
while ((length = is.read(buffer)) > 0) {
os.write(buffer, 0, length);
}
} finally {
is.close();
os.close();
}
Also, you may Apache Commons IOUtils to simplify copying from one stream to the other or if you are using Java 8 then you can just call Files.copy method.
InputStream is = null;
OutputStream os = null;
try {
is = new FileInputStream(new File("path/to/img/src"));
os = new FileOutputStream(new File("path/to/img/dest"));
byte[] buffer = new byte[1024];
int length;
while ((length = is.read(buffer)) > 0) {
os.write(buffer, 0, length);
}
} finally {
is.close();
os.close();
}
You have used BufferedImage which read the file into a image object.
instead you should read and write image file in the same way as you do with binary files (use InputStraem and OutputStream).
I have written a method that encodes a given PDF-file into a byte array:
public static byte[] encodeFileToBase64(String pathToPdfFile)
throws IOException {
File file = new File(pathToPdfFile);
InputStream input = null;
try {
input = new FileInputStream(file);
} catch (FileNotFoundException e) {
throw (e);
}
byte[] buffer = new byte[(int) file.length()];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int bytesRead;
try {
while ((bytesRead = input.read(buffer)) != -1) {
baos.write(buffer, 0, bytesRead);
}
} catch (IOException e) {
throw (e);
}
input.close();
return baos.toByteArray();
}
Now I am implementing an interface to an SOAP webservice.
The user manual demands a PDF file in Base64 encoding.
I have generated Java code from the given wsdl file with Apache Axis2 (wsdl2java). In this code, it is required to set the given PDF file as javax.activation.DataHandler:
/**
* Auto generated setter method
* #param param PdfDocument
*/
public void setPdfDocument(javax.activation.DataHandler param) {
this.localPdfDocument = param;
}
Now, my question is, how to get the Base64 encoded stuff into a DataHandler.
Can you help me?
Thank you!
try this :
DataSource fds = new FileDataSource("filePath");
request.setMessageFile(new DataHandler(fds));
the javax.activation.* package handle the base64 encoding natively.
I can't use ImageIO.read() because of my own restrictions. I can only load bytes after GET request and I need to save this bytes to file as image. But it seems to me, that there also loads some extra data, which browser usually filter (maybe response headers). So I get the array of raw bytes which I even can't open as image.
What should I do with this bytes?
Example:
byte[] buf = ContentLoader.loadBytes(new URL("http://images.visitcanberra.com.au/images/canberra_hero_image.jpg"));
try {
FileOutputStream fileOutputStream = new FileOutputStream(new File("D:\\image.jpg"));
fileOutputStream.write(buf);
fileOutputStream.flush();
} catch (IOException e) {
e.printStackTrace();
}
loadBytes() method:
public static byte[] loadBytes(URL url) {
ByteArrayOutputStream boutArray = new ByteArrayOutputStream();
try {
URLConnection connection = url.openConnection();
BufferedInputStream bin = new BufferedInputStream(connection.getInputStream());
byte[] buffer = new byte[1024 * 16];
while (bin.read(buffer) != -1) {
boutArray.write(buffer);
boutArray.flush();
}
bin.close();
} catch (Exception e) {
return null;
}
return boutArray.toByteArray();
}
Usual problems. The standard way to copy a stream in Java is:
int count;
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
out.close();
in.close();
Note that you need to store the result returned by read() into a variable; that you need to use it in the next write() call; that you shouldn't flush() inside a loop; and that you need to close the input and output streams.
And why you're using a ByteArrayInputStream at all is a mystery. It's just a waste of time and space. Read directly from the URL input stream, and write directly to the FileOutputStream.
The following code works for me:-
URL url = new URL("my url...");
InputStream is = url.openStream();
OutputStream os = new FileOutputStream("img.jpg");
byte[] b = new byte[2048];
int length;
while ((length = is.read(b)) != -1) {
os.write(b, 0, length);
}
is.close();
os.close();
I'm trying to read an image from an URL (with the Java package
java.net.URL) to a byte[]. "Everything" works fine, except that the content isn't being entirely read from the stream (the image is corrupt, it doesn't contain all the image data)... The byte array is being persisted in a database (BLOB). I really don't know what the correct approach is, maybe you can give me a tip. :)
This is my first approach (code formatted, removed unnecessary information...):
URL u = new URL("http://localhost:8080/images/anImage.jpg");
int contentLength = u.openConnection().getContentLength();
Inputstream openStream = u.openStream();
byte[] binaryData = new byte[contentLength];
openStream.read(binaryData);
openStream.close();
My second approach was this one (as you'll see the contentlength is being fetched another way):
URL u = new URL(content);
openStream = u.openStream();
int contentLength = openStream.available();
byte[] binaryData = new byte[contentLength];
openStream.read(binaryData);
openStream.close();
Both of the code result in a corrupted image...
I already read this post from Stack Overflow.
There's no guarantee that the content length you're provided is actually correct. Try something akin to the following:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
InputStream is = null;
try {
is = url.openStream ();
byte[] byteChunk = new byte[4096]; // Or whatever size you want to read in at a time.
int n;
while ( (n = is.read(byteChunk)) > 0 ) {
baos.write(byteChunk, 0, n);
}
}
catch (IOException e) {
System.err.printf ("Failed while reading bytes from %s: %s", url.toExternalForm(), e.getMessage());
e.printStackTrace ();
// Perform any other exception handling that's appropriate.
}
finally {
if (is != null) { is.close(); }
}
You'll then have the image data in baos, from which you can get a byte array by calling baos.toByteArray().
This code is untested (I just wrote it in the answer box), but it's a reasonably close approximation to what I think you're after.
Just extending Barnards's answer with commons-io. Separate answer because I can not format code in comments.
InputStream is = null;
try {
is = url.openStream ();
byte[] imageBytes = IOUtils.toByteArray(is);
}
catch (IOException e) {
System.err.printf ("Failed while reading bytes from %s: %s", url.toExternalForm(), e.getMessage());
e.printStackTrace ();
// Perform any other exception handling that's appropriate.
}
finally {
if (is != null) { is.close(); }
}
http://commons.apache.org/io/api-1.4/org/apache/commons/io/IOUtils.html#toByteArray(java.io.InputStream)
Here's a clean solution:
private byte[] downloadUrl(URL toDownload) {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
try {
byte[] chunk = new byte[4096];
int bytesRead;
InputStream stream = toDownload.openStream();
while ((bytesRead = stream.read(chunk)) > 0) {
outputStream.write(chunk, 0, bytesRead);
}
} catch (IOException e) {
e.printStackTrace();
return null;
}
return outputStream.toByteArray();
}
I am very surprised that nobody here has mentioned the problem of connection and read timeout. It could happen (especially on Android and/or with some crappy network connectivity) that the request will hang and wait forever.
The following code (which also uses Apache IO Commons) takes this into account, and waits max. 5 seconds until it fails:
public static byte[] downloadFile(URL url)
{
try {
URLConnection conn = url.openConnection();
conn.setConnectTimeout(5000);
conn.setReadTimeout(5000);
conn.connect();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
IOUtils.copy(conn.getInputStream(), baos);
return baos.toByteArray();
}
catch (IOException e)
{
// Log error and return null, some default or throw a runtime exception
}
}
byte[] b = IOUtils.toByteArray((new URL( )).openStream()); //idiom
Note however, that stream is not closed in the above example.
if you want a (76-character) chunk (using commons codec)...
byte[] b = Base64.encodeBase64(IOUtils.toByteArray((new URL( )).openStream()), true);
Use commons-io IOUtils.toByteArray(URL):
String url = "http://localhost:8080/images/anImage.jpg";
byte[] fileContent = IOUtils.toByteArray(new URL(url));
Maven dependency:
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.6</version>
</dependency>
The content length is just a HTTP header. You cannot trust it. Just read everything you can from the stream.
Available is definitely wrong. It's just the number of bytes that can be read without blocking.
Another issue is your resource handling. Closing the stream has to happen in any case. try/catch/finally will do that.
It's important to specify timeouts, especially when the server takes to respond. With pure Java, without using any dependency:
public static byte[] copyURLToByteArray(final String urlStr,
final int connectionTimeout, final int readTimeout)
throws IOException {
final URL url = new URL(urlStr);
final URLConnection connection = url.openConnection();
connection.setConnectTimeout(connectionTimeout);
connection.setReadTimeout(readTimeout);
try (InputStream input = connection.getInputStream();
ByteArrayOutputStream output = new ByteArrayOutputStream()) {
final byte[] buffer = new byte[8192];
for (int count; (count = input.read(buffer)) > 0;) {
output.write(buffer, 0, count);
}
return output.toByteArray();
}
}
Using dependencies, e.g., HC Fluent:
public byte[] copyURLToByteArray(final String urlStr,
final int connectionTimeout, final int readTimeout)
throws IOException {
return Request.Get(urlStr)
.connectTimeout(connectionTimeout)
.socketTimeout(readTimeout)
.execute()
.returnContent()
.asBytes();
}