I follow several example from SO to get image file from zip, and put each file bytes into a hashmap:
final byte[] zip_file = ((CommonsMultipartFile) zip_file).getBytes();
zip_stream = new ZipInputStream(new ByteArrayInputStream(zip_file));
try {
while ((entry = zip_stream.getNextEntry()) != null) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try {
BufferedOutputStream dest = new BufferedOutputStream(baos, BUFFER_SIZE);
try {
int count = 0;
byte[] data = new byte[BUFFER_SIZE];
while ((count = zip_stream.read(data, 0, BUFFER_SIZE)) > 0) {
dest.write(data, 0, count);
}
dest.flush();
filelist.put(entry.getName(), baos.toByteArray());
baos.reset();
} finally {
dest.close();
}
} finally {
baos.close();
}
}
} finally {
zip_stream.close();
}
Later when reading from filelist, the byte array will persist into a java bean, just like this
Customer customer = new Customer();
byte[] image = fileist.get(imageFileName);
customer.setImage(image);
Customer is an JPA entity that field image is with #Lob type. So this part shouldn't have any issue.
The sad party is after the whole transaction there did some data write into 'image' field but from Oracle (using SQL developer) the bytes cannot compose to image file, which means from oracle the file is broken. There must be something wrong make the bytes corrupted. How can I make it work?
UPDATE
change inputstream-outputstream transfer using IOUtils.copy but still not working...But I feel something wrong here but don't know how to fix. in following code the looping seems to work on each entry of the zipInputStream, entry is never visited instead of the file name, is it look normal?
try {
while ((entry = zip_stream.getNextEntry()) != null) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try {
IOUtils.copy(zip_stream, baos);
} finally {
baos.close();
}
filelist.put(entry.getName(), baos.toByteArray());
}
} finally {
zip_stream.close();
}
Remove baos.set and move the filelist.put after baos.close.
Honestly I think one should nest dest and baos inversely, and dest.close should suffice, imply closing baos.
Also instead of getBytes one could do getInputStream.
Certainly there is IOUtils with a copy; somewhere there should be a copy with a flag "keep opened."
Directory entries were not skipped and closeEntry not called.
try {
ZipInputStream zipInputStream = new ZipInputStream(
new FileInputStream("D:/dev/... .zip"));
ZipEntry zipEntry;
while ((zipEntry = zipInputStream.getNextEntry()) != null) {
System.out.println("- " + zipEntry.getName()
+ " #" + zipEntry.getSize());
if (zipEntry.isDirectory()) {
zipInputStream.closeEntry();
continue;
}
long size = zipEntry.getSize();
if (size > Integer.MAX_VALUE) {
throw new IOException("File too large: " + zipEntry.getName());
}
int reserved = size == -1L ? 8192 : (int)size;
ByteArrayOutputStream baos = new ByteArrayOutputStream(reserved);
IOUtils.copy(zipInputStream, baos);
zipInputStream.closeEntry();
baos.close();
File file = new File("D:/dev/data/temp/" + zipEntry.getName());
file.getParentFile().mkdirs();
FileUtils.writeByteArrayToFile(file, baos.toByteArray());
}
} catch (IOException ex) {
Logger.getLogger(Stackoverflow.class.getName()).log(Level.SEVERE, null, ex);
}
Related
I have a requirement to create a zip file from input stream data, and before writing to zip I need to find the checksum for the input stream.
To do that I am using below codes:
private String writeZipFileToFS(List<ResponsePacks> attachmentList) throws IOException
{
File fileToWrite = new File(getZipPath() + "fileName.zip");
try
{
FileUtils.copyInputStreamToFile(compress(attachmentList), fileToWrite);
}
catch (IOException e)
{
throw e;
}
return fileName;
}
private InputStream compress(List<ResponsePacks> attachmentList)
{
byte buffer[] = new byte[2048];
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
ZipOutputStream zipFileToSend = new ZipOutputStream(byteStream);
try
{
for (ResponsePacks info : attachmentList)
{
// only for successful requests files would need to be added
zipFileToSend.putNextEntry(new ZipEntry(info.getFileName()));
InputStream in = info.getFileContentStream();
getCheckSum(in, info.getFileName());
int length;
while ((length = in.read(buffer)) >= 0)
{
zipFileToSend.write(buffer, 0, length);
}
zipFileToSend.closeEntry();
}
zipFileToSend.close();
}
catch (IOException e)
{
throw e;
}
return new ByteArrayInputStream(byteStream.toByteArray());
}
private static void getCheckSum(InputStream is, String fileName)
{
byte[] dataCopy = null;
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
try
{
IOUtils.copy(is, outputStream);
dataCopy = outputStream.toByteArray();
printLog("Byte Array Size {}", dataCopy.length);
String checkSum = calculateChecksum(dataCopy);
printLog("Checksum for file {} {}", fileName, checkSum);
outputStream.flush();
outputStream.close();
}
catch (Exception e)
{
printLog("Error on calculationg checksum {}", e.getMessage());
}
}
private static String calculateChecksum(byte[] dataCopy)
{
try (ZipInputStream zipInputStream = new ZipInputStream(new ByteArrayInputStream(dataCopy)))
{
ZipEntry zipEntry;
MessageDigest digest = DigestUtils.getSha256Digest();
DWriter writer = new DWriter(digest);
while ((zipEntry = zipInputStream.getNextEntry()) != null)
{
byte[] entityData = IOUtils.toByteArray(zipInputStream);
if (!zipEntry.isDirectory())
{
writer.write(entityData);
}
}
if (writer.getChecksum() != null)
{
return writer.getChecksum();
}
}
catch (Exception e)
{
throw e;
}
return "";
}
static class DWriter
{
private final MessageDigest myDigest;
DWriter(MessageDigest digest)
{
myDigest = digest;
}
public void write(byte[] data)
{
myDigest.update(data);
}
public String getChecksum()
{
return new String(Hex.encodeHex(myDigest.digest()));
}
}
But problem is if I am adding code to calculate the checksum then zip file creating with empty content and if I am removing the checksum calculation code then zip file creating with proper contents.
And also when I check the log I found InputStream contents different contents but still I am getting the same checkSum (empty string) always as below:
Byte Array Size 20854
Checksum for file 20200910173919142.json e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Byte Array Size 14383
Checksum for file 1599752440405.zip e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
I am unable to find where I am doing wrong, due to which zip file is creating with empty content, and checkSum also creating same always.
Requesting to help me to find where I am doing wrong.
You consume twice the same inputstream: first you read it to get the checksum and the you read it again to write the zip entry.
getCheckSum(in, info.getFileName());
int length;
while ((length = in.read(buffer)) >= 0)
{
zipFileToSend.write(buffer, 0, length);
}
The second time you're trying to read, there's nothing to read anymore, so nothing gets written into the zip entry.
Some input streams can be reset and read multiple times, if that's not the case here you would need to save the data into a ByteArrayOutputStream (as you're already doing inside the getCheckSum() method), and then you could read that data multiple times.
I have a BLOB file which I have got from the DB team. I know that its a PDF document (I opened using Notepad++ and I could see the file name) and I need to convert the same using java. I have checked for few examples and I couldn't find any example where the BLOB file itself is taken as an input instead of taking directly from the DB (Resultset). Can anyone please give some pointers as to how I can accomplish this?
Thanks in advance!
I have tried below,
File file = new File("C:/Users/User1/Desktop/0LK54E33K1477e2MCEU25JV0G8MG418S007N45JU.BLOB0");
FileInputStream fis = new FileInputStream(file);
//System.out.println(file.exists() + "!!");
//InputStream in = resource.openStream();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for (int readNum; (readNum = fis.read(buf)) != -1;) {
bos.write(buf, 0, readNum); //no doubt here is 0
//Writes len bytes from the specified byte array starting at offset off to this byte array output stream.
System.out.println("read " + readNum + " bytes,");
}
} catch (IOException ex) {
//Logger.getLogger(genJpeg.class.getName()).log(Level.SEVERE, null, ex);
}
byte[] bytes = bos.toByteArray();
//below is the different part
File someFile = new File("C:/Users/User1/Desktop/Test.pdf");
FileOutputStream fos = new FileOutputStream(someFile);
fos.write(bytes);
fos.flush();
fos.close();
I use NanoHTTPD as web server in my Android APP, I hope to compress some files and create a InputStream in server side, and I download the InputStream in client side using Code A.
I have read Code B at How to zip and unzip the files?, but how to create a ZIP InputStream in Android without creating a ZIP file first?
BTW, I don't think Code C is good way, because it make ZIP file first, then convert ZIP file to FileInputStream , I hope to create a ZIP InputStream directly!
Code A
private Response ActionDownloadSingleFile(InputStream fis) {
Response response = null;
response = newChunkedResponse(Response.Status.OK, "application/octet-stream",fis);
response.addHeader("Content-Disposition", "attachment; filename="+"my.zip");
return response;
}
Code B
public static void zip(String[] files, String zipFile) throws IOException {
BufferedInputStream origin = null;
ZipOutputStream out = new ZipOutputStream(new BufferedOutputStream(new FileOutputStream(zipFile)));
try {
byte data[] = new byte[BUFFER_SIZE];
for (int i = 0; i < files.length; i++) {
FileInputStream fi = new FileInputStream(files[i]);
origin = new BufferedInputStream(fi, BUFFER_SIZE);
try {
ZipEntry entry = new ZipEntry(files[i].substring(files[i].lastIndexOf("/") + 1));
out.putNextEntry(entry);
int count;
while ((count = origin.read(data, 0, BUFFER_SIZE)) != -1) {
out.write(data, 0, count);
}
}
finally {
origin.close();
}
}
}
finally {
out.close();
}
}
Code C
File file= new File("my.zip");
FileInputStream fis = null;
try
{
fis = new FileInputStream(file);
} catch (FileNotFoundException ex)
{
}
ZipInputStream as per the documentation ZipInputStream
ZipInputStream is an input stream filter for reading files in the ZIP file format. Includes support for both compressed and uncompressed entries.
Earlier I answered to this question in a way that it is not possible using ZipInputStream. I am Sorry.
But after investing some time I found that it is possible as per the below code
It is very much obvious that since you are sending files in zip format
over the network.
//Create proper background thread pool. Not best but just for solution
new Thread(new Runnable() {
#Override
public void run() {
// Moves the current Thread into the background
android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_BACKGROUND);
HttpURLConnection httpURLConnection = null;
byte[] buffer = new byte[2048];
try {
//Your http connection
httpURLConnection = (HttpURLConnection) new URL("https://s3-ap-southeast-1.amazonaws.com/uploads-ap.hipchat.com/107225/1251522/SFSCjI8ZRB7FjV9/zvsd.zip").openConnection();
//Change below path to Environment.getExternalStorageDirectory() or something of your
// own by creating storage utils
File outputFilePath = new File ("/mnt/sdcard/Android/data/somedirectory/");
ZipInputStream zipInputStream = new ZipInputStream(new BufferedInputStream(httpURLConnection.getInputStream()));
ZipEntry zipEntry = zipInputStream.getNextEntry();
int readLength;
while(zipEntry != null){
File newFile = new File(outputFilePath, zipEntry.getName());
if (!zipEntry.isDirectory()) {
FileOutputStream fos = new FileOutputStream(newFile);
while ((readLength = zipInputStream.read(buffer)) > 0) {
fos.write(buffer, 0, readLength);
}
fos.close();
} else {
newFile.mkdirs();
}
Log.i("zip file path = ", newFile.getPath());
zipInputStream.closeEntry();
zipEntry = zipInputStream.getNextEntry();
}
// Close Stream and disconnect HTTP connection. Move to finally
zipInputStream.closeEntry();
zipInputStream.close();
} catch (IOException e) {
e.printStackTrace();
}finally {
// Close Stream and disconnect HTTP connection.
if (httpURLConnection != null) {
httpURLConnection.disconnect();
}
}
}
}).start();
I've got a Table where i store my pdf files as blob.
I get the InputStream and insert it like this.
pstmt.setBinaryStream(1, inputStream);
For this I created a Model with Integer ID and InputStream blob; as variables.
I read the blob like this out of my DB.
blob.setBlob(rs.getBinaryStream("blob_file"));
Now I tried to create the PDF file again with this.
byte[] buffer = new byte[4096];
File file= new File("c:\\MyPath\\myPDF.pdf");
try{
FileOutputStream output= new FileOutputStream(file);
int b = 0;
while ((b = blob.getBlob().read()) != -1) {
output.write(buffer);
}
output.close();
}catch(IOException ex){
System.err.println("Blob Error: " + ex.getMessage());
}
With this method I get a corrupt PDF file which I can't open.
I found an alternative which worked very well like this.
IOUtils.copy(blob.getBlob(), output);
But I don't get why my first Version didn't work and what's the difference between These two.
Try this:
FileOutputStream output = null;
InputStream is = blob.getBlob();
try{
output= new FileOutputStream(file);
int b = 0;
while ((b = is.read(buffer)) != -1) {
output.write(buffer, 0, b);
}
} catch(IOException ex){
System.err.println("Blob Error: " + ex.getMessage());
} finally {
is.close();
if (output != null) {
output.close();
}
}
The problem in your initial code is the fact that you don't use the value of b (which is the the total number of bytes read into the buffer) so you probably write more bytes than you should which is probably the cause of the corruption of your file.
I used ostermillerutils library to create base64 string but I get OutOfMemory error if the image is heavy. If the image I try to convert is a simple image, the code is working fine.
public String createBase64String(InputStream in) {
//collect = new ByteArrayOutputStream();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for(int readNum; (readNum = in.read(buf)) != -1; ) {
bos.write(buf, 0, readNum);
}
}
catch (IOException ex) {
Logger.getInstance().debug("XML createBase64String: IOException");
return null;
}
finally {
if (in != null) {
try {
in.close();
}
catch (IOException ex) {
;
}
}
}
byte[] ba = bos.toByteArray();
String coded = Base64.encodeToString(ba);
return coded;
}
I also tried doing this but the base64 was incorrect when I tried to decode it.
public void createBase64String(InputStream in) throws IOException {
//collect = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int readNum = 0;
try {
while((readNum = in.read(buf)) != -1)
{
smtp.addBase64(Base64.encodeBase64String(buf));
}
}
catch (IOException ex) {
Logger.getInstance().debug("XML createBase64String: IOException");
}
finally {
if (in != null) {
in.close();
}
}
}
Please suggest solutions for JDK 1.4 and also for later versions of Java.
If you like to write the encoded content straight into a file then use the following code
public void encode(File file, OutputStream base64OutputStream) {
InputStream is = new FileInputStream(file);
OutputStream out = new Base64OutputStream(base64OutputStream)
IOUtils.copy(is, out);
is.close();
out.close();
}
IOUtils class from Apache Commons IO.
EDIT
Since you want to do it using BufferedWriter, use it as follows
OutputStream out = Base64OutputStream(smtpSocket.getOutputStream());
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(out));
IOUtils.copy(is, bw);
It sounds like the problem is that you're having to manipulate too much data in memory when you read the entire image. One fix would be to increase the Java heap size until you have enough memory, but that would just be avoiding the problem rather than solving it.
A better option would be to look at a streaming implementation of a Base64 encoder. This would mean you're only working on a subset of the image at any time. I believe that Base64OutputStream from Apache Commons would do the job for you.
I've fixed my problem by using javabase64-1.3.1.jar library.
OutputStream fos2 = FileUtil.getOutputStream(base64FileName, FileUtil.HDD);
InputStream in2 = FileUtil.getInputStream(fileName, FileUtil.HDD);
Base64.encode(in2, fos2);
in2.close();
fos2.close();
I stored the base64 string to a text file first.
public void createBase64String(InputStream in) throws IOException {
baos = new ByteArrayOutputStream();
byte[] buf = new byte[BUFFER_SIZE];
int readNum = 0;
smtp.addBase64("\t\t");
try {
while ((readNum = in.read(buf)) >= 0) {
baos.write(buf, 0, readNum);
smtp.addBase64(baos.toString());
baos.reset();
}
}
catch (IOException ex) {
LogUtil.error("Sending of Base64 String to SMTP: IOException: " + ex);
}
finally {
if (in != null) {
in.close();
baos.close();
}
}
baos = null;
buf = null;
}
then send each line to smtp's socket outputstream.
From Java 8 onwards, there is a simple way to implement base64 encoding in an output stream with one line of code and no external dependencies:
import java.util.Base64;
OutputStream os = ...
OutputStream base64 = Base64.getEncoder().wrap(os);
Base64 also provides other flavors of base64 encoder; see javadocs:
Base64
Base64.Encoder.wrap