Develop Reference

Is it possible to convert a zip file to binary file to load into BLOB

I have to upload a zip file into BLOB using informatica, for this task, I am using java transformation. Using the following code, I was able to upload all the flat files and retrieve them from the database table in correct format.
This code is not working for zip files. Can you please suggest me on how to convert a zip file into a binary file so that it can be inserted into BLOB?
byte bytes[] = null;
File f1 = new File(TARGETFILE);
try (FileInputStream fis = new FileInputStream(f1)) {
try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
byte[] buffer = new byte[1024];
int read = -1;
while ((read = fis.read(buffer)) != -1) {
baos.write(buffer, 0, read);
    }
bytes = baos.toByteArray();
FILE_CONTENT=bytes;
FILE_SIZE=Double.toString(f1.length()/1024*1024);
}
catch(Exception e1) {
}
}
catch (IOException exp) {
exp.printStackTrace();
}

A very simple version, just converts the data to a hex print. Using these parts you should be able to get to binary no problem. Just save the conversion to memory rather then print the string.
package test;
import java.io.BufferedInputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.util.Arrays;
/**
*
* #author User
*/
public class Test {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
File file = new File("C:\\Users\\User\\Downloads\\ltc1298-arduino-library-master.zip");
byte[] bytes = new byte[(int)file.length()];
try {
DataInputStream dataInputStream = new DataInputStream(new BufferedInputStream(new FileInputStream("C:\\Users\\User\\Downloads\\ltc1298-arduino-library-master.zip")));
dataInputStream.readFully(bytes);
System.out.println(String.format(Arrays.toString(bytes)));
for (byte b : bytes) {
System.out.format("0x%x ", b);
}
dataInputStream.close();
}
catch (Exception e) {
System.out.println("Error");
e.printStackTrace();
}
}
}

Checking MD5 of a file

I am getting an error while trying to check the MD5 hash of a file.
The file, notice.txt has the following contents:
My name is sanjay yadav . i am in btech computer science .>>
When I checked online with onlineMD5.com it gave the MD5 as: 90F450C33FAC09630D344CBA9BF80471.
My program output is:
My name is sanjay yadav . i am in btech computer science .
Read 58 bytes
d41d8cd98f00b204e9800998ecf8427e
Here's my code:
import java.io.*;
import java.math.BigInteger;
import java.security.DigestException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
public class MsgDgt {
public static void main(String[] args) throws IOException, DigestException, NoSuchAlgorithmException {
FileInputStream inputstream = null;
byte[] mybyte = new byte[1024];
inputstream = new FileInputStream("e://notice.txt");
int total = 0;
int nRead = 0;
MessageDigest md = MessageDigest.getInstance("MD5");
while ((nRead = inputstream.read(mybyte)) != -1) {
System.out.println(new String(mybyte));
total += nRead;
md.update(mybyte, 0, nRead);
}
System.out.println("Read " + total + " bytes");
md.digest();
System.out.println(new BigInteger(1, md.digest()).toString(16));
}
}

There's a bug in your code and I believe the online tool is giving the wrong answer. Here, you're currently computing the digest twice:
md.digest();
System.out.println(new BigInteger(1, md.digest()).toString(16));
Each time you call digest(), it resets the internal state. You should remove the first call to digest(). That then leaves you with this as the digest:
2f4c6a40682161e5b01c24d5aa896da0
That's the same result I get from C#, and I believe it to be correct. I don't know why the online checker is giving an incorrect result. (If you put it into the text part of the same site, it gives the right result.)
A couple of other points on your code though:
You're currently using the platform default encoding when converting the bytes to a string. I would strongly discourage you from doing that.
You're currently converting the whole buffer to a string, instead of only the bit you've read.
I don't like using BigInteger as a way of converting binary data to hex. You potentially need to pad it with 0s, and it's basically not what the class was designed for. Use a dedicated hex conversion class, e.g. from Apache Commons Codec (or various Stack Overflow answers which provide standalone classes for the purpose).
You're not closing your input stream. You should do so in a finally block, or using a try-with-resources statement in Java 7.

I use this function:
public static String md5Hash(File file) {
try {
MessageDigest md = MessageDigest.getInstance("MD5");
InputStream is = new FileInputStream(file);
byte[] buffer = new byte[1024];
try {
is = new DigestInputStream(is, md);
while (is.read(buffer) != -1) { }
} finally {
is.close();
}
byte[] digest = md.digest();
BigInteger bigInt = new BigInteger(1, digest);
String output = bigInt.toString(16);
while (output.length() < 32) {
output = "0" + output;
}
return output;
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}

Decompressed video file is not working in Java

Basically i compress video using the customized compressor class in Java. I have assembled my complete code snippets here. My actually problem is, generated video [ A.mp4] from the decompressed byte array is not running. I actually i got this compressor class code over the internet. As i new to Java platform, i am struggling to resolve this problem. Could you please any one help me on this.?
public class CompressionTest
{
public static void main(String[] args)
{
Compressor compressor = new Compressor();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
FileInputStream fis=null;
File file=null;
try
{
URL uri=CompressionTest.class.getResource("/Files/Video.mp4");
file=new File(uri.getPath());
fis = new FileInputStream(file);
}
catch ( FileNotFoundException fnfe )
{
System.out.println( "Unable to open input file");
}
try
{
byte[] videoBytes = getBytesFromFile(file);
System.out.println("CompressionVideoToCompress is: '" +videoBytes + "'");
byte[] bytesCompressed = compressor.compress(videoBytes);
System.out.println("bytesCompressed is: '" +bytesCompressed+ "'");
byte[] bytesDecompressed=compressor.decompress(bytesCompressed);
System.out.println("bytesDecompressed is: '" +bytesDecompressed+ "'");
FileOutputStream out = new FileOutputStream("A.mp4");
out.write(bytesDecompressed,0,bytesDecompressed.length-1);
out.close();
}
catch (IOException e)
{
// TODO Auto-generated catch block
System.out.println("bytesCompressed is: '");
}
}
public static byte[] getBytesFromFile(File file) throws IOException
{
InputStream is = new FileInputStream(file);
// Get the size of the file
long length = file.length();
// You cannot create an array using a long type.
// It needs to be an int type.
// Before converting to an int type, check
// to ensure that file is not larger than Integer.MAX_VALUE.
if (length > Integer.MAX_VALUE) {
// File is too large
}
// Create the byte array to hold the data
byte[] bytes = new byte[1064];
// Read in the bytes
int offset = 0;
int numRead = 0;
while (offset < bytes.length
&& (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0)
{
offset += numRead;
}
// Ensure all the bytes have been read in
if (offset < bytes.length) {
throw new IOException("Could not completely read file "+file.getName());
}
// Close the input stream and return bytes
is.close();
return bytes;
}
}
class Compressor
{
public Compressor()
{}
public byte[] compress(byte[] bytesToCompress)
{
Deflater deflater = new Deflater();
deflater.setInput(bytesToCompress);
deflater.finish();
byte[] bytesCompressed = new byte[Short.MAX_VALUE];
int numberOfBytesAfterCompression = deflater.deflate(bytesCompressed);
byte[] returnValues = new byte[numberOfBytesAfterCompression];
System.arraycopy
(
bytesCompressed,
0,
returnValues,
0,
numberOfBytesAfterCompression
);
return returnValues;
}
public byte[] decompress(byte[] bytesToDecompress)
{
Inflater inflater = new Inflater();
int numberOfBytesToDecompress = bytesToDecompress.length;
inflater.setInput
(
bytesToDecompress,
0,
numberOfBytesToDecompress
);
int compressionFactorMaxLikely = 3;
int bufferSizeInBytes =
numberOfBytesToDecompress
* compressionFactorMaxLikely;
byte[] bytesDecompressed = new byte[bufferSizeInBytes];
byte[] returnValues = null;
try
{
int numberOfBytesAfterDecompression = inflater.inflate(bytesDecompressed);
returnValues = new byte[numberOfBytesAfterDecompression];
System.arraycopy
(
bytesDecompressed,
0,
returnValues,
0,
numberOfBytesAfterDecompression
);
}
catch (DataFormatException dfe)
{
dfe.printStackTrace();
}
inflater.end();
return returnValues;
}
}

I've tested your code by compressing and decompressing a simple TXT file. The code is broken, since the compressed file, when uncompressed, is different from the original one.
Take for granted that the code is broken at least in the getBytesFromFile function. Its logic is tricky and troublesome, since it only allows files up to length 1064 and the check (throwing IOException when a longer file is read) does not work at all. The file gets read only partially and no exception is thrown.
What you are trying to achieve (file compression/decompression) can be done this way. I've tested it and it works, you just need this library.
import java.io.*;
import java.util.zip.*;
import org.apache.commons.io.IOUtils; // <-- get this from http://commons.apache.org/io/index.html
public class CompressionTest2 {
public static void main(String[] args) throws IOException {
File input = new File("input.txt");
File output = new File("output.bin");
Compression.compress(input, output);
File input2 = new File("input2.txt");
Compression.decompress(output, input2);
// At this point, input.txt and input2.txt should be equal
}
}
class Compression {
public static void compress(File input, File output) throws IOException {
FileInputStream fis = new FileInputStream(input);
FileOutputStream fos = new FileOutputStream(output);
GZIPOutputStream gzipStream = new GZIPOutputStream(fos);
IOUtils.copy(fis, gzipStream);
gzipStream.close();
fis.close();
fos.close();
}
public static void decompress(File input, File output) throws IOException {
FileInputStream fis = new FileInputStream(input);
FileOutputStream fos = new FileOutputStream(output);
GZIPInputStream gzipStream = new GZIPInputStream(fis);
IOUtils.copy(gzipStream, fos);
gzipStream.close();
fis.close();
fos.close();
}
}
This code doesn't come from "credible and/or official sources" but at least it works. :)
Moreover, in order to get more answers, adjust the title stating your real problem: your compressed files don't decompress the right way. There is no 'video' stuff here. Moreover, zipping a .mp4 file is no achievement (compression ratio will likely be around 99.99%).

Two tips:
1) Replace getBytesFromFile with a well known API call, either using Apache commons (IOUtils) or java 7 now provides such a method, too.
2) Test compress and decompress by writing a Junit test:
Create a random huge byte array, write it out, read it back and compare it with the created one.

Converting a file to short array and vice versa

How can I read a short array from a file e.g. audio or video file? And how can I write it back to the file?

I really doubt if SHORT if possible. Nevertheless, you can check out Apache Commons File Utils for reading file as byte[] and vice verse.
public static byte[] readFileToByteArray(File file) throws IOException

This is better example:
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
/**
Converting binary data into different forms.
<P>Reads binary data into memory, and writes it back out.
(If your're actually copying a file, there are better ways to do this.)
<P>Buffering is used when reading and writing files, to minimize the number
of interactions with the disk.
*/
public final class BytesStreamsAndFiles {
/** Change these settings before running this class. */
private static final String INPUT_FILE_NAME = "C:\\TEMP\\cottage.jpg";
private static final String OUTPUT_FILE_NAME = "C:\\TEMP\\cottage_copy.jpg";
/** Run the example. */
public static void main(String... aArgs) {
BytesStreamsAndFiles test = new BytesStreamsAndFiles();
//read in the bytes
byte[] fileContents = test.read(INPUT_FILE_NAME);
//test.readAlternateImpl(INPUT_FILE_NAME);
//write it back out to a different file name
test.write(fileContents, OUTPUT_FILE_NAME);
}
/** Read the given binary file, and return its contents as a byte array.*/
byte[] read(String aInputFileName){
log("Reading in binary file named : " + aInputFileName);
File file = new File(aInputFileName);
log("File size: " + file.length());
byte[] result = new byte[(int)file.length()];
try {
InputStream input = null;
try {
int totalBytesRead = 0;
input = new BufferedInputStream(new FileInputStream(file));
while(totalBytesRead < result.length){
int bytesRemaining = result.length - totalBytesRead;
//input.read() returns -1, 0, or more :
int bytesRead = input.read(result, totalBytesRead, bytesRemaining);
if (bytesRead > 0){
totalBytesRead = totalBytesRead + bytesRead;
}
}
/*
the above style is a bit tricky: it places bytes into the 'result' array;
'result' is an output parameter;
the while loop usually has a single iteration only.
*/
log("Num bytes read: " + totalBytesRead);
}
finally {
log("Closing input stream.");
input.close();
}
}
catch (FileNotFoundException ex) {
log("File not found.");
}
catch (IOException ex) {
log(ex);
}
return result;
}
/**
Write a byte array to the given file.
Writing binary data is significantly simpler than reading it.
*/
void write(byte[] aInput, String aOutputFileName){
log("Writing binary file...");
try {
OutputStream output = null;
try {
output = new BufferedOutputStream(new FileOutputStream(aOutputFileName));
output.write(aInput);
}
finally {
output.close();
}
}
catch(FileNotFoundException ex){
log("File not found.");
}
catch(IOException ex){
log(ex);
}
}
/** Read the given binary file, and return its contents as a byte array.*/
byte[] readAlternateImpl(String aInputFileName){
log("Reading in binary file named : " + aInputFileName);
File file = new File(aInputFileName);
log("File size: " + file.length());
byte[] result = null;
try {
InputStream input = new BufferedInputStream(new FileInputStream(file));
result = readAndClose(input);
}
catch (FileNotFoundException ex){
log(ex);
}
return result;
}
/**
Read an input stream, and return it as a byte array.
Sometimes the source of bytes is an input stream instead of a file.
This implementation closes aInput after it's read.
*/
byte[] readAndClose(InputStream aInput){
//carries the data from input to output :
byte[] bucket = new byte[32*1024];
ByteArrayOutputStream result = null;
try {
try {
//Use buffering? No. Buffering avoids costly access to disk or network;
//buffering to an in-memory stream makes no sense.
result = new ByteArrayOutputStream(bucket.length);
int bytesRead = 0;
while(bytesRead != -1){
//aInput.read() returns -1, 0, or more :
bytesRead = aInput.read(bucket);
if(bytesRead > 0){
result.write(bucket, 0, bytesRead);
}
}
}
finally {
aInput.close();
//result.close(); this is a no-operation for ByteArrayOutputStream
}
}
catch (IOException ex){
log(ex);
}
return result.toByteArray();
}
private static void log(Object aThing){
System.out.println(String.valueOf(aThing));
}
}
for more detail go to : Reading and writing binary files

The snippet below reads a file using FileInputStream, and writes the file to the given path using FileOutputStream.
Java Code:
byte[] fileBArray = new byte[(int)file.length()];
FileInputStream fis = new FileInputStream(file);
fis.read(fileBArray);
FileOutputStream fos = new FileOutputStream("C:\\abc.jpg");
fos.write(fileBArray);

Why aren't two files that should be equal not equal? I download a file with Java Socket, and compare it with the same file downloaded with Mozilla

I am downloading a file using Java Socket. The code is only for testing whether the files are equal or not. It is part of a bigger project.
import java.io.BufferedOutputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintWriter;
import java.net.Socket;
import java.util.Arrays;
public class Compare {
/**
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
Socket sk = null;
try
{
sk = new Socket("wlab.cs.bilkent.edu.tr", 80);
if (sk.isConnected())
{
PrintWriter out = new PrintWriter(sk.getOutputStream(),true);
out.println("GET /" + "/PA2/test5MB.bin" + " HTTP/1.1");
out.println("Host: " + 80);
// out.println("Range: " + 0 + '-' + 5242879);
out.println("Connection: close\r\n");
out.println("");
out.flush();
}
}catch(Exception e){
}
// receive file
byte [] mybytearray = new byte [5242880]; // when i use HEAD, this size is returned.
// FileInputStream is = new FileInputStream("C:\\Users\\Eda\\workspace\\network1\\t1est5MB.bin");
InputStream is = sk.getInputStream(); //when I used this one, the files don't match, but the above declaration works.
FileOutputStream fos = new FileOutputStream( new File ("C:\\Users\\Eda\\workspace\\network1\\tesst5MB.bin"));
BufferedOutputStream bos = new BufferedOutputStream(fos);
int offset3 = 0;
int numRead3 = 0;
System.out.println("1 in length: " +mybytearray.length);
while(offset3 < mybytearray.length
&& (numRead3=is.read(mybytearray, offset3, mybytearray.length-offset3)) >= 0 )
{
offset3 += numRead3;
// is=sk.getInputStream();
}
bos.write(mybytearray);
is.close();
bos.close();
//rest is for comparing two binary files.works except for the file I downloaded in this code above.
try{
File filename=new File("C:\\Users\\Eda\\workspace\\network1\\tesst5MB" +".bin");
File filename2=new File("C:\\Users\\Eda\\workspace\\network1\\t1est5MB" +".bin");
if(filename.exists() && filename2.exists())
System.out.println("both exists");
int size = (int)filename.length();
System.out.println("size1: " + size);
byte[] byteArray1 = new byte[size];
size = (int)filename2.length();
System.out.println("size2: " + size);
byte[] byteArray2 = new byte[size];
DataInputStream infile1 = new DataInputStream(new FileInputStream(filename));
DataInputStream infile2 = new DataInputStream(new FileInputStream(filename2));
int offset1 = 0;
int numRead1 = 0;
int offset2 = 0;
int numRead2 = 0;
System.out.println("1 in length: " +byteArray1.length);
while(offset1 < byteArray1.length
&& (numRead1=infile1.read(byteArray1, offset1, byteArray1.length-offset1)) >= 0)
{
offset1 += numRead1;
}
infile1.close();
while(offset2 < byteArray2.length
&& (numRead2=infile2.read(byteArray2, offset2, byteArray2.length-offset2)) >= 0)
{
offset2 += numRead2;
}
infile2.close();
System.out.println(Arrays.equals(byteArray1,byteArray2));
}
catch(Exception e){
}
}
}
When I copy a file by reading it byte by byte and compare, they are equal. But when I download the file through Socket and compare it to the file that I download with Mozilla (they should be equal) they are not equal. I don't know what is wrong with my sk.getInputStream(). I am stuck here. Can you tell me how? I don't know what to do. They are just not equal and I don't know why.
The filesize I gave is the file size when I used HEAD request. 5 MB

When you connect with a raw socket, you are going to get HTTP headers, that you need to parse, and not save into the output file. The real data starts after you see two newlines in a row, indicating the end of the headers. You should be reading Content-length from the headers, not hard-coding.
If you open up your downloaded file (the one you made by using Socket), you'll see those HTTP headers.
If this is meant as production code (not just playing around or a one-shot deal), you should not be using Socket for HTTP. HTTP is much more complex than what you have implemented here. At the very least, you should be checking the result code to make sure you got a 200.
Take a look at java.net.URLConnection