I have a zip file, and in my Java code i want to calculate the md5 hash of the zip file. Is there any java libary i can use for this purpose ?. Some example would be really appreciated.
Thank You
I got that working a few weeks ago with this Article here:
http://www.javalobby.org/java/forums/t84420.html
Just to have it a stackoveflow:
public static void main(String[] args) throws NoSuchAlgorithmException, FileNotFoundException {
MessageDigest digest = MessageDigest.getInstance("MD5");
File f = new File("c:\\myfile.txt");
InputStream is = new FileInputStream(f);
byte[] buffer = new byte[8192];
int read = 0;
try {
while( (read = is.read(buffer)) > 0) {
digest.update(buffer, 0, read);
}
byte[] md5sum = digest.digest();
BigInteger bigInt = new BigInteger(1, md5sum);
String output = bigInt.toString(16);
System.out.println("MD5: " + output);
}
catch(IOException e) {
throw new RuntimeException("Unable to process file for MD5", e);
}
finally {
try {
is.close();
}
catch(IOException e) {
throw new RuntimeException("Unable to close input stream for MD5 calculation", e);
}
}
}
There is also an option to do that with Apache Codec like that
final String sha256 = DigestUtils.sha256Hex(new FileInputStream(file))
Related
I wrote a program that gets all bytes from an InputStream in Java 9 with
InputStream.readAllBytes()
Now, I want to export it to Java 1.8 and below. Is there an equivalent function? Couldn't find one.
InputStream.readAllBytes() is available since java 9 not java 7...
Other than that you can (no thirdparties):
byte[] bytes = new byte[(int) file.length()];
DataInputStream dataInputStream = new DataInputStream(new FileInputStream(file));
dataInputStream .readFully(bytes);
Or if you don't mind using thirdparties (Commons IO):
byte[] bytes = IOUtils.toByteArray(is);
Guava also helps:
byte[] bytes = ByteStreams.toByteArray(inputStream);
You can use the good old read method like this:
public static byte[] readAllBytes(InputStream inputStream) throws IOException {
final int bufLen = 1024;
byte[] buf = new byte[bufLen];
int readLen;
IOException exception = null;
try {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
while ((readLen = inputStream.read(buf, 0, bufLen)) != -1)
outputStream.write(buf, 0, readLen);
return outputStream.toByteArray();
} catch (IOException e) {
exception = e;
throw e;
} finally {
if (exception == null) inputStream.close();
else try {
inputStream.close();
} catch (IOException e) {
exception.addSuppressed(e);
}
}
}
Question at the bottom
I'm using netty to transfer a file to another server.
I limit my file-chunks to 1024*64 bytes (64KB) because of the WebSocket protocol. The following method is a local example what will happen to the file:
public static void rechunck(File file1, File file2) {
FileInputStream is = null;
FileOutputStream os = null;
try {
byte[] buf = new byte[1024*64];
is = new FileInputStream(file1);
os = new FileOutputStream(file2);
while(is.read(buf) > 0) {
os.write(buf);
}
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
} finally {
try {
if(is != null && os != null) {
is.close();
os.close();
}
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
}
}
}
The file is loaded by the InputStream into a ByteBuffer and directly written to the OutputStream.
The content of the file cannot change while this process.
To get the md5-hashes of the file I've wrote the following method:
public static String checksum(File file) {
InputStream is = null;
try {
is = new FileInputStream(file);
MessageDigest digest = MessageDigest.getInstance("MD5");
byte[] buffer = new byte[8192];
int read = 0;
while((read = is.read(buffer)) > 0) {
digest.update(buffer, 0, read);
}
return new BigInteger(1, digest.digest()).toString(16);
} catch(IOException | NoSuchAlgorithmException e) {
Controller.handleException(Thread.currentThread(), e);
} finally {
try {
is.close();
} catch(IOException e) {
Controller.handleException(Thread.currentThread(), e);
}
}
return null;
}
So: just in theory it should return the same hash, shouldn't it? The problem is that it returns two different hashes that do not differ with every run.. file size stays the same and the content either.
When I run the method once for in: file-1, out: file-2 and again with in: file-2 and out: file-3 the hashes of file-2 and file-3 are the same! This means the method will properly change the file every time the same way.
1. 58a4a9fbe349a9e0af172f9cf3e6050a
2. 7b3f343fa1b8c4e1160add4c48322373
3. 7b3f343fa1b8c4e1160add4c48322373
Here is a little test that compares all buffers if they are equivalent. Test is positive. So there aren't any differences.
File file1 = new File("controller/templates/Example.zip");
File file2 = new File("controller/templates2/Example.zip");
try {
byte[] buf1 = new byte[1024*64];
byte[] buf2 = new byte[1024*64];
FileInputStream is1 = new FileInputStream(file1);
FileInputStream is2 = new FileInputStream(file2);
boolean run = true;
while(run) {
int read1 = is1.read(buf1), read2 = is2.read(buf2);
String result1 = Arrays.toString(buf1), result2 = Arrays.toString(buf2);
boolean test = result1.equals(result2);
System.out.println("1: " + result1);
System.out.println("2: " + result2);
System.out.println("--- TEST RESULT: " + test + " ----------------------------------------------------");
if(!(read1 > 0 && read2 > 0) || !test) run = false;
}
} catch (IOException e) {
e.printStackTrace();
}
Question: Can you help me chunking the file without changing the hash?
while(is.read(buf) > 0) {
os.write(buf);
}
The read() method with the array argument will return the number of files read from the stream. When the file doesn't end exactly as a multiple of the byte array length, this return value will be smaller than the byte array length because you reached the file end.
However your os.write(buf); call will write the whole byte array to the stream, including the remaining bytes after the file end. This means the written file gets bigger in the end, therefore the hash changed.
Interestingly you didn't make the mistake when you updated the message digest:
while((read = is.read(buffer)) > 0) {
digest.update(buffer, 0, read);
}
You just have to do the same when you "rechunk" your files.
Your rechunk method has a bug in it. Since you have a fixed buffer in there, your file is split into ByteArray-parts. but the last part of the file can be smaller than the buffer, which is why you write too many bytes in the new file. and that's why you do not have the same checksum anymore. the error can be fixed like this:
public static void rechunck(File file1, File file2) {
FileInputStream is = null;
FileOutputStream os = null;
try {
byte[] buf = new byte[1024*64];
is = new FileInputStream(file1);
os = new FileOutputStream(file2);
int length;
while((length = is.read(buf)) > 0) {
os.write(buf, 0, length);
}
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
} finally {
try {
if(is != null)
is.close();
if(os != null)
os.close();
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
}
}
}
Due to the length variable, the write method knows that until byte x of the byte array, only the file is off, then there are still old bytes in it that no longer belong to the file.
private static String encodeFileToBase64Binary(String fileName)
throws IOException {
File file = new File(fileName);
byte[] bytes = loadFile(file);
byte[] encoded = Base64.encodeBase64(bytes);
String encodedString = new String(encoded,StandardCharsets.US_ASCII);
return encodedString;
}
private static byte[] loadFile(File file) throws IOException {
InputStream is = new FileInputStream(file);
long length = file.length();
if (length > Integer.MAX_VALUE) {
// File is too large
}
byte[] bytes = new byte[(int)length];
int offset = 0;
int numRead = 0;
while (offset < bytes.length
&& (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) {
offset += numRead;
}
if (offset < bytes.length) {
throw new IOException("Could not completely read file "+file.getName());
}
is.close();
return bytes;
}
// to get encode string
String encoded=encodeFileToBase64Binary("file.fmr");
// encoded string is :
Rk1SACAyMAAAAAFiAAABQAHgAMUAxQEAAABGNkDZADP/SEC8AD6CSECqAEcGSED+AFJtO0CgAGCKZEC6AGuFZEDgAHz1ZECzAI6HZEENAJluNEBWAJ4ZZEB1AKkTZEECALbuZEA/ALqfSECCALySSECxAMP/ZECIAMURVUAXAN2jGkCnAOD8ZEAoAOWlZEBnAOyhLkCyAP/tZECHAQMSGkD8AQTdZECfASKFGkCHASUaGkA1ASy6ZEDAAS3JZEDPAS7NZEAnATG4ZEDxATzOZEBOAUPLZEBzAVbuGkCAAWF8NEDTAWsxLkDnAXa0LkC/AX2nLkC0AYojIEBMAYvkSEDJAa0fT0CsAbwVIIDqANTsZIDIAPfnZICbAQKHO4D5AR/XZIBlASS7IIEoASbYO4CsAUetLoDvAVXSZIDaAVvDO4EHAWrLZICsAX2fNIDnAYEwNIDQAZKnT4BfAZxtZAAA
//encoded string from file using some other source.
Rk1SACAyMAAAAAFiAAABQAHgAMUAxQEAAABGNkCLACELSEDAADYDZEEYAGFxO0DGAGJ9SEC1AGkCSEA6AHWYVUDJAHp5ZEBEAHwVZECVAJgIZEEaALHrZEB4ALuOZEELAMFqZEEzAM/sNEDRANvwZEBkAN0VZECcAOIAZEEwAOjnLkEvAPXlO0CnAP71ZEB7AQYRNEBdAQ0eZED8ARDhZEDXASXcZECZAS3uGkBoAT4eO0AUAUMxSEA7AUYqZEDxAUnSZECmAVNDO0EIAXDHSEDYAXW7ZEEUAXXKSEEGAYY8IEEhAYrDNEDfAZ81ZEDQAcGqLoEBAC/7O4EGAE7zVYB+AP2QSICEARuLZIBnATUfO4D/ATXaZIDEATjSZIDRATrVZICnATvSNIBTATwnZIARAV1LGoB1AV2oO4CrAV68SIDnAWHGZIB+AWauNICVAX0ySICNAYytO4CJAZorSAAA
When i am trying to match both the encoded string , i am getting a missmatch.
please suggest method for encode file to base64 to match encoded string found from other source.
i have tried with StandardCharsets.UTF_8 and StandardCharsets.US_ASCII.
You already using apache commons-codec so I recommend adding commons-io for reading the file. That way you can remove your loadFile() method and just have:
private static String encodeFileToBase64Binary(String fileName) throws IOException {
File file = new File(fileName);
byte[] encoded = Base64.encodeBase64(FileUtils.readFileToByteArray(file));
return new String(encoded, StandardCharsets.US_ASCII);
}
Here is a solution without any required dependencies (Apacha et al.) requiring only JDK 8+:
import java.util.Base64;
import java.nio.file.Files;
private static String encodeFileToBase64(File file) {
try {
byte[] fileContent = Files.readAllBytes(file.toPath());
return Base64.getEncoder().encodeToString(fileContent);
} catch (IOException e) {
throw new IllegalStateException("could not read file " + file, e);
}
}
Since Java 8 you can use the class java.util.Base64 and the corresponding inner classes:
java.util.Base64.Encoder
java.util.Base64.Decoder
See JavaDoc: Base64-Doc
And a sample for the use: Example from Oracle
This example worked great for me: https://grokonez.com/java/java-advanced/java-8-encode-decode-an-image-base64
public static String encoder(String filePath) {
String base64File = "";
File file = new File(filePath);
try (FileInputStream imageInFile = new FileInputStream(file)) {
// Reading a file from file system
byte fileData[] = new byte[(int) file.length()];
imageInFile.read(fileData);
base64File = Base64.getEncoder().encodeToString(fileData);
} catch (FileNotFoundException e) {
System.out.println("File not found" + e);
} catch (IOException ioe) {
System.out.println("Exception while reading the file " + ioe);
}
return base64File;
}
i want to calculate the signature of classes.dex and
something like this
public byte[] computeSignature() throws IOException {
MessageDigest digest;
try {
digest = MessageDigest.getInstance("SHA-1");
} catch (NoSuchAlgorithmException e) {
throw new AssertionError();
}
byte[] buffer = new byte[8192];
ByteBuffer data = this.data.duplicate(); // positioned ByteBuffers aren't thread safe
data.limit(data.capacity());
data.position(SIGNATURE_OFFSET + SIGNATURE_SIZE);
while (data.hasRemaining()) {
int count = Math.min(buffer.length, data.remaining());
data.get(buffer, 0, count);
digest.update(buffer, 0, count);
}
return digest.digest();
}
but im confused
any help with proper code ?
The functions I'm using to convert the file to a string and then to an mdf are below. I'm outputting the file paths and file names to make sure everything is cool. Is there anything I'm not considering that could change the file's (a video mp4 actually) fingerprint? I'm checking it against md5sum on ubuntu.
private static String readFileToString(String filePath)
throws java.io.IOException{
StringBuffer fileData = new StringBuffer(1000);
BufferedReader reader = new BufferedReader(
new FileReader(filePath));
char[] buf = new char[1024];
int numRead=0;
while((numRead=reader.read(buf)) != -1){
String readData = String.valueOf(buf, 0, numRead);
fileData.append(readData);
buf = new char[1024];
}
reader.close();
System.out.println(fileData.toString());
return fileData.toString();
}
public static String getMD5EncryptedString(String encTarget){
MessageDigest mdEnc = null;
try {
mdEnc = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException e) {
System.out.println("Exception while encrypting to md5");
e.printStackTrace();
} // Encryption algorithm
mdEnc.update(encTarget.getBytes(), 0, encTarget.length());
String md5 = new BigInteger(1, mdEnc.digest()).toString(16) ;
return md5;
}
String isn't a container for binary data. Lose the two conversions between byte array and String. You should be reading the file as bytes and computing the MD5 directly in the bytes. You can do that while you're reading it: you don't need to read the entire file first.
And MD5 isn't an encryption: it's a secure hash.
Found this answer here: How to generate an MD5 checksum for a file in Android?
public static String fileToMD5(String filePath) {
InputStream inputStream = null;
try {
inputStream = new FileInputStream(filePath);
byte[] buffer = new byte[1024];
MessageDigest digest = MessageDigest.getInstance("MD5");
int numRead = 0;
while (numRead != -1) {
numRead = inputStream.read(buffer);
if (numRead > 0)
digest.update(buffer, 0, numRead);
}
byte [] md5Bytes = digest.digest();
return convertHashToString(md5Bytes);
} catch (Exception e) {
return null;
} finally {
if (inputStream != null) {
try {
inputStream.close();
} catch (Exception e) { }
}
}
}
private static String convertHashToString(byte[] md5Bytes) {
String returnVal = "";
for (int i = 0; i < md5Bytes.length; i++) {
returnVal += Integer.toString(( md5Bytes[i] & 0xff ) + 0x100, 16).substring(1);
}
return returnVal;
}