I develop an android application for encrypt files on the phone. By searching, i found this topic : How to encrypt file from SD card using AES in Android?
The method works fine but it is very slow to encrypt files...
At this line : byte[] d = new byte[8]; why only 8 bytes ? can't we set an higher value ?
Also, do you know a way to encrypt files fastly ? I heard of crypto++ for native code implementation but how can I implement JNI on my application ?
Thank you,
EDIT : Encryption function
public void encrypt (String RSAPrivateKey, String Key, byte[] iv, String zipname, ZipEncryptAsyncTask task) throws IOException, NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException, InvalidAlgorithmParameterException
{
FileInputStream fis = new FileInputStream (zipname + ".temp");
FileOutputStream fos = new FileOutputStream (zipname);
SecretKeySpec sks = new SecretKeySpec (Base64.decode(Key, Base64.DEFAULT), "AES");
IvParameterSpec ivspec = new IvParameterSpec (iv);
Cipher cipher = Cipher.getInstance ("AES/CBC/PKCS5Padding");
fos.write(String.valueOf(RSAPrivateKey.getBytes().length).getBytes());
fos.write(RSAPrivateKey.getBytes());
cipher.init(Cipher.ENCRYPT_MODE, sks, ivspec);
CipherOutputStream cos = new CipherOutputStream(fos, cipher);
long size = 0;
byte[] d = new byte[8];
for(int b; (b = fis.read(d)) != -1; )
{
cos.write(d, 0, b);
task.doProgress((size += 8));
}
cos.flush();
cos.close();
fis.close();
new File(zipname + ".temp").delete();
}
As an alternative, you could consider changing the cypher mode you are using. CBC mode must be used serially, block by block. Counter mode (CTR) can be run in parallel with different blocks being encrypted simultaneously. Of course there are overheads to parallel processing, and you will need to rework your code, but if buffer resizing does not give you enough speed gains, then it might be the next option to try.
as #CodesInChaos said, a way to do this, is to increase the size of the buffer.
Now I do a benchmark which determines the best size for the buffer and I use the optimal value.
Related
I have an android app which handles some large byte array but I am getting some OOM crash in my Firebase Crashlytics reports for devices with low memory while handling byte array whose size may go from 10 mb to 50mb. Below is my method that I have used. So could anyone help me to improve it to avoid OOM.
byte[] decrypt(File files) {
try {
FileInputStream fis = new FileInputStream(files);
SecretKeySpec sks = new SecretKeySpec(getResources().getString(R.string.encryptPassword).getBytes(),
"AES");
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.DECRYPT_MODE, sks);
CipherInputStream cis = new CipherInputStream(fis, cipher);
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int b;
byte[] d = new byte[1024];
while ((b = cis.read(d)) != -1) {
buffer.write(d, 0, b); //this is one of the line which is being referred for the OOM in firebase
}
byte[] decryptedData = buffer.toByteArray();//this is the line which is being referred for the OOM in firebase
buffer.flush();
fis.close();
cis.close();
return decryptedData;
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
EDIT
Actually I am using the above method for decrypting downloaded audio files which are encrypted during downloading.
The above methods return the content of the encrypted files to exoplayer to play its content and I am calling the above method in the following way
ByteArrayDataSource src= new ByteArrayDataSource(decrypt(some_file));
Uri uri = new UriByteDataHelper().getUri(decrypt(some_file));
DataSpec dataSpec = new DataSpec(uri);
src.open(dataSpec);
DataSource.Factory factory = new DataSource.Factory()
{
#Override public DataSource createDataSource()
{
return src;
}
};
audioSource = new ProgressiveMediaSource.Factory(factory).createMediaSource(uri);
First of all, I would make sure that the devices where this is running have enought heap memory to run this, it might be that simply the software has already been allocated with a lot of space and there might not be much more left on the heap to provide the software. This operation should not require much memory and I don't see anything obvious that would point towards trying to allocated and unexpectadely large amount of memory.
What I would recommend though, if you want to hav a quick test is actually simply lowering the byte array size, any particular reason why you are using 1024?
If possible perhaps try:
byte[] d = new byte[8];
Also, If that was me, I would store the read data temporarily, on an array perhaps and only once the read of the cypher has finished, I would call
buffer.write()
From my experience, trying to read and write at the same time tens to not be advised and can result in serveral issues, at the very least you should make sure you have the whole cypher and that it is a valid one (if you have some validation requirements) and only then send it.
Again, this should not be the core issue, the device seems that it is lacking enough availalbe memory to be allocated, perhaps too much reserved memory for other processes?
You should consider of writing the decrypted data to a tempfile and then reload the data for usage.
The main reasons for the Out of memory error are the ByteArrayOutputStream AND
byte[] decryptedData = buffer.toByteArray(), because both of them hold the complete (decrypted) data and that doubles the memory consumption of your decrypt method.
You could avoid this by decrypting the data to a tempfile in the first step and later load the data from the tempfile. I modified the decrypt method to handle the decrypted output stream and later there is a method for reloading the decrypted data (there is no propper exception handling and for my testing I setup a static encryptPassword-variable ...).
There is just one part left for you - you need to find a good place for the tempfile and I'm no Android specialist.
Just two notes: You are using the unsecure AES ECB mode and the String to byte[]conversion for your password should be
changed to
.getBytes(StandardCharsets.UTF_8)
on ecryption and decryption side to avoid errors caused by different encodings on different platforms.
public static void decryptNew(File files, File tempfiles) {
try (FileInputStream fis = new FileInputStream(files);
BufferedInputStream in = new BufferedInputStream(fis);
FileOutputStream out = new FileOutputStream(tempfiles);
BufferedOutputStream bos = new BufferedOutputStream(out)) {
byte[] ibuf = new byte[1024];
int len;
Cipher cipher = Cipher.getInstance("AES");
SecretKeySpec sks = new SecretKeySpec(encryptPassword.getBytes(),"AES"); // static password
// SecretKeySpec sks = new SecretKeySpec(getResources().getString(R.string.encryptPassword).getBytes(),"AES");
cipher.init(Cipher.DECRYPT_MODE, sks);
while ((len = in.read(ibuf)) != -1) {
byte[] obuf = cipher.update(ibuf, 0, len);
if (obuf != null)
bos.write(obuf);
}
byte[] obuf = cipher.doFinal();
if (obuf != null)
bos.write(obuf);
} catch (BadPaddingException | IllegalBlockSizeException | InvalidKeyException | IOException | NoSuchAlgorithmException | NoSuchPaddingException e) {
e.printStackTrace();
}
}
public static byte[] loadFile(File filename) throws IOException {
byte[] filecontent = new byte[0];
FileInputStream fileInputStream = null;
try {
fileInputStream = new FileInputStream(filename);
// int byteLength = fff.length();
// In android the result of file.length() is long
long byteLength = filename.length(); // byte count of the file-content
filecontent = new byte[(int) byteLength];
fileInputStream.read(filecontent, 0, (int) byteLength);
} catch (IOException e) {
e.printStackTrace();
fileInputStream.close();
return filecontent;
}
fileInputStream.close();
return filecontent;
}
After loading the tempfile content to the byte array you can delete the file with a one-liner (again no exception handling):
Files.deleteIfExists(tempFile.toPath());
I'm writing a second answer and do not edit my first answer as it is a total different approach to solve the problem.
As you post a part of your code I can see that you have a byte array with the complete and decrypted content that gets played by the exoplayer:
output:
byte[] decrypt(File files)
as input for
ByteArrayDataSource src= new ByteArrayDataSource(decrypt(some_file));
So to avoid double and moretime memory consumption when playing with large files (approx. 50mb) my approach it is to download the complete encrypted file and save it in a byte array.
On devices with a good memory equipment you can decrypt the encrypted byte array in one run to another byte array and play the music from this decrypted byte array (step 6 + 8 in my sample program).
Using a low memory device you decrypt the byte array in chunks (in my program in 16 byte long blocks) and save the decrypted chunks at the same place in the encrypted byte array. When all chunks got processed the (former) encrypted data are now decrypted and you used the memory of just one byte array length. Now you can play the music from this byte array (steps 7 + 8).
Just for explanation, steps 1-3 are on server side and in steps 3+4 the transmission takes place.
This example uses the AES CTR mode as it provides the same length for input and output data.
In the end I'm comparing the byte arrays to prove that the decryption was successful for steps 6 (direct decryption) and 7 (decryption in chunks):
output:
Decrypting 50mb data in chunks to avoid out of memory error
https://stackoverflow.com/questions/62412705/how-to-write-byte-array-without-oom
plaindata length: 52428810
cipherdata length: 52428810
decrypteddata length: 52428810
cipherdata parts in 16 byte long parts: 3276800 = rounds for decryption
cipherdata moduluo 16 byte long parts: 10 + 1 round for rest/modulus
cipherdata length: 52428810 (after decryption)
plaindata equals decrypteddata: true
plaindata equals cipherdata: true
code:
import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import java.security.GeneralSecurityException;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Random;
public class EncryptionCtrSo4 {
public static void main(String[] args) throws GeneralSecurityException {
System.out.println("Decrypting 50mb data in chunks to avoid out of memory error");
System.out.println("https://stackoverflow.com/questions/62412705/how-to-write-byte-array-without-oom");
/*
* author michael fehr, http://javacrypto.bplaced.net
* no licence applies, no warranty
*/
// === server side ===
// 1 create a 50 mb byte array (unencrypted)
byte[] plaindata = new byte[(50 * 1024 * 1024 + 10)];
// fill array with (random) data
Random random = new Random();
random.nextBytes(plaindata);
// 2 encrypt the data with aes ctr mode, create random keys
SecureRandom secureRandom = new SecureRandom();
byte[] key = new byte[32]; // 32 byte = 256 bit aes key
secureRandom.nextBytes(key);
byte[] iv = new byte[16]; // 16 byte = 128 bit
secureRandom.nextBytes(iv);
SecretKeySpec keySpecEnc = new SecretKeySpec(key, "AES");
IvParameterSpec ivParameterSpecEnc = new IvParameterSpec(iv);
Cipher cipherEnc = Cipher.getInstance("AES/CTR/NoPadding");
cipherEnc.init(Cipher.ENCRYPT_MODE, keySpecEnc, ivParameterSpecEnc);
byte[] cipherdata = cipherEnc.doFinal(plaindata);
System.out.println("plaindata length: " + plaindata.length);
System.out.println("cipherdata length: " + cipherdata.length);
// 3 transfer the cipherdata to app
// ...
// === app side ===
// 4 receive encrypted data from server
// ...
// 5 decryption setup
SecretKeySpec keySpecDec = new SecretKeySpec(key, "AES");
IvParameterSpec ivParameterSpecDec = new IvParameterSpec(iv);
Cipher cipherDec = Cipher.getInstance("AES/CTR/NoPadding");
cipherDec.init(Cipher.DECRYPT_MODE, keySpecDec, ivParameterSpecDec);
// 6 decryption in one run on high memory devices
byte[] decrypteddata = cipherDec.doFinal(cipherdata);
System.out.println("decrypteddata length: " + decrypteddata.length);
// 7 decryption in chunks using the cipherdata byte array
int cipherdataLength = cipherdata.length;
int chunksize = 16; // should be a multiple of 16, minimum 16
byte[] decryptedPart = new byte[chunksize];
int parts16byteDiv = cipherdataLength / chunksize;
int parts16byteMod = cipherdataLength % chunksize;
System.out.println("cipherdata parts in " + chunksize + " byte long parts: " + parts16byteDiv + " = rounds for decryption");
System.out.println("cipherdata moduluo " + chunksize + " byte long parts: " + parts16byteMod + " + 1 round for rest/modulus");
for (int i = 0; i < parts16byteDiv; i++) {
cipherDec.update(cipherdata, (i * chunksize), chunksize, decryptedPart);
System.arraycopy(decryptedPart, 0, cipherdata, (i * chunksize), decryptedPart.length);
}
if (parts16byteMod > 0) {
decryptedPart = new byte[parts16byteMod];
cipherDec.update(cipherdata, (parts16byteDiv * chunksize), parts16byteMod, decryptedPart);
System.arraycopy(decryptedPart, 0, cipherdata, (parts16byteDiv * chunksize), decryptedPart.length);
}
System.out.println("cipherdata length: " + cipherdata.length + " (after decryption)");
// the cipherdata byte array is now decrypted !
// 8 use cipherdata (encrypted) or decrypteddata as input for exoplayer
// compare ciphertext with decrypteddata in step 6
System.out.println("plaindata equals decrypteddata: " + Arrays.equals(plaindata, decrypteddata));
// check that (decrypted) cipherdata equals plaindata of step 7
System.out.println("plaindata equals cipherdata: " + Arrays.equals(plaindata, cipherdata));
}
}
I was trying to implement an encrypt/decrypt program using classes under javax.crypto and file streams for input/output.
To limit the memory usage, I run with -Xmx256m parameter.
It works fine with encryption and decryption with smaller files.
But when decrypt a huge file (1G in size), there is an out of memory exception:
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3236)
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
at com.sun.crypto.provider.GaloisCounterMode.decrypt(GaloisCounterMode.java:505)
at com.sun.crypto.provider.CipherCore.update(CipherCore.java:782)
at com.sun.crypto.provider.CipherCore.update(CipherCore.java:667)
at com.sun.crypto.provider.AESCipher.engineUpdate(AESCipher.java:380)
at javax.crypto.Cipher.update(Cipher.java:1831)
at javax.crypto.CipherOutputStream.write(CipherOutputStream.java:166)
Here is the decrypt code:
private final int _readSize = 0x10000;//64k
...
GCMParameterSpec gcmParameterSpec = new GCMParameterSpec(gcmTagSize, iv);
Key keySpec = new SecretKeySpec(key, keyParts[0]);
Cipher decCipher = Cipher.getInstance("AES/GCM/PKCS5Padding");
decCipher.init(Cipher.DECRYPT_MODE, keySpec, gcmParameterSpec);
try (InputStream fileInStream = Files.newInputStream(inputEncryptedFile);
OutputStream fileOutStream = Files.newOutputStream(outputDecryptedFile)) {
try (CipherOutputStream cipherOutputStream = new CipherOutputStream(fileOutStream, decCipher)) {
long count = 0L;
byte[] buffer = new byte[_readSize];
int n;
for (; (n = fileInStream.read(buffer)) != -1; count += (long) n) {
cipherOutputStream.write(buffer, 0, n);
}
}
}
The key parameters like gcmTagSize and iv are read from a key file, and it works fine with smaller files, like some one around size of 50M.
As I understand, every time there are only 64k data passed to decipher, why it runs out of heap memory?
How can I avoid this?
Edit:
Actually I have tried with 4k as buffer size, failed with same exception.
Edit 2:
With more testing, the max file size it can handle is around 1/4 of the heap size. Like, if you set -Xmx256m, files bigger than 64M will fail to decrypt.
This appears to be an issue with the implementation of the GCM mode. I'm not sure that you can work-around it.
If you look at your stack trace:
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3236)
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
at com.sun.crypto.provider.GaloisCounterMode.decrypt(GaloisCounterMode.java:505)
The out-of-memory error is happening when writing to a ByteArrayOutputStream from within GaloisCounterMode. You use a FileOutputStream, so either you're not showing the right code or this ByteArrayStream is used internally.
If you look at the source for GaloisCounterMode you'll see that it defines an internal ByteArrayOutputStream (it actually defines two, but I think this is the one that's the problem):
// buffer for storing input in decryption, not used for encryption
private ByteArrayOutputStream ibuffer = null;
Then, later down, it writes bytes to this stream. Note the code comment.
int decrypt(byte[] in, int inOfs, int len, byte[] out, int outOfs) {
processAAD();
if (len > 0) {
// store internally until decryptFinal is called because
// spec mentioned that only return recovered data after tag
// is successfully verified
ibuffer.write(in, inOfs, len);
}
return 0;
}
That buffer isn't reset until decryptFinal().
Edit: looking at this CSx answer it looks like GCM needs to buffer the entire stream. That would make it a very bad choice if you have large files and not enough memory.
I think your best solution is to switch to CBC mode.
The bad news are: IMHO the error is caused by an bad implementation of AES GCM-mode in native Java. Even if you could get it to work you will find that the decryption of a large file (1 GB or so) will take a lot of time (maybe hours ?).
But there are good news: you could/should use BouncyCastle as service provider for your decryption task - that way the decryption will work and it's much faster.
The following full example will create a sample file of 1 gb size, encrypts it with BouncyCastle and later decrypts it. In the end there is a file compare to show that plain and decrypted file contents are equal and the files will be deleted. You need temporary a total of more than 3 GB free space on your device to run this example.
Using a buffer of 64 KB I'm running this example with this data:
Milliseconds for Encryption: 14295 | Decryption: 16249
A buffer of 1 KB is a little bit slower on encryption side but much slower on decryption task:
Milliseconds for Encryption: 15250 | Decryption: 21952
A last word regarding your cipher - "AES/GCM/PKCS5Padding" is not existing and "available" in some implementations but the real used algorithm is "AES/GCM/NoPadding" (see Can PKCS5Padding be in AES/GCM mode? for more details).
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.spec.GCMParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import java.io.*;
import java.nio.file.Files;
import java.security.*;
import java.util.Arrays;
public class GcmTestBouncyCastle {
public static void main(String[] args) throws IOException, NoSuchPaddingException, InvalidAlgorithmParameterException,
NoSuchAlgorithmException, IllegalBlockSizeException, BadPaddingException, NoSuchProviderException, InvalidKeyException {
System.out.println("Encryption & Decryption with BouncyCastle AES-GCM-Mode");
System.out.println("https://stackoverflow.com/questions/61792534/out-of-memory-exception-when-decrypt-large-file-using-cipher");
// you need bouncy castle, get version 1.65 here:
// https://mvnrepository.com/artifact/org.bouncycastle/bcprov-jdk15on/1.65
Security.addProvider(new BouncyCastleProvider());
// setup files
// filenames
String filenamePlain = "plain.dat";
String filenameEncrypt = "encrypt.dat";
String filenameDecrypt = "decrypt.dat";
// generate a testfile of 1024 byte | 1 gb
//createFileWithDefinedLength(filenamePlain, 1024);
createFileWithDefinedLength(filenamePlain, 1024 * 1024 * 1024); // 1 gb
// time measurement
long startMilli = 0;
long encryptionMilli = 0;
long decryptionMilli = 0;
// generate nonce/iv
int GCM_NONCE_LENGTH = 12; // for a nonce of 96 bit length
int GCM_TAG_LENGTH = 16;
int GCM_KEY_LENGTH = 32; // 32 = 256 bit keylength, 16 = 128 bit keylength
SecureRandom r = new SecureRandom();
byte[] nonce = new byte[GCM_NONCE_LENGTH];
r.nextBytes(nonce);
// key should be generated as random byte[]
byte[] key = new byte[GCM_KEY_LENGTH];
r.nextBytes(key);
// encrypt file
startMilli = System.currentTimeMillis();
encryptWithGcmBc(filenamePlain, filenameEncrypt, key, nonce, GCM_TAG_LENGTH);
encryptionMilli = System.currentTimeMillis() - startMilli;
startMilli = System.currentTimeMillis();
decryptWithGcmBc(filenameEncrypt, filenameDecrypt, key, nonce, GCM_TAG_LENGTH);
decryptionMilli = System.currentTimeMillis() - startMilli;
// check that plain and decrypted files are equal
System.out.println("SHA256-file compare " + filenamePlain + " | " + filenameDecrypt + " : "
+ Arrays.equals(sha256File(filenamePlain), sha256File(filenameDecrypt)));
System.out.println("Milliseconds for Encryption: " + encryptionMilli + " | Decryption: " + decryptionMilli);
// clean up with files
Files.deleteIfExists(new File(filenamePlain).toPath());
Files.deleteIfExists(new File(filenameEncrypt).toPath());
Files.deleteIfExists(new File(filenameDecrypt).toPath());
}
public static void encryptWithGcmBc(String filenamePlain, String filenameEnc, byte[] key, byte[] nonce, int gcm_tag_length)
throws IOException, NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException,
InvalidAlgorithmParameterException, IllegalBlockSizeException, BadPaddingException, NoSuchProviderException {
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding", "BC");
SecretKeySpec keySpec = new SecretKeySpec(key, "AES");
GCMParameterSpec gcmSpec = new GCMParameterSpec(gcm_tag_length * 8, nonce);
cipher.init(Cipher.ENCRYPT_MODE, keySpec, gcmSpec);
try (FileInputStream fis = new FileInputStream(filenamePlain);
BufferedInputStream in = new BufferedInputStream(fis);
FileOutputStream out = new FileOutputStream(filenameEnc);
BufferedOutputStream bos = new BufferedOutputStream(out)) {
//byte[] ibuf = new byte[1024];
byte[] ibuf = new byte[0x10000]; // = 65536
int len;
while ((len = in.read(ibuf)) != -1) {
byte[] obuf = cipher.update(ibuf, 0, len);
if (obuf != null)
bos.write(obuf);
}
byte[] obuf = cipher.doFinal();
if (obuf != null)
bos.write(obuf);
}
}
public static void decryptWithGcmBc(String filenameEnc, String filenameDec, byte[] key, byte[] nonce, int gcm_tag_length)
throws IOException, NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException,
InvalidAlgorithmParameterException, IllegalBlockSizeException, BadPaddingException, NoSuchProviderException {
try (FileInputStream in = new FileInputStream(filenameEnc);
FileOutputStream out = new FileOutputStream(filenameDec)) {
//byte[] ibuf = new byte[1024];
byte[] ibuf = new byte[0x10000]; // = 65536
int len;
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding", "BC");
SecretKeySpec keySpec = new SecretKeySpec(key, "AES");
GCMParameterSpec gcmSpec = new GCMParameterSpec(gcm_tag_length * 8, nonce);
cipher.init(Cipher.DECRYPT_MODE, keySpec, gcmSpec);
while ((len = in.read(ibuf)) != -1) {
byte[] obuf = cipher.update(ibuf, 0, len);
if (obuf != null)
out.write(obuf);
}
byte[] obuf = cipher.doFinal();
if (obuf != null)
out.write(obuf);
}
}
// just for creating a large file within seconds
private static void createFileWithDefinedLength(String filenameString, long sizeLong) throws IOException {
RandomAccessFile raf = new RandomAccessFile(filenameString, "rw");
try {
raf.setLength(sizeLong);
} finally {
raf.close();
}
}
// just for file comparing
public static byte[] sha256File(String filenameString) throws IOException, NoSuchAlgorithmException {
byte[] buffer = new byte[8192];
int count;
MessageDigest md = MessageDigest.getInstance("SHA-256");
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(filenameString));
while ((count = bis.read(buffer)) > 0) {
md.update(buffer, 0, count);
}
bis.close();
return md.digest();
}
}
I have done Encryption and Decryption in android when file downloading
but I want to improve time performance when file decrypted.
My problem is when I am downloading any file so I have add encryption over there but at this stage I am showing Progress loader so it looks good but but when file completely download and try to open that file then it is decrypted that file this time it's taking too much time .
which is look very bad. How can I reduce decryption time? Here is my code
Encryption Code
byte data[] = new byte[1024];
String seed = "password";
byte[] rawKey = getRawKey(seed.getBytes());
SecretKeySpec skeySpec = new SecretKeySpec(rawKey, "AES");
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, skeySpec);
output = new CipherOutputStream(output, cipher);
long total = 0;
while ((count = input.read(data)) != -1) {
total += count;
publishProgress("" + (int) ((total * 100) / lenghtOfFile));
output.write(data, 0, count);
}
Decryption Code Here:
String newPath = sdCardPath + "/" + dPdfName;
File f1 = new File(newPath);
if (!f1.exists())
try {
f1.createNewFile();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
try {
InputStream fis = new FileInputStream(f);
OutputStream fos = new FileOutputStream(f1);
String seed = "password";
byte[] rawKey = getRawKey(seed.getBytes());
SecretKeySpec skeySpec = new SecretKeySpec(rawKey,
"AES");
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.DECRYPT_MODE, skeySpec);
fis = new CipherInputStream(fis, cipher);
int b;
byte[] data = new byte[4096];
while ((b = fis.read(data)) != -1) {
// fos.write(cipher.doFinal(data), 0, b);
fos.write(data, 0, b);
}
fos.flush();
fos.close();
fis.close();
} catch (Exception e) {
// TODO: handle exceptionpri
e.printStackTrace();
}
Get Row Key Method:
private static byte[] getRawKey(byte[] seed) throws Exception {
KeyGenerator kgen = KeyGenerator.getInstance("AES");
SecureRandom sr = SecureRandom.getInstance("SHA1PRNG");
sr.setSeed(seed);
kgen.init(128, sr);
SecretKey skey = kgen.generateKey();
byte[] raw = skey.getEncoded();
return raw;
}
Suggestions:
1) move the decryption to another asynctask and add another progress indicator.
2) reserve last, say, 10% of the progress indicator to decryption. This is what I actually did once, but I was doing an integrity check (against an MD5 hash IIRC), not decryption.
3) move the decryption to the downloading asynctask, decrypt each received portion of data immediately and so hide the decryption time behind the download time.
4) not sure this will be any faster, but you may have two service threads: one downloading file and another decrypting it. It's better no to use AsyncTask here, because they may behave differently on different versions of Android (including sequential execution on a single thread, see Is AsyncTask really conceptually flawed or am I just missing something? for the discussion, and my note https://stackoverflow.com/a/14602486/755804 )
Note also that the thread responsible for downloading and decryption belongs to Model (in MVC sense) and must not be owned by an Activity which is a Controller that cannot outlive a screen turn: https://stackoverflow.com/a/14603375/755804
If your download takes a long time, you may be interested in resuming interrupted downloads, and it's better to think about it from the very beginning. It's always easier to modify simple solutions, and multi-threaded solutions are rather complex. If you transfer a number of files, it may happen that one of them gets broken during transfer, and you may want an option to retransfer only the file(s) that are broken. You may also want an integrity check.
Make sure you align the data buffer to the block size of the encryption.
For a Example see: https://stackoverflow.com/a/33171612/475496
Using this method has speedup our encryption enormous.
I am unable to find any working example for AES CTR encryption for random access. Can someone guide me how to use the counter in CTR MODE also how to implement jumping to a specific position in the stream?
The default stream implementation (CipherInputStream) doesn't skip the stream and it corrupts the plain text.
I am trying to decrypt encrypted Video file stored on sdcard in Android. An embedded HTTP file server decrypts it on the fly. Everything is working fine until a user performs a seek within the video: the video stops immediately because it receives corrupt video stream.
I am using following code to initialize and encrypt/ decrypt stream (for sake of simplicity i hard coded the keys. it will not be hard coded in production)
ByteBuffer bb = ByteBuffer.allocate(16);
bb.put("1234567891230000".getBytes());
byte[] ivString = bb.array();
// INITIALISATION
String keyString = "1234567812345678";
IvParameterSpec iv = new IvParameterSpec(ivString);
SecretKeySpec keySpec = new SecretKeySpec(keyString.getBytes(), "AES");
// FOR ENCRYPTION
Cipher cipher = Cipher.getInstance("AES/CTR/NoPadding");
cipher.init(Cipher.ENCRYPT_MODE, keySpec, new IvParameterSpec(ivString));
Inputstream encrypted_is = new CipherInputStream(in, cipher);
// FOR DECRYPTION
cipher.init(Cipher.DECRYPT_MODE, keySpec, new IvParameterSpec(ivString));
Inputstream decrypted_is = new CipherInputStream(in, cipher);
You should not implement this using a stream. A stream is used for sequential access to data. Skipping should only be used for jumping short distances forward and mark/reset only for jumping small distances back.
Using a file map is probably the most efficient. For a slightly easier approach but less efficient approach you could use RandomAccessFile instead. You should furthermore use Cipher.getInstance("AES/CTR/NoPadding") using an "IV" that is set to the counter you expect at the location you start within the file.
Sample code for using CTR with offset:
private static final int AES_BLOCK_SIZE = 16;
public static final void jumpToOffset(final Cipher c,
final SecretKey aesKey, final IvParameterSpec iv, final long offset) {
if (!c.getAlgorithm().toUpperCase().startsWith("AES/CTR")) {
throw new IllegalArgumentException(
"Invalid algorithm, only AES/CTR mode supported");
}
if (offset < 0) {
throw new IllegalArgumentException("Invalid offset");
}
final int skip = (int) (offset % AES_BLOCK_SIZE);
final IvParameterSpec calculatedIVForOffset = calculateIVForOffset(iv,
offset - skip);
try {
c.init(Cipher.ENCRYPT_MODE, aesKey, calculatedIVForOffset);
final byte[] skipBuffer = new byte[skip];
c.update(skipBuffer, 0, skip, skipBuffer);
Arrays.fill(skipBuffer, (byte) 0);
} catch (ShortBufferException | InvalidKeyException
| InvalidAlgorithmParameterException e) {
throw new IllegalStateException(e);
}
}
private static IvParameterSpec calculateIVForOffset(final IvParameterSpec iv,
final long blockOffset) {
final BigInteger ivBI = new BigInteger(1, iv.getIV());
final BigInteger ivForOffsetBI = ivBI.add(BigInteger.valueOf(blockOffset
/ AES_BLOCK_SIZE));
final byte[] ivForOffsetBA = ivForOffsetBI.toByteArray();
final IvParameterSpec ivForOffset;
if (ivForOffsetBA.length >= AES_BLOCK_SIZE) {
ivForOffset = new IvParameterSpec(ivForOffsetBA, ivForOffsetBA.length - AES_BLOCK_SIZE,
AES_BLOCK_SIZE);
} else {
final byte[] ivForOffsetBASized = new byte[AES_BLOCK_SIZE];
System.arraycopy(ivForOffsetBA, 0, ivForOffsetBASized, AES_BLOCK_SIZE
- ivForOffsetBA.length, ivForOffsetBA.length);
ivForOffset = new IvParameterSpec(ivForOffsetBASized);
}
return ivForOffset;
}
I have an encrypted mp4 using Rijndael and I am decrypting in C# in the following manner.
System.Security.Cryptography.Rijndael crypt = System.Security.Cryptography.Rijndael.Create();
crypt.Key = convertedSecureString;
byte[] initializationVectorLength = new byte[sizeof(int)];
CryptoStream cryptostream = new CryptoStream(inputStream, crypt.CreateDecryptor(), CryptoStreamMode.Read);
byte[] buffer = new byte[1024];
int len;
while ((len = cryptostream.Read(buffer, 0, buffer.Length)) > 0)
{
outputStream.Write(buffer, 0, len);
buffer = new byte[1024];
}
outputStream.Flush();
cryptostream.Close();
inputStream.Close();
Now I need to convert this code to Java/Android equivalent. I am not sure where to start frankly. I am confused by so many options - some say use Bouncy Castle, some say Apache Commons, some the native Java lib. How do I do this. And what do I do about CryptoStream etc?
UPDATE
I am using the following the code in C# for assigning the key
byte[] convertedSecureString = new byte[this.key.Length];
IntPtr ptr = System.Runtime.InteropServices.Marshal.SecureStringToBSTR(this.key);
for (int i = 0, j = 0; i < this.key.Length * UnicodeByteLength; i = i + UnicodeByteLength, j++)
{
convertedSecureString[j] = System.Runtime.InteropServices.Marshal.ReadByte(ptr, i);
}
try
{
crypt.Key = convertedSecureString;
}
where key is secure. I have the equivalent unsecure key in Java. How do i convert this piece of code to Java
UPDATE
Rfc2898DeriveBytes newKey = new Rfc2898DeriveBytes(crypt.Key.ToString(), crypt.IV);
Array.Copy(newKey.GetBytes((int)crypt.KeySize / 8), crypt.Key, (int)crypt.KeySize / 8);
Array.Copy(newKey.GetBytes((int)crypt.BlockSize / 8), crypt.IV, (int)crypt.BlockSize / 8);
I am using this in C# to mod the key using Rfc 2898 and derive the bytes - I cant find an equivalent in Java - I found here Java equivalent of C#'s Rfc2898DerivedBytes in the second comment - but what values do I give for iterator and dklen?
You need to get Cipher object. Here is one way of getting it, using byte[] aesKey, and byte[] iv (initialization vector, must always be 16 bytes for AES).
// aesKey is 128, 196 or 256-bit key (8, 12 or 16 byte array)
SecretKeySpec key = new SecretKeySpec(aesKey, "AES");
// initialization vector
IvParameterSpec ivSpec = new IvParameterSpec(iv);
// create and initialize cipher object
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING");
cipher.init(Cipher.DECRYPT_MODE, key, ivSpec);
Once you have Cipher object in decrypt mode, you can feed it with encrypted data using update methods, and it will return you plain-text data. When you are done, you must call one of doFinal methods to get final block.
Alternatively, you can create CipherInputStream using your Cipher object, and original input stream that supplies encrypted data. You read data from CipherInputStream, which in turn reads data from original input stream, decrypts it, and returns you the plain-text data.
For encrypting, you need to pass Cipher.ENCRYPT_MODE into Cipher.init method, and use CipherOutputStream instead.
Update: full example, more or less equivalent to original C# code:
// Algorithm, mode and padding must match encryption.
// Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING");
// If you have Bouncycastle library installed, you can use
// Rijndael/CBC/PKCS7PADDING directly.
Cipher cipher = Cipher.getInstance("Rijndael/CBC/PKCS7PADDING");
// convertedSecureString and initVector must be byte[] with correct length
cipher.init(Cipher.DECRYPT_MODE, new SecretKeySpec(convertedSecureString, "AES"),
new IvParameterSpec(initVector));
CipherInputStream cryptoStream = new CipherInputStream(inputStream, cipher);
byte[] buffer = new byte[1024];
int len = cryptoStream.read(buffer, 0, buffer.length);
while (len > 0) {
outputStream.write(buffer, 0, len);
len = cryptoStream.read(buffer, 0, buffer.length);
}
outputStream.flush();
cryptoStream.close();
// no need to close inputStream here, as cryptoStream will close it
By default, Java doesn't support Rijndael algorithm (AES may or may not work, see the other answer) and PKCS7 padding. You will need to install Bouncycastle extension for that and then just use Cipher.getInstance("Rijndael/CBC/PKCS7PADDING");. Why CBC and PKCS7 Padding? Those seem to be defaults for System.Security.Cryptography.Rijndael class. If I got that wrong, use correct mode and padding for your situation.
Don't worry about the implementation in C#. Go through the below link for encrypting/decrypting for Java Language.RjIndeal Implementation in Java