Develop Reference

How to write byte array without OOM?

I have an android app which handles some large byte array but I am getting some OOM crash in my Firebase Crashlytics reports for devices with low memory while handling byte array whose size may go from 10 mb to 50mb. Below is my method that I have used. So could anyone help me to improve it to avoid OOM.
byte[] decrypt(File files) {
try {
FileInputStream fis = new FileInputStream(files);
SecretKeySpec sks = new SecretKeySpec(getResources().getString(R.string.encryptPassword).getBytes(),
"AES");
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.DECRYPT_MODE, sks);
CipherInputStream cis = new CipherInputStream(fis, cipher);
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int b;
byte[] d = new byte[1024];
while ((b = cis.read(d)) != -1) {
buffer.write(d, 0, b); //this is one of the line which is being referred for the OOM in firebase
}
byte[] decryptedData = buffer.toByteArray();//this is the line which is being referred for the OOM in firebase
buffer.flush();
fis.close();
cis.close();
return decryptedData;
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
EDIT
Actually I am using the above method for decrypting downloaded audio files which are encrypted during downloading.
The above methods return the content of the encrypted files to exoplayer to play its content and I am calling the above method in the following way
ByteArrayDataSource src= new ByteArrayDataSource(decrypt(some_file));
Uri uri = new UriByteDataHelper().getUri(decrypt(some_file));
DataSpec dataSpec = new DataSpec(uri);
src.open(dataSpec);
DataSource.Factory factory = new DataSource.Factory()
{
#Override public DataSource createDataSource()
{
return src;
}
};
audioSource = new ProgressiveMediaSource.Factory(factory).createMediaSource(uri);

First of all, I would make sure that the devices where this is running have enought heap memory to run this, it might be that simply the software has already been allocated with a lot of space and there might not be much more left on the heap to provide the software. This operation should not require much memory and I don't see anything obvious that would point towards trying to allocated and unexpectadely large amount of memory.
What I would recommend though, if you want to hav a quick test is actually simply lowering the byte array size, any particular reason why you are using 1024?
If possible perhaps try:
byte[] d = new byte[8];
Also, If that was me, I would store the read data temporarily, on an array perhaps and only once the read of the cypher has finished, I would call
buffer.write()
From my experience, trying to read and write at the same time tens to not be advised and can result in serveral issues, at the very least you should make sure you have the whole cypher and that it is a valid one (if you have some validation requirements) and only then send it.
Again, this should not be the core issue, the device seems that it is lacking enough availalbe memory to be allocated, perhaps too much reserved memory for other processes?

You should consider of writing the decrypted data to a tempfile and then reload the data for usage.
The main reasons for the Out of memory error are the ByteArrayOutputStream AND
byte[] decryptedData = buffer.toByteArray(), because both of them hold the complete (decrypted) data and that doubles the memory consumption of your decrypt method.
You could avoid this by decrypting the data to a tempfile in the first step and later load the data from the tempfile. I modified the decrypt method to handle the decrypted output stream and later there is a method for reloading the decrypted data (there is no propper exception handling and for my testing I setup a static encryptPassword-variable ...).
There is just one part left for you - you need to find a good place for the tempfile and I'm no Android specialist.
Just two notes: You are using the unsecure AES ECB mode and the String to byte[]conversion for your password should be
changed to
.getBytes(StandardCharsets.UTF_8)
on ecryption and decryption side to avoid errors caused by different encodings on different platforms.
public static void decryptNew(File files, File tempfiles) {
try (FileInputStream fis = new FileInputStream(files);
BufferedInputStream in = new BufferedInputStream(fis);
FileOutputStream out = new FileOutputStream(tempfiles);
BufferedOutputStream bos = new BufferedOutputStream(out)) {
byte[] ibuf = new byte[1024];
int len;
Cipher cipher = Cipher.getInstance("AES");
SecretKeySpec sks = new SecretKeySpec(encryptPassword.getBytes(),"AES"); // static password
// SecretKeySpec sks = new SecretKeySpec(getResources().getString(R.string.encryptPassword).getBytes(),"AES");
cipher.init(Cipher.DECRYPT_MODE, sks);
while ((len = in.read(ibuf)) != -1) {
byte[] obuf = cipher.update(ibuf, 0, len);
if (obuf != null)
bos.write(obuf);
}
byte[] obuf = cipher.doFinal();
if (obuf != null)
bos.write(obuf);
} catch (BadPaddingException | IllegalBlockSizeException | InvalidKeyException | IOException | NoSuchAlgorithmException | NoSuchPaddingException e) {
e.printStackTrace();
}
}
public static byte[] loadFile(File filename) throws IOException {
byte[] filecontent = new byte[0];
FileInputStream fileInputStream = null;
try {
fileInputStream = new FileInputStream(filename);
// int byteLength = fff.length();
// In android the result of file.length() is long
long byteLength = filename.length(); // byte count of the file-content
filecontent = new byte[(int) byteLength];
fileInputStream.read(filecontent, 0, (int) byteLength);
} catch (IOException e) {
e.printStackTrace();
fileInputStream.close();
return filecontent;
}
fileInputStream.close();
return filecontent;
}
After loading the tempfile content to the byte array you can delete the file with a one-liner (again no exception handling):
Files.deleteIfExists(tempFile.toPath());

I'm writing a second answer and do not edit my first answer as it is a total different approach to solve the problem.
As you post a part of your code I can see that you have a byte array with the complete and decrypted content that gets played by the exoplayer:
output:
byte[] decrypt(File files)
as input for
ByteArrayDataSource src= new ByteArrayDataSource(decrypt(some_file));
So to avoid double and moretime memory consumption when playing with large files (approx. 50mb) my approach it is to download the complete encrypted file and save it in a byte array.
On devices with a good memory equipment you can decrypt the encrypted byte array in one run to another byte array and play the music from this decrypted byte array (step 6 + 8 in my sample program).
Using a low memory device you decrypt the byte array in chunks (in my program in 16 byte long blocks) and save the decrypted chunks at the same place in the encrypted byte array. When all chunks got processed the (former) encrypted data are now decrypted and you used the memory of just one byte array length. Now you can play the music from this byte array (steps 7 + 8).
Just for explanation, steps 1-3 are on server side and in steps 3+4 the transmission takes place.
This example uses the AES CTR mode as it provides the same length for input and output data.
In the end I'm comparing the byte arrays to prove that the decryption was successful for steps 6 (direct decryption) and 7 (decryption in chunks):
output:
Decrypting 50mb data in chunks to avoid out of memory error
https://stackoverflow.com/questions/62412705/how-to-write-byte-array-without-oom
plaindata length: 52428810
cipherdata length: 52428810
decrypteddata length: 52428810
cipherdata parts in 16 byte long parts: 3276800 = rounds for decryption
cipherdata moduluo 16 byte long parts: 10 + 1 round for rest/modulus
cipherdata length: 52428810 (after decryption)
plaindata equals decrypteddata: true
plaindata equals cipherdata: true
code:
import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import java.security.GeneralSecurityException;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Random;
public class EncryptionCtrSo4 {
public static void main(String[] args) throws GeneralSecurityException {
System.out.println("Decrypting 50mb data in chunks to avoid out of memory error");
System.out.println("https://stackoverflow.com/questions/62412705/how-to-write-byte-array-without-oom");
/*
* author michael fehr, http://javacrypto.bplaced.net
* no licence applies, no warranty
*/
// === server side ===
// 1 create a 50 mb byte array (unencrypted)
byte[] plaindata = new byte[(50 * 1024 * 1024 + 10)];
// fill array with (random) data
Random random = new Random();
random.nextBytes(plaindata);
// 2 encrypt the data with aes ctr mode, create random keys
SecureRandom secureRandom = new SecureRandom();
byte[] key = new byte[32]; // 32 byte = 256 bit aes key
secureRandom.nextBytes(key);
byte[] iv = new byte[16]; // 16 byte = 128 bit
secureRandom.nextBytes(iv);
SecretKeySpec keySpecEnc = new SecretKeySpec(key, "AES");
IvParameterSpec ivParameterSpecEnc = new IvParameterSpec(iv);
Cipher cipherEnc = Cipher.getInstance("AES/CTR/NoPadding");
cipherEnc.init(Cipher.ENCRYPT_MODE, keySpecEnc, ivParameterSpecEnc);
byte[] cipherdata = cipherEnc.doFinal(plaindata);
System.out.println("plaindata length: " + plaindata.length);
System.out.println("cipherdata length: " + cipherdata.length);
// 3 transfer the cipherdata to app
// ...
// === app side ===
// 4 receive encrypted data from server
// ...
// 5 decryption setup
SecretKeySpec keySpecDec = new SecretKeySpec(key, "AES");
IvParameterSpec ivParameterSpecDec = new IvParameterSpec(iv);
Cipher cipherDec = Cipher.getInstance("AES/CTR/NoPadding");
cipherDec.init(Cipher.DECRYPT_MODE, keySpecDec, ivParameterSpecDec);
// 6 decryption in one run on high memory devices
byte[] decrypteddata = cipherDec.doFinal(cipherdata);
System.out.println("decrypteddata length: " + decrypteddata.length);
// 7 decryption in chunks using the cipherdata byte array
int cipherdataLength = cipherdata.length;
int chunksize = 16; // should be a multiple of 16, minimum 16
byte[] decryptedPart = new byte[chunksize];
int parts16byteDiv = cipherdataLength / chunksize;
int parts16byteMod = cipherdataLength % chunksize;
System.out.println("cipherdata parts in " + chunksize + " byte long parts: " + parts16byteDiv + " = rounds for decryption");
System.out.println("cipherdata moduluo " + chunksize + " byte long parts: " + parts16byteMod + " + 1 round for rest/modulus");
for (int i = 0; i < parts16byteDiv; i++) {
cipherDec.update(cipherdata, (i * chunksize), chunksize, decryptedPart);
System.arraycopy(decryptedPart, 0, cipherdata, (i * chunksize), decryptedPart.length);
}
if (parts16byteMod > 0) {
decryptedPart = new byte[parts16byteMod];
cipherDec.update(cipherdata, (parts16byteDiv * chunksize), parts16byteMod, decryptedPart);
System.arraycopy(decryptedPart, 0, cipherdata, (parts16byteDiv * chunksize), decryptedPart.length);
}
System.out.println("cipherdata length: " + cipherdata.length + " (after decryption)");
// the cipherdata byte array is now decrypted !
// 8 use cipherdata (encrypted) or decrypteddata as input for exoplayer
// compare ciphertext with decrypteddata in step 6
System.out.println("plaindata equals decrypteddata: " + Arrays.equals(plaindata, decrypteddata));
// check that (decrypted) cipherdata equals plaindata of step 7
System.out.println("plaindata equals cipherdata: " + Arrays.equals(plaindata, cipherdata));
}
}

Recursive encryption with CipherOutputStream gives an empty byte[]

I'd like to encrypt a string multiple times. But I don't know why I'm ending with an empty byte array.
One public key is ok but adding another one returns an empty result.. Does anyone know why ?
private static byte[] encrypt(LinkedList<PublicKey> keys, byte[] input) throws Exception {
System.out.println("Input length : " + input.length);
if (keys.isEmpty()) {
return input;
}
PublicKey publicKey = keys.poll();
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
Cipher cipher = Cipher.getInstance("RSA");
cipher.init(Cipher.ENCRYPT_MODE, publicKey);
try (CipherOutputStream cipherOutputStream = new CipherOutputStream(byteArrayOutputStream, cipher)) {
cipherOutputStream.write(input);
}
byteArrayOutputStream.flush();
byteArrayOutputStream.close();
return encrypt(keys, byteArrayOutputStream.toByteArray());
}
public static void main(String[] args) throws Exception {
KeyPair pair1 = createPair();
KeyPair pair2 = createPair();
LinkedList<PublicKey> keys = new LinkedList<>();
keys.add(pair1.getPublic());
keys.add(pair2.getPublic());
byte[] result = encrypt(keys, "Just testing".getBytes(Charset.forName("UTF-8")));
System.out.println(new String(result, Charset.forName("UTF-8")));
}
public static KeyPair createPair() throws NoSuchAlgorithmException {
KeyPairGenerator keyPairGen = KeyPairGenerator.getInstance("RSA");
keyPairGen.initialize(2048);
return keyPairGen.generateKeyPair();
}
The output is
Input length : 12
Input length : 256
Input length : 0
After Topaco' answer.. a working version is :
private static BufferedInputStream encryptS(LinkedList<PublicKey> keys, BufferedInputStream inputStream) throws Exception {
if (keys.isEmpty()) {
return inputStream;
}
PublicKey publicKey = keys.poll();
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
cipher.init(Cipher.ENCRYPT_MODE, publicKey);
int currentPos = 0;
while (inputStream.available() > 0) {
int sizeToRead = Math.min(inputStream.available(), 245);
try (CipherOutputStream cipherOutputStream = new CipherOutputStream(byteArrayOutputStream, cipher)) {
byte[] array = new byte[sizeToRead];
inputStream.read(array, 0, sizeToRead);
cipherOutputStream.write(array);
currentPos += sizeToRead;
}
}
byteArrayOutputStream.close();
return encryptS(keys, new BufferedInputStream(new ByteArrayInputStream(byteArrayOutputStream.toByteArray())));
}

For RSA, the following must be taken into account:
The length of the message plus padding must not exceed the key length (= size of the modulus) [0], [1], [2], [3]. Padding means that additional bytes are added to the message according to a certain scheme [4].
The length of the ciphertext corresponds to the key length (=size of the modulus), [5].
This means that already after the first encryption the maximum allowed length is reached. Thus, without padding the maximum length is not exceeded, with padding it is exceeded.
Creating the cipher instance with
Cipher.getInstance("RSA")
corresponds to
Cipher.getInstance("RSA/ECB/PKCS1Padding")
for the SunJCE-Provider ([6], [7]), i.e. PKCS1 v1.5 padding is used with a padding of at least 11 characters, so that with a key size of 256 bytes the maximum size of the message must not exceed 245 bytes.
This is the reason why the recursive encryption in the current code doesn't work. If the cipher instance is created with
Cipher.getInstance("RSA/ECB/NoPadding")
(no padding used), the current code works.
However, for security reasons a padding must always be used in practice!

How to handle BadPaddingException During AES256 encryption in C# and decryption in Java

I don't know why an error is coming up.
Exception in thread "main" javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
I understand that this error occurs when the incorrect key is used during the decryption. However, if you look at the test results result below, you can see that both C# and Java are the same (Key, IV, Salt is Base64 encoded).
C# Test Result
Java Test Result
It's the same!(Key, IV, Salt)
But the current BadpaddingException error is generated. What could be the problem?
I am attaching my source file.
C# (Encryption)
class AES {
private readonly static string keyStr = "This is Key";
private readonly static string vector = "This is Vector";
public static Rfc2898DeriveBytes MakeKey(string password){
byte[] keyBytes = System.Text.Encoding.UTF8.GetBytes(password);
byte[] saltBytes = SHA512.Create().ComputeHash(keyBytes);
Rfc2898DeriveBytes result = new Rfc2898DeriveBytes(keyBytes, saltBytes, 65536);
return result;
}
public static Rfc2898DeriveBytes MakeVector(string vector){
byte[] vectorBytes = System.Text.Encoding.UTF8.GetBytes(vector);
byte[] saltBytes = SHA512.Create().ComputeHash(vectorBytes);
Rfc2898DeriveBytes result = new Rfc2898DeriveBytes(vectorBytes, saltBytes, 65536);
return result;
}
public static void Encrypt(String inputFile, String outputFile) {
using (RijndaelManaged aes = new RijndaelManaged()){
//Create Key and Vector
Rfc2898DeriveBytes key = AES.MakeKey(AES.keyStr);
Rfc2898DeriveBytes vector = AES.MakeVector(AES.vector);
//AES256
aes.BlockSize = 128;
aes.KeySize = 256;
// It is equal in java
// Cipher _Cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING");
aes.Mode = CipherMode.CBC;
aes.Padding = PaddingMode.PKCS7;
aes.Key = key.GetBytes(32); //256bit key
aes.IV = vector.GetBytes(16); //128bit block size
//processing Encrypt
ICryptoTransform encryptor = aes.CreateEncryptor(aes.Key, aes.IV);
byte[] encrypted;
using (MemoryStream msEncrypt = new MemoryStream()) {
using (CryptoStream csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write)) {
byte[] inputBytes = File.ReadAllBytes(inputFile);
csEncrypt.Write(inputBytes, 0, inputBytes.Length);
}
encrypted = msEncrypt.ToArray();
}
string encodedString = Convert.ToBase64String(encrypted);
File.WriteAllText(outputFile, encodedString);
}
}
}
Java (Decryption)
public class AES256File {
private static final String algorithm = "AES";
private static final String blockNPadding = algorithm+"/CBC/PKCS5Padding";
private static final String password = "This is Key";
private static final String IV = "This is Vector";
private static IvParameterSpec ivSpec;
private static Key keySpec;
public static void MakeKey(String password) throws NoSuchAlgorithmException, UnsupportedEncodingException, InvalidKeySpecException{
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
MessageDigest digest = MessageDigest.getInstance("SHA-512");
byte[] keyBytes = password.getBytes("UTF-8");
// C# : byte[] saltBytes = SHA512.Create().ComputeHash(keyBytes);
byte[] saltBytes = digest.digest(keyBytes);
//256bit
PBEKeySpec pbeKeySpec = new PBEKeySpec(password.toCharArray(), saltBytes, 65536, 256);
Key secretKey = factory.generateSecret(pbeKeySpec);
byte[] key = new byte[32];
System.arraycopy(secretKey.getEncoded(), 0, key, 0, 32);
SecretKeySpec secret = new SecretKeySpec(key, "AES");
setKeySpec(secret);
}
public static void MakeVector(String IV) throws NoSuchAlgorithmException, UnsupportedEncodingException, InvalidKeySpecException{
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
MessageDigest digest = MessageDigest.getInstance("SHA-512");
byte[] vectorBytes = IV.getBytes("UTF-8");
byte[] saltBytes = digest.digest(vectorBytes);
// 128bit
PBEKeySpec pbeKeySpec = new PBEKeySpec(IV.toCharArray(), saltBytes, 65536, 128);
Key secretIV = factory.generateSecret(pbeKeySpec);
byte[] iv = new byte[16];
System.arraycopy(secretIV.getEncoded(), 0, iv, 0, 16);
IvParameterSpec ivSpec = new IvParameterSpec(iv);
setIvSpec(ivSpec);
}
public void decrypt(File source, File dest) throws Exception {
Cipher c = Cipher.getInstance(blockNPadding);
c.init(Cipher.DECRYPT_MODE, keySpec, ivSpec);
fileProcessing(source, dest, c);
}
public void fileProcessing(File source, File dest, Cipher c) throws Exception{
InputStream input = null;
OutputStream output = null;
try{
input = new BufferedInputStream(new FileInputStream(source));
output = new BufferedOutputStream(new FileOutputStream(dest));
byte[] buffer = new byte[input.available()];
int read = -1;
while((read = input.read(buffer)) != -1){
output.write(c.update(buffer, 0, read));
}
byte[] deryptedBytes = c.doFinal(buffer); // -----------------------> Error!! Showing!
byte[] decodedBytes = Base64.getDecoder().decode(deryptedBytes);
String decodeString = new String(decodedBytes, "UTF-8");
decodedBytes = decodeString.getBytes(StandardCharsets.UTF_8);
output.write(decodedBytes);
}finally{
if(output != null){
try{output.close();}catch(IOException e){}
}
if(input != null){
try{input.close();}catch(IOException e){}
}
}
}
I have verified as below.
Verification Key and IV in C#
//Key Verification
var salt = Convert.ToBase64String(saltBytes);
Console.Write("Salt Result : ");
Console.WriteLine(salt);
var result_test = Convert.ToBase64String(result.GetBytes(32));
Console.Write("Key Test Result: ");
Console.WriteLine(result_test);
//IV Verification (Salt is Using same code)
var result_test = Convert.ToBase64String(result.GetBytes(16));
Console.Write("IV Test Result: ");
Console.WriteLine(result_test);
Verification Key and IV in Java
//Key Verification
/* print Salt */
String base64 = Base64.getEncoder().encodeToString(saltBytes);
System.out.println("Salt Result : " + base64);
/* print Key */
String result_test = Base64.getEncoder().encodeToString(key);
System.out.println("Key Test Result : " + result_test);
/* print generated Key */
System.out.println("Secret Key Result : " + Base64.getEncoder().encodeToString(secret.getEncoded()));
//IV Verification (Salt is Using same code)
/* print IV */
String result_test = Base64.getEncoder().encodeToString(iv);
System.out.println("IV Test Result : " + result_test);
/* print generated IV */
System.out.println("IV Result : " + Base64.getEncoder().encodeToString(ivSpec.getIV()));
Updated
c# .netframework 4.5 / Java8 modified what #Topaco said and confirmed that it worked well.
I want to say thank you very much to #Topaco and #Gusto2, and I'm going to make changes to the parts that have been modified in security, just as #Gusto2 said!

1) In the C# Encrypt-method the plain text is encrypted first and then Base64-encoded. Thus, in the decryption process the data must be Base64-decoded first and then decrypted. Currently this is handled in the wrong order i.e. the data are decrypted first and then decoded. Therefore, in the Java fileProcessing-method replace
while((read = input.read(buffer)) != -1){
output.write(c.update(buffer, 0, read));
}
with
while((read = input.read(buffer)) != -1) {
byte[] bufferEncoded = buffer;
if (read != buffer.length) {
bufferEncoded = Arrays.copyOf(buffer, read);
}
byte[] bufferDecoded = Base64.getDecoder().decode(bufferEncoded);
output.write(c.update(bufferDecoded));
}
2) It's not necessary to pass buffer (or bufferDecoded) to the doFinal-method, since that was already done in the update-method. Thus,
byte[] deryptedBytes = c.doFinal(buffer);
must be replaced with
output.write(c.doFinal());
3) Since the Base64-decoding is already done in 1) in the try-block all lines following the doFinal-statement have to be removed. Overall, this results in
try {
input = new BufferedInputStream(new FileInputStream(source));
output = new BufferedOutputStream(new FileOutputStream(dest));
byte[] buffer = new byte[input.available()];
int read = -1;
while((read = input.read(buffer)) != -1) {
byte[] bufferEncoded = buffer;
if (read != buffer.length) {
bufferEncoded = Arrays.copyOf(buffer, read);
}
byte[] bufferDecoded = Base64.getDecoder().decode(bufferEncoded);
output.write(c.update(bufferDecoded));
}
output.write(c.doFinal());
}
4) The size of the buffer has to be a multiple of 4 in order to ensure a proper Base64-decoding. Thus, it's more reliable to replace
byte[] buffer = new byte[input.available()];
with
byte[] buffer = new byte[4 * (input.available() / 4)];
As long as the data are read in one chunk (which is not guaranteed, see e.g. https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/io/InputStream.html#available()) there is no problem. However, if the data are read in several chunks it's important to read a multiple of 4 bytes, otherwise the Base64-decoding will fail. That can be easily proved by using a buffer size which isn't a multiple of 4. This point must also be considered if the buffer size is explicitly defined with regard to larger files.

while((read = input.read(buffer)) != -1){
output.write(c.update(buffer, 0, read));
}
byte[] deryptedBytes = c.doFinal(buffer)
you are decrypting the input to a file, then you are using the same cipher instance to decrypt the the last read chunk (again) into a separate array not to the file
quick fix:
while((read = input.read(buffer)) != -1){
output.write(c.update(buffer, 0, read));
}
output.write(c.doFinal()); // write the padded block
if you want to create and print a decrypted String, you need to create a new Cipher instance (or maybe it will be enough to reinitialize the instance, I am not sure) assuming the buffer contains the whole input
c.init(Cipher.DECRYPT_MODE, keySpec, ivSpec);
// assuming the buffer contains the whole input again
byte[] deryptedBytes = c.doFinal(buffer); // decrypting the whole file again
correct approach:
IV is used to securely reuse the same encryption key for multiple encryptions. So if your key is not random, you should generate new random IV for each encryption (and pass the IV along the ciphertext, most often prepended). Otherwise the encryption is not semantically secure and you may create opening for the two pad attack. So deriving IV from the key may not be very secure.
I advice to use any MAC (authentication code) passed along the ciphertext to ensure integrity (e.g. HMAC)
you are still reading all the file input fully into memory, what would not work for REALLY LARGE files. You may initialize the buffer to an arbitrary length (a few MB?) and process the input file as chunked

RSA keyblock wrapper: javax.crypto.BadPaddingException: Decryption error

In a larger application doing other things - I need to encrypt and decrypt a file. So I have been looking around and have implemented these two core functions that basically use RSA keys to wrap a random AES key that encrypts a file. The symmetric key and iv are written to the start of the file.
I'm getting an exception ("javax.crypto.BadPaddingException: Decryption error") in the decrypt function portion of below. On the unpackKeyandIV line -- the doFinal. Specifically this line is the Exception point:
Object[] keyIv = unpackKeyAndIV(xCipher.doFinal(keyBlock));
I've checked and remade the RSA key pairs. I've also checked the save/load of the keyBlock.
My gut is the problem has something to do with how I write/read the keyBlock --- or encoding perhaps?
One goal is to keep the RSA/AES instance as generic as possible so as not to need Bouncy Castle or extra Java security unlimited strength extensions.
Any thoughts on where I might be going wrong.
Thanks in advance.
[Final update: This code below is working. Error was passing in a corrupted privKey]
// RSA_INSTANCE = "RSA";
// ASSYM_CRYPTO_STR = 1024;
// SYM_CRYPTO_STR = 128;
// SYM_CRYPTO = "AES";
// AES_INSTANCE = "AES/CTR/NoPadding";
//
// File in = plain input file
// File out = encrypted output file
// Key pubKey = public Key (that wraps a random AES key)
public static void encryptFile(File in, File out, Key pubKey) throws Exception {
FileInputStream fin;
FileOutputStream fout;
int nread = 0;
byte[] inbuf = new byte[1024];
fout = new FileOutputStream(out);
fin = new FileInputStream(in);
SecureRandom random = new SecureRandom();
// symmetric wrapping
Key sKey = createKeyForAES(Config.SYM_CRYPTO_STR, random);
IvParameterSpec sIvSpec = createCtrIvForAES(0, random);
// encrypt symmetric key with RSA/pub key
Cipher xCipher = Cipher.getInstance(Config.RSA_INSTANCE);
xCipher.init(Cipher.ENCRYPT_MODE, pubKey, random);
byte[] keyBlock = xCipher.doFinal(packKeyAndIv(sKey, sIvSpec));
fout.write(keyBlock);
// encrypt data with symmetric key
Cipher sCipher = Cipher.getInstance(Config.AES_INSTANCE);
sCipher.init(Cipher.ENCRYPT_MODE, sKey, sIvSpec);
// Now read our file and encrypt it.
while((nread = fin.read(inbuf)) > 0) {
fout.write(sCipher.update(inbuf, 0, nread)); // cannot be null, by construction
}
// NB doFinal() cannot return null, but can return a zero-length array, which is benign below.
fout.write(sCipher.doFinal());
fout.flush();
fin.close();
fout.close();
}
// Decrypt File
public static void decryptFile(File in, File out, Key privKey) throws Exception {
FileInputStream fin;
FileOutputStream fout;
int nread = 0;
byte[] inbuf = new byte[1024];
fout = new FileOutputStream(out);
fin = new FileInputStream(in);
byte[] keyBlock = new byte[128];
nread = fin.read(keyBlock);
Cipher xCipher = Cipher.getInstance(Config.RSA_INSTANCE);
Cipher sCipher = Cipher.getInstance(Config.AES_INSTANCE);
// symmetric key/iv unwrapping step
xCipher.init(Cipher.DECRYPT_MODE, privKey);
Object[] keyIv = unpackKeyAndIV(xCipher.doFinal(keyBlock));
// decryption step
sCipher.init(Cipher.DECRYPT_MODE, (Key)keyIv[0], (IvParameterSpec)keyIv[1]);
while((nread = fin.read(inbuf)) >0) {
fout.write(sCipher.update(inbuf,0,nread));
}
fout.write(sCipher.doFinal());
fout.flush();
fin.close();
fout.close();
}
public static byte[] packKeyAndIv(Key key, IvParameterSpec ivSpec) throws IOException {
ByteArrayOutputStream bOut = new ByteArrayOutputStream();
bOut.write(ivSpec.getIV());
bOut.write(key.getEncoded());
return bOut.toByteArray();
}
public static Object[] unpackKeyAndIV(byte[] data) {
byte[] keyD = new byte[16];
byte[] iv = new byte[data.length - 16];
return new Object[] {
new SecretKeySpec(data, 16, data.length - 16, "AES"),
new IvParameterSpec(data, 0, 16)
};
}

Per edits and comments. Error was a corrupted privKey being passed into the decrypt function. Above code works fine.

try adding the following under your constructor -
Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());

Increase speed of encryption AES android

I develop an android application for encrypt files on the phone. By searching, i found this topic : How to encrypt file from SD card using AES in Android?
The method works fine but it is very slow to encrypt files...
At this line : byte[] d = new byte[8]; why only 8 bytes ? can't we set an higher value ?
Also, do you know a way to encrypt files fastly ? I heard of crypto++ for native code implementation but how can I implement JNI on my application ?
Thank you,
EDIT : Encryption function
public void encrypt (String RSAPrivateKey, String Key, byte[] iv, String zipname, ZipEncryptAsyncTask task) throws IOException, NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException, InvalidAlgorithmParameterException
{
FileInputStream fis = new FileInputStream (zipname + ".temp");
FileOutputStream fos = new FileOutputStream (zipname);
SecretKeySpec sks = new SecretKeySpec (Base64.decode(Key, Base64.DEFAULT), "AES");
IvParameterSpec ivspec = new IvParameterSpec (iv);
Cipher cipher = Cipher.getInstance ("AES/CBC/PKCS5Padding");
fos.write(String.valueOf(RSAPrivateKey.getBytes().length).getBytes());
fos.write(RSAPrivateKey.getBytes());
cipher.init(Cipher.ENCRYPT_MODE, sks, ivspec);
CipherOutputStream cos = new CipherOutputStream(fos, cipher);
long size = 0;
byte[] d = new byte[8];
for(int b; (b = fis.read(d)) != -1; )
{
cos.write(d, 0, b);
task.doProgress((size += 8));
}
cos.flush();
cos.close();
fis.close();
new File(zipname + ".temp").delete();
}

As an alternative, you could consider changing the cypher mode you are using. CBC mode must be used serially, block by block. Counter mode (CTR) can be run in parallel with different blocks being encrypted simultaneously. Of course there are overheads to parallel processing, and you will need to rework your code, but if buffer resizing does not give you enough speed gains, then it might be the next option to try.

as #CodesInChaos said, a way to do this, is to increase the size of the buffer.
Now I do a benchmark which determines the best size for the buffer and I use the optimal value.