So I have these large files (6GB+) that I need to decrypt on a 32 bit computer. The general procedure that I used previously was to read the entire file in memory, then pass it on to the decrypt function and then write it all back to a file. This doesn't really work due to memory limitations. I did try passing the file in parts to the decrypt function but it seems to mess up around the boundaries of where I break up the file before sending it to the decrypt function.
I've tried breaking up the file in parts relative to key size but that doesnt seem to matter. I tried a byte array of size 2048 as well as a byte aray of size 294 thinking that might be the special boundary but, no luck. I can see parts of the file correctly decrypted but parts which are total gibberish.
Is it just NOT POSSIBLE to decrypt the file in chunks? If there is a way, then how?
Here is my decryption function / my attempt to decrypt in parts.
private Path outFile;
private void decryptFile(FileInputStream fis, byte[] initVector, byte[] aesKey, long used) {
//Assume used = 0 for this function.
byte[] chunks = new byte[2048]; //If this number is greater than or equal to the size of the file then we are good.
try {
if (outFile.toFile().exists())
outFile.toFile().delete();
outFile.toFile().createNewFile();
FileOutputStream fos = new FileOutputStream(outFile.toFile());
OutputStreamWriter out = new OutputStreamWriter(fos);
IvParameterSpec spec = new IvParameterSpec(Arrays.copyOfRange(initVector, 0, 16));
SecretKeySpec key = new SecretKeySpec(aesKey, "AES");
Cipher cipher = Cipher.getInstance("AES/CFB/NoPadding");
cipher.init(Cipher.DECRYPT_MODE, key, spec);
int x;
while ((x = fis.read(chunks, 0, chunks.length)) != -1) {
byte[] dec = cipher.doFinal(Arrays.copyOfRange(chunks, 0, x));
out.append(new String(dec));
}
out.close();
fos.close();
} catch (Exception e) {
e.printStackTrace();
LOG.error(ExceptionUtils.getStackTrace(e));
}
}
Consider using Cipher#update(byte[], int, int, byte[], int) instead of doFinal() for multipart operations. This will take care of part boundaries for you.
The last part of the deciphered data can be obtained by calling the doFinal(byte[] output, int outputOffset) method.
Related
I have an android app which handles some large byte array but I am getting some OOM crash in my Firebase Crashlytics reports for devices with low memory while handling byte array whose size may go from 10 mb to 50mb. Below is my method that I have used. So could anyone help me to improve it to avoid OOM.
byte[] decrypt(File files) {
try {
FileInputStream fis = new FileInputStream(files);
SecretKeySpec sks = new SecretKeySpec(getResources().getString(R.string.encryptPassword).getBytes(),
"AES");
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.DECRYPT_MODE, sks);
CipherInputStream cis = new CipherInputStream(fis, cipher);
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int b;
byte[] d = new byte[1024];
while ((b = cis.read(d)) != -1) {
buffer.write(d, 0, b); //this is one of the line which is being referred for the OOM in firebase
}
byte[] decryptedData = buffer.toByteArray();//this is the line which is being referred for the OOM in firebase
buffer.flush();
fis.close();
cis.close();
return decryptedData;
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
EDIT
Actually I am using the above method for decrypting downloaded audio files which are encrypted during downloading.
The above methods return the content of the encrypted files to exoplayer to play its content and I am calling the above method in the following way
ByteArrayDataSource src= new ByteArrayDataSource(decrypt(some_file));
Uri uri = new UriByteDataHelper().getUri(decrypt(some_file));
DataSpec dataSpec = new DataSpec(uri);
src.open(dataSpec);
DataSource.Factory factory = new DataSource.Factory()
{
#Override public DataSource createDataSource()
{
return src;
}
};
audioSource = new ProgressiveMediaSource.Factory(factory).createMediaSource(uri);
First of all, I would make sure that the devices where this is running have enought heap memory to run this, it might be that simply the software has already been allocated with a lot of space and there might not be much more left on the heap to provide the software. This operation should not require much memory and I don't see anything obvious that would point towards trying to allocated and unexpectadely large amount of memory.
What I would recommend though, if you want to hav a quick test is actually simply lowering the byte array size, any particular reason why you are using 1024?
If possible perhaps try:
byte[] d = new byte[8];
Also, If that was me, I would store the read data temporarily, on an array perhaps and only once the read of the cypher has finished, I would call
buffer.write()
From my experience, trying to read and write at the same time tens to not be advised and can result in serveral issues, at the very least you should make sure you have the whole cypher and that it is a valid one (if you have some validation requirements) and only then send it.
Again, this should not be the core issue, the device seems that it is lacking enough availalbe memory to be allocated, perhaps too much reserved memory for other processes?
You should consider of writing the decrypted data to a tempfile and then reload the data for usage.
The main reasons for the Out of memory error are the ByteArrayOutputStream AND
byte[] decryptedData = buffer.toByteArray(), because both of them hold the complete (decrypted) data and that doubles the memory consumption of your decrypt method.
You could avoid this by decrypting the data to a tempfile in the first step and later load the data from the tempfile. I modified the decrypt method to handle the decrypted output stream and later there is a method for reloading the decrypted data (there is no propper exception handling and for my testing I setup a static encryptPassword-variable ...).
There is just one part left for you - you need to find a good place for the tempfile and I'm no Android specialist.
Just two notes: You are using the unsecure AES ECB mode and the String to byte[]conversion for your password should be
changed to
.getBytes(StandardCharsets.UTF_8)
on ecryption and decryption side to avoid errors caused by different encodings on different platforms.
public static void decryptNew(File files, File tempfiles) {
try (FileInputStream fis = new FileInputStream(files);
BufferedInputStream in = new BufferedInputStream(fis);
FileOutputStream out = new FileOutputStream(tempfiles);
BufferedOutputStream bos = new BufferedOutputStream(out)) {
byte[] ibuf = new byte[1024];
int len;
Cipher cipher = Cipher.getInstance("AES");
SecretKeySpec sks = new SecretKeySpec(encryptPassword.getBytes(),"AES"); // static password
// SecretKeySpec sks = new SecretKeySpec(getResources().getString(R.string.encryptPassword).getBytes(),"AES");
cipher.init(Cipher.DECRYPT_MODE, sks);
while ((len = in.read(ibuf)) != -1) {
byte[] obuf = cipher.update(ibuf, 0, len);
if (obuf != null)
bos.write(obuf);
}
byte[] obuf = cipher.doFinal();
if (obuf != null)
bos.write(obuf);
} catch (BadPaddingException | IllegalBlockSizeException | InvalidKeyException | IOException | NoSuchAlgorithmException | NoSuchPaddingException e) {
e.printStackTrace();
}
}
public static byte[] loadFile(File filename) throws IOException {
byte[] filecontent = new byte[0];
FileInputStream fileInputStream = null;
try {
fileInputStream = new FileInputStream(filename);
// int byteLength = fff.length();
// In android the result of file.length() is long
long byteLength = filename.length(); // byte count of the file-content
filecontent = new byte[(int) byteLength];
fileInputStream.read(filecontent, 0, (int) byteLength);
} catch (IOException e) {
e.printStackTrace();
fileInputStream.close();
return filecontent;
}
fileInputStream.close();
return filecontent;
}
After loading the tempfile content to the byte array you can delete the file with a one-liner (again no exception handling):
Files.deleteIfExists(tempFile.toPath());
I'm writing a second answer and do not edit my first answer as it is a total different approach to solve the problem.
As you post a part of your code I can see that you have a byte array with the complete and decrypted content that gets played by the exoplayer:
output:
byte[] decrypt(File files)
as input for
ByteArrayDataSource src= new ByteArrayDataSource(decrypt(some_file));
So to avoid double and moretime memory consumption when playing with large files (approx. 50mb) my approach it is to download the complete encrypted file and save it in a byte array.
On devices with a good memory equipment you can decrypt the encrypted byte array in one run to another byte array and play the music from this decrypted byte array (step 6 + 8 in my sample program).
Using a low memory device you decrypt the byte array in chunks (in my program in 16 byte long blocks) and save the decrypted chunks at the same place in the encrypted byte array. When all chunks got processed the (former) encrypted data are now decrypted and you used the memory of just one byte array length. Now you can play the music from this byte array (steps 7 + 8).
Just for explanation, steps 1-3 are on server side and in steps 3+4 the transmission takes place.
This example uses the AES CTR mode as it provides the same length for input and output data.
In the end I'm comparing the byte arrays to prove that the decryption was successful for steps 6 (direct decryption) and 7 (decryption in chunks):
output:
Decrypting 50mb data in chunks to avoid out of memory error
https://stackoverflow.com/questions/62412705/how-to-write-byte-array-without-oom
plaindata length: 52428810
cipherdata length: 52428810
decrypteddata length: 52428810
cipherdata parts in 16 byte long parts: 3276800 = rounds for decryption
cipherdata moduluo 16 byte long parts: 10 + 1 round for rest/modulus
cipherdata length: 52428810 (after decryption)
plaindata equals decrypteddata: true
plaindata equals cipherdata: true
code:
import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import java.security.GeneralSecurityException;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Random;
public class EncryptionCtrSo4 {
public static void main(String[] args) throws GeneralSecurityException {
System.out.println("Decrypting 50mb data in chunks to avoid out of memory error");
System.out.println("https://stackoverflow.com/questions/62412705/how-to-write-byte-array-without-oom");
/*
* author michael fehr, http://javacrypto.bplaced.net
* no licence applies, no warranty
*/
// === server side ===
// 1 create a 50 mb byte array (unencrypted)
byte[] plaindata = new byte[(50 * 1024 * 1024 + 10)];
// fill array with (random) data
Random random = new Random();
random.nextBytes(plaindata);
// 2 encrypt the data with aes ctr mode, create random keys
SecureRandom secureRandom = new SecureRandom();
byte[] key = new byte[32]; // 32 byte = 256 bit aes key
secureRandom.nextBytes(key);
byte[] iv = new byte[16]; // 16 byte = 128 bit
secureRandom.nextBytes(iv);
SecretKeySpec keySpecEnc = new SecretKeySpec(key, "AES");
IvParameterSpec ivParameterSpecEnc = new IvParameterSpec(iv);
Cipher cipherEnc = Cipher.getInstance("AES/CTR/NoPadding");
cipherEnc.init(Cipher.ENCRYPT_MODE, keySpecEnc, ivParameterSpecEnc);
byte[] cipherdata = cipherEnc.doFinal(plaindata);
System.out.println("plaindata length: " + plaindata.length);
System.out.println("cipherdata length: " + cipherdata.length);
// 3 transfer the cipherdata to app
// ...
// === app side ===
// 4 receive encrypted data from server
// ...
// 5 decryption setup
SecretKeySpec keySpecDec = new SecretKeySpec(key, "AES");
IvParameterSpec ivParameterSpecDec = new IvParameterSpec(iv);
Cipher cipherDec = Cipher.getInstance("AES/CTR/NoPadding");
cipherDec.init(Cipher.DECRYPT_MODE, keySpecDec, ivParameterSpecDec);
// 6 decryption in one run on high memory devices
byte[] decrypteddata = cipherDec.doFinal(cipherdata);
System.out.println("decrypteddata length: " + decrypteddata.length);
// 7 decryption in chunks using the cipherdata byte array
int cipherdataLength = cipherdata.length;
int chunksize = 16; // should be a multiple of 16, minimum 16
byte[] decryptedPart = new byte[chunksize];
int parts16byteDiv = cipherdataLength / chunksize;
int parts16byteMod = cipherdataLength % chunksize;
System.out.println("cipherdata parts in " + chunksize + " byte long parts: " + parts16byteDiv + " = rounds for decryption");
System.out.println("cipherdata moduluo " + chunksize + " byte long parts: " + parts16byteMod + " + 1 round for rest/modulus");
for (int i = 0; i < parts16byteDiv; i++) {
cipherDec.update(cipherdata, (i * chunksize), chunksize, decryptedPart);
System.arraycopy(decryptedPart, 0, cipherdata, (i * chunksize), decryptedPart.length);
}
if (parts16byteMod > 0) {
decryptedPart = new byte[parts16byteMod];
cipherDec.update(cipherdata, (parts16byteDiv * chunksize), parts16byteMod, decryptedPart);
System.arraycopy(decryptedPart, 0, cipherdata, (parts16byteDiv * chunksize), decryptedPart.length);
}
System.out.println("cipherdata length: " + cipherdata.length + " (after decryption)");
// the cipherdata byte array is now decrypted !
// 8 use cipherdata (encrypted) or decrypteddata as input for exoplayer
// compare ciphertext with decrypteddata in step 6
System.out.println("plaindata equals decrypteddata: " + Arrays.equals(plaindata, decrypteddata));
// check that (decrypted) cipherdata equals plaindata of step 7
System.out.println("plaindata equals cipherdata: " + Arrays.equals(plaindata, cipherdata));
}
}
We implemented chunked file decryption in a way that the initialization vector is added at the beginning of the file and followed by the encrypted data. The following decryption method decrypts the file and write:
private void decrypt_AES_CBC_PKCS7(final byte[] symKeyBytes, final FileInputStream inputStream, final FileOutputStream outputStream) throws Exception {
Security.addProvider(new BouncyCastleProvider());
// Read init vector
final byte[] iv = new byte[16];
inputStream.read(iv, 0, 16);
// Prepare for decryption
final SecretKeySpec secretKeySpec = new SecretKeySpec(symKeyBytes, "AES");
final Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7Padding", new BouncyCastleProvider());
cipher.init(Cipher.DECRYPT_MODE, secretKeySpec, new IvParameterSpec(iv));
// Decrypt chunk by chunk
int chunkLen = 0;
final byte[] buffer = new byte[CHUNK_SIZE_DECRTPY]; // CHUNK_SIZE_DECRTPY = 20 * 1024 * 1024;
while ((chunkLen = inputStream.read(buffer)) > 0) {
byte[] decrypted = cipher.doFinal(buffer, 0, chunkLen);
outputStream.write(decrypted, 0, decrypted.length);
}
// close streams
inputStream.close();
outputStream.close();
}
The code worked fine in earlier Android versions (L & M), but when we tried it on Nexus 5X with Android N, 16 "junk" bytes were inserted at the begginning of the resulting file. This happens only in files that consist of one chunk only, i.e., files with size larger than one chunk would be decrypted correctly and no extra bytes would be prepended. Interestingly, when the code is run with Android studio debugger attached, with a breakpoint between reading IV and reading input stream, the decryption works fine and no extra bytes appear in the output file.
Example encrypted file (IV is visible as first 16 bytes, i.e., first row):
Example encrypted file
Example decrypted file, first 16 bytes are added only in Android N:
Example decrypted file
All suggestions are welcome!
As suggested by #martijno, I checked the result of inputStream.read(iv, 0, 16) and it turned out that the first invocation of it returns -1 in Android N. So, I modified the part of the code that reads the IV:
final byte[] iv = new byte[16];
int result;
int numTries = 0;
int MAX_NUM_TRIES = 2;
do {
result = inputStream.read(iv, 0, 16);
++numTries;
} while (result == -1 && numTries < MAX_NUM_TRIES);
if (result == -1) {
throw new Exception("Failed to read IV from file!");
}
It turns out that two tries for reading IV are enough, we'll test this further when more Android N devices become available.
With the following Code - I'm generally able to decrypt a token that I'm being passed. The token string when decoded to json will look like the following:
\"id\":\"9efef759-15a3-4cd0-b1f1-fceab7ad0a6e\",
\"exp\":\"2016-07-23T15:27:50.758+12:00\",
\"iv\":\"OOqNpy9puM5jPjTwrWHSNb+d5NYDEwIq2pZFqx6mraI14Kkh0bzEWADoU2d/KGu6cp9/FrVt4epheIP5Fw9qUFrdVcNYjLO5HWdJ0V5GhpdLJlFbMnFy4vS1rJ+4X1qTNZrqPwZh2deLceoHmxnqw7ml8JVFeIaz9H8BQXkgcNo=\",
\"ver\":\"1\",
\"iat\":\"2016-07-13T15:27:50.758+12:00\",
\"key\":\"d7R9blmqBYMywOEdYpRbd+gvKPfOqmxsRQMlDipkuGoWZobJ0dnK0MGBFAXq4wOdHbHVbfisjqm+6HoRSZ2w0KcfY+enPoKL5yptvlULkwpDtATEP8pnRmCh6ycWntbanL1gJI7RoNWTkomItBp/yODdL5kSMue76xAtIzc9+no=\",
\"sig\":\"X6A58tRDSUC5HJEP1VVmQjo17Qk2rJC9pYZiV5ccIjdcLmz7HPIkpm0ZCsFcQX4ps1k32asSojqOyegYFIdDqHypdrV9c5sHchIrp6Ak8MOjNTpy+SweTGPzkjlEHCMkWLVHjrkBq9mmoMk2o0sYyZes+/ARuYB8IjtAINtbAQE=\",
\"enc\":\"n+exbDhicBLuUtbYPXrrKESIktgyaidSreD5FWAxErGJeOyjTWv9QOqCGfEou5yJq2njCddf0mu0JOEP9i1mlhe1MUUa1hE4J+qnqxre+tSxWRNszHQL8Pk+0FV6cZ1nqk+aCfw9VOjlOLYXYmNF0NSZBqQIqzpobM3twHIf5u7pvJkvbnfP8Db0S83ZchNgMWyH1t+UEb+jbpcg1Um3U7Yb8Q==\"
The IV and key are pulled from the token and Asymmetrically decrypted then the enc text is symmetrically decrypted before they are passed to a gzip decompression.
internal virtual UserObj decrypt(string jsonToken, UserObj cls, System.Security.Cryptography.AsymmetricAlgorithm certPrivateKey)
{
Org.BouncyCastle.Crypto.AsymmetricCipherKeyPair bcPrivateKey;
try
{
//Make a bouncyCastle private key for feeding to the rsa Engine.
bcPrivateKey = Org.BouncyCastle.Security.DotNetUtilities.GetKeyPair(certPrivateKey);
// Attempt to unmarshal the JSON token
Token token = encoder.unmarshalJsonString<Token>(jsonToken);
// Asymmetrically encrypt the symmetric encryption iv
Org.BouncyCastle.Crypto.Engines.RsaEngine rsaCipher = new Org.BouncyCastle.Crypto.Engines.RsaEngine();
Org.BouncyCastle.Crypto.Encodings.Pkcs1Encoding rsaEncoder = new Org.BouncyCastle.Crypto.Encodings.Pkcs1Encoding(rsaCipher);
rsaEncoder.Init(false, bcPrivateKey.Private);
// Asymmetrically decrypt the symmetric encryption key
byte[] encryptedAesKeyBytes = encoder.fromBase64String(token.Get(Token.ENCRYPTED_KEY));
byte[] aesKeyBytes = rsaEncoder.ProcessBlock(encryptedAesKeyBytes, 0, encryptedAesKeyBytes.Length);
// Asymmetrically decrypt the symmetric encryption IV
byte[] encryptedAesIvBytes = encoder.fromBase64String(token.Get(Token.IV));
byte[] aesIvBytes = rsaEncoder.ProcessBlock(encryptedAesIvBytes, 0, encryptedAesIvBytes.Length);
//Setting equivalent excyption to "AES/CTR/NoPadding"
Org.BouncyCastle.Crypto.Engines.AesEngine aes = new Org.BouncyCastle.Crypto.Engines.AesEngine();
Org.BouncyCastle.Crypto.Modes.SicBlockCipher blockCipher = new Org.BouncyCastle.Crypto.Modes.SicBlockCipher(aes);
Org.BouncyCastle.Crypto.Paddings.PaddedBufferedBlockCipher aesCipher = new Org.BouncyCastle.Crypto.Paddings.PaddedBufferedBlockCipher(blockCipher, new Org.BouncyCastle.Crypto.Paddings.ZeroBytePadding());
Org.BouncyCastle.Crypto.Parameters.KeyParameter keyParam2 = new Org.BouncyCastle.Crypto.Parameters.KeyParameter(aesKeyBytes);
// Symmetrically decrypt the data
Org.BouncyCastle.Crypto.Parameters.ParametersWithIV keyParamWithIv = new Org.BouncyCastle.Crypto.Parameters.ParametersWithIV(keyParam2, aesIvBytes, 0, TokenEncryptor.IV_SIZE_BYTES);
//
// Symmetrically decrypt the data
aesCipher.Init(false, keyParamWithIv);
string encryptedData = token.Get(Token.ENCRYPTED_DATA);
byte[] inputBytes = encoder.fromBase64String(encryptedData);
byte[] compressedJsonBytes = new byte[aesCipher.GetOutputSize(inputBytes.Length)];
//Do the decryption. length is the proper size of the compressed data, compressedJsonBytes will
//contain extra nulls at the end.
int length = aesCipher.ProcessBytes(inputBytes, compressedJsonBytes, 0);
//String to look at the compressed data (debug)
string compressed = encoder.toBase64String(compressedJsonBytes);
byte[] compressedJsonBytesProperSize = new byte[length];
Array.Copy(compressedJsonBytes, compressedJsonBytesProperSize, length);
//String to look at the compressed data (debug)
compressed = encoder.toBase64String(compressedJsonBytesProperSize);
byte[] jsonBytes = null;
try
{
jsonBytes = encoder.decompress(compressedJsonBytesProperSize);
}
catch (Exception)
{
jsonBytes = encoder.decompress(compressedJsonBytes);
}
string tmep = System.Text.Encoding.UTF8.GetString(jsonBytes, 0, jsonBytes.Length);
UserObj dataObj = encoder.fromJsonBytes<UserObj>(jsonBytes);
return dataObj;
}
catch (Exception e)
{
throw new Exceptions.TokenDecryptionException(e);
}
}
The final decrypted token looking something like this:
\"domain\":\"GLOBAL\",
\"user\":\"someuser\",
\"groups\":[\"GROUP1\",\"GROUP2\",\"GROUP3\"],
\"branchId\":\"0000\"
My problem occurs when depending on the number of items in the group, the GZ Decompression will fail. On some tokens if I pass the full compressedJsonByte array (with nulls at the end), it complains about a CRC error (why I have the try/catch around the decompression), so then I pass it the trimmed byte array. But for other tokens with more groups, it decompresses with the full byte array.
I have a comparable encryption routine and found that with everything else being equal if a alerted the user name from 17 to 19 characters I'd need to use the untrimmed byte array for decompression. But have since found the problem runs deeper.
Any help would be appreciated. I'm hoping its a decompression issue, but I suspect maybe something in the decrypting is fudging the end of the output byte array.
I'm not able to change the decryption type as it's coming from an external entity and their side is written in Java.
For reference the decompression routine is:
public virtual byte[] decompress(byte[] compressedData)
{
try
{
//Push to a file for debug
System.IO.FileStream fs = new System.IO.FileStream(#"C:\temp\file.gz", System.IO.FileMode.OpenOrCreate);
fs.Write(compressedData,0,compressedData.Length);
fs.Flush();
fs.Close();
byte[] outputBytes = new byte[4096];
byte[] buffer = new byte[4096];
using (System.IO.MemoryStream msInput = new System.IO.MemoryStream(compressedData))
{
System.IO.MemoryStream msOutput = new System.IO.MemoryStream();
//using (System.IO.Compression.GZipStream gzs = new System.IO.Compression.GZipStream(msInput, System.IO.Compression.CompressionMode.Decompress))
using (ZLibNet.GZipStream gzs = new ZLibNet.GZipStream(msInput, ZLibNet.CompressionMode.Decompress))
{
int nRead;
bool canR = gzs.CanRead;
while ((nRead = gzs.Read(buffer, 0, buffer.Length)) > 0)
{
msOutput.Write(buffer, 0, nRead);
}
}
outputBytes = msOutput.ToArray();
if (outputBytes.Length == 0)
throw new Exception("Could not decompress");
}
return outputBytes;
}
catch (Exception e)
{
throw new Exceptions.ServiceException(e);
}
}
You're using ZeroBytePadding while CTR mode doesn't require any padding at all, you might as well directly use the SicCipher instance.
Zero byte padding will strip all the zero valued bytes from the end of the data, which is why you may end up with compromised data.
Zero byte padding is not deterministic and should not be used unless you're either:
sure of the data not ending with a zero byte or bit;
able to determine the plaintext length in other ways.
I may have just cracked it. For the larger payload token, ProcessBytes would work in one pass.
For the smaller ones it took me a long while to notice that the length reported from ProcessBytes was less that the length of the inputBytes array.
I tried a DoFinal after the process bytes:
int length = aesCipher.ProcessBytes(inputBytes, compressedJsonBytes, 0);
int length2= aesCipher.DoFinal(compressedJsonBytes, length);
And for a token I encoded it worked...but for the given ones I have, it posted a Org.BouncyCastle.Crypto.DataLengthException: last block incomplete in decryption
So finally I tried
int length2 = aesCipher.ProcessBytes(inputBytes,length,inputBytes.Length-length,compressedJsonBytes,length);
To process the remain data - and this worked.
So my resulting code now looks as follows - replacing the single line:
int length = aesCipher.ProcessBytes(inputBytes, compressedJsonBytes, 0);
With:
int length = 0;
while (length < inputBytes.Length)
{
length += aesCipher.ProcessBytes(inputBytes, length, inputBytes.Length-length, compressedJsonBytes, length);
}
And this seems to have removed the need to take the full length compressed byte array. The UncompressedProperSize byte array now down it's job properly.
I'm currently working on an encryption program, and I'm having an issue when decrypting. The resulting file is blank, and I have been trying to find the reason for this for about an hour. My decryption code is below...
Can someone please tell me why my data might come out blank?
file = x;
FileInputStream fis = new FileInputStream(file.getAbsolutePath());
file = new File(file.getAbsolutePath().substring(0,
file.getAbsolutePath().length() - 4));
FileOutputStream fos = new FileOutputStream(file);
byte k[] = Hash.MD5(password).getBytes("UTF-8");
SecretKeySpec key = new SecretKeySpec(k, "AES");
Cipher cipher = Cipher.getInstance(algorithm);
byte[] iv = batchIV;
IvParameterSpec ivSpec = new IvParameterSpec(iv);
cipher.init(Cipher.DECRYPT_MODE, key, ivSpec);
CipherInputStream cin = new CipherInputStream(fis,
cipher);
byte[] buffer = new byte[1024];
int read = 0;
while ((read = cin.read(buffer)) != -1) {
fos.write(buffer, 0, read);
}
fos.flush();
fos.close();
cin.close();
Links go to larger code portions.
Decrypt Method Here: http://pastebin.com/2p2juUTa
Full Class Here: http://pastebin.com/hgZHT4wg
I've found that the CipherInputStream is returning -1 when you try to read from it... I'm still unsure as to what might cause this, if anyone can help.
You don't state what Hash.MD5() and Hash.MD5R() do, let alone supply the source code, but evidently MD5() returns a String, which is already an error (see below), and that MD5R() only returns the input argument, which is basically pointless. Unless you think you've discovered a way to reverse MD5? You haven't.
If you make the following changes to your code:
change MD5() to return the byte[] resulting from the MessageDigest.digest() operation that it must perform, instead of wrapping that in a String, and make the corresponding adjustments at the callings sites (i.e. remove .toByteArray("UTF-8") in a couple of places, and
change MD5R() to return the input argument, or just remove the method,
you will find that your code works.
NB:
Repeat after me: 'String is not a container for binary data'.
flush() before close() is redundant.
The File variable file should be method-local, not static.
You don't need all those File.getAbsolutePath() method calls. getPath() will work just as well in most cases, or just the File object itself in others.
I am new to blackberry development and got to complete a task of encryption and decryption with AES/ECB/NoPadding. I used below code, from internet.
Encryption method:
public static byte[] encrypt( byte[] keyData,String message )
throws Exception
{
byte[] data = message.getBytes("UTF-8");
// Create the AES key to use for encrypting the data.
// This will create an AES key using as much of the keyData
// as possible.
if ((data.length % 16) != 0 ) {
StringBuffer buffer = new StringBuffer(message);
int moduleOut = data.length % 16;
int padding = 16 - moduleOut;
for(int i = 0 ; i < padding; i++){
buffer.append(" ");
}
data = buffer.toString().getBytes("UTF-8");
}
AESKey key = new AESKey( keyData);
NoCopyByteArrayOutputStream out = new NoCopyByteArrayOutputStream(data.length);
AESEncryptorEngine engine = new AESEncryptorEngine(key);
BlockEncryptor encryptor = new BlockEncryptor(engine, out);
encryptor.write(data,0,data.length);
int finalLength = out.size();
byte[] cbytes = new byte[finalLength];
System.arraycopy(out.toByteArray(), 0, cbytes, 0, finalLength);
// encryptor.close();
// out.close();
return cbytes;
}
Decryption method:
public static byte[] decrypt(byte[] keyData, byte[] base64EncodedData)
throws CryptoException, IOException
{
// String base64EncodedData=new String(base64EncodedData);
byte[] cipherText =Base64ToBytes(new String(base64EncodedData));
// First, create the AESKey again.
AESKey key = new AESKey(keyData);
// Now, create the decryptor engine.
AESDecryptorEngine engine = new AESDecryptorEngine(key);
// Create the BlockDecryptor to hide the decryption details away.
ByteArrayInputStream input = new ByteArrayInputStream(cipherText);
BlockDecryptor decryptor = new BlockDecryptor(engine, input);
// Now, read in the data.
byte[] temp = new byte[100];
DataBuffer buffer = new DataBuffer();
for (;;)
{
int bytesRead = decryptor.read(temp);
buffer.write(temp, 0, bytesRead);
if (bytesRead < 100)
{
// We ran out of data.
break;
}
}
byte[] plaintext = buffer.getArray();
return plaintext;
}
Base64 to Bytes convert method:
private static byte[] Base64ToBytes(String code) {
byte[] aesString = null;
try
{
aesString = Base64InputStream.decode(code);
}
catch (IOException ioe)
{
}
return aesString;
}
Now the problem is when i encrypt my string with above method i get padded unicodes at the end of the string, which is not tolerable at Server side.
I know it is due to PKCS5FormatterEngine and its normal to have characters appended at end of the string while using this class.
But what if i want to encrypt and decrypt string with AES/ECB method and that too with NoPadding. I know ECB is not a secure mode and all that but server is with PHP and ready, works great in Android, and J2ME.
Please guide. How to bypass PKCS5FormatterEngine or encrypt without any padding.
Update:
I tried to use Cipher class in Blackberry as i used in Android and J2ME but it seems unavailable in net_rim_api.jar, even i tried downloading bouncy castle jar file and dependent class NoSuchAlogrithmException so java.security jar (org.osgi.foundation-1.0.0.jar), compiles but when i try to run it stops saying duplicate classes found. It has a problem with few duplicate classes in jar i have kept for java.security.
If you have solution towards this, please let me know.
Update for Answer:
I have update my code with full encryption and decryption code and also check the answer for better understanding.
Not sure this is really an answer in general, but it might help in this specific case, so I am adding it as such.
You can't really use AES without some padding, because the AES processing does not wish to assume the data you supply will be a multiple of 16 bytes. However if you actually always supply a buffer that is a multiple of 16 bytes, then you can just encrypt your data with code like this:
AESEncryptorEngine engine = new AESEncryptorEngine( key );
for ( int j = 0; j < ciphertext.length - 15; ) {
engine.encrypt(plainText, j, ciphertext, j);
j = j+16;
}
So how do you make sure this works OK at the other end? It may or may not be possible to do this - it actually depends on what is being transferred.
But if, for example, you were passing XML data, then you can append spaces to make the data up to a 16 byte boundary and these will be decrypted as spaces by the Server, and then ignored by the parsing. You can add redundant padding bytes to a variety of file formats and the padding will be ignored, it all depends on the file format being processed.
Update
Given that the actual data is JSON and that I believe that JSON will ignore trailing spaces, the approach I would take is append spaces to the JSON data before encrypting it.
So convert the JSON string to bytes:
byte [] jsonBytes = jsonString.getBytes("UTF-8");
pad this if you need too:
if ( (jsonBytes.length % 16) != 0 ) {
// Now pad this with spaces
}
and you can encrypt the result with no worries about padding bytes.