Develop Reference

Problem with ANSI Encoding file with Encryption in Linux, Java

I export a database with mysqldump a database in Ubuntu with java, then I encrypt and decrypt it with Java. I doing that with the following classes Encrypt and Decrypt with Java. But after the decryption some characters at the start of the file is wrong. Here is the problem:
At the first image is the file which programmatically have mysqldump, encrypt and decrypt. At the second one is just the mysqldump from the same command line. Can you point me the direction what to do? Thanks
EDIT
I have create a salt and stored it in a file like this:
Encryption:
FileInputStream saltFis = new FileInputStream("salt.enc");
byte[] salt = new byte[8];
saltFis.read(salt);
saltFis.close();
// reading the iv
FileInputStream ivFis = new FileInputStream("iv.enc");
byte[] iv = new byte[16];
ivFis.read(iv);
ivFis.close();
SecretKeyFactory factory = SecretKeyFactory.getInstance(secretAlgorithm1);
KeySpec keySpec = new PBEKeySpec(rsaSecret.toCharArray(), salt, 65536, 256);
SecretKey secretKey = factory.generateSecret(keySpec);
SecretKey secret = new SecretKeySpec(secretKey.getEncoded(), secretAlgorithm2);
//
Cipher cipher = Cipher.getInstance(algorithmEncryption);
cipher.init(Cipher.ENCRYPT_MODE, secret);
// file encryption
byte[] input = new byte[64];
int bytesRead;
while ((bytesRead = inFile.read(input)) != -1) {
byte[] output = cipher.update(input, 0, bytesRead);
if (output != null)
outFile.write(output);
}
byte[] output = cipher.doFinal();
if (output != null)
outFile.write(output);
inFile.close();
outFile.flush();
outFile.close();
Decryption:
FileInputStream saltFis = new FileInputStream("salt.enc");
byte[] salt = new byte[8];
saltFis.read(salt);
saltFis.close();
// reading the iv
FileInputStream ivFis = new FileInputStream("iv.enc");
byte[] iv = new byte[16];
ivFis.read(iv);
ivFis.close();
SecretKeyFactory factory = SecretKeyFactory.getInstance(secretAlgorithm1);
KeySpec keySpec = new PBEKeySpec(rsaSecret.toCharArray(), salt, 65536, 256);
SecretKey secretKey = factory.generateSecret(keySpec);
SecretKey secret = new SecretKeySpec(secretKey.getEncoded(), secretAlgorithm2);
// file decryption
Cipher cipher = Cipher.getInstance(algorithmEncryption);
cipher.init(Cipher.DECRYPT_MODE, secret, new IvParameterSpec(iv));
FileInputStream fis = new FileInputStream(decodedB64);
FileOutputStream fos = new FileOutputStream(outputFile);
byte[] in = new byte[64];
int read;
while ((read = fis.read(in)) != -1) {
byte[] output = cipher.update(in, 0, read);
if (output != null)
fos.write(output);
}
byte[] output = cipher.doFinal();
if (output != null)
fos.write(output);
fis.close();
fos.flush();
fos.close();
System.out.println("File Decrypted.");

Oh, that one is simple. That idiotic (but funny enough, seeming largely correct otherwise) method of file encryption using CBC stores the IV in a separate file, overwriting any old one. So if you overwrite or take the wrong IV file then you'll get 16 random bytes at the start after decryption. So unless you can find the IV file that hopefully makes sense, your first 16 bytes (/characters) are now lost forever.
Of course, any sane encryption program stores the salt (a password & PBKDF2 is used for key derivation) and IV in the same file as the ciphertext.
Still, if you managed to lose the salt file or password then all the data would have been lost, so there's that...
With the added code the issue becomes even more clear. In the encryption mode you are forgetting to create & use an IvParameterSpec entirely during initialization:
cipher.init(Cipher.ENCRYPT_MODE, secret);
however, because of the way the IV data is read, you don't get any warning that the variable isn't used:
ivFis.read(iv);
If you would have created a nice method such as IvParameterSpec iv = readIvFromFile() then you would have caught this error.
Note that Java (by default in the included provider for Cipher) uses an all zero IV, so you're lucky and your data isn't partially gone.

Compress then encrypt via buffer in Java

I'm new to encryption/compression in Java and I'm working on a test project where the goal is to compress and then encrypt files via a buffered input in Java. At no point should the file be stored on disk in a non-encrypted format, therefore I want to do the compression and encryption solely on a buffer until the file is fully written.
So the progression would be: read part of file into memory (buffer, 1024 bytes) -> compress (~32 bytes)-> encrypt -> output to disk -> repeat until entire file is written
The issue I'm facing is that once I perform the reverse operations to read the compressed/encrypted file back, only part of the data is there.
To accomplish my goal, I've been combining the Inflater/Deflater classes and a block cipher with AES 256 encryption.
Encryption setup:
byte[] randomSalt = new byte[8];
SecureRandom secRand = new SecureRandom();
secRand.nextBytes(randomSalt);
String randomPassword = new BigInteger(130, secRand).toString(32);
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
KeySpec spec = new PBEKeySpec(randomPassword.toCharArray(), randomSalt, 65536, 256);
SecretKey tmp = factory.generateSecret(spec);
SecretKey secret = new SecretKeySpec(tmp.getEncoded(), "AES");
Cipher cipher = Cipher.getInstance("AES/CTR/NoPadding");
cipher.init(Cipher.ENCRYPT_MODE, secret);
AlgorithmParameters params = cipher.getParameters();
byte[] iv = params.getParameterSpec(IvParameterSpec.class).getIV();
Getting input / writing output:
BufferedInputStream bufferedInput = new BufferedInputStream(
new FileInputStream("file.txt"));
BufferedOutputStream bufferedOutput = new BufferedOutputStream(
new FileOutputStream("encrypted file"));
byte[] buffer = new byte[1024];
try {
while (bufferedInput.read(buffer) != -1) {
byte[] encryptedBuffer = cipher.doFinal(compress(buffer));
bufferedOutput.write(encryptedBuffer);
bufferedOutput.flush();
}
} catch (Exception e) {
//snip
} finally {
bufferedInput.close();
bufferedOutput.close();
}
Compress method:
public static byte[] compress(byte[] data) throws IOException{
Deflater deflater = new Deflater();
deflater.setInput(data);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(data.length);
deflater.finish();
byte[] buffer = new byte[1024];
while(!deflater.finished()){
int count = deflater.deflate(buffer);
outputStream.write(buffer, 0, count);
}
outputStream.close();
return outputStream.toByteArray();
}
What can I do to be able to compress and encrypt a file 1KB at a time and get the file back in its entirety when I perform the reverse operations?

Decrypting of AES/CBC/PKCS5Padding Error: Given final block not properly padded

I'm getting Given final block not properly padded error while decrypting AES/CBC/PKCS5Padding cipher on large encrypted file.
I think this issue is caused by adding wrong initialization vector in cipher.init() method.
I can't read whole file at runtime, so i need to encrypt fixed-size blocks. At this point I'm creating IV and storing it to .txt file. But in decrypting method I'm using the same IV every decryption cycle. How should I change this?
Encryption:
void encrypt() throws Exception{
char[] password = passwordText.getText().toCharArray();
byte[] salt = new byte[8];
/* Creating and saving salt */
salt = saveSalt(salt);
/* Securing password */
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
KeySpec spec = new PBEKeySpec(password, salt, 65536, 128);
SecretKey tmp = factory.generateSecret(spec);
SecretKey secret = new SecretKeySpec(tmp.getEncoded(), "AES");
if (choosedFile != null) {
/* Choosing algorithm for decryption */
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
/* Getting plain file */
CipherInputStream fis = new CipherInputStream(new FileInputStream(choosedFile), cipher);
CipherOutputStream fos = new CipherOutputStream(new FileOutputStream(choosedFile+".encrypted"), cipher);
/* Encrypting and Measuring */
long startTime = System.currentTimeMillis();
cipher.init(Cipher.ENCRYPT_MODE, secret);
byte[] rawText = new byte[128];
int count;
while((count = fis.read(rawText)) > 0) {
System.out.println(count);
byte[] encryptedText = cipher.doFinal(rawText);
fos.write(encryptedText, 0, count);
}
long stopTime = System.currentTimeMillis();
fis.close();
fos.close();
/* Creating initialization vector and storing*/
byte[] iVector = cipher.getIV();
saveIVector(iVector);
text.setText(text.getText() + "File was encrypted in " + (stopTime - startTime) + "ms.\n");
}
}
Decryption:
void decrypt() throws Exception {
/* Getting salt */
byte[] salt = getSalt();
/* Getting initialization vector */
byte[] iVector = getIVector();
/* Getting user password */
char[] password = passwordText.getText().toCharArray();
/* Securing password */
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
KeySpec spec = new PBEKeySpec(password, salt, 65536, 128);
SecretKey tmp = factory.generateSecret(spec);
SecretKey secret = new SecretKeySpec(tmp.getEncoded(), "AES");
if (choosedFile != null) {
/* Choosing algorithm for decryption */
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
/* Getting ciphered file */
CipherInputStream fis = new CipherInputStream(new FileInputStream(choosedFile), cipher);
CipherOutputStream fos = new CipherOutputStream(new FileOutputStream(choosedFile+".decrypted"), cipher);
/* Decrypting and Measuring */
long startTime = System.currentTimeMillis();
cipher.init(Cipher.DECRYPT_MODE, secret, new IvParameterSpec(iVector));
byte[] rawText = new byte[128];
int count;
while((count = fis.read(rawText)) > 0) {
byte[] encryptedText = cipher.doFinal(rawText);
fos.write(encryptedText, 0, count);
}
long stopTime = System.currentTimeMillis();
fis.close();
fos.close();

When using CipherInputStream and CipherOutputStream, the streams handle all the calls to the cipher (that's why you pass the cipher to it on initialization). You just need to initialize it correctly, and stream the data through the stream, and the cipher stream will do the needed calls to update() and doFinal(). Remember to close the steams to trigger the doFinal().
Currently your code passes the data through the cipher several times in an uncontrolled way, and the data is messed up.
Also, you only need a CipherInputStream for decrypt, and a CipherOutputStream for encrypt. In your current code you use both for both encrypt and decrypt.
Encrypt could be something like this (this don't handle the iv ..):
...
cipher.init(Cipher.ENCRYPT_MODE, secret);
InputStream is = new FileInputStream(choosedFile);
OutputStream os = new CipherOutputStream(new FileOutputStream(choosedFile+".encrypted"), cipher);
byte[] buffer = new byte[1024];
int len;
while ((len = is.read(buffer)) != -1) {
os.write(buffer, 0, len);
}
is.close();
os.close();
...

Worried about "until I had to cut large file into fixed-size blocks".
Using "chunk" in place of "block" above because "block"has a specific meaning in block ciphers such as AES.
What are toy doing with the chunks, concatenating them?
With CBC mode, after the first block the the previous encrypted block value is effectively used as the IV for the next block. So when splitting and then concatenating the chunks the value of the last block of the previous chunk is the IV for the next chunk.
See CBC mode.
Or are you doing something completely different?

incomplete RSA decryption with large XML file type - JAVA

I have a problem when decrypting XML type my file my returns incomplete data algorithm and rare symbols.
public File decryptFile(File fileInput, X509Certificate certificate) throws BadPaddingException, Exception {
try (DataInputStream dis = new DataInputStream(new FileInputStream(fileInput))) {
byte[] encryptedKeyBytes = new byte[dis.readInt()];
dis.readFully(encryptedKeyBytes);
PublicKey publicKey = certificate.getPublicKey();
rsaCipher.init(Cipher.DECRYPT_MODE, publicKey);
byte[] rijndaelKeyBytes = rsaCipher.doFinal(encryptedKeyBytes);
SecretKey rijndaelKey = new SecretKeySpec(rijndaelKeyBytes, "Rijndael");
byte[] iv = new byte[16];
dis.read(iv);
IvParameterSpec spec = new IvParameterSpec(iv);
Cipher cipher = Cipher.getInstance("Rijndael/CBC/PKCS5Padding");
cipher.init(Cipher.DECRYPT_MODE, rijndaelKey, spec);
try (CipherInputStream cis = new CipherInputStream(dis, cipher)) {
try (FileOutputStream fos = new FileOutputStream(fileInput.getAbsolutePath() + ".xml")) {
byte[] data = new byte[16];
int theByte;
while ((theByte = cis.read(data)) != -1) {
System.out.print(new String(data));
fos.write(data, 0, theByte);
}
System.out.println("\n\n");
}
}
}
return new File(fileInput.getAbsolutePath() + ".xml");
}
this code returns me the data
</ctaAbonBenef><distPago>00000</distPago><item>00000</item><pagoPoder>N</p�|���[�[W�Z�5��Q�
I think this has to do with UTF-8, but I can not solve.
Now I can also believe that it is the encryption algorithm to use, I leave just in case.
public static void generateFileEncrypt(File fileInput, PrivateKey privateKey, String folderSave) throws Exception {
String fileOutput = folderSave + "\" + fileInput.getName() + ENCRYPTED_FILENAME_SUFFIX;
DataOutputStream output = new DataOutputStream(new FileOutputStream(fileOutput));
Cipher rsaCipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
rsaCipher.init(Cipher.ENCRYPT_MODE, privateKey);
KeyGenerator rijndaelKeyGenerator = KeyGenerator.getInstance("Rijndael");
rijndaelKeyGenerator.init(128);
Key rijndaelKey = rijndaelKeyGenerator.generateKey();
byte[] encodedKeyBytes = rsaCipher.doFinal(rijndaelKey.getEncoded());
output.writeInt(encodedKeyBytes.length);
output.write(encodedKeyBytes);
SecureRandom random = new SecureRandom();
byte[] iv = new byte[16];
random.nextBytes(iv);
output.write(iv);
IvParameterSpec spec = new IvParameterSpec(iv);
Cipher symmetricCipher = Cipher.getInstance("Rijndael/CBC/PKCS5Padding");
symmetricCipher.init(Cipher.ENCRYPT_MODE, rijndaelKey, spec);
try (
CipherOutputStream cos = new CipherOutputStream(output, symmetricCipher);
FileInputStream fis = new FileInputStream(fileInput)) {
int theByte;
byte[] data = new byte[16];
while ((theByte = fis.read(data)) != -1) {
System.out.print(new String(data));
cos.write(data, 0, theByte);
}
System.out.println("\n\n");
cos.flush();
}
}
Thanks in advance.

I haven't digested all your code; I stopped when I saw you trying to decrypt with the public key, and encrypting with the private key. That's sort of like a digital signature, but your padding will be all wrong and you should use the Signature class if that is what your really want to do.
The public key is used to encrypt, or to verify a digital signature. Use the private key to decrypt, and see if that resolves your problem.
You are still doing it wrong. Don't call it "encryption" if the key isn't private.
But anyway, I think the printing to stdout looks wrong because you are converting the entire buffer to text. The last block is likely to be padded, so it won't decode to valid text—it's padding; it wasn't part of the input file, and you aren't writing it to the decrypted file, but you are printing it.
Change to encrypt with the public key, decrypt with the private key, and then change your printing to this:
System.out.print(new String(data, 0, theByte));
Even better would be to specify the character set of the data (probably UTF-8, since it's the default for XML).

I think u should do the opposite. encrypt with the public key and decrypt with the private key..

Android File Decryption and Encryption taking time

I have done Encryption and Decryption in android when file downloading
but I want to improve time performance when file decrypted.
My problem is when I am downloading any file so I have add encryption over there but at this stage I am showing Progress loader so it looks good but but when file completely download and try to open that file then it is decrypted that file this time it's taking too much time .
which is look very bad. How can I reduce decryption time? Here is my code
Encryption Code
byte data[] = new byte[1024];
String seed = "password";
byte[] rawKey = getRawKey(seed.getBytes());
SecretKeySpec skeySpec = new SecretKeySpec(rawKey, "AES");
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, skeySpec);
output = new CipherOutputStream(output, cipher);
long total = 0;
while ((count = input.read(data)) != -1) {
total += count;
publishProgress("" + (int) ((total * 100) / lenghtOfFile));
output.write(data, 0, count);
}
Decryption Code Here:
String newPath = sdCardPath + "/" + dPdfName;
File f1 = new File(newPath);
if (!f1.exists())
try {
f1.createNewFile();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
try {
InputStream fis = new FileInputStream(f);
OutputStream fos = new FileOutputStream(f1);
String seed = "password";
byte[] rawKey = getRawKey(seed.getBytes());
SecretKeySpec skeySpec = new SecretKeySpec(rawKey,
"AES");
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.DECRYPT_MODE, skeySpec);
fis = new CipherInputStream(fis, cipher);
int b;
byte[] data = new byte[4096];
while ((b = fis.read(data)) != -1) {
// fos.write(cipher.doFinal(data), 0, b);
fos.write(data, 0, b);
}
fos.flush();
fos.close();
fis.close();
} catch (Exception e) {
// TODO: handle exceptionpri
e.printStackTrace();
}
Get Row Key Method:
private static byte[] getRawKey(byte[] seed) throws Exception {
KeyGenerator kgen = KeyGenerator.getInstance("AES");
SecureRandom sr = SecureRandom.getInstance("SHA1PRNG");
sr.setSeed(seed);
kgen.init(128, sr);
SecretKey skey = kgen.generateKey();
byte[] raw = skey.getEncoded();
return raw;
}

Suggestions:
1) move the decryption to another asynctask and add another progress indicator.
2) reserve last, say, 10% of the progress indicator to decryption. This is what I actually did once, but I was doing an integrity check (against an MD5 hash IIRC), not decryption.
3) move the decryption to the downloading asynctask, decrypt each received portion of data immediately and so hide the decryption time behind the download time.
4) not sure this will be any faster, but you may have two service threads: one downloading file and another decrypting it. It's better no to use AsyncTask here, because they may behave differently on different versions of Android (including sequential execution on a single thread, see Is AsyncTask really conceptually flawed or am I just missing something? for the discussion, and my note https://stackoverflow.com/a/14602486/755804 )
Note also that the thread responsible for downloading and decryption belongs to Model (in MVC sense) and must not be owned by an Activity which is a Controller that cannot outlive a screen turn: https://stackoverflow.com/a/14603375/755804
If your download takes a long time, you may be interested in resuming interrupted downloads, and it's better to think about it from the very beginning. It's always easier to modify simple solutions, and multi-threaded solutions are rather complex. If you transfer a number of files, it may happen that one of them gets broken during transfer, and you may want an option to retransfer only the file(s) that are broken. You may also want an integrity check.

Make sure you align the data buffer to the block size of the encryption.
For a Example see: https://stackoverflow.com/a/33171612/475496
Using this method has speedup our encryption enormous.