Develop Reference

SHA1 sum on multilanguage strings

I have an android app that comminicates with a nodejs server via http.
To each request I add a SHA1 checksum that iterates over the key value pairs I'm sending to the server so that it would validate it on arrival.
I have tested this and it works okay as long as the values I'm sending are in English. Sending a request that contains a string value in Hebrew for example or that includes an unstandard character (like é, ç or à) will fail the request base on a checksum mismatch.
The checksum is calculated first in the andoird client (JAVA) and than in the nodejs server for verification.
I eliminated the possibility that this is caused due to the Right To Left nature of the Hebrew language by sending a single letter value, and it still failed.
What could cause this? I assume this might be because the encoding of a JAVA string for non standard characters is different from that of nodejs. But how can I solve this?
Here is the relevant code from the server side:
var shasum = crypto.createHash('sha1');
while (fieldsPointer < allFields.length
|| filesPointer < allFiles.length) {
shasum.update(fieldKey);
let fieldKey = allFields[fieldsPointer];
shasum.update(fieldKey);
let fieldValue = fields[allFields[fieldsPointer]];
shasum.update(fieldValue);
}
and the client side:
try {
MessageDigest md = MessageDigest.getInstance("SHA-1");
for(Entry<String, Object> entry : allParams.entrySet()) {
md.update(entry.getKey().getBytes());
if(entry.getValue() instanceof String) {
String value = (String) entry.getValue();
md.update(value.getBytes());
} else (...) // some other instance checking for non String values
}
String checksum = bytesToHex(md.digest());
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}

I did some research based on Williams's comment.
It turns out all I need to do was to change this:
shasum.update(fieldValue);
to this:
shasum.update(fieldValue, 'utf8');
as stated in the nodejs crypto documentation for hash encoding: http://nodejs.org/api/crypto.html#crypto_hash_update_data_input_encoding
While this works, it is also a good idea to state the charset in the client side, as William stated in the comment below.
I did this by changing this:
md.update(value.getBytes());
to this
md.update(value.getBytes("utf8"));

InvalidKeyException: Key Spec Not Recognised

I'm getting a
InvalidKeyException: Illegal key size or default parameters
when trying to run a web app that is was a deployed WAR. I am hosting it on Tomcat in a Linux environment. I have already put the two UnlimitedJCEPolicy files into the destination /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.55.x86_64/jre/lib/security, and it seems that the error is still occuring. NOTE that this only is thrown when I am running in the linux environment. Locally, it works fine. Here is my code:
public static final void decryptFile(File inputFile, File outputFile) throws
IOException, PGPException {
// Add Bouncy Castle provider
Security.addProvider(new BouncyCastleProvider());
// Grab secret key that's in folder with AE classes
Resource resource = new ClassPathResource(Env.getSecretKeyAE());
log.debug("Resource: " + Env.getSecretKeyAE());
File keyFileName = resource.getFile();
log.debug("Key File Name: " + keyFileName);
// Decryption password
String pass = "pass";
char[] passwd = pass.toCharArray();
// Read files into streams
log.info("Reading files into streams");
InputStream keyIn = new BufferedInputStream(new FileInputStream(keyFileName));
InputStream in = PGPUtil.getDecoderStream(new BufferedInputStream(new
FileInputStream(inputFile)));
// I don't even know what these do
PGPObjectFactory pgpObjFactory = new PGPObjectFactory(in);
PGPEncryptedDataList pgpEncryptedDataList = null;
Object o = pgpObjFactory.nextObject();
log.info("Checking instance of PGPEncryptedDataList");
if (o instanceof PGPEncryptedDataList) {
pgpEncryptedDataList = (PGPEncryptedDataList)o;
}
else {
pgpEncryptedDataList = (PGPEncryptedDataList)pgpObjFactory.nextObject();
}
// This will be the PGPPrivateKey we use to decrypt
log.info("Initializing secret key");
PGPPrivateKey secretKey = null;
PGPPublicKeyEncryptedData publicKeyEncryptedData = null;
PGPSecretKeyRingCollection pgpSecretKeyRingCollection = new
PGPSecretKeyRingCollection(PGPUtil.getDecoderStream(keyIn));
// This iterates the key file as if it has many keys, this file has only one
// This is the only way I could find to construct a PGPPrivateKey
log.info("Iterating through key file");
Iterator<?> it = pgpEncryptedDataList.getEncryptedDataObjects();
while(it.hasNext() && secretKey == null) {
publicKeyEncryptedData = (PGPPublicKeyEncryptedData) it.next();
PGPSecretKey pgpSecKey =
pgpSecretKeyRingCollection.getSecretKey(publicKeyEncryptedData.getKeyID());
if (pgpSecKey != null) {
Provider provider = Security.getProvider("BC");
secretKey = pgpSecKey.extractPrivateKey(new
JcePBESecretKeyDecryptorBuilder(new
JcaPGPDigestCalculatorProviderBuilder().setProvider(provider)
.build()).setProvider(provider).build(passwd));
}
}
log.info("PGPPrivateKey has been constructed");
if (secretKey == null) {
throw new IllegalArgumentException("secret key for message not found.");
}
log.info("Secret Key found!");
if(publicKeyEncryptedData == null) {
throw new NullPointerException("cannot continue with null public key encryption
data.");
}
log.info("Public Key Encrypted Data found!");
// More stuff I don't fully understand, I think this is just standard way to
decrypt files once the above is all set up
log.info("Starting actual decryption");
//get data stream where our publicKeyDataDecrypterFactory sets ours provider to BC
and we build our secretKey
//secretkey is our PGPPrivateKey
log.info("start");
//=====================================================================
//ERROR IS OCCURRING HERE
InputStream clear = publicKeyEncryptedData.getDataStream(new
JcePublicKeyDataDecryptorFactoryBuilder().setProvider("BC").build(secretKey));
log.info("1");
PGPObjectFactory plainFact = new PGPObjectFactory(clear);
log.info("2");
PGPCompressedData compressedData = (PGPCompressedData)plainFact.nextObject();
log.info("3");
InputStream compressedStream = new
BufferedInputStream(compressedData.getDataStream());
log.info("4");
PGPObjectFactory pgpFact = new PGPObjectFactory(compressedStream);
log.info("5");
Object message = pgpFact.nextObject();
log.info("6");
if (message instanceof PGPLiteralData) {
log.info("Our message is an instance of PGP Literal Data.");
PGPLiteralData literalData = (PGPLiteralData)message;
InputStream literalDataInputStream = literalData.getInputStream();
OutputStream out = new BufferedOutputStream(new FileOutputStream(outputFile));
Streams.pipeAll(literalDataInputStream, out);
out.close();
}
else if (message instanceof PGPOnePassSignatureList) {
log.error("encrypted message contains a signed message - not literal data.");
throw new PGPException("encrypted message contains a signed message - not
literal data.");
}
else {
log.error("message is not a simple encrypted file - type unknown.");
throw new PGPException("message is not a simple encrypted file - type
unknown.");
}
log.info("Checking if public key encrypted data is integrity protected");
if (publicKeyEncryptedData.isIntegrityProtected()) {
if (!publicKeyEncryptedData.verify()) {
throw new PGPException("message failed integrity check");
}
}
keyIn.close();
in.close();
}
Using logs, I was able to find that the error was occurring when
InputStream clear = publicKeyEncryptedData.getDataStream(new
JcePublicKeyDataDecryptorFactoryBuilder().setProvider("BC").build(secretKey));
But I have no clue why. Like I said, I already placed the JCEUnlimited files appropriately and the error still occurs.
EDIT I fixed the illegal key size problem, but am now getting "key spec not recognized"
EDIT More elaboration on the error "key spec not recognized":
So as I said, Illegal key size is gone, but "key spec not recognized" seems to be a problem still. The weird thing is that my encryptFile method works perfectly, but decryptFile is throwing the error. I'm not entirely sure why. Before I left work, I tested one more time and it seems that the error wasn't thrown. I almost seems like this error occurs randomly depending on the deployment of the WAR to tomcat. If I deploy my WAR, the error wont occur at some points, but if I undeploy and redeploy with an updated WAR file, the error is thrown. I have no clue what is causing this, and the based off research no one really knows either. Apparently this used to be a bug in Bouncy Castle before 1.5, but 1.5 is the version I'm running so that's not the problem. I will post if I find anything that can possibly fix this error.

To solve:
java.security.spec.InvalidKeySpecException: key spec not recognised
Modify security providers:
sudo nano $JAVA_HOME/jre/lib/security/java.security
Add security provider:
security.provider.10=org.bouncycastle.jce.provider.BouncyCastleProvider
Copy bcprov-jdk15on-1.54.jar to:
$JAVA_HOME/jre/lib/ext/bcprov-jdk15on-1.54.jar
Restart Tomcat.

if (o instanceof PGPEncryptedDataList) {
pgpEncryptedDataList = (PGPEncryptedDataList)o;
if o is already an instance of PGPEncryptedDataList, why are you casting it to PGPEncryptedDataLIST?
I don't know enough about the specifics of what you're doing so I just figured I'd provide some general code analysis. Sorry I couldn't be of more help.

To prevent the error "Illegal key size or default parameters", I simply had to put the UnlimitedJCEPolicy files in my working java directory, /opt/jre1.7.0_60/lib/security. After putting the files there, and redeploying my war file, I was not experiencing this problem anymore.
To prevent the "key spec not recognized" error, I had to restart my tomcat server when redeploying my WAR file.

Java checksum md5 on a file situated on a FTP server

I got a program which needs to compare a file that situated on the Local disk and one on a FTP server.
I've decided to go with md5 checksum. I am able to do it with the local file, but I am having problems with the ftp one. Also, I am using Apache FTPClient common.
MessageDigest digest = MessageDigest.getInstance("MD5");
FileInputStream is = new FileInputStream(FTP_listFiles[i]); //ERROR HERE
//FTP_files is a FTPFile from FTPClient apache commons.
byte[] buffer = new byte[8192];
int read = 0;
try {
while( (read = is.read(buffer)) > 0) {
digest.update(buffer, 0, read);
}
byte[] md5sum = digest.digest();
BigInteger bigInt = new BigInteger(1, md5sum);
String output = bigInt.toString(16);
System.out.println("MD5: \n" + output);
}
catch(IOException e) {
throw new RuntimeException("Unable to process file for MD5", e);
} finally {
try {
is.close();
}
catch(IOException e) {
throw new RuntimeException("Unable to close input stream for MD5 calculation", e);
}
}
NB: If impossible, do you know any equivilant to md5 but can do the same?

What are trying to achieve? Are.you aware that to compute the md5 locally, you will need to download the full file ?
There is a special ftp extension that provide a server-side md5 to the client but it'd not generally supported. Some server also implement some similar proprietary functionality; you''ll have to check for your specific server.
You may want to have a look at the XCRC or XMD5 or XSHA1 commands, for example.

CRC32 (Ethernet) Calculation vs .ZIP/.PNG in Java?

I can only calculate the CRC32 values of .ZIP/.PNG Strings, but not Ethernet related ones. The Java CRC32 class only seems to allow for one type of calculation.
String str = textField.getText();
Checksum checksum = new CRC32();
byte bytes[] = null;
try {
bytes = str.getBytes("ASCII");
} catch (UnsupportedEncodingException ex) {
Logger.getLogger(GUI.class.getName()).log(Level.SEVERE, null, ex);
}
checksum.update(bytes, 0, bytes.length);
long lngChecksum = checksum.getValue();
crc32bField.setText(Long.toHexString(lngChecksum));
This is the code I've written to calculate my CRC, could anyone help me achieve the same values as one calculated on this website?
http://hash.online-convert.com/crc32-generator
Just as an example,
"hello world" =
7813f744 (website)
D4A1185 (My Code)
Thanks :)

It seems that the algorithm used in the website you provided is CRC32 whereas the one you are using is CRC32B. Both of them are completely different algorithms that is why you are getting different values.
You try the CRC32B algorithm from the same website. It is inline with what the Checksum class is giving you.

org.jasypt.exceptions.EncryptionOperationNotPossibleException in Tomcat

I'm using the Jasypt encryption library to encrypt/decrypt some text. This code is embedded in a WAR file and deployed to a server.
When running locally, and in unit tests, the encrypt/decrypt cycle works perfectly. I use Jetty to develop the application. The code works perfectly in that server. For some reason, deploying to Tomcat breaks it with the following exception:
FYI, I have the strong encryption libraries installed in both my local and server environments and I'm using the latest 1.6 version (patch level 25).
org.jasypt.exceptions.EncryptionOperationNotPossibleException
The exception has no message.
The code is fully symmetric. I pasted it here for examination. Here are the relevant bits:
I found one old Nabble post where a user had a very similar problem. Code worked everywhere except inside Tomcat. No solution was given.
Any insights would be most appreciated.
**Update: ** Running in Tomcat on my local system, it appears to work. So there's something about my server. On the server, I'm using a 64-bit JVM on Windows Server 2008. I'm using a 32-bit JVM locally (due to my system being a bit older). I wonder if this has something to do with the issue.
public void initializeService() {
binaryEncryptor = new BasicBinaryEncryptor();
binaryEncryptor.setPassword(keyBase64);
}
#Override
public <T extends Serializable> String simpleEncrypt(T objectToEncrypt) throws EncryptionException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
try {
ObjectOutputStream oos = new ObjectOutputStream(bos);
oos.writeObject(objectToEncrypt);
byte[] bytes = binaryEncryptor.encrypt(bos.toByteArray());
return new String(Base64.encodeBase64(bytes));
} catch (IOException e) {
LOGGER.error("failed to encrypt String: " + e.getMessage());
throw new EncryptionException(e.getMessage(), e);
} catch (Exception e) {
LOGGER.error("failed to encrypt String: " + e.getMessage());
throw new EncryptionException(e.getMessage(), e);
}
};
#SuppressWarnings("unchecked")
#Override
public <T> T simpleDecrypt(String objectToDecrypt) throws EncryptionException {
try {
byte[] bytes = Base64.decodeBase64(objectToDecrypt);
byte[] decryptedBytes = binaryEncryptor.decrypt(bytes);
ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(decryptedBytes));
T object = (T)ois.readObject();
return object;
} catch (IOException e) {
LOGGER.error("failed to decrypt String: '" + objectToDecrypt + "', mesage = " + e.getMessage());
throw new EncryptionException(e.getMessage(), e);
} catch (Exception e) {
LOGGER.error("failed to decrypt String: '" + objectToDecrypt + "', mesage = " + e.getMessage());
throw new EncryptionException(e.getMessage(), e);
}
}

Here is a link to the docs: http://www.jasypt.org/faq.html#i-keep-on-receiving-encryption-operation-not-possible
Is encryption and decryption config identical
Check to make sure table columns are large enough
Base64 encoding and urlencoding can conflict, so it has to be done just right.

#biniam_Ethiopia
I would have commented your answer but I have not enough reputation, so I write my own answer:
I had a very similiar problem, but in my case it was because of changing the encryption algorithm (PBEWithMD5AndTripleDES), while entries in the db were saved with a different one before (PBEWithMD5AndDES).
So I got a EncryptionOperationNotPossibleException too, which is without information because of #Nathan Feger's comment above.
I hope this could help somebody someday too ;)

I faced similar problem.
For me, it was because it was trying to decrypt a password which could not have been decrypted using the decrypting mechanism.
Hence, I encrypted the password and stored it in database before the decrypt method tries to decrypt it.
I hope it helps someone.