I have an access-controlled PHP system that HTTP redirects clients to a Java-based reporting system with a unique security model (it's awful). To get around the report security model I'm using a Tomcat filter to validate all requests before they reach the reporting system. I'm using an encrypted token passed from PHP to Java which tells the reporting system who the client is. The filter checks the requested report name against a restricted list and returns a 403 if the client's role is insufficient.
The encrypted token stores a timestamp and the user's role, e.g.
1365549482|SysAdmin
When encrypted it looks something like this
vSEFgBYd30Ik5p4PZlG968cvdg==
The PHP system acts as a proxy for all reporting requests. When a user requests a report the request goes to PHP, which generates an encrypted token, URL encodes it, then appends it to the report URL and makes the GET request to the reporting system. My Java filter decrypts the token, pulls it apart, and figures out what to do.
9 times out of 10 this is fine, but occasionally the token cannot be properly decrypted. The above (unencrypted) example is converted to something like this
1365549482q??YZ7
And everything goes wrong.
I'm a bit out of my depth with encryption, decryption, and the particulars of character encoding, but unfortunately I'm the only developer available to work on this. Any thoughts at all on what might be going wrong here would be hugely appreciated. I don't expect any big code changes as it works most of the time, but there is clearly a time-sensitive component in the mix that I don't understand. Code snippets below
EDIT
I've spent a while debugging this now and it just got stranger. I wrote a small Java program to request a token from PHP via HTTP GET. The PHP script returns the same (URL-encoded) value that is passed to Java via a URL parameter in the normal workflow. The Java program decodes and decrypts this in the same way as the code snippet below and checks the result. Over thousands of iterations (so far, and counting) it is working as expected. However, while this test is going on I can see the same failures happening in the filter's log file.
Whatever's causing this intermittent problem is seemingly related the Java class being a Tomcat filter or the data being passed by URL via Tomcat. Does this give anyone a hint as to what could be going on here? I'm so very confused right now.
PHP
$presentAsSeconds = time();
$message = strval($presentAsSeconds + Configure::read('Reporting.Authentication.ExpireInSeconds')) . '|' . $userDetails['role'];
return base64_encode(
mcrypt_encrypt(
MCRYPT_RIJNDAEL_128,
md5(Configure::read('Reporting.Authentication.Key')), // matches "the key" in Java function
$message,
MCRYPT_MODE_CFB,
Configure::read('Reporting.Authentication.IVector') // matches "the vector" in Java function
)
);
Java
private String decrypt(String initial) throws Exception {
SecretKeySpec skeySpec = new SecretKeySpec(md5("the key").getBytes("UTF-8"), "AES");
IvParameterSpec initialVector = new IvParameterSpec("the vector".getBytes("UTF-8"));
Cipher cipher = Cipher.getInstance("AES/CFB8/NoPadding");
cipher.init(Cipher.DECRYPT_MODE, skeySpec, initialVector);
byte[] encryptedByteArray = (new org.apache.commons.codec.binary.Base64()).decode(initial.getBytes("UTF-8"));
byte[] decryptedByteArray = cipher.doFinal(encryptedByteArray);
return (new String(decryptedByteArray, "UTF8"));
}
private String md5(String input) throws NoSuchAlgorithmException {
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] messageDigest = md.digest(input.getBytes("UTF-8"));
BigInteger number = new BigInteger(1, messageDigest);
return number.toString(16);
}
The problem may be with your getBytes() methods - these use the default system character encoding, which is NOT the same across every JVM. Use getBytes("UTF-8") instead.
I think the issue is in how you've used the mcrypt library in PHP and then you're base64 encoding the encrypted data? We've definitily had some issues doing something similar and dropped the base64 encode and it worked after that.
Can you use another method to pass the token instead of the url? Such as a cookie or auth header?
Here's a snippet of my encryption/decryption in PHP (I'm not great with Java) because I think your mcrypt isn't right.
ENCODING:
$userObjectJson = json_encode($this);
//encrypt the user session object
$mcrypt = mcrypt_module_open(MCRYPT_3DES, '', MCRYPT_MODE_CFB, '');
$_SESSION['iv'] = mcrypt_create_iv(mcrypt_enc_get_iv_size($mcrypt), MCRYPT_RAND);
$keySize = mcrypt_enc_get_key_size($mcrypt);
$key = substr(MYAWESOME_KEY, 0, $keySize);
mcrypt_generic_init($mcrypt, $key, $_SESSION['iv']);
$_SESSION['user'] = mcrypt_generic($mcrypt, $userObjectJson);
mcrypt_generic_deinit($mcrypt);
mcrypt_module_close($mcrypt);
DECODING:
//decrypt the user session object
$mcrypt = mcrypt_module_open(MCRYPT_3DES, '', MCRYPT_MODE_CFB, '');
$keySize = mcrypt_enc_get_key_size($mcrypt);
$key = substr(MYAWESOME_KEY, 0, $keySize);
mcrypt_generic_init($mcrypt, $key, $_SESSION['iv']);
$userObjectJson = mdecrypt_generic($mcrypt, $_SESSION['user']);
mcrypt_generic_deinit($mcrypt);
mcrypt_module_close($mcrypt);
Related
I started using SellingPartner (SP) recently and I am kind confused how they provide us S3 reports to download.
When I fetch a Report Document from SP API I get this return (omitted):
GetReportDocumentResponse class:
{
"payload": {
"reportDocumentId": "amzn1.tortuga.3.OMITTED.OMITTED",
"url": "https://tortuga-prod-na.s3-external-1.amazonaws.com/%2FOMITED/amzn1.tortuga.3.OMITTED.OMITTED?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201025T163212Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=OMITED%2F20201025%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=OMITED",
"encryptionDetails": {
"standard": "AES",
"initializationVector": "OMITED==",
"key": "+OMITED="
},
"compressionAlgorithm": null
},
"errors": null
}
If I copy/paste the payload.url directly in my browser, it downloads an encrypted document, which looks fine (I couldn't decrypt it though, snippet in the end).
I am trying to download using the AWS S3 Java SDK and I keep getting software.amazon.awssdk.services.s3.model.S3Exception: Access Denied
I have this snippet:
public String getReportFile(String reportDocumentId) throws IOException {
GetReportDocumentResponse response = getReport(reportDocumentId);
ReportDocumentEncryptionDetails encryptionDetails = response.getPayload().getEncryptionDetails();
GetObjectRequest request =
GetObjectRequest.builder()
.key(reportDocumentId)
.bucket("tortuga-prod-na") //hardcoding here, thats the bucket on the URL, right?
.sseCustomerAlgorithm(encryptionDetails.getStandard())
.sseCustomerKey(encryptionDetails.getKey())
// .sseCustomerKeyMD5() should I apply it? Is that the Initialization Vector field?
.build();
//I tried both without Credentials, and using accessKey and secretKey from my personal account, not sure if should be another one related to the URL, what should I use for credentials if the URL works fine in my browser?
StaticCredentialsProvider credentialsProvider =
StaticCredentialsProvider.create(AwsBasicCredentials.create(accessKey, secretKey));
BufferedReader br =
new BufferedReader(
new InputStreamReader(
S3Client.builder()
.credentialsProvider(credentialsProvider)
.region(Region.US_EAST_1)
.build()
.getObject(request)));
My end goal is to download this file in chunks (as it may have over 500mb) and process a few hundred lines at a time. Would that be possible if it's encrypted? I would like to download it already decrypted and be able to process it in chunks.
I wonder how to make the same request using S3Client like the URL coming from the JSON. Do we have a way to just paste a URL on S3Client, include the encryption settings and make a call?
About the downloaded file from the browser, I tried to decrypt it doing this:
byte[] bytes = FileUtils.readFileToByteArray(new File("encrypted_file"));
Cipher cipher = Cipher.getInstance("AES/ECB/PKCS5PADDING");
SecretKeySpec secretKey = new SecretKeySpec(Base64.getDecoder().decode(<payload.encryptionDetails.key String value>), "AES");
cipher.init(Cipher.DECRYPT_MODE, secretKey);
System.out.println(new String(cipher.doFinal(bytes)));
which throws exception:
Exception in thread "main" javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
Thanks in advance.
If I copy/paste the payload.url directly in my browser, it downloads an encrypted document, which looks fine (I couldn't decrypt it though, snippet in the end).
It means the object is not using SSE-C, otherwise you wouldn't be able to download it. Seems the content is encrypted in the client side (outside s3 api) and the encrypted content is uploaded as a normal object. So please check in your code, where the content is uploaded, for encryption parameters .
If you are able to directly download the object from the browser, then just download the content as a normal object.
Note: proper use of the AWS S3 Client Side Encryption
About the downloaded file from the browser, I tried to decrypt it doing this
wherever you got this code, please do not use it. Just using the AES/ECB/PKCS5PADDING mode is not safe.
Using the IV (initializationVector) implies using different encryption mode of operation. You have to find out which is it is from the code or service which encrypts the content.
private static final String SYMMETRIC_KEY_ALG = "AES";
// find out the correct value, could be AES/CBC/PKCS5Padding
private static final String SYMMETRIC_CIPHER_NAME = "???";
IvParameterSpec ivParamSpec = new IvParameterSpec(encryptionParams.getIv());
SecretKey symmetricKey = new SecretKeySpec(encryptionParams.getKey(), SYMMETRIC_KEY_ALG);
Cipher cipher = Cipher.getInstance(SYMMETRIC_CIPHER_NAME);
cipher.init(Cipher.DECRYPT_MODE, symmetricKey, ivParamSpec);
byte[] decrypted = cipher.doFinal(encryptionParams.getCiphertext());
However - you really need to find out how the content is encrypted. It could be possible the aes-gcm mode is used and then part of the ciphertext can be an authentication hash. So here you should not make assumptions and find the real data.
I have a jhipster spring boot application which accepts a token generated by a third party which has been encrypted with our public key. I have a JWTFilter which decrypts the token using our private key and creates an authentication object which is stored in the security context. Once execution reaches the controller, I intend to pull the username and password from the security context so I can make API calls back to the third party application.
This is working to some degree in our integration environment where the third-party has a link to a running instance of our application. To test locally, I am hitting the link in the integration environment, and copying the token. Then I make a request from Postman to an instance of our application I'm running locally with the token added in the headers, the same as our js client would.
I am using "com.nimbusds:nimbus-jose-jwt:4.23" for decryption, and I am getting a 'MAC check failed' error. I can change the value of macCheckPassed to true in the debugger, and the decryption will complete, allowing me to see the claims and load them into the security context. However, some other filter is catching my hack, and the request gets rejected with an authorization error.
public static byte[] decryptAuthenticated(final SecretKey secretKey,
final byte[] iv,
final byte[] cipherText,
final byte[] aad,
final byte[] authTag,
final Provider ceProvider,
final Provider macProvider)
throws JOSEException {
// Extract MAC + AES/CBC keys from input secret key
CompositeKey compositeKey = new CompositeKey(secretKey);
// AAD length to 8 byte array
byte[] al = AAD.computeLength(aad);
// Check MAC
int hmacInputLength = aad.length + iv.length + cipherText.length + al.length;
byte[] hmacInput = ByteBuffer.allocate(hmacInputLength).
put(aad).
put(iv).
put(cipherText).
put(al).
array();
byte[] hmac = HMAC.compute(compositeKey.getMACKey(), hmacInput, macProvider);
byte[] expectedAuthTag = Arrays.copyOf(hmac, compositeKey.getTruncatedMACByteLength());
boolean macCheckPassed = true;
if (! ConstantTimeUtils.areEqual(expectedAuthTag, authTag)) {
// Thwart timing attacks by delaying exception until after decryption
macCheckPassed = false;
}
byte[] plainText = decrypt(compositeKey.getAESKey(), iv, cipherText, ceProvider);
if (! macCheckPassed) {
throw new JOSEException("MAC check failed");
}
return plainText;
}
What is this MAC check? I thought it had to do with the origin of the token. Something along the lines of the token being encrypted with the MAC id of the source system, which throws an error when it doesn't synch up with my current host.
What other filter would be rejecting the request if the decryption passed? Is there some other flag I'm supposed to be setting so the framework will honor the request?
JWE spec mandates authenticated encryption, to ensure the plain text is not just encrypted, but also protected against tampering. To ensure that an HMAC is applied after the content encryption.
The "Mac check failed" error can mean two things - the library that produced the original JWE / JWT has applied the HMAC incorrectly, or, the JWE / JWT was modified in transit.
I need to use AEAD to share information between two users, part of which must be encrypted and part of which should be kept in cleartext.
Is there an API to check the ciphertext tag and access the associated data once a message has been encrypted with AES/GCM ?
In more detail:
I'm using Java 7 with bouncycastle as a provider and I have managed to encrypt and decrypt my data successfully, using the corresponding API:
private byte[] encrypt(SecretKey key, byte[] nonce, byte[] message, byte[] associatedData) throws ... {
Cipher aeadCipher = Cipher.getInstance(AES_GCM_NOPADDING);
aeadCipher.init(Cipher.ENCRYPT_MODE, kint, new GCMParameterSpec(GCM_MAC_SIZE, nonce);
aeadCipher.updateAAD(associatedData);
return aeadCipher.doFinal(message);
}
private byte[] decrypt(SecretKey key, byte[] nonce, byte[] cipherText, byte[] associatedData) throws ... {
Cipher aeadCipher = Cipher.getInstance(AES_GCM_NOPADDING);
aeadCipher.init(Cipher.DECRYPT_MODE, kint, new GCMParameterSpec(GCM_MAC_SIZE, nonce);
aeadCipher.updateAAD(associatedData);
return aeadCipher.doFinal(cipherText);
}
However, it is my understanding that AES/GCM ciphertexts should already contain the parameters that could affect decryption (nonce and associatedData).
Therefore, I would like to be able to retrieve them from the ciphertext, rather than having to store them alongside the ciphertext and pass them along to the decryption function. Furthermore, I'd like to be able to run integrity checks (computing the tag) and run some checks on the associated data without having to completely decrypt the message.
Is there an API that would allow this and that I might have missed ?
So far, I've checked:
The Cipher API: http://docs.oracle.com/javase/7/docs/api/javax/crypto/Cipher.html
This very informative blog post on AES/GCM implementations in Java: http://blog.philippheckel.com/2014/03/01/cipherinputstream-for-aead-modes-is-broken-in-jdk7-gcm/
Since the Java API automatically places the tag at the end, you only have to extract this tag from the your encryption result as follow:
private byte[] getTag(SecretKey key, byte[] nonce, byte[] message, byte[] associatedData) throws ... {
Cipher aeadCipher = Cipher.getInstance(AES_GCM_NOPADDING);
aeadCipher.init(Cipher.ENCRYPT_MODE, kint, new GCMParameterSpec(GCM_MAC_SIZE, nonce);
aeadCipher.updateAAD(associatedData);
byte[] encrypted = aeadCipher.doFinal(message);
// Assuming you have an AAD_SIZE = 128 bits (16 bytes)
return Arrays.copyOfRange (encrypted, encrypted.length-16, encrypted.length)
}
There is no implicit format that stores all the input data of GCM in a specific location. The Java API is already a bit strange in
the sense that it automatically places the tag at the end. This makes the
algorithm more compatible with the Cipher class, but in principle the tag just
needs to be kept with the ciphertext - where does not matter. Now you have the
issue that you don't know where the AAD ends and the ciphertext starts for instance.
So what you can do is either to create your own format (maybe your AAD has a
static size, so you can just concatenate) or you can use a predefined container format.
There is an internet draft
that specifies how to use both modes in the Cryptographic Message Syntax (CMS).
The AAD can then be stored in authenticated atributes, which should also include the required
parameters (including the IV comprising of the nonce).
If you feel masochistic you could also try and use XML-encryption with GCM mode, but beware of the many pitfalls with regard to verifying XML authenticity (e.g. beware that you are actually verifying the data you are going to use).
Bouncy Castle seems to offer support for CMS using GCM.
I am writing a code for oauth with twitter, and i have 401 error code and "Failed to validate oauth signature and token" response from twitter when i fetch post request to https://api.twitter.com/oauth/request_token. This is my data which i have:
Consumer/api key - ffo9MLdRF8XOd9DKB0HeA
Consumer/api secret - fUJtvIpujTslQOlVbZY6QU8cNEMdwoxzTG1gh93SUgs
Url callback - https://oauth.vk.com/blank.html
My steps:
1.Prepared string for making signature
POST&https%3A%2F%2Fapi.twitter.com%2Foauth%2Frequest_token&oauth_callback%3Dhttps%3A%2F%2Foauth.vk.com%2Fblank.html%26oauth_consumer_key%3Dffo9MLdRF8XOd9DKB0HeA%26oauth_nonce%3Dfb0e9383f0c84326a124dd4ccfddd2d2%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1393951866%26oauth_version%3D1.0
2.Created signature qQwIvFao9yeIQpi9ouz0oFi7/v8= by code:
public String calculateSign(String stringToEncode, String secret) throws Exception{
byte[] keyBytes = secret.getBytes();
byte[] text = stringToEncode.getBytes();
SecretKey secretKey = new SecretKeySpec(keyBytes, "HmacSHA1");
Mac mac = Mac.getInstance("HmacSHA1");
mac.init(secretKey);
byte[] encoded = mac.doFinal(text);
return new String(Base64.encode(encoded, Base64.DEFAULT)).trim();
}
3.Final Authorization header (with escaped quotes):
OAuth oauth_nonce="fb0e9383f0c84326a124dd4ccfddd2d2", oauth_callback="https%3A%2F%2Foauth.vk.com%2Fblank.html", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1393951866", oauth_consumer_key="ffo9MLdRF8XOd9DKB0HeA", oauth_signature="qQwIvFao9yeIQpi9ouz0oFi7%2Fv8%3D", oauth_version="1.0"
Question to people who maybe knows how to work with twitter. What is wrong with my data?
I can provide more code, but i think it is not useful.
P.S. - i provided working credentials, i will change them after 3 - 4 hours.
UPD: This is my key which used in calculateSign function
fUJtvIpujTslQOlVbZY6QU8cNEMdwoxzTG1gh93SUgs%26
There is an error in your signature base string. The callback url should be double encoded so it should be:
... &oauth_callback%3Dhttps%253A%252F%252Foauth.vk.com%252Fblank.html ...
Also you do not need to percent encode the ampersand ('&') character in the signing key, so you should use:
fUJtvIpujTslQOlVbZY6QU8cNEMdwoxzTG1gh93SUgs&
You can check your signature using the online OAuth Test Console.
Check update#1
This logic is a candidate for a authentication procedure, done by simple HTTP requests:
I'm sending: userName + encrypted_userName (encrypted_userName is actually the encrypted result of userName, done using AES & as key i use the md5 hash of the password). NOTE: I'm not sending the md5 hashed Password.
on the server I'm comparing: encrypted_userName with own_encrypted_userName (since on server i have access to full info on user, i calculate own encrypted_userName).
Question: is this a security flaw? Say bad guy captures full HTTP request, can he extract password from this 2 infos?
CODE DETAILS, if needed:
private static Cipher getCipher(String key, int mode) throws Exception{
byte[] rawKey = getRawKey(key.getBytes("UTF-8"));
SecretKeySpec skeySpec = new SecretKeySpec(rawKey, "AES");
Key key2 = skeySpec;
Cipher cipher = Cipher.getInstance("AES/ECB/PKCS5PADDING");
cipher.init(mode, key2);
return cipher;
}
private static byte[] getRawKey(byte[] seed) throws Exception {
/* BEFORE:
KeyGenerator kgen = KeyGenerator.getInstance("AES");
SecureRandom sr = SecureRandom.getInstance("SHA1PRNG");
sr.setSeed(seed);
kgen.init(128, sr); // 192 and 256 bits may not be available
SecretKey skey = kgen.generateKey();
byte[] raw = skey.getEncoded();
*/
byte[] raw = MD5Util.getMD5HashRaw(seed);
return raw;
}
(NOTE: reason why i use the hash of the password is that code is compatible among platforms (client is Android device), while the commented version is NOT)
UPDATE#1
Short answer:
Presented logic is not even close to be considered a secure authentication mecanism
(for Why? check Michael's answer bellow)
Decided to use Kerberos (AND not https, since I am not familiar + seems complicated to setup):
It is not a true version of Kerberos (like v4 or v5), it is just my own implementation so lets call it "similar with Kerberos" (I know, I know: DONT "roll your own encryption"!!!),
Here are some details:
it works on UDP (now)
authentication is done only once, by:
client sending a Authenticator message (contains: [userId] in plain text & [something_ecrypted] with [entered_user_password] (curently [something_ecrypted] contains just a timestamp, call it [authenticator_creation_timestamp])) NOTE: password is not transmited
server upon receiving message, tryies to decrypt the [something_ecrypted] with [actual_user_password] -> if SUCCESS then client is who it pretends to be, so i send him back a OK response (as in Kerberos this response contains some stuff, like a [public_key] (a RSA key, but encrypted with user_password) + ticket granting ticket (call it [TGT], encrypted with a password known only by server, currently it doenst expire, this [TGT] also contains some stuff, like these 2 timestamps: [TGT_creation_time_stamp] + [authenticator_creation_timestamp] (the one received in the Authenticator message))
after receiving this OK message, client has procured a valid [public_key].. so nice!
protection agains "reply attack" is not a 100% guarantee, but I see it "safe enought":
on each next HTTP reaquest, i attach as headers these 2 guys [new_request_creation_timestamp] (encrypted with [public_key], procured above) + the [TGT] (untouched, as received above)
on server I just need to validate [new_request_creation_timestamp] agains some math (obvious [TGT] needs to be valid too):
** i expect that the following variables to be almost equal
delta1 = [TGT_creation_time_stamp] - [authenticator_creation_timestamp]
delta2 = now()-[new_request_creation_timestamp]
(I actually allow a difference between them of 5 seconds, but from my tests, its just a matter of some 10-20 millis,
** So initial delta (calculated when creating OK response to Authenticator) should perpetuate on next interactions.
I do find this new approach quite trust-worthy, but if you have an opinion or see a BUG in logic, please share.. Thanks
Yes, this is a weak security mechanism.
Anyone who captures the information sent to the server can easily replay it to authenticate themselves (replay attack).
It is vulnerable to offline password guessing - anyone who captures the information sent to the server can then very quickly test a password list to find what password your user has chosen (by encrypting the observed username using the hash of each password in turn). The hashing can even be precomputed, speeding the attack up further.
Password based authentication protocols should be resistant to replay attacks and offline password guessing attacks.
Simply using an HTTPS (TLS) connection to your server and sending the username and password in plaintext would normally be a better solution.
In response to your update 1:
I strongly advise using HTTPS. It is used everywhere for a reason - it has undergone immense security review and been found to be (largely) secure - far better than what you can get through a SO post.
I haven't considered your updated scheme thoroughly, but as it is based on Kerberos it is also subject to offline password guessing attacks as I described above.
Having successfully authenticated, don't forget about then actually protecting your data - you'll likely need to derive a shared symmetric key then use authentication + encryption on your data...
What i understand is : you are sending Username + Encrypted Username to the server.
Ans:
Since you are sending the Username and the encrypted Username which is : UserName + AES(UserName + MD5 Hashed Password)
If anyone knows or find that you give the Username and also gets the Username from your data to server: No worries. There you stand with AES. If you have doubt in AES encryption check this. Your data is secure.
I don't think this is a security flaw per se because even knowing both the plaintext message and the encrypted one, it is practically impossible to get the AES key. But I still wouldn't recommend to store the passwords hashed with MD5.