I have an android app that comminicates with a nodejs server via http.
To each request I add a SHA1 checksum that iterates over the key value pairs I'm sending to the server so that it would validate it on arrival.
I have tested this and it works okay as long as the values I'm sending are in English. Sending a request that contains a string value in Hebrew for example or that includes an unstandard character (like é, ç or à) will fail the request base on a checksum mismatch.
The checksum is calculated first in the andoird client (JAVA) and than in the nodejs server for verification.
I eliminated the possibility that this is caused due to the Right To Left nature of the Hebrew language by sending a single letter value, and it still failed.
What could cause this? I assume this might be because the encoding of a JAVA string for non standard characters is different from that of nodejs. But how can I solve this?
Here is the relevant code from the server side:
var shasum = crypto.createHash('sha1');
while (fieldsPointer < allFields.length
|| filesPointer < allFiles.length) {
shasum.update(fieldKey);
let fieldKey = allFields[fieldsPointer];
shasum.update(fieldKey);
let fieldValue = fields[allFields[fieldsPointer]];
shasum.update(fieldValue);
}
and the client side:
try {
MessageDigest md = MessageDigest.getInstance("SHA-1");
for(Entry<String, Object> entry : allParams.entrySet()) {
md.update(entry.getKey().getBytes());
if(entry.getValue() instanceof String) {
String value = (String) entry.getValue();
md.update(value.getBytes());
} else (...) // some other instance checking for non String values
}
String checksum = bytesToHex(md.digest());
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
I did some research based on Williams's comment.
It turns out all I need to do was to change this:
shasum.update(fieldValue);
to this:
shasum.update(fieldValue, 'utf8');
as stated in the nodejs crypto documentation for hash encoding: http://nodejs.org/api/crypto.html#crypto_hash_update_data_input_encoding
While this works, it is also a good idea to state the charset in the client side, as William stated in the comment below.
I did this by changing this:
md.update(value.getBytes());
to this
md.update(value.getBytes("utf8"));
Related
I'm trying to authenticate a webhook from starling bank on a PHP 7.0.22 (Apache/2.4.6 (Red Hat Enterprise Linux)) server.
I've been told by support that the following java code is being used to generate the digest
private String calculateSignature(String sharedSecret, String requestJson) {
try {
String contentToDigest = sharedSecret + requestJson;
MessageDigest messageDigest = MessageDigest.getInstance("SHA-512");
byte[] digest = messageDigest.digest(contentToDigest.getBytes());
return Base64.getEncoder().encodeToString(digest);
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException("Error calculating digest for payload [" + requestJson + "]", e);
}
}
The sharedSecret I already have and the requestJson I take from the webhook POST using:
$requestJson=file_get_contents('php://input') ;
my php code to generate the hash is as follows:
$concatenated_string=$sharedSecret . json_encode($requestJson) ;
$generated_hash=base64_encode(hash('sha512', $concatenated_string ));
This doesn't give the same hash. Whilst hacking to try and find an answer, I've also tried the following :
$concatenated_string=$sharedSecret . $requestJson ;
and different hash types and options:
$generated_hash=base64_encode(hash('sha512', $concatenated_string, true ))
$generated_hash=base64_encode(openssl_digest($concatenated_string, 'sha512')) ;
base64_encode and hash are effectively doing the same thing in this case:
https://stackoverflow.com/a/11195855/3323777
You should specify third argument as TRUE at your php code to match the java version:
raw_output - Setting to TRUE will return as raw output data, otherwise the return value is binhex encoded.
http://php.net/manual/ru/function.openssl-digest.php
I've ran your both snippets on java and php and found not difference when encoding a string "test". I advise you to output the json payloads to two files on both environments and use diff to compare them.
I am writing an App which does the following with a given String str:
encode(encrypt(encode(stringToBytearray(str))));
The Server receives the Base64 encoded String, which is then decoded -> decrypted -> decoded, to get the sent String str from the App.
Unfortunately it doesnt work for all Strings, long Strings lead to a long Base64 String and my Server throws the following Exception:
Exception in thread "main" java.lang.IllegalArgumentException: Illegal base64 character 5b
at java.util.Base64$Decoder.decode0(Base64.java:714)
at java.util.Base64$Decoder.decode(Base64.java:526)
at Main.decode(Main.java:113)
at Main.main(ain.java:33)
The String has the format "[string, string, ..., string]" without "s.
Like I mentioned above, for Strings which are not too long (sorry I cant quantify the length yet), this works. So I think I implemented it right.
Whats weird is, that if I dont send it, but decode(decrypt(decode(stringToBytearray(str)))); the String on the Device itself, it all works perfectly.
My Setup:
JDK 7, eclipse (adt-bundle, android development) (Windows 7)
JDK 8, eclipse ('normal' java) (Linux (xubuntu))
Is it because both Classes (Base64) are implemented diffrently? If so, how can I make it work?
Here are the Encoding/Decoding Methods:
Encoding (Device: on Windows 7, adt-bundle, Android-Dev, jdk 7):
import android.util.Base64
public byte[] encode(byte[] bytearrayToEncode){
return Base64.encode(bytearrayToEncode, Base64.NO_WRAP|Base64.URL_SAFE);
}
Decoding (Server: on Linux, jdk 8):
import java.util.Base64
public byte[] decode(byte[] bytearrayToEncode){
return Base64.getUrlDecoder().decode(bytearrayToDecode);
}
Strings are all De-/Encoded with the same charset (utf-8)!
Encoding/Decoding: Base64
Crypto: AES
If you need more Information, just ask, but I think I provided all neccessary informations.
Edit:
public String bytearrayToString(byte[] bytearray){
String str = null;
try {
str = new String(bytearray, "UTF-8");
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return str;
}
public byte[] stringToBytearray(String str){
byte[] bytearray = null;
try {
bytearray = str.getBytes("UTF-8");
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return bytearray;
}
You must be using JSONObjects to send the Base64 String to the server. If so then the JSONObject will add " \ " escape characters to the string which are illegal Base64 characters. All you need to do is
String rectifiedString = Base64String.replace("\\","");
on the server end before using the acquired string.
Optional
Also on the server side use
import javax.xml.bind.DatatypeConverter;
and
String result = DatatypeConverter.printBase64Binary(bytearrayToDecode);
byte[] result = DatatypeConverter.parseBase64Binary(str);
instead.
It will not only get the job done but is much faster at encoding/decoding (as benchmarked here).
If you have a space in your encoded string, which is an illegal character in base64. If you remove it, it still generates the same image/data in the converter you linked to, and can be now decoded by your code as well.
UPDATE:
Some decoders (like the one you linked, or Base64.getMimeDecoder() in Java) ignore illegal characters, others (like Base64.getDecoder()) don't allow them.
Use for encode:
Base64.getEncoder().encodeToString(yourString.getBytes("UTF-8"));
and decode:
byte[] decodedBytes = Base64.getDecoder().decode(yourString);
String stringDecode = new String(decodedBytes, "UTF-8");
I have a legacy system that uses hibernate interceptor to encrypt (and encode) and decrypt (and decode) some fields on some database tables. It makes use of the OnSave, OnLoad and OnFlushDirty methods. This code turns out to be buggy as data read from this system, when transferred to another application still has some of the records encrypted and encoded (some encrypted multiple times). The challenge for me here is that I could perform the decryption and decoding (as many times as necessary) when the receiving application is on a Windows machine. I get a BadPaddingException when I try to repeat the same thing when the receiving application is a linux VM.
Any help/suggestions will be greatly appreciated
here is a snippet of the hibernate interceptor
public boolean onLoad(Object entity, Serializable arg1, Object[] state, String[] propertyNames, Type[] arg4) throws CallbackException {
if (key != null){
try {
if (entity instanceof BasicData) {
for (int i = 0; i < state.length; i++) {
if (state[i] instanceof String){
String cipherText = (String)state[i];
byte[] cipherTextBytes = Base64Coder.decode(cipherText);
byte[] plainTextBytes = dCipher.doFinal(cipherTextBytes);
state[i] = new String(plainTextBytes, "UTF8");
}
}
return true;
}
} catch (Exception e) {
e.printStackTrace();
}}return false;}
I'd have to guess here but if you mean this Base64Coder the problem might be the following:
It is unclear how the base64 string has been created, i.e. which encoding had been used.
If you use UTF-8 to get the bytes of a string and create a base64 from those bytes you'll get a different result than if you'd use ISO Latin-1, for example.
Afterwards you create a string from those bytes using UTF-8, but if the base64 string had not been created using UTF-8, you'll get wrong results.
Just a quote from the linked source (if this is the correct one):
public static String encodeString (String s) {
return new String(encode(s.getBytes())); }
Here, s.getBytes() will use the system's/jvm's default encoding, so you should really ensure it is UTF-8!
If you control both sides, encode and decode, better way to use DatatypeConverter:
String buffer = DatatypeConverter.printBase64Binary( symKey );
byte[] supposedSymKey = DatatypeConverter.parseBase64Binary( buffer );
After a day of long programming I want to post something useful for someone else.
Few days ago I was wondering how to handle JPasswordField's method getPassword() with a correct procedure to pass a value to a server and get an answer.
and this is the question:
How can I correctly get a value from a JPasswordField in a safe way and handle it to create a login procedure with a server?
This is the solution I reached.
First of all I decided the procedure of the login that was safe enough to my purpose, I didn't want to send a plain password to the server and I didn't want to store a plain password (obviously) in my database.
The first thing to say is that a good way to secure a password is that it is never ever exchanged over a network in a plain and readable form, this is obviously because of a possible "man in the middle", that in few words is someone reading your messages in their way to the server.
The password need to be hashed, that means that it is transformed to a quite long sequence of hexadecimal characters. The good thing of the hash is that is (hopefully) one-way. You can't de-hash a password.
There are many algorithms to do this, I choose the SHA256.
The password is then hashed, but just between us, this can be not enough. If an hacker is able to steal the hash there are some techniques that can bring him to a successful "translation" of it. Just to add a variable in his equation, and to make his life harder, we can add a salt to the password prior to hash it. A salt is a piece of string that is added in any position we desire to the password. This avoids some kind of attacks based on dictionary and most used password.
But if an hacker that is better trained than me can't read the password, how can I?
The answer is simple, I don't have to.
But to understand this we need to jump for a moment in the "Registration procedure" that is the moment when a new user is added to my database. This is it:
The client ask the server to be registered sending the nickname.
The server answer with a token that is the salt for the password.
The client salt the password, hash it and send it back to the server.
The server now receive something that is unreadable so there's no security problem, and store it with the nickname and the salt. The salted hashed password is the "common secret".
So the login procedure will be like this:
The client ask the server to login
the server answer with the salt
the client salts the password then hashes it and sends it back to the server.
the server compare the shared secret with the received string. If they are equals the user is allowed to login.
This should be quite fine, but in this case if an hacker knows the shared secret can access the server without any problem because doing so we just changed the password, not to be readable, but still usable directly.
To avoid this behavior we just have to add a passage in our chain:
The client ask the server to login
the server answers with the salt and a random session-salt
the client salts the password, hashes it. At this point it salts again the hash and re-hash it. Then it send the hashed-salted-hash-of-salted-password back to the server
the server takes the shared secret, salts it with the random session salt and then hashes it. If the two strings are equals then the user is allowed to login.
Now that the procedure is clear we have an issue to solve. If I handle any kind of String this can persist in the memory for long time, so if I put my password in a String it can be readable in a plain form for long time. This is not so good for us, but we are not the first to think about it, java indeed created a way to avoid this password persisting. The solution is use an array of characters. This is because even if the array is persisting in the memory, its datas are spread with no order in the memory and is very difficult to re-create the original password.
Re inventing hot water? Yep, just use the getPassword() method in a JPasswordField.
But this is quite difficult for newbie. We get a char[] array, and it is strange for a not expert.
The first thing that reach our mind can be to transform that array in a plain string .......
But is just what I want to avoid. So I need to handle the array as-is.
We need then a method to salt and hash the password, the result can be this:
public static String digestSalted(String salt, char[] password) throws NoSuchAlgorithmException {
MessageDigest md = MessageDigest.getInstance("SHA-256");
ArrayList<Byte> list = new ArrayList<Byte>();
for (int i = 0; i < password.length; i++) {
//String ch = String.valueOf(password[i]);
//byte[] b = ch.getBytes();
//for (int j = 0; j < b.length; j++) {
// list.add(b[j]);
//}
list.add((byte)password[i]);
}
byte[] saltInBytes = salt.getBytes();
byte[] toBeHashed = new byte[(saltInBytes.length + list.size())];
for (int i = 0; i < saltInBytes.length; i++) {
toBeHashed[i] = saltInBytes[i];
}
for (int i = saltInBytes.length; i < list.size() + saltInBytes.length; i++) {
toBeHashed[i] = list.get(i - saltInBytes.length);
}
md.update(toBeHashed);
byte byteData[] = md.digest();
StringBuffer hexString = new StringBuffer();
for (int i = 0; i < byteData.length; i++) {
String hex = Integer.toHexString(0xff & byteData[i]);
if (hex.length() == 1) {
hexString.append('0');
}
hexString.append(hex);
}
return hexString.toString();
}
this method create an array of bytes passing through many little strings then append the salt. Once salted it hash the result with SHA256.
Now the return can be a string because is hashed and there is no problem of security.
This give a solution for the first part of the question.
The second part is just to implements our protocol between server and client.
I will only show the code in the client that is significant enough to understand the procedure.
I am using a blocking queue where the messages is put when read from the socket. this is the code:
public void login(String nickname, char[] password) {
if (cl == null) {
throw new RuntimeException();
}
long s = Sys.getTime();
cl.send("NICK " + nickname);
IncomingMessage reply = null;
try {
reply = this.mh.getMessage(); //The response to NICK msg
if (reply.getCommand().equalsIgnoreCase("LOGIN")) {
ArrayList<String> params = reply.getParams();
String accountSalt = params.get(0);
String randomSalt = params.get(1);
try {
String sharedSecret = SHAHash.digestSalted(accountSalt, password);
String saltedSharedSecret = SHAHash.digestSalted(randomSalt, sharedSecret);
if (saltedSharedSecret != null) {
cl.send("PASS " + saltedSharedSecret);
reply = this.mh.getMessage();
if (reply.getCommand().equalsIgnoreCase("WELCOME") && reply.getParams().get(0).equals(nickname)) {
// ************ LOG ************ //
LOG.config("Logged in.");
// ***************************** //
this.running = true;
this.loggedIn = true;
mh.startExecutor();
LOG.config("Time passed: " + (Sys.getTime() - s));
mh.startGame();
} else {
// ************ LOG ************ //
LOG.warning("A problem has occured while trying to login to the server.");
// ***************************** //
JOptionPane.showMessageDialog(null, "Error while logging to the server, shutting down.\n- ERROR 006 -");
System.exit(0);
}
}
} catch (NoSuchAlgorithmException e) {
// ************ LOG ************ //
LOG.warning("Error while SHA hashing the password, shutting down.");
// ***************************** //
JOptionPane.showMessageDialog(null, "Error while SHA hashing the password, shutting down.\n- ERROR 005 -");
System.exit(0);
}
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
The code, now that we have clear how the protocol works, it's easy to understand the thing that should be considered is that this.mh.getMessage() is a blocking method, this means that the thread will wait until something is available in the queue before trying to get it.
This (almost) how I solved my problem. Let me know if there is any error in the answer or if you need some clarification.
I hope this will be useful to someone. Have a nice programming
I'm using the Jasypt encryption library to encrypt/decrypt some text. This code is embedded in a WAR file and deployed to a server.
When running locally, and in unit tests, the encrypt/decrypt cycle works perfectly. I use Jetty to develop the application. The code works perfectly in that server. For some reason, deploying to Tomcat breaks it with the following exception:
FYI, I have the strong encryption libraries installed in both my local and server environments and I'm using the latest 1.6 version (patch level 25).
org.jasypt.exceptions.EncryptionOperationNotPossibleException
The exception has no message.
The code is fully symmetric. I pasted it here for examination. Here are the relevant bits:
I found one old Nabble post where a user had a very similar problem. Code worked everywhere except inside Tomcat. No solution was given.
Any insights would be most appreciated.
**Update: ** Running in Tomcat on my local system, it appears to work. So there's something about my server. On the server, I'm using a 64-bit JVM on Windows Server 2008. I'm using a 32-bit JVM locally (due to my system being a bit older). I wonder if this has something to do with the issue.
public void initializeService() {
binaryEncryptor = new BasicBinaryEncryptor();
binaryEncryptor.setPassword(keyBase64);
}
#Override
public <T extends Serializable> String simpleEncrypt(T objectToEncrypt) throws EncryptionException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
try {
ObjectOutputStream oos = new ObjectOutputStream(bos);
oos.writeObject(objectToEncrypt);
byte[] bytes = binaryEncryptor.encrypt(bos.toByteArray());
return new String(Base64.encodeBase64(bytes));
} catch (IOException e) {
LOGGER.error("failed to encrypt String: " + e.getMessage());
throw new EncryptionException(e.getMessage(), e);
} catch (Exception e) {
LOGGER.error("failed to encrypt String: " + e.getMessage());
throw new EncryptionException(e.getMessage(), e);
}
};
#SuppressWarnings("unchecked")
#Override
public <T> T simpleDecrypt(String objectToDecrypt) throws EncryptionException {
try {
byte[] bytes = Base64.decodeBase64(objectToDecrypt);
byte[] decryptedBytes = binaryEncryptor.decrypt(bytes);
ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(decryptedBytes));
T object = (T)ois.readObject();
return object;
} catch (IOException e) {
LOGGER.error("failed to decrypt String: '" + objectToDecrypt + "', mesage = " + e.getMessage());
throw new EncryptionException(e.getMessage(), e);
} catch (Exception e) {
LOGGER.error("failed to decrypt String: '" + objectToDecrypt + "', mesage = " + e.getMessage());
throw new EncryptionException(e.getMessage(), e);
}
}
Here is a link to the docs: http://www.jasypt.org/faq.html#i-keep-on-receiving-encryption-operation-not-possible
Is encryption and decryption config identical
Check to make sure table columns are large enough
Base64 encoding and urlencoding can conflict, so it has to be done just right.
#biniam_Ethiopia
I would have commented your answer but I have not enough reputation, so I write my own answer:
I had a very similiar problem, but in my case it was because of changing the encryption algorithm (PBEWithMD5AndTripleDES), while entries in the db were saved with a different one before (PBEWithMD5AndDES).
So I got a EncryptionOperationNotPossibleException too, which is without information because of #Nathan Feger's comment above.
I hope this could help somebody someday too ;)
I faced similar problem.
For me, it was because it was trying to decrypt a password which could not have been decrypted using the decrypting mechanism.
Hence, I encrypted the password and stored it in database before the decrypt method tries to decrypt it.
I hope it helps someone.