Develop Reference

Generate Ethereum addresses in HD Wallet using public key only (bitcoinj/web3j)

I trying to generate Ethereum addresses for the HD Wallet keys implemented with bitcoinj library, but I got confused:
DeterministicSeed seed = new DeterministicSeed("some seed code here", null, "", 1409478661L);
DeterministicKeyChain chain = DeterministicKeyChain.builder().seed(seed).build();
DeterministicKey addrKey = chain.getKeyByPath(HDUtils.parsePath("M/44H/60H/0H/0/0"), true);
System.out.println("address from pub=" + Keys.getAddress(Sign.publicKeyFromPrivate(addrKey.getPrivKey())));
this code prints a correct Ethereum address accordingly to https://iancoleman.io/bip39/. Everything is fine here.
But when I trying to avoid private key usage and generate non-hardened keys using public keys only I getting different results, i.e. the call returns another result:
System.out.println("address from pub=" + Keys.getAddress(addrKey.getPublicKeyAsHex()));
And it looks like the issue is in the "different public keys", i.e. result of the Sign.publicKeyFromPrivate(addrKey.getPrivKey()) and addrKey.getPublicKeyAsHex() are different.
I'm not experienced with cryptography, thus it may be a silly question... but I would appreciate any advice here.

Like Bitcoin, Ethereum uses secp256k1. Ethereum addresses are derived as follows:
Step 1: The 32 bytes x and y coordinate of the public key are concatenated to 64 bytes (where both the x and y coordinate are padded with leading 0x00 values if necessary).
Step 2: From this the Keccak-256 hash is generated.
Step 3: The last 20 bytes are used as the Ethereum address.
For the examples used here, the key is generated with:
String mnemonic = "elevator dinosaur switch you armor vote black syrup fork onion nurse illegal trim rocket combine";
DeterministicSeed seed = new DeterministicSeed(mnemonic, null, "", 1409478661L);
DeterministicKeyChain chain = DeterministicKeyChain.builder().seed(seed).build();
DeterministicKey addrKey = chain.getKeyByPath(HDUtils.parsePath("M/44H/60H/0H/0/0"), true);
This corresponds to the following public key and Ethereum address:
X: a35bf0fdf5df296cc3600422c3c8af480edb766ff6231521a517eb822dff52cd
Y: 5440f87f5689c2929542e75e739ff30cd1e8cb0ef0beb77380d02cd7904978ca
Address: 23ad59cc6afff2e508772f69d22b19ffebf579e7
as can also be verified with the website https://iancoleman.io/bip39/.
Step 1:
In the posted question, the expressions Sign.publicKeyFromPrivate() and addrKey.getPublicKeyAsHex() provide different results. Both functions return the public key in different types. While Sign.publicKeyFromPrivate() uses a BigInteger, addrKey.getPublicKeyAsHex() provides a hex string. For a direct comparison, BigInteger can be converted to a hex string with toString(16). When the results of both expressions are displayed with:
System.out.println(Sign.publicKeyFromPrivate(addrKey.getPrivKey()).toString(16));
System.out.println(addrKey.getPublicKeyAsHex());
the following result is obtained:
a35bf0fdf5df296cc3600422c3c8af480edb766ff6231521a517eb822dff52cd5440f87f5689c2929542e75e739ff30cd1e8cb0ef0beb77380d02cd7904978ca
02a35bf0fdf5df296cc3600422c3c8af480edb766ff6231521a517eb822dff52cd
The output of Sign.publicKeyFromPrivate() has a length of 64 bytes and corresponds to the concatenated x and y coordinate as defined in step 1. Therefore, the address generated with this is a valid Ethereum address, as also described in the posted question.
The output of addrKey.getPublicKeyAsHex(), on the other hand, corresponds to the x coordinate prefixed with a 0x02 value. This is the compressed format of the public key. The leading byte has either the value 0x02 if the y value is even (as in this example), or the value 0x03. Since the compressed format does not contain the y coordinate, this cannot be used to directly infer the Ethereum address, or if it is done anyway, it will result in a wrong address (indirectly, of course, it would be possible since the y coordinate can be derived from a compressed public key).
The uncompressed format of the public key can be obtained, e.g. with addrKey.decompress():
System.out.println(addrKey.decompress().getPublicKeyAsHex());
which gives this result:
04a35bf0fdf5df296cc3600422c3c8af480edb766ff6231521a517eb822dff52cd5440f87f5689c2929542e75e739ff30cd1e8cb0ef0beb77380d02cd7904978ca
The uncompressed format consists of a leading marker byte with the value 0x04 followed by the x and y coordinates. So if the leading marker byte is removed, just the data according to step 1 is obtained, which is needed for the derivation of the Ethereum address:
System.out.println(addrKey.decompress().getPublicKeyAsHex().substring(2));
which results in:
a35bf0fdf5df296cc3600422c3c8af480edb766ff6231521a517eb822dff52cd5440f87f5689c2929542e75e739ff30cd1e8cb0ef0beb77380d02cd7904978ca
Steps 2 and 3:
Steps 2 and 3 are performed by Keys.getAddress(). This allows the Ethereum address to be obtained using the uncompressed public key as follows:
System.out.println(Keys.getAddress(addrKey.decompress().getPublicKeyAsHex().substring(2)));
System.out.println(Keys.getAddress(Sign.publicKeyFromPrivate(addrKey.getPrivKey()))); // For comparison
which gives the Ethereum address:
23ad59cc6afff2e508772f69d22b19ffebf579e7
23ad59cc6afff2e508772f69d22b19ffebf579e7
Overloads of Keys.getAddress():
Keys.getAddress() provides various overloads for the data types BigInteger, hex string and byte[]. If the uncompressed key is given as byte[], e.g. with addrKey.getPubKeyPoint().getEncoded(false), the byte[] can be used directly after removing the marker byte. Alternatively, the byte[] can be converted to a BigInteger with the marker byte removed:
byte[] uncompressed = addrKey.getPubKeyPoint().getEncoded(false);
System.out.println(bytesToHex(Keys.getAddress(Arrays.copyOfRange(uncompressed, 1, uncompressed.length))).toLowerCase()); // bytesToHex() from https://stackoverflow.com/a/9855338
System.out.println(Keys.getAddress(new BigInteger(1, uncompressed, 1, uncompressed.length - 1)));
System.out.println(Keys.getAddress(Sign.publicKeyFromPrivate(addrKey.getPrivKey()))); // For comparison
which as expected returns the same Ethereum address:
23ad59cc6afff2e508772f69d22b19ffebf579e7
23ad59cc6afff2e508772f69d22b19ffebf579e7
23ad59cc6afff2e508772f69d22b19ffebf579e7
One thing to note here is that Keys.getAddress(byte[]) does not pad the passed byte[], while the overloads for BigInteger or hex strings implicitly pad. This can be relevant e.g. when converting a BigInteger (e.g. provided by Sign.publicKeyFromPrivate(addrKey.getPrivKey())) to a byte[], since the result can also have less than 64 bytes (which would lead to different Keccak-256 hashes). If Keys.getAddress(byte[]) is used in this case, it must be explicitly padded with leading 0x00 values up to a length of 64 bytes.

how to convert a number to base32 in java as per RFC-4648

Actually I'm trying to use TOTP in my app and google authenticator requires the key to be in base32 format. This is the reason I'm trying to convert a key to base32 format
Let's say I have a number = 150820200825235.
This wikipedia page says that RFC-4648 is the most common base32 alphabet used.
Here's my java code where I'm trying to convert a number to base 32:
long key=150820200825235L;
String base32Key = Long.toString(key,32);
System.out.println(base32Key);
Now it is printing this as the output :
495e87tgcj
It contains 9 which is invalid according to RFC-4648.
how do I convert to base32 number as per RFC-4648 in java?
One more thing If the most widely used Base32 alphabet is really as per RFC-4648 then why doesn't java support it as default? or is something is wrong in my understanding/code ?

"Base32" as defined in RFC-4648 is a way to encode binary data ("arbitrary sequences of octets") in ASCII text using only numbers, upper case letters, and = for padding. You can find an implementation in the Apache Commons Codec library
If you start with a number like 150820200825235 and want to convert it into RFC 4648 base32, first you need to decide how to convert the number into a sequence of bytes.
Foe example, One way to convert a long value into bytes is using the DataOutputStream.writeLong method. This method uses 8 bytes. Using this method to write 150820200825235L and encoding the resulting bytes in base32 gives you AAAISK4QP3AZG===.

RC4 Encryption/Decryption for integers Java

In short, I have a integer value about 10 digits long. I would like to encrypt it using rc4 algorithm in Java. I went online and search, but I could only find encryption for string values/plaintext. Please advise. Thanks!

I assume that you are using the JavaSE API, in particular the javax.crypto.Cipher class. The encryption API is concerned with generic data, not interpreted in some way; this is why Cipher#doFinal() takes a byte array. (You may interpret that as a string, given the common terms "plaintext"/"ciphertext".)
The solution to your problem is to convert the integer to a byte array. If "integer" in your case means int (32-bit), then you need 4 bytes (8-bit). See this question for (multiple good) solutions to this.

Can't you just convert the integer to a String and then encrypt the string?
String myIntegerString = Integer.toString(myInteger);
encrypt myIntegerString;
store the encrypted myIntegerString;
read the encrypted myIntegerString;
decrypt myIntegerString;
Integer.parseInt(myIntegerString).

Share and store RSA - public key in java server and vice versa

My requirements are:
Requirement 1: Share public key to java server.
Steps:
Generate public-private keys in iOS app.
Store the generated keys in keychain.
Send generated public key to java server.
Java server shall be able to store shared public key in database.
Requirement 2: Store public key sent by java server.
Steps:
Java server sends public key of other user.
Process data sent by java server and generate public key from it.
Store generated key in keychain, which can be later retrieved for encrypting message to be transferred.
I am able to achieve steps 1-2 in requirement 1 by using below method defined in SecKeyWrapper class (CommonCrypto sample):
- (void)generateKeyPair:(NSUInteger)keySize
Question 1: Now problem is- how shall I send that key to java server?
We have getPublicKeyBits method in the same class, which returns an NSData object, on some googling I found that it is in DER encoded format.
Question 2: If I send the same NSData object to server, which I guess it will interpret as ByteBuffer object, will it be possible for other devices, in my case it could be android, to interpret that data?
Question 3: What is the best way to share public key in above scenarios?
This is what I am trying to achieve from some days now:
Approach #1: Trying to generate public key from exponent and modulus
Android End
Generated public and private key at an android device (using an
openssl wrapper)
Got modulus and exponent from the generated public key
iOS End
Generated public key from modulus and exponent, at ios end, using
code specified in this link:
https://stackoverflow.com/a/10643962/217586
Converted some sample string to an object of NSData using
NSUTF8StringEncoding
Used - wrapSymmetricKey:keyRef: method defined in SecKeyWrapper
class (CryptoExercise example) for encryption, and passed key obtained
from step 1 and data to encrypt obtained from step 2 to it
Converted NSData (encrypted data) obtained in previous step to
base64encoded string, shared the same to android guy
Android End
Tried to decrypt the base64encoded string, using related private key
Problem:
getting error - too much data for RSA block
Approach #2: (Got to know from this link that -
https://github.com/superwills/iOSRSAPublicKeyEncryption, we are not
supposed to load public keys in iOS from anything other than a
certificate, so tried a different approach)
Terminal End
Generated certificate using openssl commands specified in this url:
https://stackoverflow.com/a/17295321/217586
iOS End
Obtained public key as specified in above url
Used below code to encrypt the data:
SecKeyWrapper *secKeyWrapper = [SecKeyWrapper sharedWrapper];
SecKeyRef obtainedPublicKey = [secKeyWrapper
getPublicKeyRefFromDerCertificate:kCertificatePath];
NSData *dataToBeEncrypted = [kStringToBeEncrypted
dataUsingEncoding:NSUTF8StringEncoding];
NSData *encryptedText = [secKeyWrapper
wrapSymmetricKey:dataToBeEncrypted keyRef:obtainedPublicKey];
Converted NSData to base64encoded string
Terminal End
Used below command to convert it back to original string:
echo | openssl rsautl
-decrypt -inkey rsaPrivate.pem
Problem:
getting error - rsa routines:RSA_EAY_PRIVATE_DECRYPT:data greater than
mod
len:/SourceCache/OpenSSL098/OpenSSL098-47.1/src/crypto/rsa/rsa_eay.c
Any suggestions?

Your first approach is almost the same I used in a project time ago. The difference is that I was not sharing the modulus and exponent but the whole public key encoded in base 64 (without the -----BEGIN/END PUBLIC KEY----- header/footer), converting it back and forth to the appropriate classes in the end device. Basically the same concept is applied to encrypted data, always share Base64 encoded data to avoid conversion issues.
So far your approach thus should be fine if you hadn't any conversion issue (and you can find issues easily by converting the public keys to its base64 representation, on both android and iOS the string should be exactly the same, otherwise something bad happened).
The problem you have (getting error - too much data for RSA block) is because the text you are encrypting is too large for the key size you provided. RSA can only encrypt messages that are several bytes shorter than the modulus of the key pair thus the error. What you should be doing is, generate a one-time encryption symmetric key used to encrypt/decrypt the data, exchange the data encrypted with the symmetric key and the key encrypted using RSA. This is how roughly any correctly designed RSA encryption scheme works.
Anyway I think that if you try your first approach with a shorter fixed string you should observe that the first approach is probably working (unless you implemented things wrong, as you explained are theoretically correct but is hard to tell without seeing some code).
Antonio

Why doesn't my implementation of ElGamal work for long text strings?

I'm playing with the El Gamal cryptosystem, and my goal is to be able to encipher and decipher long sequences of text.
El Gamal requires the plaintext to be an integer. I have turned my string into a byte[] using the .getBytes() method for Strings, and then created a BigInteger out of the byte[]. After encryption/decryption, I turn the BigInteger into a byte[] using the .toByteArray() method for BigIntegers, and then create a new String object from the byte[].
I am using a 1035 bit key, and this works perfectly when I encipher/decipher with strings up to 129 characters. With 130 or more characters, the output produced from my decipher method is garbled.
Can someone suggest how to solve this issue?

Just like in RSA, you cannot encrypt a value larger than the modulus in ElGamal.

You can try
BigInteger pText = new BigInteger(plaintext.getBytes("UTF-8"));
to make the encoding/decoding and enciphering/deciphering more symmetric, but I'm not sure if that's the root cause.
By the way, you should never silently consume an Exception. The very least you can do is just catch (UnsupportedEncodingException e).

You need to use positive numbers for your operations. So you must construct BigInteger like this,
BigInteger pText = new BigInteger(1, plaintext.getBytes());
// 1: select a random integer k such that 1 <= k <= p-2
BigInteger k = abs(new BigInteger(p.bitLength() - 2, sr));

If you want to encrypt certain data with asymmetric cryptographic algorithm, you can do this only for really short data block. The reasons are both "technical" (the algorithm works this way) and "practical" (asymmetric cryptography is slow).
The right way to encrypt the large block of data using asymmetric cryptographic algorithm is
generate random ("session") key for some symmetric algorithm (AES, RC4, 3DES, you name it).
use this algorithm to encrypt the data
use your asymmetric algorithm to encrypt the session key
store the encrypted key near the data.
stop reinventing the wheel