Develop Reference

How to url encode data in Java

So Im trying to translate a working python code into Java. One of the steps required is to url encode the data. But when I encode the data in Java it looks different than the one in encoded in Python.
In one of the block of Python code theres this:
data = {'request-json': json}
print('Sending form data:', data)
data = urlencode(data)
data = data.encode('utf-8')
print('Sending data:', data)
The Output
Sending form data: {'request-json': '{"apikey": "xewpjipcpovwiiql"}'}
The output after being encoded
Sending data: b'request-json=%7B%22apikey%22%3A+%22xewpjipcpovwiiql%22%7D'
So this is what im trying to do in Java. As you can imagine Java is more involved. I used gson to convert to Json
Gson gson = new Gson();
API_Key key = new API_Key("xewpjipcpovwiiql");
String jsonInputString = gson.toJson(key);
Data data = new Data(key);
String request_form = gson.toJson(data);
System.out.println(request_form);
String urlencoded = URLEncoder.encode(request_form,StandardCharsets.UTF_8);
System.out.println(urlencoded);
The output:
Sending form data: {"request-json":{"apikey":"xewpjipcpovwiiql"}}
The output of the encoded string:
%7B%22requestjson%22%3A%7B%22apikey%22%3A%22xewpjipcpovwiiql%22%7D%7D
So they dont look the same so why are they coming differently ? How do I get the same python encoded String in Java ? I noticed in Python it used a combination of single and double quotes and in Java its only Double quotes so I dont know if that makes a difference.
Thank You!

On the Python side: The data.encode('utf-8') call is not necessary or at least the documentation describes with a different intention compared to this use https://docs.python.org/3/library/stdtypes.html#str.encode (and that's why there's a b' at the beggining).
The outer brackets are missing because it is interpreting request-json as the URL parameter name (it may be easier to understand if you add a second property at the json's top/first property level, you'll see you end with request-json=%7B%22apikey%22%3A+%22xewpjipcpovwiiql%22%7D&second-property=<second-property-value>).
On the Java side: the request_form is being completely interpreted as a single value to encode so you can put the encoded value as part of some parameter in a URL, as in: https://host:port?some-parameter-name=%7B%22requestjson%22%3A%7B%22apikey%22%3A%22xewpjipcpovwiiql%22%7D%7D

Equivalent Java function Base64.encode(byte, 0) in Swift

I have the following function in Java(write inside an Android app)
Bitmap bm = BitmapFactory.decodeFile(stringPath);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bm.compress(Bitmap.CompressFormat.JPEG, 100, baos); // bm is the bitmap object era quality 100
byte[] byteData = baos.toByteArray();
byte[] newB = Base64.encode(byteData,0);
It get an Image file and convert it to a byte array.
I need that to work on Swift.
I was able to convert the byteData to Int8, the equivalent of byte array in Swift.
I use the code below:
let filename = "RES4010110001"
let test_image:UIImage = UIImage(named: filename)!
let dataImage = UIImageJPEGRepresentation(test_image, 1)! as Data
var bytes: [Int8] = dataImage.map{Int8(bitPattern: $0)}
When I print this data using print(bytes) I got the same results on iOS and Android when I compare the "byteData" from Android and "bytes" from iOS
But I don't know what is the Java equivalent function below on Swift
Base64.encode(byteData,0);
How can I create an equivalent function on Swift of the function above?
For me, it appears that this function is encoding a byte array. But I have no idea of how I can encode a Int8 Array.
I said Int8 because for me, Int8 is the Swift equivalent of byte type in Java.
edit: I want to encode a Int8 array, I guess this is what the Java function (the function in the post title) is doing inside the Android app.

You can use base64EncodedData to encode your Data to get an encoded Data, rather than working with [Int8].
The Java code passes 0 (DEFAULT) as the flags. According to the docs, this means it's compliant with RFC 2045. RFC 2045 says that lines should be no more than 76 characters, so we should pass lineLength76Characters. RFC 2045 also requires CRLF as line endings, but base64EncodedData seems to insert them automatically even if we don't pass endLineWithCarriageReturn and endLineWithLineFeed.
let filename = "RES4010110001"
if let testImage = UIImage(named: filename),
let dataImage = testImage.jpegData(compressionQuality: 1) {
let encodedData = dataImage.base64EncodedData(options: [.lineLength76Characters])
} else {
// failed to get the UIImage, or the JPEG data, handle the error here...
}
I suggest that you do not work with [Int8] here. The byte[]s in the Java code are clearly not just "lists of numbers between -128 and 127". They represent data buffers, and those are represented by the Data struct in Swift. Don't worry, Both Data and [Int8] share a very similar set of protocol conformances, like MutableCollection, RangeReplaceableCollection, RandomAccessCollection etc. You can do almost everything you can do to an Int8 array, to a Data.

I was able to find an definite answer using part of the answer created by
#Sweeper and from another answer published here on Stackoverflow
let filename = "RES4010110001"
let test_image:UIImage = UIImage(named: filename)!
let dataImage = UIImageJPEGRepresentation(test_image, 1)! as Data // use apenas este que é correto, o galvez usa jpg no android
let dataImageStringBase64:String = dataImage.base64EncodedString(options: [.lineLength76Characters])
let byteArray = [UInt8](dataImageStringBase64.utf8)
The answer posted by #Sweeper just miss this line
let byteArray = [UInt8](dataImageStringBase64.utf8)
This last line of code made the code works perfectly for me.

ColdFusion or Java equivalent to PHP encryption/decryption

I have an API spec that reads:
Encryption Algorithm
The API utilizes the AES-128 (also known as Rijndael-128) algorithm with a 192-bit key in
CBC mode for encryption and decryption of sensitive pieces of information – the password parameter in the user/signin and user/signup methods, the authentication token, etc. The steps of the algorithm are listed below:
Encryption
Pad the input data so that its size is a multiple of the encryption algorithm block size – 16 bytes. In case the length of input data is a multiple of 16, a block of additional 16 bytes needs to be appended. The value of each pad byte is the number of pad bytes as an 'unsigned char'. That is, the last byte of the padded data should always be between 0x01 and 0x10.
Generate a 16-byte long initialization vector (IV) for the encryption algorithm.
Encrypt the padded data using AES-128 with the EK and the generated IV.
Combine the IV with the encrypted data.
Encode the result with urlsafe Base64. The urlsafe Base46 alphabet uses '–' instead of '+' and '_' instead of '/'.
Decryption
Base64-decode the input data.
Extract the first 16 bytes – these are the IV for the AES algorithm.
Decrypt the data using AES-128 with the EK and IV.
Read the value of the last byte of the decrypted data and remove that many bytes off its tail.
The only example provided by the supplier of this API is in PHP, using mcrypt. I know absolutely nothing about PHP, and am not an encryption expert. I need to be able to represent the above algorithm using ColdFusion 10.
I started by trying to take the example PHP files and find equivalents in either the ColdFusion tag or function library, and then by looking for a Java library with the same interface. I just don't know enough to make this happen.
Is there someone here that can point me in the right direction, or work with me offline to assist?
EDIT:
Here's the example given, for the basic task of doing a "check" on the keys (partner key and encryption key) provided to me for use with the API.
Object Client.php, has this constructor:
public function __construct($hostApiUrl, $partnerKey, $encryptionKey, $token = null)
{
$this->_pk = $partnerKey;
$this->_ek = $encryptionKey;
$this->_crypt = new Crypt($encryptionKey);
$this->_url = rtrim($hostApiUrl, '/') . self::BASE_URL;
if ($token) {
$this->setUserSession($token);
}
}
and this is the function I'm attempting to use:
public function checkKeys()
{
$secret = $this->_encodeParam($this->_ek);
$result = $this->call('partner/checkkeys', array(
'secret' => $secret
));
if (!$result || !$this->_isCodeOk($result->code)) {
return false;
}
return true;
}
So the client object already has the partner key and encryption key when this method is called, obviously.
so the "secret" is created by "encoding" the encryption key provided, using _encodeParam() method. that looks like this:
protected function _encodeParam($secret)
{
$secret = "{$secret}:{$this->_pk}";
return $this->_crypt->encrypt($secret);
}
so the secret is appended with the partner key. and then encrypted using this method in the crypt object (AES_BLOCK_SIZE is set as 16):
public function encrypt($data)
{
$pad = self::AES_BLOCK_SIZE - strlen($data) % self::AES_BLOCK_SIZE;
$data .= str_repeat(chr($pad), $pad);
if (stristr(PHP_OS, 'win') !== false) {
$random_source = MCRYPT_RAND;
} else {
$random_source = MCRYPT_DEV_URANDOM;
}
$iv = mcrypt_create_iv(self::AES_BLOCK_SIZE, $random_source);
mcrypt_generic_init($this->_td, $this->_key, $iv);
$data = $iv . mcrypt_generic($this->_td, $data);
mcrypt_generic_deinit($this->_td);
return self::urlsafe_b64encode($data);
}
this is returned back to the above checkKeys() function which sends the request to the API, which then returns a response. That actual API call is a POST which is easy enough to generate of course, but all those encryption hoops, including the MCRYPT library calls, are where I get stuck trying to determine the equivalent in CF10 or Java.
If I were to get an example thus far, I think I'd stand a chance of replicating the other functions in the crypt object (the ones that are even necessary, which may not be, since some may be built right into the CF encrypt() and decrypt() functions). This seems like a reasonable starting point, however.

Protocol Buffer: From Java to Objective-C using byte[]

I am currently using google's protocol buffers. It works painlessly between Java and C#, however I am running into problems trying to achieve the same use with Obj-c.
The Java WS returns a byte[]. The code that uses the protocol buffer API is simple enough:
productGroup.toByteArray();
I am able to recreate the object with Skeet's C# port, using:
byte[] result = searchWebService.SearchProductsProtocolBuffer(search);
ProductProtoGroup products = ProductProtoGroup.ParseFrom(result);
However, on the obj-c side, I am struggling to work with the return value.
The NSString I receive from the same web service RPC is this:
CmYKEzgwMDAwMUFELTEzMjUyNzk5MTQySUZPT0QgJiBCRV...
I'm not quite sure what to do with this, because I don't know what it is, save it was generated from an array of bytes. I tried parsing it directly to NSData using
NSData* data = [returnValue dataUsingEncoding:NSUTF8StringEncoding];
but on [ProductProtoGroup parseFromData:data];, I get an InvalidProtocolBuffer
I've checked on the Java side what byte string/hexadecimal representations of the original byte[], and it doesn't match the string I receive from the ws.
Hexadecimal is 0-F. Perhaps each byte was converted to a char? No, that doesn't match.
Any help would be appreciated.
Dane

With a fresher mind and some helpful comments, I finally got it.
I was wondering how the web service automagically sent a byte[] in Java, then reconstructed it in C# later, and also how to store this message later.
Turns out the string,
CmYKEzgwMDAwMUFELTEzMjUyNzk5MTQySUZPT0QgJiBCRV...
was indeed what was being sent in the soap envelopes. So inbetween creation of a byte[] in Java and transmission, something was happening. Looking at the message handler in C#,
[System.Web.Services.Protocols.SoapDocumentMethodAttribute("",
RequestNamespace="x", ResponseNamespace="x",
Use=System.Web.Services.Description.SoapBindingUse.Literal,
ParameterStyle=System.Web.Services.Protocols.SoapParameterStyle.Wrapped)]
[return: System.Xml.Serialization.XmlElementAttribute("return",
Form=System.Xml.Schema.XmlSchemaForm.Unqualified,
DataType="base64Binary", IsNullable=true)]
public byte[] searchProductProtocolBuffer([System.Xml.Serialization.XmlElementAttribute(Form=System.Xml.Schema.XmlSchemaForm.Unqualified)] string arg0)
{
object[] results = this.Invoke("searchProductProtocolBuffer", new object[] {
arg0});
return ((byte[])(results[0]));
}
So base64Binary!, which I admittedly am meeting for the first time. Poking around, Skeet says it is the safest way:
How can I safely convert a byte array into a string and back?
So, knowing the encoding, the solution becomes straight forward. Using the algorithm presented in an answer to this question: How do I do base64 encoding on iphone-sdk?, my final code becomes:
NSString* returnValue = [WebServiceUtil processStringReturnValue:value];
NSData* data = [Encoding base64DataFromString:returnValue];
ProductProtoGroup* products = [ProductProtoGroup parseFromData:data];

I know very little about Objective C, but an NSString is no byte array. Have you tried converting the NSString to char* using -[NSString UTF8String]?

Android HMAC-SHA1 Different than Standard Java HMAC-SHA1

I'm having issues with some HMAC on android. I am using the SHA1 algorithm with the following code which shows up all over the web when searching for android hmac-sha1.
String base_string = "This is a test string";
String key = "testKey";
try {
Mac mac = Mac.getInstance("HmacSHA1");
SecretKeySpec secret = new SecretKeySpec(key.getBytes("UTF-8"), mac.getAlgorithm());
mac.init(secret);
byte[] digest = mac.doFinal(base_string.getBytes());
String enc = new String(digest);
// Base 64 Encode the results
String retVal = Base64.encodeBase64String(enc.getBytes());
Log.v(TAG, "String: " + base_string);
Log.v(TAG, "key: " + key);
Log.v(TAG, "result: " + retVal);
} catch (Exception e) {
System.out.println(e.getMessage());
}
To test this code I created a simple standard Java program with it (replacing the Log.v calls with System.out.println calls of course) so I can compare vs. the android version. In both instances I'm using the same test values for the base_string and key.
In addition I have verified the encoded results from the standard Java with some PHP functions and a validation server (using some OAuth tokens). The code works fine in the standard Java program however it does not work in the Android program. I have done a lot of searching and cant figure out what is wrong. Anyone ever experience this?
Here are the results from standard java and android...
Java (and PHP): fH/+pz0J5XcPZH/d608zGSn7FKA=
Android Program: fH/vv73vv709Ce+/vXcPZH/vv73vv71PMxkp77+9FO+/vQ==
Looking into it a bit more I am sure it is the hmac function and not the Base64 encode where it gets messed up as comparing those hmac values the Android version has all sorts of extra spaces and other unknown character symbols vs. the Java program.
Any help is appreciated!

I suppose this is a String encoding problem.
What are you doing here?
String enc = new String(digest);
// Base 64 Encode the results
String retVal = Base64.encodeBase64String(enc.getBytes());
You turn the bytes in a string, and then back to a byte array again (which you then base-64 encode).
Instead, do this:
String retVal = Base64.encodeBase64String(digest);
In general, never ever user String.getBytes() or new String(byte[]) if you want a portable program. And never try to convert an arbitrary byte arrays (which was not a string before) to a string (other than something like Base64).