Develop Reference

iOS and Android AES Encryption (No UINT in Java)

All,
I am new to encryption so I'm not sure all of the information I need to share to get help; but I'll edit this question as I learn more about how to ask this question well :)
I am performing AES encryption on both an iOS and an android app that communicate over bluetooth to a device. I am using AES CTR encryption and it is fully implemented and functional on iOS. The problem I'm running into is that when I convert items such as my IV to a byte array; java bytes are signed and swift bytes are unsigned so while I can encrypt and decrypt my string on Java; it is a different result than what I would see in iOS.
How are other people dealing with this unsigned int issue? I feel like there's got to be some straight-forward thing I'm doing wrong. I'm really not sure what code to post. For android I'm using hex string to byte conversion functions I found here on stack overflow and they are working correctly...they're just signed instead of unsigned so the values are different than the unsigned byte arrays in iOS.
iOS Implementation:
let aesPrivateKey = "********************************"
print("MacAddress:-> \(macAddress)")
var index = 0
let aesPrivateKeyStartIndex = aesPrivateKey.startIndex
let macAddressStartIndex = macAddress.startIndex
//Perform an XOR to get the device key
var deviceKeyArray: Array<Character> = Array(repeating: "?", count: 32)
for _ in macAddress {
let nextPrivateKeyIndex = aesPrivateKey.index(aesPrivateKeyStartIndex, offsetBy: index)
let nextMacAddressIndex = macAddress.index(macAddressStartIndex, offsetBy: index)
let nextPrivateKeyString = String(aesPrivateKey[nextPrivateKeyIndex])
let nextMacAddressString = String(macAddress[nextMacAddressIndex])
let nextPrivateKeyByte = Int(nextPrivateKeyString, radix: 16)
let nextMacAddressByte = Int(nextMacAddressString, radix: 16)
let nextCombinedByte = nextPrivateKeyByte! ^ nextMacAddressByte!
let nextCombinedString = nextCombinedByte.hexString
deviceKeyArray[index] = nextCombinedString[nextCombinedString.index(nextCombinedString.startIndex, offsetBy: 1)]
index+=1
}
while(index < 32) {
let nextPrivateKeyIndex = aesPrivateKey.index(aesPrivateKeyStartIndex, offsetBy: index)
deviceKeyArray[index] = aesPrivateKey[nextPrivateKeyIndex]
index += 1
}
//Convert the device key to a byte array
let deviceKey = "0x" + String(deviceKeyArray)
let deviceKeyByte = Array<UInt8>(hex: deviceKey)
//Convert the password to a byte array
let passwordByte : Array<UInt8> = password.bytes
//Convert the initialization vector to a byte array
let aesIVHex = "0x" + AESIV
let aesIVByte = Array<UInt8>(hex: aesIVHex)
//Encrypt the password
var encrypted = [Unicode.UTF8.CodeUnit]()
do{
encrypted = try AES(key: deviceKeyByte, blockMode: CTR(iv: aesIVByte)).encrypt(passwordByte)
}
catch{
print(error)
}
print("The Encrypted Password Data: \(encrypted)")
let encryptedData = encrypted.toHexString()
//Write password to bluetooth and check result
UserDefaultUtils.setObject(encryptedData as AnyObject, key: userDefaults.password)
DeviceLockManager.shared().isEncrypted = false.
DeviceLockManager.share().setVerifyPasswordForDevice(isGunboxDevice:true)
Android implementation:
System.out.println("ble_ Password:"+str_password+"\nble_ AesKey:"+aesDeviceKey+"\nble_ AesIV:"+aesIV);
byte[] encryptedData = encrypt(
str_password.getBytes(),
Utility.getInstance().hexStringToByteArray(aesDeviceKey),
Utility.getInstance().hexStringToByteArray(aesIV));
String encryptedPassword = Utility.getInstance().bytesToHexString(encryptedData);
System.out.println("ble_ AES Encrypted password " + encryptedPassword);
byte[] decryptedData = decrypt(encryptedData, aesDeviceKey.getBytes(), aesIV.getBytes());
System.out.println("ble_ Cipher Decrypt:"+new String(decryptedData));
//Write password to bluetooth and check result
deviceManager.writePassword(encryptedPassword);
Utility.getInstance().sleep(100);
deviceManager.readPasswordResult();
All input values match exactly until I call the function: hextStringtoByteArray. At this point, the iOS byte arrays are unsigned and the android byte arrays are signed.
Here is that function for reference:
public static byte[] hexStringToByteArray(String s){
byte[] b = new byte[s.length() / 2];
for (int i = 0; i < b.length; i++) {
int index = i * 2;
int v = Integer.parseInt(s.substring(index, index + 2), 16);
b[i] = (byte) v;
}
return b;
}
Sample IV Byte Array:
iOS vs Android:
43, 34, 95, 101, 57, 150, 75, 100, 250, 178, 194, 70, 253, 236, 92, 70
43, 34, 95, 101, 57, -106, 75, 100, -6, -78, -62, 70, -3, -20, 92, 70

You might notice a difference between the two printed arrays because java by default displays a byte as a signed value. But in reality those are actually equal. To make it more clear I'll add a little table with the last 5 values of the example IV array you provided.
|----------------------------------------|
| hex | 46 | FD | EC | 5C | 46 |
|----------------------------------------|
| unsigned | 70 | 253 | 236 | 92 | 70 |
|----------------------------------------|
| signed | 70 | -3 | -20 | 92 | 70 |
|----------------------------------------|
So they are actually the same (bit wise), only printed diffently as they are interpreted as different values. If you want to make sure things are correct, I would suggest looking at a few numbers with a calculator on programming mode. Usually there is a way to set the byte/word length so you can play around with signed vs unsigned interpretation of the same Hexadecimal value (there should also be a bit-representation of the value).
As an alternative I found a small website containing a signed vs unsigned type-bit/hex converter, which will do the trick as well. (make sure you select either char-type, otherwise the signed values will be incorrect)
So in the IV-bytes part of the code there shouldn't be any problem. There might be one however when you create your String using only a byte-array as parameter. e.i:
byte[] decryptedData = decrypt(encryptedData, aesDeviceKey.getBytes(), aesIV.getBytes());
System.out.println("ble_ Cipher Decrypt:" + new String(decryptedData));
Since most likely the used Charset is not UTF-8. (you can determine that by calling Charset#defaultCharset, and check its value). The alternative would be:
new String(decryptedData, StandardCharsets.UTF_8)
or:
new String(decryptedData, "UTF-8");

AES CMAC calculation - output length for host cryptogram incorrect length

I have below function which is supposed to return 8 byte host cryptogram based on length of derived data "L" but I am getting 16 bytes data. Although key is 128 bits, I was expecting BC AESCMAC function will return data based on value of L in the derivation data. If this is not the case, do I need to extract MS 8 Bytes from output. Below is my function -
private String scp03CalculatehostCryptogram(byte[] derivedSMACSessionKey, String hostChallenge, String cardChallenge) throws InvalidKeyException, NoSuchAlgorithmException, NoSuchProviderException, UnsupportedEncodingException {
// Reference : GPC_2.2_D_SCP03_v1.1.1 > 6.2.2.3 Host Authentication Cryptogram - The host cryptogram (8 bytes) is calculated using the data derivation scheme defined in section 4.1.5 with the session key S-MAC and the derivation constant set to “host authentication cryptogram generation”. The length of the cryptogram shall be reflected in the parameter “L” (i.e. '0040'). The “context” parameter shall be set to the concatenation of the host challenge (8 bytes) and the card challenge (8 bytes).
String labelForSMAC = "000000000000000000000001";
String separationIndicator = "00";
String lInteger = "0040";
String counter = "01";
String context = hostChallenge.concat(cardChallenge);
String hostCryptogramDerivationData = labelForSMAC.concat(separationIndicator).concat(lInteger).concat(counter).concat(context);
byte[] hostCryptogramDerivationDataBytes = DatatypeConverter.parseHexBinary(hostCryptogramDerivationData);
System.out.println(" Host Cryptogram Derivation data : "+DatatypeConverter.printHexBinary(hostCryptogramDerivationDataBytes));
Mac aescmac = Mac.getInstance("AESCMAC", "BC");
SecretKey scpENCKeyObject = new SecretKeySpec(derivedSMACSessionKey, "AES");
aescmac.init(scpENCKeyObject);
aescmac.update(hostCryptogramDerivationDataBytes);
byte[] hostCryptogram = aescmac.doFinal();
System.out.println(" Calculated Host Cryptogram : "+DatatypeConverter.printHexBinary(hostCryptogram));
return DatatypeConverter.printHexBinary(hostCryptogram);
}
Output :
Host Cryptogram Derivation data : 0000000000000000000000010000400161BD435249EC20B7AA984A2D47AD4302
Calculated Host Cryptogram : 6F405B9FD1438A4633A4289B618A1FB5
Example - derived smac session key : 47297387E512687FBEB37D1C1F4B8F4C
what am I doing wrong?

The length L is included in the input of the crytogram to make the output of the cryptogram as specific as possible.
Obviously the MAC algorithm won't pay any respect to the input. A MAC simply takes a key, input and then produces a predefined amount of data. Your function is supposed to create the cryptogram. This cryptogram requires the size of the output data L as parameter. So if you're not producing the required amount of output data then that's up to you.
And yes, in general if an output of a PRF (e.g. your function) needs to be resized then the leftmost bytes are taken.

Is this possible to convert EBCDIC Comp-3 file to ASCII file values using java?

I am trying to convert the EBCDIC COMP-3 fields to ASCII values but which is not working.But Binary COMP-3 fields could be converted to ASCII values.Please help me to understand is this possible or not? Even using any other java library is ok for me.I tried and searched may but no concrete answer I could see.
Update:
In my previous one binary should be the one which will work.This what
I received as answer but there was no clarity about EBCDIC COMP-3.
COPYBOOK:
001700 01 EMP-RECORD.
001900 10 EMP-NO PIC 9(10).
002000 10 EMP-NAME PIC X(30).
002100 10 EMP-ADDRESS PIC X(30).
002200 10 EMP-SALARY PIC S9(8)V9(2) COMP-3.
002200 10 EMP-ZIPCODE PIC 9(4).
BINARY COMP-3 file: could be converted
ËÍ>ÁÁ% ,Í_/Ê Ê Â/>Å/%?ÊÁ Á~ ¢|ëá&ç ïçñèá ãñá<à ÊÊ>
EBCDIC COMP-3:not able to convert
0000001001suneel kumar r city e¡5671
Program:
public static void main(String args[]) throws Exception {
String salesFile = "empcompnewbinary.txt";
String copybookName = "EMPCOPYBOOK.txt";
AbstractLine saleRecord;
int fileStructure = Constants.IO_FIXED_LENGTH;
CobolIoProvider ioProvider = CobolIoProvider.getInstance();
AbstractLineReader reader = ioProvider.getLineReader(fileStructure, Convert.FMT_MAINFRAME,
CopybookLoader.SPLIT_NONE, copybookName, salesFile);
while ((saleRecord = reader.read()) != null) {
System.out.print(saleRecord.getFieldValue("EMP-NO").asString() + "-"
+ saleRecord.getFieldValue("EMP-NAME").asString() + "-"
+ saleRecord.getFieldValue("EMP-ADDRESS").asString() + "-"
+ saleRecord.getFieldValue("EMP-SALARY").asDouble() + "-"
+ saleRecord.getFieldValue("EMP-ZIPCODE").asString());
}
reader.close();
}

There is no such thing as an "EBCDIC COMP-3 field", and it has no equivalent in ASCII code points. It is a binary format. So understand you have a record mixed of character and binary formats.
Comp-3 is packed decimal. It can vary a bit on different machine architectures as to where they put the sign nibble and whatnot, but it is a binary format, so any attempt to convert it using character set rules will always fail.
The easiest way to deal with this, by far, is to convert any packed decimal data to a display format, made up of characters. So instead of x'0123456C', you actually convert that to c'01234.56', and then your standard EBCDIC to ASCII conversion will work fine.

Talend - generating n multiple rows from 1 row

Background: I'm using Talend to do something (I guess) that is pretty common: generating multiple rows from one. For example:
ID | Name | DateFrom | DateTo
01 | Marco| 01/01/2014 | 04/01/2014
...could be split into:
new_ID | ID | Name | DateFrom | DateTo
01 | 01 | Marco | 01/01/2014 | 02/01/2014
02 | 01 | Marco | 02/01/2014 | 03/01/2014
03 | 01 | Marco | 03/01/2014 | 04/01/2014
The number of outcoming rows is dynamic, depending on the date period in the original row.
Question: how can I do this? Maybe using tSplitRow? I am going to check those periods with tJavaRow. Any suggestions?

Expanding on the answer given by Balazs Gunics
Your first part is to calculate the number of rows one row will become, easy enough with a date diff function on the to and from dates
Part 2 is to pass that value to a tFlowToIterate, and pick it up with a tJavaFlex that will use it in its start code to control a for loop:
tJavaFlex start:
int currentId = (Integer)globalMap.get("out1.id");
String currentName = (String)globalMap.get("out1.name");
Long iterations = (Long)globalMap.get("out1.iterations");
Date dateFrom = (java.util.Date)globalMap.get("out1.dateFrom");
for(int i=0; i<((Long)globalMap.get("out1.iterations")); i++) {
Main
row2.id = currentId;
row2.name = currentName;
row2.dateFrom = TalendDate.addDate(dateFrom, i, "dd");
row2.dateTo = TalendDate.addDate(dateFrom, i+1, "dd");
End
}
and sample output:
1|Marco|01-01-2014|02-01-2014
1|Marco|02-01-2014|03-01-2014
1|Marco|03-01-2014|04-01-2014
2|Polo|01-01-2014|02-01-2014
2|Polo|02-01-2014|03-01-2014
2|Polo|03-01-2014|04-01-2014
2|Polo|04-01-2014|05-01-2014
2|Polo|05-01-2014|06-01-2014
2|Polo|06-01-2014|07-01-2014
2|Polo|07-01-2014|08-01-2014
2|Polo|08-01-2014|09-01-2014
2|Polo|09-01-2014|10-01-2014
2|Polo|10-01-2014|11-01-2014
2|Polo|11-01-2014|12-01-2014
2|Polo|12-01-2014|13-01-2014
2|Polo|13-01-2014|14-01-2014
2|Polo|14-01-2014|15-01-2014
2|Polo|15-01-2014|16-01-2014
2|Polo|16-01-2014|17-01-2014
2|Polo|17-01-2014|18-01-2014
2|Polo|18-01-2014|19-01-2014
2|Polo|19-01-2014|20-01-2014
2|Polo|20-01-2014|21-01-2014
2|Polo|21-01-2014|22-01-2014
2|Polo|22-01-2014|23-01-2014
2|Polo|23-01-2014|24-01-2014
2|Polo|24-01-2014|25-01-2014
2|Polo|25-01-2014|26-01-2014
2|Polo|26-01-2014|27-01-2014
2|Polo|27-01-2014|28-01-2014
2|Polo|28-01-2014|29-01-2014
2|Polo|29-01-2014|30-01-2014
2|Polo|30-01-2014|31-01-2014
2|Polo|31-01-2014|01-02-2014

You can use tJavaFlex to do this.
If you have a small amount of columns the a tFlowToIterate -> tJavaFlex options could be fine.
In the begin part you can start to iterate, and in the main part you assign values to the output schema. If you name your output is row6 then:
row6.id = (String)globalMap.get("id");
and so on.

I came here as I wanted to add all context parameters into an Excel data sheet. So the solution bellow works when you are taking 0 input lines, but can be adapted to generate several lines for each line in input.
The design is actually straight forward:
tJava –trigger-on-OK→ tFileInputDelimited → tDoSomethingOnRowSet
↓ ↑
[write into a CSV] [read the CSV]
And here is the kind of code structure usable in the tJava.
try {
StringBuffer wad = new StringBuffer();
wad.append("Key;Nub"); // Header
context.stringPropertyNames().forEach(
key -> wad.
append(System.getProperty("line.separator")).
append(key + ";" + context.getProperty(key) )
);
// Here context.metadata contains the path to the CSV file
FileWriter output = new FileWriter(context.metadata);
output.write(wad.toString());
output.close();
} catch (IOException mess) {
System.out.println("An error occurred.");
mess.printStackTrace();
}
Of course if you have a set of rows as input, you can adapt the process to use a tJavaRow instead of a tJava.
You might prefer to use an Excel file as an on disk buffer, but dealing with this file format asks more work at least the first time when you don’t have the Java libraries already configured in Talend. Apache POI might help you if you nonetheless chose to go this way.

java reading numbers, interpreting as octal, want interpreted as string

i am having an issue, where java is reading an array list from a YAML file of numbers, or strings, and it is interpreting the numbers as octal if it has a leading 0, and no 8-9 digit.
is there a way to force java to read the yaml field as a string?
code:
ArrayList recordrarray = (ArrayList) sect.get("recordnum");
if (recordrarray != null) {
recno = join (recordrarray, " ");
}
HAVE ALSO TRIED:
Iterator<String> iter = recordrarray.iterator();
if (iter.hasNext()) recno = " " +String.valueOf(iter.next());
System.out.println(" this recnum:" + recno);
while (iter.hasNext()){
recno += ""+String.valueOf(iter.next()));
System.out.println(" done recnum:" + String.valueOf(iter.next()));
}
the input is such:
061456 changes to 25390
061506 changes to 25414
061559 -> FINE
it took a while to figure out what it was doing, and apparently this is a common issue for java,
ideas?
thanks
edit: using jvyaml
yaml:
22:
country_code: ' '
description: ''
insection: 1
recordnum:
- 061264
type: misc
yaml loading:
import org.jvyaml.YAML;
Map structure = new HashMap();
structure = (Map) YAML.load(new FileReader(structurefn)); // load the structure file

Where are you reading the file? The problem lies in where the file contents are being read. Most likeley the recordarray list contains integers, ie. they have alreadey been parsed. Find the place where the records are being read. Maybe you are doing something like this:
int val = Integer.parseInt(record);
Use this instead:
int val = Integer.parseInt(record, 10);