Develop Reference

HEX generating issue from Java cryptography versus And .NET

I am generating modulus and exponent from Java and .NET system but there are differences in both. I need the out come like .NET in java. Java is adding two extra zeros on hex conversion of modulus and in exponent removing 1 zero but .NET is generating correct. Please see below results from .Net and Java.
If I use toString(16) then its generate below results. toString(16) is not adding two zeros in modulus but removing a zero from exponent where .NET add a 0 in exponent and remove two zeros from modulus which is I want.
String modlusHexString = publicKey.getModulus().toString(16).toUpperCase();
String exponentHexString = publicKey.getPublicExponent().toString(16).toUpperCase();
ModlusHex toString(16): D9B4E023A7CEF604499E184CBA7B7847FE35A824D15FF902EADB952FB54620158A564EFDDB0A66A7647CBDB339359BF6756F5851A73CC1D24859A064DD7AE30B2330F965C54682B10E886D35FE005F42B056C7ABF64D3F6D592AEDC0234417507A0A1432E51C7867E3ACC4867A1AE03EF9E62050180882B18771D5703C8BADCB3AC767CD1A1C0F9344F10B8C82EF5D0ACA4422512EA3ECCB5B71097BDEDAD9BBBE11697D1E61814CF3BBDEB48BDC2C95AA10DFC3F7F794E307D49B5455F928A9BB3ED2F28D6E2974238EFB2D9A822EC1832177CB988206204DF1D9DB7D291E2816576BEBD669184894B526F0B5D10C7D19FA67E79DADDF97D4A3082D4812A27B
ExponentHex toString(16): 10001
I tried below method also to convert BigInteger of modulus and exponent to Hex but no luck-
static String toHex(byte[] ba) {
StringBuilder hex = new StringBuilder(ba.length * 2);
for (byte b : ba) {
hex.append(String.format("%02x", b, 0xff));
}
return hex.toString().toUpperCase();
}
Modlus Hex: 00D9B4E023A7CEF604499E184CBA7B7847FE35A824D15FF902EADB952FB54620158A564EFDDB0A66A7647CBDB339359BF6756F5851A73CC1D24859A064DD7AE30B2330F965C54682B10E886D35FE005F42B056C7ABF64D3F6D592AEDC0234417507A0A1432E51C7867E3ACC4867A1AE03EF9E62050180882B18771D5703C8BADCB3AC767CD1A1C0F9344F10B8C82EF5D0ACA4422512EA3ECCB5B71097BDEDAD9BBBE11697D1E61814CF3BBDEB48BDC2C95AA10DFC3F7F794E307D49B5455F928A9BB3ED2F28D6E2974238EFB2D9A822EC1832177CB988206204DF1D9DB7D291E2816576BEBD669184894B526F0B5D10C7D19FA67E79DADDF97D4A3082D4812A27B
Exponent : 010001
Following is .NET generated HEX of public key modulus and exponent which is correct
.NET
Modulus HEX:
F86020AFD75A03911BE8818BCB506B5DAC2760C68BB46F2A53E10E3E0972A7FFFFF71EAC0E0D73E8FDB4C332C759E781E54C0F1F4637656E6E995873B9580027F49606811C7B5DB458C1BAC3E3D6EB7B77BFE1E55A822F23797E4CB27C7BD88C1B782AEF5235DAE55B937ABB0FFD30AF64F8D69DA07946441D9E4704FA98BD026E00A92851FABC8AB347AB75615ACC8A7CCFDE56B0797DDB70FCA1F28F23F86548AABE6DD89B5CC859BC6352D077F765AAFCF15695215850A8D7E1F9DF187AE0EF1934E096B739E884757F810B3320EFA72BBE2A957CE465E2010A5FD9C96A5F6456658D3BA0DF51B472AFEBEF31C3609B58C6A03C671DA33650039822465179
Exponent : 010001

The problem you are facing results from the behavior of the methods BigInteger.toString(int radix) and BigInteger.toByteArray().
When you call the BigInteger.toString(int radix) method, it returns only the significant digits of the number. So if the value is supposed to be, for example, 05ABFF, it returns only 5ABFF. This is natural when the radix is 10 (we don't expect the big integer 13 to have be converted to something like 013), but this is somewhat counter-intuitive when the radix is 16, as you expect the output to have an even length, exactly two characters for each byte. But that's not how it works.
But when you call your own toHex() method, it is based on the value returned from BigInteger.toByteArray(). Here you have your other problem. This method always returns the number of bytes necessary to represent the number, including a sign bit. Now consider the number 0xD9B4E023. This is actually a negative number if it is considered an integer, but if it is considered as positive by BigInt, you need an extra byte that represents the sign. Hence the additional byte that translates to 00 in your method.
I can think of two possible solutions:
static String toHex(byte[] ba) {
StringBuilder hex = new StringBuilder(ba.length * 2);
boolean skipZeroBytes = true;
for (byte b : ba) {
// As soon as we hit the first non-zero byte, we stop skipping bytes
if (b != 0) {
skipZeroBytes = false;
}
// If the current byte is zero, and we are in skipping mode, skip
if (skipZeroBytes) {
continue;
}
hex.append(String.format("%02X", b, 0xff));
}
if (skipZeroBytes) {
// If we are still in skipping mode, it means all the bytes in the
// array were zero and we skipped them all. So just return the
// representation of a zero.
return "00";
} else {
return hex.toString();
}
}
What we do here is skip all the initial zero bytes until we hit the first non-zero byte, and only then we start interpreting it. Small note: using the format %02X with a capital X gives you uppercase hexadecimal digits and saves the need to call toUpperCase() later.
The other, simpler method is to add the missing zero to the result of BigInteger.toString(int radix):
static String toHex2(BigInteger bi) {
String hex = bi.toString(16).toUpperCase();
if (hex.length() % 2 == 1) {
return "0" + hex;
} else {
return hex;
}
}

Represent long in least amount of characters

I need to represent both very large and small numbers in the shortest string possible. The numbers are unsigned. I have tried just straight Base64 encode, but for some smaller numbers, the encoded string is longer than just storing the number as a string. What would be the best way to most optimally store a very large or short number in the shortest string possible with it being URL safe?

I have tried just straight Base64 encode, but for some smaller numbers, the encoded string is longer than just storing the number as a string
Base64 encoding of binary byte data will make it longer, by about a third. It is not supposed to make it shorter, but to allow safe transport of binary data in formats that are not binary safe.
However, base 64 is more compact than decimal representation of a number (or of byte data), even if it is less compact than base 256 (the raw byte data). Encoding your numbers in base 64 directly will make them more compact than decimal. This will do it:
private static final String base64Chars =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_";
static String encodeNumber(long x) {
char[] buf = new char[11];
int p = buf.length;
do {
buf[--p] = base64Chars.charAt((int)(x % 64));
x /= 64;
} while (x != 0);
return new String(buf, p, buf.length - p);
}
static long decodeNumber(String s) {
long x = 0;
for (char c : s.toCharArray()) {
int charValue = base64Chars.indexOf(c);
if (charValue == -1) throw new NumberFormatException(s);
x *= 64;
x += charValue;
}
return x;
}
Using this encoding scheme, Long.MAX_VALUE will be the string H__________, which is 11 characters long, compared to its decimal representation 9223372036854775807 which is 19 characters long. Numbers up to about 16 million will fit in a mere 4 characters. That's about as short as you'll get it. (Technically there are two other characters which do not need to be encoded in URLs: . and ~. You can incorporate those to get base 66, which would be a smidgin shorter for some numbers, although that seems a bit pedantic.)

To extend on Stephen C's answer, here is a piece of code to convert to base 62 (but you can increase this by adding more characters to the digits String (just pick what characters are valid for you):
public static String toString(long n) {
String digits = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
int base = digits.length();
String s = "";
while (n > 0) {
long d = n % base;
s = digits.charAt(d) + s;
n = n / base;
}
return s;
}
This will never result in the string representation being longer than the digit one.

Assuming that you don't do any compression, and that you restrict yourself to URL safe characters, then the following procedure will give you the most compact encoding possible.
Make a list of all URL safe characters
Count them. Suppose you have N.
Represent your number in base N, representing 0 by the first character, 1 by the 2nd and so on.
So, what about compression ...
If you assume that the numbers you are representing are uniformly distributed across their range, then there is no real opportunity for compression.
Otherwise, there is potential for compression. If you can reduce the size of the common numbers then you can typically achieve a saving by compression. This is how Huffman encoding works.
But the downside is that compression at this level is not perfect across the range of numbers. It reduces the size of some numbers, but it inevitably increases the size of others.
So what does this mean for your use-case?
I think it means that you are looking at the problem the wrong way. You should not be aiming for a minimal encoded size for every number. You should be aiming to minimize the size on average ... averaged over the actual distribution of your numbers.

How many bytes does a string contains?

public class ClassToTestSnippets {
private static ClassToTestSnippets ctts;
public static void main(String[] args) {
ctts = new ClassToTestSnippets();
ctts.testThisMethod();
}
public void testThisMethod() {
System.out.println("\u2014".length()); //answer is 1
}
}
Above code prints 1. But \u2014 is E2 80 94 i.e. 3 bytes. How do I know how many bytes does a string contains?

Depends. What encoding do you want to use?
System.out.println("äö".getBytes("UTF-8").length);
Prints 4, but if I change UTF-8 to ISO-8859-1 (for example), it'll print 2. Other encodings may print other values (try UTF-32).

Internally - it contains (number of chars) * 2 bytes, as each char in Java takes up two bytes (a normal character in Java is 16 bits unicode). The actual bytes are 0x20 and 0x14.
However, the length function returns the number of characters, not the number of bytes.

Decode encoded text with ASCII

I'm using below method to encode given text.
static long encodeText(String text) {
long l = 31;
for (int i = 0; i < text.length(); i++) {
l = l * 47 + text.getBytes()[i] % 97;
}
return l;
}
When i call above method as encodeText("stackoverflow"), return the encoded text 3818417496786881978.
Now i want to provide encoded text and get String value. For example, if i give 3818417496786881978 to decodeText(long encoded), i need to get output as stackoverflow.
static String decodeText(long encoded) {
String str = null;
// decode steps here
return str;
}
How can i do this ?

Think this through logically: the clear text "stackoverflow" when represented as 7-bit ASCII represents 13 times 7 bits (=91 bits) of information. Thats more than a long (64 bits) can hold. So your encoding will lose information, making decoding impossible.
That should be also quite apparent from the formula you use:
l = l * 47 + text.getBytes()[i] % 97;
For each charcter you get a number between 0 and 96 (you already loosing information in the modulo, reducing information to 97 possible characters (e.g. you cannot distinguish between the bytes 1 and 98 after the modulo any more). Then you multiply your long by a number less than 97 (47), so two consecutive characters will overlap in terms of information distribution in the cyphertext.
Finally, after adding more an more characters, the long simply overflows and the topmost bits are simply lost.
In conclusion: If you want to decode the cyphertext ever again, fix the loss of information in these three places.

Reading Image from j2me to c++

I am learning c++ in order to create a little application which displays image stream. The images are coming from a j2me device which aren't stored as a file (so I just have the bytes).
I am thinking I need to send the size of the image as an int first so then the client knows how much to read for that particular image in the stream.
My problem is the size is always way too big - not the size of the image when I initially just try to read the size (I send this length in java server in a socket.write(int) and have tried dataoutputstream.writeInt). I will post some code as it's probably pretty simple.
Why is the size different to what I send?
ssize_t
readLine(int fd, char *buffer, size_t n)
{
ssize_t numRead, tt; /* # of bytes fetched by last read() */
size_t totRead; /* Total bytes read so far */
char *buf;
char ch;
if (n <= 0 || buffer == NULL) {
return -1;
}
buf = buffer; /* No pointer arithmetic on "void *" */
totRead = 0;
for (;;) {
numRead = read(fd, &ch, 1);
tt += numRead;
if (numRead == -1) {
return -1; /* Some other error */
} else if (numRead == 0) { /* EOF */
if (totRead == 0) /* No bytes read; return 0 */
return 0;
else /* Some bytes read; add '\0' */
break;
} else { /* 'numRead' must be 1 if we get here */
if (totRead < n - 1) { /* Discard > (n - 1) bytes */
totRead++;
*buf++ = ch;
}
if (ch == '\n')
break;
}
}
printf("read line %s ", buf);
fflush(stdout);
int test = (int)buf;
printf("read line int %i ", tt);
fflush(stdout);
*buf = '\0';
return totRead;
}

WBXML defines a platform independent way to write int values: Multy-byte integers.
A multi-byte integer consists of a series of octets, where the most significant bit is the continuation flag and the remaining seven bits are a scalar value. The continuation flag indicates that an octet is not the end of the multi-byte sequence. A single integer value is encoded into a sequence of N octets. The first N-1 octets have the continuation flag set to a value of one (1). The final octet in the series has a continuation flag value of zero (0).
The remaining seven bits in each octet are encoded in a big-endian order, eg, most significant bit first. The octets are arranged in a big-endian order, eg, the most significant seven bits are transmitted first. In the situation where the initial octet has less than seven bits of value, all unused bits must be set to zero (0).
For example, the integer value 0xA0 would be encoded with the two-byte sequence 0x81 0x20. The integer value 0x60 would be encoded with the one-byte sequence 0x60.
I did it for Java ME and Bada but it is pretty straightforward to implement in any language.

Your reading code handles text files, it works one char after the other, it checks for newlines, etc.
The image ("so i just have the bytes") seem to be binary data. When you interpret binary data as text, you get all sorts of random errors. These binary data may include for example a "\n", when the value of a pixel happen to be 13. It may also include "\0", which will end the string before the real end.
When store the size first, you send it as int, which is represent with 4 bytes. When you read it as 4 separate characters, you get some garbage.
You need also beware of order/endian of bytes. java uses "network order", on x86, C may read it just the other way around.
You are using the old C standard lib. It may be easier to use the C++ iostreams.