I use Java and Log4j..
I want to log a string with german special characters, like for example Ü Ä ä etc..
But in my LogFile it appears like this:
<E4><FC><F6>
log4j.properties
log4j.rootLogger = ALL, rollingFile
log4j.appender.rollingFile=org.apache.log4j.RollingFileAppender
log4j.appender.rollingFile.File=/home/tomcat/logs/debug.log
log4j.appender.rollingFile.MaxFileSize=10MB
log4j.appender.rollingFile.MaxBackupIndex=2
log4j.appender.rollingFile.layout = org.apache.log4j.PatternLayout
log4j.appender.rollingFile.layout.ConversionPattern=%d %p %c - %m%n
log4j.appender.rollingFile.encoding=UTF-8
You should try the below code for storing and retriving the values in the unicode format
import java.io.UnsupportedEncodingException;
public class Test {
public static void printBytes(byte[] array, String name) {
for (int k = 0; k < array.length; k++) {
System.out.println(name + "[" + k + "] = " + "0x"
+ UnicodeFormatter.byteToHex(array[k]));
}
}
public static void main(String[] args) {
System.out.println(System.getProperty("file.encoding"));
String original = new String("A" + "\u00ea" + "\u00f1" + "\u00fc" + "C");
System.out.println("original = " + original);
System.out.println();
try {
byte[] utf8Bytes = original.getBytes("UTF8");
byte[] defaultBytes = original.getBytes();
String roundTrip = new String(utf8Bytes, "UTF8");
System.out.println("roundTrip = " + roundTrip);
System.out.println();
printBytes(utf8Bytes, "utf8Bytes");
System.out.println();
printBytes(defaultBytes, "defaultBytes");
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
} // main
}
class UnicodeFormatter {
static public String byteToHex(byte b) {
// Returns hex String representation of byte b
char hexDigit[] = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
'a', 'b', 'c', 'd', 'e', 'f' };
char[] array = { hexDigit[(b >> 4) & 0x0f], hexDigit[b & 0x0f] };
return new String(array);
}
static public String charToHex(char c) {
// Returns hex String representation of char c
byte hi = (byte) (c >>> 8);
byte lo = (byte) (c & 0xff);
return byteToHex(hi) + byteToHex(lo);
}
}
Output
Cp1252
original = AêñüC
roundTrip = AêñüC
utf8Bytes[0] = 0x41
utf8Bytes[1] = 0xc3
utf8Bytes[2] = 0xaa
utf8Bytes[3] = 0xc3
utf8Bytes[4] = 0xb1
utf8Bytes[5] = 0xc3
utf8Bytes[6] = 0xbc
utf8Bytes[7] = 0x43
defaultBytes[0] = 0x41
defaultBytes[1] = 0xea
defaultBytes[2] = 0xf1
defaultBytes[3] = 0xfc
defaultBytes[4] = 0x43
According to the most posted issues about encoding with Log4J there doesnt seem to be any known issues, therefor i assume you are using a wrong encoding while opening the file, try to check the editor and system encoding maby you will find there an issue.
Related
I am trying to encrypt and decrypt a string of data using 3DES and it is working fine. However I want that the size of the data after encryption to be limited to length of 16 bits.
This is the code I am referring from https://gist.github.com/riversun/6e15306cd6e3b1b37687a0e5cec1cef1 :
import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
public class DesedeCrypter {
private static final String CRYPT_ALGORITHM = "DESede";
private static final String PADDING = "DESede/CBC/NoPadding";
private static final String CHAR_ENCODING = "UTF-8";
private static final byte[] MY_KEY = "5oquil2oo2vb63e8ionujny6".getBytes();//24-byte
private static final byte[] MY_IV = "3oco1v52".getBytes();//8-byte
public static void main(String[] args) {
String srcText = "M3A1B2C3D4HHG393";
final DesedeCrypter crypter = new DesedeCrypter();
String encryptedText = crypter.encrypt(srcText);
System.out.println("sourceText=" + srcText + " -> encryptedText=" + encryptedText + "\n");
System.out.println("encrypted-text=" + encryptedText + " -> decrypted-text(source text)="
+ crypter.decrypt(encryptedText));
}
public String encrypt(String text) {
String retVal = null;
try {
final SecretKeySpec secretKeySpec = new SecretKeySpec(MY_KEY, CRYPT_ALGORITHM);
final IvParameterSpec iv = new IvParameterSpec(MY_IV);
final Cipher cipher = Cipher.getInstance(PADDING);
cipher.init(Cipher.ENCRYPT_MODE, secretKeySpec, iv);
final byte[] encrypted = cipher.doFinal(text.getBytes(CHAR_ENCODING));
retVal = new String(encodeHex(encrypted));
} catch (Exception e) {
e.printStackTrace();
}
return retVal;
}
public String decrypt(String text) {
String retVal = null;
try {
final SecretKeySpec secretKeySpec = new SecretKeySpec(MY_KEY, CRYPT_ALGORITHM);
final IvParameterSpec iv = new IvParameterSpec(MY_IV);
final Cipher cipher = Cipher.getInstance(PADDING);
cipher.init(Cipher.DECRYPT_MODE, secretKeySpec, iv);
final byte[] decrypted = cipher.doFinal(decodeHex(text.toCharArray()));
retVal = new String(decrypted, CHAR_ENCODING);
} catch (Exception e) {
e.printStackTrace();
}
return retVal;
}
private byte[] decodeHex(char[] data) throws Exception {
int len = data.length;
if ((len & 0x01) != 0) {
throw new Exception("Odd number of characters.");
}
byte[] out = new byte[len >> 1];
// two characters form the hex value.
for (int i = 0, j = 0; j < len; i++) {
int f = toDigit(data[j], j) << 4;
j++;
f = f | toDigit(data[j], j);
j++;
out[i] = (byte) (f & 0xFF);
}
return out;
}
private int toDigit(char ch, int index) throws Exception {
int digit = Character.digit(ch, 16);
if (digit == -1) {
throw new Exception("Illegal hexadecimal character " + ch + " at index " + index);
}
return digit;
}
private char[] encodeHex(byte[] data) {
final char[] DIGITS = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f' };
int l = data.length;
char[] out = new char[l << 1];
// two characters form the hex value.
for (int i = 0, j = 0; i < l; i++) {
out[j++] = DIGITS[(0xF0 & data[i]) >>> 4];
out[j++] = DIGITS[0x0F & data[i]];
}
return out;
}
}
Currently this is the output I am getting:
sourceText=M3A1B2C3D4HHG393 -> encryptedText=afc1d48ea5cc703253cbc1a88a198103
encrypted-text=afc1d48ea5cc703253cbc1a88a198103 -> decrypted-text(source text)=M3A1B2C3D4HHG393
Is there any way that the size of the encryptedText be limited to 16 as I want to add the encrypted text back into a message which has 16 digits space for encrypted text.
Please suggest some way or any other change that is required to achieve this. Thanks !
For one, I highly recommend not supporting (3)DES anymore, as it's officially unsecure in favour of AES/ChaCha, which I must say before answering this question.
3DES has a block size of 64 bits (or 8 bytes). With that also comes that encryption ≠ compression. So if you want to encrypt 16 bytes, provide 16 bytes of input, unless:
You apply a padding scheme (which it appears you're not doing)(taken from the DESede/CBC/NoPadding
You apply a (random) initialisation vector (which it appears you're doing)
The latter one should, but I'm not to sure of Android's implementation, create a 64 bits (8 byte) iv, as the iv should be as big as the block size of the cipher (again, 64 bits for 3DES).
So if you want 16 bytes of output, you can, and should, only provide 8 bytes to encrypt. Now, if 8 bytes is not enough, you might choose to drop the iv from being in the ciphertext, which you could do if you use a fixed iv (such as an all zero iv) with random keys, as reusing the same key with iv is not secure.
Now if you do take security in consideration, keep in mind that AES has a block size of 16 bytes. AES and ChaCha come with the same constraints regarding the iv.
Then you might also might want to consider changing the message (protocol) instead, so it can take more than 16 bytes of data or use those 16 bytes in such a way that it indicates that there is more data to handle, as like an attachment to an e-mail.
How to convert hex string to ansi (window 1252) and ansi (window 1252)to hex string in Java.
python (Works perfectly)
q = "hex string value"
x = bytes.fromhex(q).decode('ANSI')
a = x.encode("ANSI")
a = a.hex()
if q==a:
print("Correct")
Java (This code has a problem)
String hexOri = "hex string value";
StringBuilder output = new StringBuilder();
for (int i = 0; i < hexOri.length(); i+=2) {
String str = hexOri.substring(i, i+2);
output.append((char)Integer.parseInt(str, 16));
}
System.out.println("ANSI = " + output);
char [] chars = output.toString().toCharArray();
StringBuffer hexOutput = new StringBuffer();
for(int i = 0; i < chars.length; i++){
hexOutput.append(Integer.toHexString((int)chars[i]));
}
System.out.println("HexOutput = " + hexOutput.toString());
System.out.println(hexOri.equals(hexOutput.toString()));
Output from Python
Correct
Expected Output from Python
Correct
Output from Java
False
Expected Output from Java
Correct
In java the strings are encoded in UTF-16, so you can't read simply read/write the bytes of a string to get the encoding representation you desire.
You should use String#getBytes(String str, String charset) to get the string converted in the encoding you need and serialized to a byte array.
The same thing must be done to decode a byte array, using new String(buffer,encoding).
In both cases if you use the method without the charset it will use the default encoding for the JVM instance (which should be the system charset).
public static void main(String[] args) {
String str = "\tSome text [à]";
try {
System.out.println(str); // Some text [à]
String windowsLatin1 = "Cp1252";
String hexString = toHex(windowsLatin1, str);
System.out.println(hexString); // 09536f6d652074657874205be05d
String winString = toString(windowsLatin1, hexString);
System.out.println(winString); // Some text [à]
} catch (UnsupportedEncodingException e) {
// Should not happen.
}
}
public static String toString(String encoding, String hexString) throws UnsupportedEncodingException {
int length = hexString.length();
byte [] buffer = new byte[length/2];
for (int i = 0; i < length ; i+=2) {
String hexVal = hexString.substring(i,i+2);
byte code = (byte) Integer.parseInt(hexVal,16);
buffer[i/2]=code;
}
String winString = new String(buffer,encoding);
return winString;
}
public static String toHex(String encoding, String str) throws UnsupportedEncodingException {
byte[] bytes = str.getBytes(encoding);
StringBuilder builder = new StringBuilder();
for (int i = 0; i < bytes.length; i++) {
byte b = bytes[i];
String hexChar = Integer.toHexString(b & 0xff);
if(hexChar.length()<2) {
builder.append('0');
}
builder.append(hexChar);
}
String hexString = builder.toString(); // 09536f6d652074657874205be05d
return hexString;
}
I have to sum the ASCII in a string and put the last character of the sum in a byte to send with the string on Bluetooth.
es: String s = "R002"
sum: R+0+0+2 = 000000000000000000000000000000000000c3a4 = ä
I try to send R(52)+0(30)+0(30)+2(32)+¤(a4)
but I send R(52)+0(30)+0(30)+2(32)+Ä(c3)+¤(a4) ,
In which way can i send ¤ without Ä?
the code:
String pergolato = "ä";
String pesto= String.format("%040x", new BigInteger(1, pergolato.substring(0, 1).getBytes(/*YOUR_CHARSET?*/)));
int zaino = Integer.parseInt(pesto, 16);
char c = (char) (zaino & 0xff);
String sum="R002"+c;
for(int i=0;i<sum.length();i++){
String s= String.format("%040x", new BigInteger(1, sum.substring(i, i+1).getBytes(/*YOUR_CHARSET?*/)));
Log.i(TAG, sum.charAt(i)+" "+s);
}
the LogCat:
R 0000000000000000000000000000000000000052
0 0000000000000000000000000000000000000030
0 0000000000000000000000000000000000000030
2 0000000000000000000000000000000000000032
¤ 000000000000000000000000000000000000c2a4
I modified the code a bit for my purpose.
Results with different encoding :
UTF-8 --> ¤ 000000000000000000000000000000000000c2a4
UTF-16 --> ¤ 00000000000000000000000000000000feff00a4
UTF-32 --> ¤ 00000000000000000000000000000000000000a4
import java.math.BigInteger;
import java.nio.charset.Charset;
public class Test2 {
public static void main(String[] args) {
String pergolato = "ä";
String pesto= String.format("%040x", new BigInteger(1, pergolato.getBytes()));
int zaino = Integer.parseInt(pesto, 16);
char c = (char) (zaino & 0xff);
String sum="R002"+c;
for(int i=0;i<sum.length();i++){
String s= String.format("%040x", new BigInteger(1, sum.substring(i, i+1).getBytes(Charset.forName("UTF-32"))));
System.out.println(sum.charAt(i)+" "+s);
}
}
}
resolved:
somma=(char) (somma%128);
I want to use percent value encoding/decoding for some string operations in my Android application. I don't want to use a URI encode/decode function pair because I want to encode every character, not just the ones that are encoded when encoding URI's for making web requests. Are there any built-in functions in the Android libraries or Java libraries that can do this?
-- roschler
There's nothing built in to the API to do this directly, but it's pretty simple. It's better to use a specific character encoding (like UTF-8) to convert characters to bytes. This should do the trick for encoding:
static final String digits = "0123456789ABCDEF";
static void convert(String s, StringBuffer buf, String enc)
throws UnsupportedEncodingException {
byte[] bytes = s.getBytes(enc);
for (int j = 0; j < bytes.length; j++) {
buf.append('%');
buf.append(digits.charAt((bytes[j] & 0xf0) >> 4));
buf.append(digits.charAt(bytes[j] & 0xf));
}
}
Oh yes, you asked for decoding as well:
static String decode(String s, String enc)
throws UnsupportedEncodingException {
StringBuffer result = new StringBuffer(s.length());
ByteArrayOutputStream out = new ByteArrayOutputStream();
for (int i = 0; i < s.length();) {
char c = s.charAt(i);
if (c == '%') {
out.reset();
do {
if (i + 2 >= s.length()) {
throw new IllegalArgumentException(
"Incomplete trailing escape (%) pattern at " + i);
}
int d1 = Character.digit(s.charAt(i + 1), 16);
int d2 = Character.digit(s.charAt(i + 2), 16);
if (d1 == -1 || d2 == -1) {
throw new IllegalArgumentException(
"Illegal characters in escape (%) pattern at " + i
+ ": " + s.substring(i, i+3));
}
out.write((byte) ((d1 << 4) + d2));
i += 3;
} while (i < s.length() && s.charAt(i) == '%');
result.append(out.toString(enc));
continue;
} else {
result.append(c);
}
i++;
}
}
This is pretty trivial, and doesn't need library functions:
public static String escapeString(String input) {
String output = "";
for (byte b : input.getBytes()) output += String.format("%%%02x", b);
return output;
}
public static String unescapeString(String input) {
String output = "";
for (String hex: input.split("%")) if (!"".equals(hex)) output += (char)Integer.parseInt(hex, 16);
return output;
}
public static String unescapeMultiByteString(String input, String charset) {
ByteArrayOutputStream output = new ByteArrayOutputStream();
String result = null;
for (String hex: input.split("%")) if (!"".equals(hex)) output.write(Integer.parseInt(hex, 16));
try { result = new String(output.toByteArray(), charset); }
catch (Exception e) {}
return result;
}
I have been working on this for hours, but I can't get it to work.
Basically I am developing a REST client in Java for a REST server in PHP. Both the client and the server have to compute the md5 of a string and the server will compare them for authentication (kinda).
On the server, the PHP code is:
md5("getTokenapi_keybf8ddfs845jhre980543jhsjfro93fd8capi_ver1tokeniud9ER£jdfff");
that generates:
4d7b2e42c3dfd11de3e77b9fe2211b87
Nice!
Here is the code for the client:
import java.security.*;
....
String s = "getTokenapi_keybf8ddfs845jhre980543jhsjfro93fd8capi_ver1tokeniud9ER£jdfff";
byte[] bytesOfMessage = s.getBytes("UTF-8");
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] thedigest = md.digest(bytesOfMessage);
System.out.println("String2: " + thedigest);
System.out.println("String3: " + new String(thedigest));
That generates:
String2: [B#42e816
String3: M{.B�����{��!�
How can I get Java to compute the md5 sum the same way PHP does, please?
Thanks,
Dan
Give this a try:
public static String md5(String input) throws NoSuchAlgorithmException {
String result = input;
if(input != null) {
MessageDigest md = MessageDigest.getInstance("MD5"); //or "SHA-1"
md.update(input.getBytes());
BigInteger hash = new BigInteger(1, md.digest());
result = hash.toString(16);
while(result.length() < 32) { //40 for SHA-1
result = "0" + result;
}
}
return result;
}
code from http://web.archive.org/web/20140209230440/http://www.sergiy.ca/how-to-make-java-md5-and-sha-1-hashes-compatible-with-php-or-mysql/
Found myself:
import java.math.BigInteger;
..
public static String md5(String input) throws NoSuchAlgorithmException {
String result = input;
if(input != null) {
MessageDigest md = MessageDigest.getInstance("MD5"); //or "SHA-1"
md.update(input.getBytes());
BigInteger hash = new BigInteger(1, md.digest());
result = hash.toString(16);
if ((result.length() % 2) != 0) {
result = "0" + result;
}
}
return result;
}
Source: http://www.sergiy.ca/how-to-make-java-md5-and-sha-1-hashes-compatible-with-php-or-mysql/
You are outputting the raw md5 output, which is just a bunch of bytes. You would get the same result in php if you said md5("some string", true).
You need to convert the bytes to ascii characters instead.
You need to convert the result into the HEX representation. This is how it is done in Fast MD5 library:
private static final char[] HEX_CHARS = { '0', '1', '2', '3', '4', '5',
'6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f', };
/**
* Turns array of bytes into string representing each byte as unsigned hex
* number.
*
* #param hash
* Array of bytes to convert to hex-string
* #return Generated hex string
*/
public static String asHex(byte hash[]) {
char buf[] = new char[hash.length * 2];
for (int i = 0, x = 0; i < hash.length; i++) {
buf[x++] = HEX_CHARS[(hash[i] >>> 4) & 0xf];
buf[x++] = HEX_CHARS[hash[i] & 0xf];
}
return new String(buf);
}
So you will need to call System.out.println("String3: " + asHex(thedigest));
if you use spring security framework , just do :
import org.springframework.security.authentication.encoding.*
new Md5PasswordEncoder().encodePassword("myWord",null)
The same result as PHP::md5(). I confirm
See more examples