I want to transfer a file from C# to a java webservice which accepts base64 strings. The problem is that when I encode the file using the c# Convert class, it produces a string based on a little endian unsigned byte[].
In Java byte[] are signed / big endian. When I decode the delivered string, I get a different byte[] and therefor the file is corrupt.
How can I encode a byte[] in C# to a base64, which is equal to the byte[] that is decoded in java using the same string?
C# side:
byte[] attachment = File.ReadAllBytes(#"c:\temp\test.pdf");
String attachmentBase64 = Convert.ToBase64String(attachment, Base64FormattingOptions.None);
Java side:
#POST
#Path("/localpdf")
#Consumes("text/xml")
#Produces("text/xml")
public String getExtractedDataFromEncodedPdf(#FormParam("file") String base64String) {
if(base64String == null) return null;
byte[] data = Base64.decodeBase64(base64String.getBytes());
FileOutputStream ms;
try {
ms = new FileOutputStream(new File("C:\\Temp\\test1234.pdf"));
ms.write(data);
ms.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
File test1234.pdf is corrupt.
"Signed" and "big-endian" are very different things, and I believe you're confusing yourself.
Yes, bytes are signed in Java and unsigned in C# - but I strongly suspect you're getting the same actual bits in both cases... it's just that a bit pattern of (say) 11111111 represents 255 in C# and -1 in Java. Unless you're viewing the bytes as numbers (which is rarely useful) it won't matter - it certainly doesn't matter if you just use the bytes to write out a file on the Java side.
Related
I have a file /logs/file.log which I currently need to get from Point A to point B to point C.
Currently, I'm getting the file, streaming it to point B then saving it to a temporary directory and file, then I stream it to point C (browser) to download.
How can I stream this directly from point A via point B to C? Saving these files can be inefficient because they can be quite large.
Encode the bytes of your file in a Base64 encoded string. Do this in A and then send it over to B. The moment B receives this string, it simply forwards it to C and C decodes it. Here is an example of how to encode (and decode) a file into a Base64 String:
import java.util.Base64;
import java.nio.file.Files;
private static String encodeFileToBase64(File file) {
try {
byte[] fileContent = Files.readAllBytes(file.toPath());
return Base64.getEncoder().encodeToString(fileContent);
} catch (IOException e) {
throw new IllegalStateException("Could not perform the encoding." + file, e);
}
}
private static byte[] decodeBase64String(String base64String) {
byte[] decodedBytes = Base64.getDecoder().decode(base64String);
return decodedBytes;
}
I am stuck with conversion of base 64 binary converted using shptosql into sql server. now I want to convert the data into postgres, which supports wkb- well known binary. How can I convert base64 bytes into suppoorted format which is well known binary in java? I have decoded base64 but wkb is still far from my point.
Part 1 : Read XML Part (Done)
Part 2 : Extract Binary data Part (Done)
Part 3 : Convert into wkb (Left)
Part 4 : Store wkb in postgresSQL
File f = new File(request.getRealPath("/")+"themes\\ProfileImages\\temp.xml");
ArrayList<byte[]> byteArray = new ArrayList<byte[]>();
if(f.exists()){
List<String> list = geoserver.getShapes(f);
for(String s : list){
BASE64Decoder decoder = new BASE64Decoder();
byte[] decodedBytes=null;
try {
decodedBytes = decoder.decodeBuffer(s);
byteArray.add(decodedBytes);
//Here I want to convert bytes to wkb format and want to save in postgres
} catch (IOException e) {
e.printStackTrace();
}
}
List<String> objects =geoserver.getObjectIds(f);
List<ShapeFile> shapes = new ArrayList<ShapeFile>();
for(int i=0;i<byteArray.size();i++){
try {
shapes.add(new ShapeFile(objects.get(i), byteArray.get(i),IOUtils.toString(byteArray.get(i))));
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Part 3 is left for me.
Can someone please tell me how and using which libraries I can do that?
I'm trying to decompress a json object in Java that was initially compressed in PHP. Here's how it gets compressed into PHP:
function zip_json_encode(&$arr) {
$uncompressed = json_encode($arr);
return pack('L', strlen($uncompressed)).gzcompress($uncompressed);
}
and decoded (again in PHP):
function unzip_json_decode(&$data) {
$uncompressed = #gzuncompress(substr($data,4));
return json_decode($uncompressed, $array_instead_of_object);
}
That gets put into MySQL and now it must be pulled out of the db by Java. We pull it out from the ResultSet like this:
String field = rs.getString("field");
I then pass that string to a method to decompress it. This is where it falls apart.
private String decompressHistory(String historyString) throws SQLException {
StringBuffer buffer = new StringBuffer();
try {
byte[] historyBytes = historyString.substring(4).getBytes();
ByteArrayInputStream bin = new ByteArrayInputStream(historyBytes);
InflaterInputStream in = new InflaterInputStream(bin, new Inflater(true));
int len;
byte[] buf = new byte[1024];
while ((len = in.read(buf)) != -1) {
// buf should be decoded, right?
}
} catch (IOException e) {
e.getStackTrace();
}
return buffer.toString();
}
Not quite sure what's going wrong here, but any pointers would be appreciated!
You need to get rid of the true in Inflater(true). Use just Inflater(). The true makes it expect raw deflate data. Without the true, it is expecting zlib-wrapped deflate data. PHP's gzcompress() produces zlib-wrapped deflate data.
Gzipped data is binary, byte[]. Using String, Unicode text, not only needs conversion, but is faulty.
For instance this involves a conversion:
byte[] historyBytes = historyString.substring(4).getBytes();
byte[] historyBytes = historyString.substring(4).getBytes("ISO-8859-1");
The first version uses the default platform encoding, making the application non-portable.
The first to-do is to use binary data in the database as VARBINARY or BLOB.
ImputStream field = rs.getBinaryStream("field");
try (InputStream in = new GZIPInputStream(field)) {
...
}
Or so. Mind the other answer.
In the end, neither of the above solutions worked, but both have merits. When we pulled the data out of mysql and cast it to bytes we have a number of missing character bytes (67). This made it impossible to decompress on the java side. As for the answers above. Mark is correct that gzcompress() uses zlib and therefore you should use the Inflater() class in Java.
Joop is correct that the data conversion is faulty. Our table was too large to convert it to varbinary or blob. That may have solved the problem, but didn't work for us. We ended up having java make a request to our PHP app, then simply unpacked the compressed data on the PHP side. This worked well. Hopefully this is helpful to anyone else that stumbles across it.
I am writing an App which does the following with a given String str:
encode(encrypt(encode(stringToBytearray(str))));
The Server receives the Base64 encoded String, which is then decoded -> decrypted -> decoded, to get the sent String str from the App.
Unfortunately it doesnt work for all Strings, long Strings lead to a long Base64 String and my Server throws the following Exception:
Exception in thread "main" java.lang.IllegalArgumentException: Illegal base64 character 5b
at java.util.Base64$Decoder.decode0(Base64.java:714)
at java.util.Base64$Decoder.decode(Base64.java:526)
at Main.decode(Main.java:113)
at Main.main(ain.java:33)
The String has the format "[string, string, ..., string]" without "s.
Like I mentioned above, for Strings which are not too long (sorry I cant quantify the length yet), this works. So I think I implemented it right.
Whats weird is, that if I dont send it, but decode(decrypt(decode(stringToBytearray(str)))); the String on the Device itself, it all works perfectly.
My Setup:
JDK 7, eclipse (adt-bundle, android development) (Windows 7)
JDK 8, eclipse ('normal' java) (Linux (xubuntu))
Is it because both Classes (Base64) are implemented diffrently? If so, how can I make it work?
Here are the Encoding/Decoding Methods:
Encoding (Device: on Windows 7, adt-bundle, Android-Dev, jdk 7):
import android.util.Base64
public byte[] encode(byte[] bytearrayToEncode){
return Base64.encode(bytearrayToEncode, Base64.NO_WRAP|Base64.URL_SAFE);
}
Decoding (Server: on Linux, jdk 8):
import java.util.Base64
public byte[] decode(byte[] bytearrayToEncode){
return Base64.getUrlDecoder().decode(bytearrayToDecode);
}
Strings are all De-/Encoded with the same charset (utf-8)!
Encoding/Decoding: Base64
Crypto: AES
If you need more Information, just ask, but I think I provided all neccessary informations.
Edit:
public String bytearrayToString(byte[] bytearray){
String str = null;
try {
str = new String(bytearray, "UTF-8");
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return str;
}
public byte[] stringToBytearray(String str){
byte[] bytearray = null;
try {
bytearray = str.getBytes("UTF-8");
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return bytearray;
}
You must be using JSONObjects to send the Base64 String to the server. If so then the JSONObject will add " \ " escape characters to the string which are illegal Base64 characters. All you need to do is
String rectifiedString = Base64String.replace("\\","");
on the server end before using the acquired string.
Optional
Also on the server side use
import javax.xml.bind.DatatypeConverter;
and
String result = DatatypeConverter.printBase64Binary(bytearrayToDecode);
byte[] result = DatatypeConverter.parseBase64Binary(str);
instead.
It will not only get the job done but is much faster at encoding/decoding (as benchmarked here).
If you have a space in your encoded string, which is an illegal character in base64. If you remove it, it still generates the same image/data in the converter you linked to, and can be now decoded by your code as well.
UPDATE:
Some decoders (like the one you linked, or Base64.getMimeDecoder() in Java) ignore illegal characters, others (like Base64.getDecoder()) don't allow them.
Use for encode:
Base64.getEncoder().encodeToString(yourString.getBytes("UTF-8"));
and decode:
byte[] decodedBytes = Base64.getDecoder().decode(yourString);
String stringDecode = new String(decodedBytes, "UTF-8");
I can only calculate the CRC32 values of .ZIP/.PNG Strings, but not Ethernet related ones. The Java CRC32 class only seems to allow for one type of calculation.
String str = textField.getText();
Checksum checksum = new CRC32();
byte bytes[] = null;
try {
bytes = str.getBytes("ASCII");
} catch (UnsupportedEncodingException ex) {
Logger.getLogger(GUI.class.getName()).log(Level.SEVERE, null, ex);
}
checksum.update(bytes, 0, bytes.length);
long lngChecksum = checksum.getValue();
crc32bField.setText(Long.toHexString(lngChecksum));
This is the code I've written to calculate my CRC, could anyone help me achieve the same values as one calculated on this website?
http://hash.online-convert.com/crc32-generator
Just as an example,
"hello world" =
7813f744 (website)
D4A1185 (My Code)
Thanks :)
It seems that the algorithm used in the website you provided is CRC32 whereas the one you are using is CRC32B. Both of them are completely different algorithms that is why you are getting different values.
You try the CRC32B algorithm from the same website. It is inline with what the Checksum class is giving you.