Develop Reference

Is there a way to read files in bytes?

I am working on changing the code completed with php to java.
I have a test.tpf file encrypted with SEED and the file is encoded in ANSI.
The test.tpf file contains a string and image information for base64 encoding and output.
I have to cut 16 bytes each and read the file to decrypt test.tpf.
So I used fileInputStream to save and decrypt the bytes that I read in a 16-size byte array.
int nReadCur=0;
byte[] outbuf = new byte[16];
int nRead =0;
FileInputStream fis = new FileInputStream(tpf);
while (true) {
byte[] fileContentArray=new byte[16];
nRead = fis.read(fileContentArray);
nReadCur = nReadCur + nRead;
seed.SeedDecrypt(fileContentArray, pdwRoundKey, outbuf); //Decryption
String dbgmsg =new String(outbuf,"MS949");
mergeStr+=dbgmsg;
if(nFileSize<=nReadCur || nRead==-1)
break;
}//while
Then, they encoded the part corresponding to the image information in base64.
In js, the base64 code was changed to a string to receive the string and base64 information in json and display it on the screen.
String[] dataExplode=mergeStr.split("<TextData>");
String[] dataExplode1=dataExplode[1].split("</FileInfo>");
String[] dataExplode2=dataExplode1[0].split("</TextData>");
String textData = null;
String imageData = null;
textData=dataExplode2[0];
imageData=dataExplode1[1];
Encoder encoder=Base64.getEncoder();
imageData=encoder.encodeToString(imageData.getBytes());
JSONArray ja=new JSONArray();
ja.put(textData);
ja.put(imageData);
result.put("imageContent", ja);
However, it seems that the file cannot be read properly.
Compared to the result value of the php code I have, the string is incorrectly entered.
My Eclipse basic encoding is UTF8, so I think this problem is due to encoding.When I read files using fileInputStream, I wanted to set up characters and read them.
I don't know how to read bytes at this time.
How can I read files 16 bytes at a time after setting up the encoding?
Also, I would like to know if there is a mistake in my code.
My java version is 1.8 and I use spring 3.1.1
++)add
I succeeded in making a 16-size outbuf array into one array using the ByteArrayOutputStream.
ByteArrayOutputStream baos =new ByteArrayOutputStream();
.
.
.
seed.SeedDecrypt(fileContentArray, pdwRoundKey, outbuf);
baos.write(outbuf, 0, 16);
break;
}
}//while
mergeStr=new String(baos.toByteArray(),"MS949");
.
.
.
However, compared to the php code I have, I found that the result value of php is different from the result value of java.
in java:System.out.println("mergeStr:"+mergeStr.length()+" / image:"+imageData.length());
java console: mergeStr:69716 / image:168092
in php:alog("mergeStr: ".strlen($mergeStr)." / imageData : ".strlen($imageData ));
php log: mergeStr: 85552 / imageData: 111860
Since the result string decoded by java and php is different, the result of java and php is different for the value encoded by imageData as base64

When you read a file in bytes, there is no encoding.
Only when you convert/interprete the bytes as characters, encoding becomes important.
So if you want to stick to reading bytes, FileInputStream is the way.
If you want to read characters, a FileReader is the way.
Note you can specify the encoding to be used on the constructor - which is necessary if the file is not in the system default encoding.
Edit:
How can I read files 16 bytes at a time after setting up the encoding?
You simply cannot. But you can read 16 bytes, then when you convert to String specify the encoding.

Sending Images via JSON using base64 Encoding

I am testing out if it is possible to send a JSON file with a image or two. Currently, I have the images converted into bytes and I use
Base64.encodeToString(temp_arr, Base64.NO_WRAP);
(this is Androids base64 class, and I have to use .NO_WRAP feature to make it work after reading other stack overflow pages)
to convert it to a string. At this point, I pass that string object into my JSON file (Using GSON library) and add the string go it. This data will be sent to a PHP page.
I have test converting the bytes into base64 and saving to a text file, copying that text file into my php page, running it through my php page using
base64_decode($);
and that is able to properly recreate the image just fine (sha hashes match). So now I needed to test it sending it over a network and using json. Is the only difference is that base64 string is put int other json file rather then a text file, the json is sent to php, i grab the data and decode it in PHP.
now the problem is the image is corrupt, looking at both files in a hex editor, the first 20 lines or so in the hex editor match fine, but after that it does not match. Oddly enough the very end of the files have same data except the uploaded copy has extra characters making it larger in size.
So my problem is trying to find out, can it be GSON (JSON) causing the problem or something else, and if so, what can I do about it.
Sadly, the way my work is, my boss needs the data (json with text, data and etc) to be sent at same time, to same php page with the images, this is why I am sending the images via json.

Try this work for me,convert image into base64
public static String getStringImage(Bitmap bmp)
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bmp.compress(Bitmap.CompressFormat.JPEG, 60, baos);
byte[] imageBytes = baos.toByteArray();
String encodedImage = Base64.encodeToString(imageBytes, Base64.DEFAULT);
Log.e("SignUp", "Image Decode : " + encodedImage);
String asa = encodedImage;
return encodedImage;
}
//pass bitmap image and return string of base64

I believe I found my solution, problem was the Base64.encodeToString() would encode the images bytes to characters that included "+", where on the PHP side, the data would be send except all the "+" became spaces. I just had to replace all spaces with the + and it worked.

Uploading image to server corrupts the image

I have in my application a image upload method that need to send a image and a string to my server.
The problem is that the server receives the content (image and string) but when it saves the image on the disk it is corrupted and can't be opened.
This is the relevant part of the script.
HttpPost httpPost = new HttpPost(url);
Bitmap bmp = ((BitmapDrawable) imageView.getDrawable()).getBitmap();
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bmp.compress(Bitmap.CompressFormat.PNG, 100, stream);
byte[] byteArray = stream.toByteArray();
String byteStr = new String(byteArray);
StringBuilder stringBuilder = new StringBuilder();
stringBuilder.append("--"+boundary+"\r\n");
stringBuilder.append("Content-Disposition: form-data; name=\"content\"\r\n\r\n");
stringBuilder.append(message+"\r\n");
stringBuilder.append("--"+boundary+"\r\n");
stringBuilder.append("Content-Disposition: form-data; name=\"image\"; filename=\"image.jpg\"\r\n");
stringBuilder.append("Content-Type: image/jpeg\r\n\r\n");
stringBuilder.append(byteStr);
stringBuilder.append("\r\n");
stringBuilder.append("--"+boundary+"--\r\n");
StringEntity entity = new StringEntity(stringBuilder.toString());
httpPost.setEntity(entity);
I can't change the server because other clients use it and it works for them. I just need to understand why the image is being corrupted.

When you do new String(byteArray), it's converting binary into the default character set (which is typically UTF-8). Most character sets aren't a suitable encoding for binary data. In other words if you were to encode certain binary strings to UTF-8 and then decode back to binary, you would not get the same binary string.
Since you're using multipart encoding, you need to write directly to the stream of the entity. Apache HTTP Client has helpers for doing this. See this guide, or this Android guide to uploading with multipart.
If you NEED to using strings only, you can safely convert your byte array to a string with
String byteStr = android.util.Base64.encode(byteArray, android.util.Base64.DEFAULT);
But it's important to note that your server will need to Base64 decode the string back to a byte array and save it to an image. Further, the transfer size will be greater because Base64 encoding isn't as space efficient as raw binary.

Your solutions above is not working because you are using new String(byteArray). The constructor encodes the byte array using the default encoding - see What is the default encoding - and it is very likely, that you have byte sequences in your data that cannot be encoded into a character.
To be more precise, a charset defines how characters are represented as bytes.
Most charsets have more than 256 characters. That is why you need more than one byte to represent a character. UTF-8 and UTF-16 uses up to four bytes.
So you have a mapping between the number space and the character space and this mapping is not bejectiv a priori. So it is very likely that there exist a number in the number space that have no character mapped to it.
The solution #Samuel suggested is foolproof because Base64 uses A–Z, a–z, 0–9, + , / and terminates with = to represent a byte. I would prefer this solution!
If you don't want or cannot use Base64, than you can try just to throw in every byte as it is into the StringBuilder hoping that the server does not do any encoding before you get it.
for (byte b : byteArray) {
stringBuilder.append((char)b);
}
I do not recommand that solution in general, but it may help you to get your stuff done.

Reading a binary file from the file system as a BLOB to use in rhino with javascript

I'm planing to use SheetJS with rhino. And sheetjs takes a binary object(BLOB if i'm correct) as it's input. So i need to read a file from the system using stranded java I/O methods and store it into a blob before passing it to sheetjs. eg :-
var XLDataWorkBook = XLSX.read(blobInput, {type : "binary"});
So how can i create a BLOB(or appropriate type) from a binary file in java in order to pass it in.
i guess i cant pass streams because i guess XLSX needs a completely created object to process.

I found the answer to this by myself. i was able to get it done this way.
Read the file with InputStream and then write it to a ByteArrayOutputStream. like below.
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
...
buffer.write(bytes, 0, len);
Then create a byte array from it.
byte[] byteArray = buffer.toByteArray();
Finally i did convert it to a Base64 String (which is also applicable in my case) using the "Base64.encodeBase64String()" method in apache.commons.codec.binary package. So i can pass Base64 String as a method parameter.
If you further need there are lot of libraries(3rd-party and default) available for Base64 to Blob conversion as well.

How do you convert binary data to Strings and back in Java?

I have binary data in a file that I can read into a byte array and process with no problem. Now I need to send parts of the data over a network connection as elements in an XML document. My problem is that when I convert the data from an array of bytes to a String and back to an array of bytes, the data is getting corrupted. I've tested this on one machine to isolate the problem to the String conversion, so I now know that it isn't getting corrupted by the XML parser or the network transport.
What I've got right now is
byte[] buffer = ...; // read from file
// a few lines that prove I can process the data successfully
String element = new String(buffer);
byte[] newBuffer = element.getBytes();
// a few lines that try to process newBuffer and fail because it is not the same data anymore
Does anyone know how to convert binary to String and back without data loss?
Answered: Thanks Sam. I feel like an idiot. I had this answered yesterday because my SAX parser was complaining. For some reason when I ran into this seemingly separate issue, it didn't occur to me that it was a new symptom of the same problem.
EDIT: Just for the sake of completeness, I used the Base64 class from the Apache Commons Codec package to solve this problem.

String(byte[]) treats the data as the default character encoding. So, how bytes get converted from 8-bit values to 16-bit Java Unicode chars will vary not only between operating systems, but can even vary between different users using different codepages on the same machine! This constructor is only good for decoding one of your own text files. Do not try to convert arbitrary bytes to chars in Java!
Encoding as base64 is a good solution. This is how files are sent over SMTP (e-mail). The (free) Apache Commons Codec project will do the job.
byte[] bytes = loadFile(file);
//all chars in encoded are guaranteed to be 7-bit ASCII
byte[] encoded = Base64.encodeBase64(bytes);
String printMe = new String(encoded, "US-ASCII");
System.out.println(printMe);
byte[] decoded = Base64.decodeBase64(encoded);
Alternatively, you can use the Java 6 DatatypeConverter:
import java.io.*;
import java.nio.channels.*;
import javax.xml.bind.DatatypeConverter;
public class EncodeDecode {
public static void main(String[] args) throws Exception {
File file = new File("/bin/ls");
byte[] bytes = loadFile(file, new ByteArrayOutputStream()).toByteArray();
String encoded = DatatypeConverter.printBase64Binary(bytes);
System.out.println(encoded);
byte[] decoded = DatatypeConverter.parseBase64Binary(encoded);
// check
for (int i = 0; i < bytes.length; i++) {
assert bytes[i] == decoded[i];
}
}
private static <T extends OutputStream> T loadFile(File file, T out)
throws IOException {
FileChannel in = new FileInputStream(file).getChannel();
try {
assert in.size() == in.transferTo(0, in.size(), Channels.newChannel(out));
return out;
} finally {
in.close();
}
}
}

If you encode it in base64, this will turn any data into ascii safe text, but base64 encoded data is larger than the orignal data

See this question, How do you embed binary data in XML?
Instead of converting the byte[] into String then pushing into XML somewhere, convert the byte[] to a String via BASE64 encoding (some XML libraries have a type to do this for you). The BASE64 decode once you get the String back from XML.
Use http://commons.apache.org/codec/
You data may be getting messed up due to all sorts of weird character set restrictions and the presence of non-priting characters. Stick w/ BASE64.

How are you building your XML document? If you use java's built in XML classes then the string encoding should be handled for you.
Take a look at the javax.xml and org.xml packages. That's what we use for generating XML docs, and it handles all the string encoding and decoding quite nicely.
---EDIT:
Hmm, I think I misunderstood the problem. You're not trying to encode a regular string, but some set of arbitrary binary data? In that case the Base64 encoding suggested in an earlier comment is probably the way to go. I believe that's a fairly standard way of encoding binary data in XML.