Develop Reference

Convert JMS BytesMessage to String in java and use the same bytesmessage in another process

My code is listening to an IBM MQ. Receives JMS BytesMessage, converts it to string in the receiver class and pass on the same JMS BytesMessage to another processor class. Processor class again converts it into String. Both receiver class and processor class use the same code like below to get the string from BytesMessage. I am getting the right string in the Receiver class but when trying to get the string from the BytesMessage in Processor class its returning empty string. Please advise what has to be done in addition to preserve the JMS BytesMessage so that it gets converted to String in the Processor class as well.
Code that sends message to processor:
String strMessage = null;
strMessage = getStringFromMessage(Message message)
process(message)
Code used for String Conversion:
if (message instanceof BytesMessage){
BytesMessage byteMessage = (BytesMessage) message;
byte[] byteData = null;
byteData = new byte[(int) byteMessage.getBodyLength()];
byteMessage.readBytes(byteData);
stringMessage = new String(byteData);
}

I found the solution. I added the below code after reading the message for the first time
byteMessage.reset()
This has reset the cursor position to the beginning and hence i am able to read it in the processor. So my final code in the receiver will look like below
if (message instanceof BytesMessage){
BytesMessage byteMessage = (BytesMessage) message;
byte[] byteData = null;
byteData = new byte[(int) byteMessage.getBodyLength()];
byteMessage.readBytes(byteData);
byteMessage.reset();
stringMessage = new String(byteData);
}
The reason to read it again is the i started reading it in the receiver to perform some recovery functionality. I wanted to implement that without touching the framework. Initial framework is to read the message only in the processor.

The answer from #Shankar Anand will work but, I would like to refactor the code to accommodate what it exactly needs to do
public String readIbmMqMessageAsString(BytesMessage message) throws JMSException, UnsupportedEncodingException {
message.reset(); //Puts the message body in read-only mode and repositions the stream of bytes to the beginning
int msgLength = ((int) message.getBodyLength());
byte[] msgBytes = new byte[msgLength];
message.readBytes(msgBytes, msgLength);
String encoding = message.getStringProperty(JMS_IBM_CHARACTER_SET);
return new String(msgBytes, encoding).trim();
}
Before reading the message, we need to reposition the stream of bytes to the beginning. Hence the message.reset() should happen before the actual reading of the message.
Then we can read the message and put it in the bytes array
When we create the String from the bytes, It's better to provide the encoding character set which character set is used in convert message to bye
We may not need the extra trailing spaces. In that case, you can trim it as well.
I throw the exceptions to the parent method to handle

Sending pdf data through rest api inside json

I have made a webservice that send multiple pdfs as response to client using mutlipart/formdata but as it happens one of the client is salesforce which does not support mutlipart/ formdata.
They want a json in response like -
{ "filename": xyzname,
"fileContent": fileContent
}
I tried encoding data in Base64 using apache codec library but pdf on client side seems to get corrupted and I am unable to open it using acrobat.
Please find code below -
import org.apache.commons.io.FileUtils;
//------Server side ----------------
#POST
#Consumes(MULTIPART_FORM_DATA)
#Produces(MediaType.APPLICATION_JSON)
#Path("somepath")
public Response someMethod(someparam) throws Exception
{
....
JSONArray filesJson = new JSONArray();
String base64EncodedData = Base64.encodeBase64URLSafeString(loadFileAsBytesArray(tempfile));
JSONObject fileJSON = new JSONObject();
fileJSON.put("fileName",somename);
fileJSON.put("fileContent", base64EncodedData);
filesJson.put(fileJSON);
.. so on ppopulate jsonArray...
//sending reponse
responseBuilder = Response.ok().entity(filesJson.toString()).type(MediaType.APPLICATION_JSON_TYPE) ;
response = responseBuilder.build();
}
//------------Client side--------------
Response clientResponse = webTarget.request()
.post(Entity.entity(entity,MediaType.MULTIPART_FORM_DATA));
String response = clientResponse.readEntity((String.class));
JSONArray fileList = new JSONArray(response);
for(int count= 0 ;count< fileList.length();count++)
{
JSONObject fileJson = fileList.getJSONObject(count);
byte[] decodedBytes = Base64.decodeBase64(fileJson.get("fileContent").toString());
outputFile = new File("somelocation/" + fileJson.get("fileName").toString() + ".pdf");
FileUtils.writeByteArraysToFile(outputFile, fileJson.get("fileContent").toString().getBytes());
}
-------------------------------
Kindly advise.

Yes so the problem was with the client.
while decoding we should use
byte[] decodedBytes = Base64.decodeBase64(fileJson.getString("fileContent"));
rather than
byte[] decodedBytes = Base64.decodeBase64(fileJson.get("fileContent").toString());
Since encoded data.toString() yields some think else
Also replaced encodeBase64URLSafeString with encodeBase64String
Well quite a simple solution :)

We are doing the same, basically sending PDF as JSON to Android/iOS and Web-Client (so Java and Swift).
The JSON Object:
public class Attachment implements Serializable {
private String name;
private String content;
private Type contentType; // enum: PDF, RTF, CSV, ...
// Getters and Setters
}
And then from byte[] content it is set the following way:
public Attachment createAttachment(byte[] content, String name, Type contentType) {
Attachment attachment = new Attachment();
attachment.setContentType(contentType);
attachment.setName(name);
attachment.setContent(new String(Base64.getMimeEncoder().encode(content), StandardCharsets.UTF_8));
}
Client Side Java we create our own file type object first before mapping to java.io.File:
public OurFile getAsFile(String content, String name, Type contentType) {
OurFile file = new OurFile();
file.setContentType(contentType);
file.setName(name);
file.setContent(Base64.getMimeDecoder().decode(content.getBytes(StandardCharsets.UTF_8)));
return file;
}
And finally:
public class OurFile {
//...
public File getFile() {
if (content == null) {
return null;
}
try {
File tempDir = Files.createTempDir();
File tmpFile = new File(tempDir, name + contentType.getFileEnding());
tempDir.deleteOnExit();
FileUtils.copyInputStreamToFile(new ByteArrayInputStream(content), tmpFile);
return tmpFile;
} catch (IOException e) {
throw new RuntimeException(e);
}
}

In my REST application with PHP:
  1. Send the data encoded in base64 $data = base64_encode($data) to REST.
  2. Before writing the file, I decode $data = base64_decode($data).
  3. Therefore, when the file is downloaded, it is already in the correct format.

I would change from using "Safe" to just using "string". So change:
encodeBase64URLSafeString(...)
to:
encodeBase64String(...)
The reason is that the "safe" versions actually change the content to preserve URLs before encrypting - I'm totally uncertain what that would do to a PDF, but suspect it is the source of your problem.
If that doesn't do it for you, I suggest encrypting/decrypting right on the server (or a separate test app) and comparing the results while you try to work it out. That way you can see if what you are doing is working, but don't have to go through the whole "start the server, start the client, connect..." process each time, and it will speed your debugging.

inputStream and utf 8 sometimes shows "?" characters

So I've been dealing with this problem for over a months now and I also checked almost every possible related solution over here in and over google but I couldn't find anything that really solved my case.
my problem is that i'm trying to download an html source from a website but what i'm getting in most cases is that some of the text shows some "?" characters in it,most likely beacuse the site is in Hebrew.
Here's my code,
public static InputStream openHttpGetConnection(String url)
throws Exception {
InputStream inputStream = null;
HttpClient httpClient = new DefaultHttpClient();
HttpResponse httpResponse = httpClient.execute(new HttpGet(url));
inputStream = httpResponse.getEntity().getContent();
return inputStream;
}
public static String downloadSource(String url) {
int BUFFER_SIZE = 1024;
InputStream inputStream = null;
try {
inputStream = openHttpGetConnection(url);
} catch (Exception e) {
// TODO: handle exception
}
int bytesRead;
String str = "";
byte[] inpputBuffer = new byte[BUFFER_SIZE];
try {
while ((bytesRead = inputStream.read(inpputBuffer)) > 0) {
String read = new String(inpputBuffer, 0, bytesRead,"UTF-8");
str +=read;
}
} catch (Exception e) {
// TODO: handle exception
}
return str;
}
Thanks.

To read characters from a byte stream with a given encoding, use a Reader. In your case it would be something like:
InputStreamReader isr = new InputStreamReader(inpputStream, "UTF-8");
char[] inputBuffer = new char[BUFFER_SIZE];
while ((charsRead = isr.read(inputBuffer, 0, BUFFER_SIZE)) > 0) {
String read = new String(inputBuffer, 0, charsRead);
str += read;
}
You can see that the bytes will be read in directly as characters --- it's the reader's problem to know if it needs to read one or two bytes, e.g., to create the character in the buffer. It's basically your approach but decoding as the bytes are being read in, instead of after.

Converting an InputStream to a String entails specifying an encoding, just as you do at new String(inpputBuffer, 0, bytesRead,"UTF-8");.
But your approach as several drawbacks.
How do you know you have to use UTF8 ?
When retreiving HTTP content, generally speaking, you can not know in advance what encoding will be used in the HTTP response. But HTTP provides a mechanism for specifying that, using the Content-Type header.
More specifically, your response object should have a Content-Type "header", that has an "attribute" called encoding. In the response, it should look something like :
Content-Type: text/html; encoding=UTF-8
You should use whatever is after the encoding= part to transform your bytes to chars.
Seeing you seem to use Apache HTTPClient, their documentation states :
You can set the content type header for a request with the addRequestHeader method in each method and retrieve the encoding for the response body with the getResponseCharSet method.
If the response is known to be a String, you can use the getResponseBodyAsString method which will automatically use the encoding specified in the Content-Type header or ISO-8859-1 if no charset is specified..
Alternate way
If there is no Content-Type header, and if you know your content is HTML, then you can try to convert it as a String using some encoding (UTF or ISO Latin preferably), and try to find some content matching <meta charset="UTF-8">, and use that as the charset. This should only be a fail-over.
Any byte sequence is not convertible to a String
Drawback number two is that you read any number of bytes from your stream, and try to convert it to a String, which may not be possible.
In practice, UTF-8 can encode some "characters" across several bytes. For example "é" can be encoded as 0xC3A9. So say for example that the response consists of two "é" characters. If your first call to read returns :
[c3, a9, c3]
Your conversion to a String using new String(byte[], off, enc) will leave the last byte apart, because it does not match a valid UTF8 sequence.
Your following read will get what's left to read
[a9]
Which is (whatever that is) not a "é" character.
Bottom line : you can not convert even a valid UTF-8 sequence to byte using your pattern.
Going forward : you use HTTPClient, use their method of HTTP Response to String conversion.
If you wish to do it yourself, the easy way is to copy your input to a byte array, and then convert the byte array. Something along the lines of (pseudo code) :
ByteArrayOutputStream responseContent = new ByteArrayOutputStream()
copyAllBytes(responseInputStream, responseContent)
byte[] rawResponse = responseContent.toByteArray();
String stringResponse = new String(rawResponse, encoding);
But you could also use a CharsetDecoder if you want a fully streamed implementation (one that does not buffer the response fully into memory), or as #jas answers, wrap your inputStream to a reader and concatenate the output (preferably into a StringBuilder, which should be faster if a high number of concatenation is to occur).

Content length for MultipartFormDataInput request always -1

I am using RESTEasy to PUT some data by a key and value. I'd like to only accept data that's less than 4k in size, which I figured I'd check by getting the content length of the request. Here's the method I'm working with:
#PUT
#Path("/{key}")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response addData(
final #PathParam("key") String dataKey,
MultipartFormDataInput data) throws IOException, ClassNotFoundException {
final HttpServletRequest request = ....getHttpRequest(); //returns the request
final Map<String, List<InputPart>> parts = data.getFormDataMap();
final List<InputPart> inputParts = parts.get("data");
final InputPart part= inputParts.get(0);
final InputStream dataStream = part.getBody(InputStream.class, null);
int length = request.getContentLength();
String length2 = request.getHeader("Content-Length");
System.out.println(length);
System.out.println(length2);
service.addData(key, dataStream);
return Response.status(204).build();
}
However, length and length2 return -1 and null respectively. If I monitor Fiddler and look at the request, I notice that the contentLength on the request is not -1 but it rather a correct value instead.
Any ideas on why I can't get the request content length? Thanks.
Edit: My PUT request look like this using the Advanced Rest Client Chrome Extension: http://i.imgur.com/aI6WNDy.png
Edit: Not sure I have much else to add, but any thoughts on this would be great.

see if #HeaderParam works for you
http://www.mkyong.com/webservices/jax-rs/get-http-header-in-jax-rs/

posting data over network doesn't work in java.io API

I have the below piece of code that works fine when run from eclipse
String str = "testing";
InputStream is = new ByteArrayInputStream(str.getBytes());
int length = is.available();
byte[] data = new byte[length];
is.read(data, 0, length);
System.out.println("output "+new String(data));
When I break this code to post the data over network using Jerser client API and try read it back on the server using Jersey I don't get the value that I posted. The client code looks like this
public static void main(String s[]) throws IOException {
ClientConfig config = new DefaultClientConfig();
Client client = Client.create(config);
String str = "testingthi";
InputStream is = new ByteArrayInputStream(str.getBytes());
WebResource webResource = client.resource("http://192.168.1.15:8090/JersySample/resources/stream/upload");
ClientResponse response = webResource.type(MediaType.APPLICATION_OCTET_STREAM).post(ClientResponse.class,is);
is.close();
}
The code on the server looks like this
#Consumes(MediaType.APPLICATION_OCTET_STREAM)
public Response upload( InputStream is) {
try{
int size = is.available();
byte data[] = new byte[size];
is.read(data,0,size);
System.out.println(" Posted data is "+ new String(data)+" length is "+size+" stream size "+is.available());
is.close();
}catch (Exception e) {
e.printStackTrace();
}
return Response.ok().entity("Done").build();
}
When I read the string using apache-commons-io API I do get the string value that I posted. Can someone explain why it doesn't work using the java.io API which was working in non-network situation?

You're making the usual mistake of assuming that read() fills the buffer. It isn't obliged to do that. See the Javadoc. It's only obliged to read at least one byte and return. The byte count, return -1, or throw an IOException.
You're also misusing available(). It doesn't return the total number of bytes in the stream, and the way you're using it is specifically warned against in the Javadoc.