Develop Reference

InputStream for HTTP Header does not terminate

I am trying to read the http header from the client using input stream but it does not seem to terminate. I wonder if there is any problem to read the request this way. I wanted this to work on both GET and POST request. Here is my code
private static byte[] readClientRequest(byte[] clientRequest, InputStream fromClient) throws IOException{
int bytesRead;
ByteArrayOutputStream clientByteArray = new ByteArrayOutputStream();
while((bytesRead = fromClient.read()) != -1) {
clientByteArray.write(bytesRead);
}
clientByteArray.flush();
clientRequest = clientByteArray.toByteArray();
return clientRequest;
}
Thanks a lot.

In general network socket does not known where the end is. It may be application protocol specific. For HTTP you can read one header line until end of line (bytesRead != '\n'). Then you should use available method to check that there is next part of data.

My base64 encoded byte[] stream has extra characters after sent through a http response

I encode a pdf into a base64 byte[] stream and I want to send it as a http response to the browser. The problem is that the browser fails to load pdf.
I compared the base 64 string which I printed into the IDE console and the one from the browser console. The one from the IDE console is correct and the one from the browser has extra characters.
So, my base64 byte[]stream gets broken somehow when it's sent as a http response ? How do I solve this?
L.e. : The code
FileInputStream inputFileInputStream = null;
ServletOutputStream outputFileOutputStream = null;
File exportFile = new File(exportedReport);
int fileSize = (int) exportFile.length();
String fullPathToExport = exportFile.getAbsolutePath();
File fullPathFile = new File(fullPathToExport);
try {
// test to see if the path of the file is correct
System.out.println("The file is located at: "
+ exportFile.getAbsolutePath());
response.reset();
response.setContentType("application/pdf");
response.setContentLength(fileSize);
response.addHeader("Content-Transfer-Encoding", "base64");
response.setHeader( "Content-Disposition", "inline; filename=\"" + exportedReport +"\"");
inputFileInputStream = new FileInputStream(fullPathFile);
outputFileOutputStream = response.getOutputStream();
if (bytesToRead == -1) {
bytesToRead = (int)fullPathFile.length();
}
byte[] buffer = new byte[bytesToRead];
int bytesRead = -1;
while((inputFileInputStream != null) && ((bytesRead = inputFileInputStream.read(buffer)) != -1)){
if (codec.equals("base64")) {
//outputFileOutputStream.write(Base64.encodeBytes(buffer).getBytes("UTF-8"), 0, bytesToRead);
outputFileOutputStream.write(org.apache.commons.codec.binary.Base64.encodeBase64(buffer));
} else {
outputFileOutputStream.write(buffer, 0, bytesToRead);
}
}
inputFileInputStream.close();
outputFileOutputStream.flush();
outputFileOutputStream.close();

Your code has one major problem:
You are not sending one base64 encoded data part but many base64 encoded data parts (concatenated). But two or more base64 encoded data parts are not equal to one base64 encoded data part.
Example:
base64("This is a test") -> "VGhpcyBpcyBhIHRlc3Q="
base64("This ")+base64("is a ")+base64("test") -> "VGhpcyA=aXMgYSA=dGVzdA=="
You should use the org.apache.commons.codec.binary.Base64InputStream instead of the Base64.encodeBase64() utility method. Read the whole FileInputStream through it and you will get a valid base64 encoded data stream.
Anyway what you are doing is not necessary. You can transfer any 8 bit data via HTTP without further encoding.

inputStream and utf 8 sometimes shows "?" characters

So I've been dealing with this problem for over a months now and I also checked almost every possible related solution over here in and over google but I couldn't find anything that really solved my case.
my problem is that i'm trying to download an html source from a website but what i'm getting in most cases is that some of the text shows some "?" characters in it,most likely beacuse the site is in Hebrew.
Here's my code,
public static InputStream openHttpGetConnection(String url)
throws Exception {
InputStream inputStream = null;
HttpClient httpClient = new DefaultHttpClient();
HttpResponse httpResponse = httpClient.execute(new HttpGet(url));
inputStream = httpResponse.getEntity().getContent();
return inputStream;
}
public static String downloadSource(String url) {
int BUFFER_SIZE = 1024;
InputStream inputStream = null;
try {
inputStream = openHttpGetConnection(url);
} catch (Exception e) {
// TODO: handle exception
}
int bytesRead;
String str = "";
byte[] inpputBuffer = new byte[BUFFER_SIZE];
try {
while ((bytesRead = inputStream.read(inpputBuffer)) > 0) {
String read = new String(inpputBuffer, 0, bytesRead,"UTF-8");
str +=read;
}
} catch (Exception e) {
// TODO: handle exception
}
return str;
}
Thanks.

To read characters from a byte stream with a given encoding, use a Reader. In your case it would be something like:
InputStreamReader isr = new InputStreamReader(inpputStream, "UTF-8");
char[] inputBuffer = new char[BUFFER_SIZE];
while ((charsRead = isr.read(inputBuffer, 0, BUFFER_SIZE)) > 0) {
String read = new String(inputBuffer, 0, charsRead);
str += read;
}
You can see that the bytes will be read in directly as characters --- it's the reader's problem to know if it needs to read one or two bytes, e.g., to create the character in the buffer. It's basically your approach but decoding as the bytes are being read in, instead of after.

Converting an InputStream to a String entails specifying an encoding, just as you do at new String(inpputBuffer, 0, bytesRead,"UTF-8");.
But your approach as several drawbacks.
How do you know you have to use UTF8 ?
When retreiving HTTP content, generally speaking, you can not know in advance what encoding will be used in the HTTP response. But HTTP provides a mechanism for specifying that, using the Content-Type header.
More specifically, your response object should have a Content-Type "header", that has an "attribute" called encoding. In the response, it should look something like :
Content-Type: text/html; encoding=UTF-8
You should use whatever is after the encoding= part to transform your bytes to chars.
Seeing you seem to use Apache HTTPClient, their documentation states :
You can set the content type header for a request with the addRequestHeader method in each method and retrieve the encoding for the response body with the getResponseCharSet method.
If the response is known to be a String, you can use the getResponseBodyAsString method which will automatically use the encoding specified in the Content-Type header or ISO-8859-1 if no charset is specified..
Alternate way
If there is no Content-Type header, and if you know your content is HTML, then you can try to convert it as a String using some encoding (UTF or ISO Latin preferably), and try to find some content matching <meta charset="UTF-8">, and use that as the charset. This should only be a fail-over.
Any byte sequence is not convertible to a String
Drawback number two is that you read any number of bytes from your stream, and try to convert it to a String, which may not be possible.
In practice, UTF-8 can encode some "characters" across several bytes. For example "é" can be encoded as 0xC3A9. So say for example that the response consists of two "é" characters. If your first call to read returns :
[c3, a9, c3]
Your conversion to a String using new String(byte[], off, enc) will leave the last byte apart, because it does not match a valid UTF8 sequence.
Your following read will get what's left to read
[a9]
Which is (whatever that is) not a "é" character.
Bottom line : you can not convert even a valid UTF-8 sequence to byte using your pattern.
Going forward : you use HTTPClient, use their method of HTTP Response to String conversion.
If you wish to do it yourself, the easy way is to copy your input to a byte array, and then convert the byte array. Something along the lines of (pseudo code) :
ByteArrayOutputStream responseContent = new ByteArrayOutputStream()
copyAllBytes(responseInputStream, responseContent)
byte[] rawResponse = responseContent.toByteArray();
String stringResponse = new String(rawResponse, encoding);
But you could also use a CharsetDecoder if you want a fully streamed implementation (one that does not buffer the response fully into memory), or as #jas answers, wrap your inputStream to a reader and concatenate the output (preferably into a StringBuilder, which should be faster if a high number of concatenation is to occur).

Use an ASN.1 sequence with more then one argument

I tried this code to send and receive an Integer with ASN.1 generated classes
Client sending an Integer:
ClientFirstRequest h = new ClientFirstRequest();
h.clientInt.setValue(9);
BerOutputStream bos = new BerOutputStream(_socket.getOutputStream());
h.encode(bos);
Server receiving it:
ClientFirstRequest h = new ClientFirstRequest();
BerInputStream in = new BerInputStream(socket.getInputStream());
h.decode(in);
ASN1Integer ClientNumber= h.clientInt;
int clientNumbervalue = (int)ClientNumber.getValue();
It work perfectly, but in the second sequence I have to send two argument, an Int and a String
Server sending an Integer and a String
ServerFirstResponse response1 = new ServerFirstResponse();
response1.serverInt.setValue(clientNumbervalue);
response1.serverString.setValue(randomString);
BerOutputStream bos = new BerOutputStream(socket.getOutputStream());
h.encode(bos);
Client receiving them
ServerFirstResponse response1 = new ServerFirstResponse();
BerInputStream in = new BerInputStream(_socket.getInputStream());
response1.decode(in);
But I got an error
com.chaosinmotion.asn1.AsnFatalException:
In decoding process, one of the elements of your SEQUENCE (or an element of an inner sequnce/set) is not OPTIONAL and not initialized!
(If exists)name of this element is : serverString at
com.turkcelltech.jac.Sequence.check_OptionalAndInitialized_Status(Sequence.java:259)
at
com.turkcelltech.jac.Sequence.fillSequenceVariables(Sequence.java:246)
at com.turkcelltech.jac.Sequence.decode(Sequence.java:105) at
Client.main(Client.java:54)

Please contact the vendor of the ASN.1 Tool you are using. They should be better able to how to handle errors in use of their ASN.1 Tool. Each ASN.1 vendor writes code differently even though the end result should be the same encoded stream of bytes regardless of which tool you are using. Note that you have not indicated here which ASN.1 Tool you are using.

Fetch attachment content using javamail

I am using javamail to automate some email handling.
I managed to get a connection to the pop3 server and fetch the messages. Some of them contains an attachment. Base on the email title I am able to "predict" the filename of the attachment that I need to fetch.
But I can't get its content :(
I have a function
public byte[] searchForContent(Part part,String fileName){
if(part.getFileName()!=null){
if(part.getFileName().equals(fileName)){
byte[] content = new byte[part.getSize()];
part.getInputStream().read(content);
return content[]
}
}
return null;
}
The function works very well (ie: return content only if the part was the attachment described by fileName). But the array its returns is too big.
The downloaded attachment is 256 bytes long and the function return a 352 bytes long content.
I think that the problem comes from the headers, but I can't be sure.
How would you proceed to get the content only ?
Thank you.

For what it's worth, the API documentation for javax.mail.Part.getSize() says
Note that the size may not be an exact measure of the content size and may or may not account for any transfer encoding of the content. The size is appropriate for display in a user interface to give the user a rough idea of the size of this part.
Assuming that you know the type of content, you can probably fetch it from Part.getContent() and process that. getContent() returns an Object, but if you know the content type you can cast it appropriately (e.g. to a String for text).

I finally found a solution.
As eaj told : part.getSize() returns the size of the part object. Not the size of the attachment itself.
However the InputStream returned by part.getInputStream() contains the content of the attachment only.
So the code below does give the expected result :
public byte[] searchForContent(Part part,String fileName){
if(part.getFileName()!=null){
if(part.getFileName().equals(fileName)){
InputStream stream=part.getInputStream();
byte[] buffer = new byte[512];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int bytesRead;
while ((bytesRead = stream.read(buffer) != -1)
{
baos.write(buffer, 0, bytesRead);
}
return baos.toByteArray();
}
}
return null;
}