How to read HTTP request using InputStream? I used to read it like this:
InputStream in = address.openStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder result = new StringBuilder();
String line;
while((line = reader.readLine()) != null) {
result.append(line);
}
System.out.println(result.toString());
But reader.readLine() could be blocked, because there is no guarantee that null line will be reached. Of course I can read Content-Length header and then read request in a loop:
for (int i = 0; i < contentLength; i++) {
int a = br.read();
body.append((char) a);
}
But if Content-Length is set too big (I guess it could be set manually for purpose), br.read() will be blocked.
I try to read bytes directly from InputStream like this:
byte[] bytes = getBytes(is);
public static byte[] getBytes(InputStream is) throws IOException {
int len;
int size = 1024;
byte[] buf;
if (is instanceof ByteArrayInputStream) {
size = is.available();
buf = new byte[size];
len = is.read(buf, 0, size);
} else {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
buf = new byte[size];
while ((len = is.read(buf, 0, size)) != -1)
bos.write(buf, 0, len);
buf = bos.toByteArray();
}
return buf;
}
But it waits forever. What do?
If you are implementing HTTP server you should detect the end of the request according to HTTP specification. Wiki - https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol
First of all, you should read a request line, it is always a single line.
Then read all request headers. You read them until you have an empty line (i.e. two line endings - <CR><LF>).
After you have a status line and headers you should decide do you need to read body or no because not all requests might have a body - summary table
Then, if you need a body, you should parse your headers (which you already got) and get Content-Length. If it is - just read as many bytes from the stream as it is specified.
When Content-Length is missing the length is determined in other ways. Chunked transfer encoding uses a chunk size of 0 to mark the end of the content. Identity encoding without Content-Length reads content until the socket is closed.
Create a request wrapper which extends HttpServletRequestWrapper, which will override the getInputStream() which in turn return ServletInputStream , which has the safe read method. try that
Related
I have an inputStream object that comes from a socket.
This stream could be all plain text, or it could contain binary data. It has a plain text "prefix" that tells me which type of data it has. I use BufferedReader to read that prefix. Then, if the data type is plain text, I continue to use BufferedReader to read the rest which works just fine.
The problem is when I switch back to using InputStream if the prefix indicates non-text data. The stream seems to be lost somewhere, and I get a java.net.SocketTimeoutException: Read timed out instead.
Code goes something like this:
InputStream instream = mysocket.getInputStream();
BufferedReader br = (new InputStreamReader(instream, "UTF-8")));
byte[] prefixArray = new byte[4];
br.read(prefixArray, 0, 4);
String prefix = new String(prefixArray);
if("text".equals(prefix)) {
// continue to use br.read() here, which works fine...
}else{
byte[] barray = new byte[arraysize]
instream.read(barray, 0, arraysize); // THROWS "Read timed out" exception
}
Is this simply not allowed? Once instream is wrapped in a BufferedReader, it can't be used directly any longer?
A BufferedReader reads from a Reader and stores the data in a buffer for subsequent reads to use. It reads as much as it can at one time, and each time the buffer is emptied, it reads more into the buffer.
Your binary reads time out because the BufferedReader has already read the bytes you are trying to read.
It takes only 4 bytes for you to detect the stream type. Using a Reader for those 4 bytes is overkill. Read the bytes directly from the InputStream first, then create a Reader only for the text data, eg:
InputStream instream = mysocket.getInputStream();
byte[] prefixArray = new byte[4];
int offset = 0;
int numread;
do
{
numread = instream.read(prefixArray, offset, 4-offset);
if (numread == -1) return;
offset += numread;
}
while (offset < 4);
String prefix = new String(prefixArray);
if ("text".equals(prefix))
{
BufferedReader br = new BufferedReader(new InputStreamReader(instream, "UTF-8"));
//...
}
else
{
byte[] barray = new byte[arraysize];
// consider using BufferedInputStream here ...
do
{
numread = instream.read(barray, 0, arraysize);
if (numread == -1) break;
//...
}
while (true);
}
Alternatively, consider using BufferedInputStream for the socket reading, and add other classes on top of it as needed, eg:
InputStream instream = mysocket.getInputStream();
BufferedInputStream bis = new BufferedInputStream(instream);
byte[] prefixArray = new byte[4];
DataInputStream dis = new DataInputStream(bis);
dis.readFully(prefixArray);
String prefix = new String(prefixArray);
if ("text".equals(prefix))
{
InputStreamReader isr = new InputStreamReader(bis, "UTF-8"));
//...
}
else
{
byte[] barray = new byte[arraysize];
do
{
numread = bis.read(barray, 0, arraysize);
if (numread == -1) break;
//...
}
while (true);
}
Aim : To read a Url which containing information in Json.
Question: I got a code of reading Url Which is given Below. I have a complete Understanding what code is doing but I do not have any idea why the size of char array is 1024 not 2048 or something else . How to decide what character size array is good at the time of reading Url ?
private static String readUrl(String urlString) throws Exception {
BufferedReader reader = null;
try {
URL url = new URL(urlString);
reader = new BufferedReader(new InputStreamReader(url.openStream()));
StringBuffer buffer = new StringBuffer();
int read;
char[] chars = new char[1024]; ???
while ((read = reader.read(chars)) != -1)
buffer.append(chars, 0, read);
return buffer.toString();
} finally {
if (reader != null)
reader.close();
}
}
As the BufferedReader already has an internal buffer of 4096 characters, implementation-dependent, and as the socket already has a considerably larger receive buffer, it really doesn't make much difference what value you choose. The returns on buffering diminish geometrically with size.
I'm building my own HTTP webserver in java and would like to implement some security measures while reading the http request header from a socket inputstream.
I'm trying to prevent scenario's where someone sending extremely long single line headers or absurd amounts of header lines would cause memory overflows or other things you wouldn't want.
I'm currently trying to do this by reading 8kb of data into a byte array and parse all the headers within the buffer I just created.
But as far as I know this means your inputstream's current offset is always already 8kb from it's starting point, even if you had only 100bytes of header.
the code I have so far:
InputStream stream = socket.getInputStream();
HashMap<String, String> headers = new HashMap<String, String>();
byte [] buffer = new byte[8*1024];
stream.read( buffer , 0 , 8*1024);
ByteArrayInputStream bytestream = new ByteArrayInputStream( buffer );
InputStreamReader streamReader = new InputStreamReader( bytestream );
BufferedReader reader = new BufferedReader( streamReader );
String requestline = reader.readLine();
for ( ;; )
{
String line = reader.readLine();
if ( line.equals( "" ) )
break;
String[] header = line.split( ":" , 2 );
headers.put( header[0] , header[1] ); //TODO: check for bad header
}
//if contentlength > 0
// read body
So my question is, how can I be sure that I'm reading the body data (if any) starting from the correct position in the inputstream?
I don't exactly use streams a lot so I don't really have a feel for them and google hasn't been helpful so far
I figured out an answer myself. (was easier than I thought it would be)
If I were to guess it's not buffered (I've no idea when something is buffered anyway) but it works.
public class SafeHttpHeaderReader
{
public static final int MAX_READ = 8*1024;
private InputStream stream;
private int bytesRead;
public SafeHttpHeaderReader(InputStream stream)
{
this.stream = stream;
bytesRead = 0;
}
public boolean hasReachedMax()
{
return bytesRead >= MAX_READ;
}
public String readLine() throws IOException, Http400Exception
{
String s = "";
while(bytesRead < MAX_READ)
{
String n = read();
if(n.equals( "" ))
break;
if(n.equals( "\r" ))
{
if(read().equals( "\n" ))
break;
throw new Http400Exception();
}
s += n;
}
return s;
}
private String read() throws IOException
{
byte b = readByte();
if(b == -1)
return "";
return new String( new byte[]{b} , "ASCII");
}
private byte readByte() throws IOException
{
byte b = (byte) stream.read();
bytesRead ++;
return b;
}
}
I am working in J2ME.
I am getting json response from server then I try to parse this json data as per my requirement.
this is my code for getting response from server:-
InputStream inputStream = null;
OutputStreamWriter out = null;
byte[] readData = new byte[50000];
String response = "no";
try {
// --- write ---
out = new OutputStreamWriter(connection.openOutputStream(), "UTF-8");
out.write(data.toString());
out.close();
// --- read ---
int responseCode = connection.getResponseCode();
if (responseCode != HttpConnection.HTTP_OK) {
throw new IOException("HTTP response code: " + responseCode);
}
inputStream = connection.openInputStream();
int actual = inputStream.read(readData);
response = new String(readData, 0, actual, "UTF-8");
return response;
}
when response from server is small then it works fine, but if response is large then it get half of response and return to my another method. Please suggest me what should I do to get large amount of data into my readData and response variable.
Thank you in advance.
You will need to read all data before (there can be more data in the stream).
As you have noticed the call to InputStream.read doesn't guarantee you to fill your buffer (readData) it return the number of bytes it could read at the time. It just doesn't matter how big your buffer is on your side.
You will need to read re-call the InputStream.read method to check that you have all the data available in the stream. The read method will return -1 when no more data is available.
This is an example how you can do it:
....
ByteArrayOutputStream bos = new ByteArrayOutputStream();
int read;
byte[] tmp = new byte[1024];
while ((read = inputStream.read(tmp)) != -1)
bos.write(tmp, 0, read);
return new String(bos.toByteArray(), "UTF-8");
Also, when you are done with the connection you should call close on it so that the system knows that you don't need it anymore.
You nead to flush and close your input stream after reading!!!
inputStream.close();
I want to send an image from a J2ME client to a Servlet.
I am able to get a byte array of the image and send it using HTTP POST.
conn = (HttpConnection) Connector.open(url, Connector.READ_WRITE, true);
conn.setRequestMethod(HttpConnection.POST);
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
os.write(bytes, 0, bytes.length); // bytes = byte array of image
This is the Servlet code:
String line;
BufferedReader r1 = new BufferedReader(new InputStreamReader(in));
while ((line = r1.readLine()) != null) {
System.out.println("line=" + line);
buf.append(line);
}
String s = buf.toString();
byte[] img_byte = s.getBytes();
But the problem I found is, when I send bytes from the J2ME client, some bytes are lost. Their values are 0A and 0D hex. Exactly, the Carriage Return and Line Feed.
Thus, either POST method or readLine() are not able to accept 0A and 0D values.
Any one have any idea how to do this, or how to use any another method?
That's because you're using a BufferedReader to read the binary stream line by line. The readLine() basically splits the content on CRLF. Those individual lines doesn't contain the CRLF anymore.
Don't use the BufferedReader for binary streams, it doesn't make sense. Just write the obtained InputStream to an OutputStream of any flavor, e.g. FileOutputStream, the usual Java IO way.
InputStream input = null;
OutputStream output = null;
try {
input = request.getInputStream();
output = new FileOutputStream("/path/to/file.ext");
byte[] buffer = new byte[10240];
for (int length = 0; (length = input.read(buffer()) > 0;) {
output.write(buffer, 0, length);
}
} finally {
if (output != null) output.close();
if (input != null) input.close();
}
That said, the Content-Type you're using is technically wrong. You aren't sending a WWW-form URL-encoded value in the request body. You are sending a binary stream. It should be application/octet-stream or maybe image. This is not the cause of this problem, but it is just plain wrong.