Develop Reference

How to submit HTTP request with an INTENTIONAL syntax error?

I'm trying to write a simple test where I submit a request to http://localhost:12345/%, knowing that this is an illegal URI, because I want to assert that my HTTP Server's error-handling code behaves correctly. However, I am having a hard time forcing Java to do this.
If I try to create a Java 11 HttpRequest with URI.create("localhost:12345/%"), I get a URISyntaxException, which is correct and not helpful.
Similarly, using a ws-rs WebTarget:
ClientBuilder.newBuilder().build().target("http://localhost:12345").path("/%")
builds me a WebTarget pointing to /%25, which would normally be very helpful, but is not what I want in this particular situation.
Is there a way to test my error-handling behavior without resorting to low-level bytestream manipulation?

Another possibility is just to use plain Socket - it's easy enough if you know the protocol (especially if using the new text-block feature). This will allow you to misformat the request in any way you like. Reading the response and analysing the result is - of course - a bit more involved:
String request = """
GET %s HTTP/1.1\r
Host: localhost:%s\r
Connection: close\r
\r
""".formatted("/%", port);
try (Socket client = new Socket("localhost", port);
OutputStream os = client.getOutputStream();
InputStream in = client.getInputStream()) {
os.write(request.getBytes(StandardCharsets.US_ASCII));
os.flush();
// This is optimistic: the server should close the
// connection since we asked for it, and we're hoping
// that the response will be in ASCII for the headers
// and UTF-8 for the body - and that it won't use
// chunk encoding.
byte[] bytes = in.readAllBytes();
String response = new String(bytes, StandardCharsets.UTF_8);
System.out.println("response: " + response);
}

Noah's comment lead me down the right path; I was able to do this with the URL class:
#Test
public void testUriMalformed() throws Exception {
final URL url = new URL(TEST_URI + "/%");
final HttpURLConnection connection = (HttpURLConnection)url.openConnection();
final int code = connection.getResponseCode();
final String contentType = connection.getHeaderField("Content-Type");
final String entity = IOUtils.toString(connection.getErrorStream(), Charsets.UTF_8);
assertEquals(500, code);
assertEquals(MediaType.APPLICATION_JSON, contentType);
assertTrue(entity.contains("error_id"));
}

HTTP chunked-encoding buffering Tomcat

Please take a look at the following java Servlet doGet() method:
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException
{
response.setContentType("text/html;charset=utf-8");
OutputStreamWriter osw = new OutputStreamWriter(response.getOutputStream(),"UTF-8");
int j= 0;
while(j < 2)
{
String s = "";
int i = 0;
while(i<10000)
{
s = s + "a";
i++;
}
System.out.println(s.length());
osw.write(s,0,s.length());
j++;
}
osw.flush();
}
Using Tomcat as the Servlet container, the following HTTP response gets generated:
I'm aware of the fact that response.getOutputStream() gives you a reference to
a decorated version of the actual OutputStream of the socket.
Tomcat decorates it in order to handle persistent connections using a chunked-encoding of the HTTP response body.
I wonder why chunks are 2000 hex bytes (8192 dec bytes) it looks like Tomcat always buffers the bytes before sending them to the socket output stream, wich seems to me an inefficient way of doing the job.
In other words, when I make the call:
osw.write(s,0,s.length());
where s.lenght() > buffer_size
I would expect an HTTP chunk of size s.lenght() and not of the dimension of
the buffer Tomcat uses to handle HTTP chunked-encoding.
Hope this is clear.

RE-Stream flash video from server to clients

I have an ipcamera that whenever multiple of users are connecting to it it becomes too slow.
I was thinking about getting the stream from the camera with my server and multiple of clients should be able to stream from the server instead of the poor ipcamera.
i set up a quick and dirty servlet just too see if it works :
#RequestMapping(value = "/", method = RequestMethod.GET, produces = "application/x-shockwave-flash")
public String getVideoStream(Locale locale, Model model, HttpServletRequest request, HttpServletResponse response) throws IOException {
logger.info("Start");
// An IPcamera stream example
URL url = new URL("http://www.earthcam.com/swf/ads5.swf");
URLConnection yc = url.openConnection();
OutputStream out = response.getOutputStream();
InputStream in = yc.getInputStream();
String mimeType = "application/x-shockwave-flash";
byte[] bytes = new byte[100000];
int bytesRead;
response.setContentType(mimeType);
while ((bytesRead = in.read(bytes)) != -1) {
out.write(bytes, 0, bytesRead);
}
logger.info("End");
I believe this might work, my problem right now is that :
bytesRead = in.read(bytes)
reads only 61894 bytes and that's it :( why is that happening? am i trying to get the stream wrong?
btw: i tried to do this with xuggler, but i had an error that compressed-SWF not supported.
thanks

Your code is working perfectly. I just fetched ads5.swf from your server and it is, indeed, 61894 bytes in length. The problem you're facing is that the SWF file is just the movie player. After being downloaded, the player then fetches the video stream from the server. By default (if this is some kind of turn-key streaming solution), it's probably trying to get the stream from the same server where the SWF comes from.

Trying to speed up reads from a Socket in java

I'm trying to make a client that can send HTTP requests and receive responses from web servers. I tried using Java's HttpURLConnection class but it doesn't give me enough control over what actually gets sent to the server, so I'd like to compose my own HTTP request messages and send them over a Socket. However, reading from the Socket's InputStream is prohibitively slow for some servers, and I'd like to speed that up if possible. Here's some code that I used to test how slow the reads were for the socket as compared to the HttpURLConnection:
public static void useURLConnection() throws Exception
{
URL url = new URL("http://" + hostName + "/");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
InputStream in = conn.getInputStream();
byte[] buffer = new byte[buffersize];
long start = System.currentTimeMillis();
while(in.read(buffer) != -1) { }
System.out.println(System.currentTimeMillis() - start);
}
public static void useSocket() throws Exception
{
byte[] request = ("GET / HTTP/1.1\r\nHost: " + hostName + "\r\n\r\n").getBytes();
Socket socket = new Socket(hostName, 80);
OutputStream out = socket.getOutputStream();
InputStream in = socket.getInputStream();
out.write(request);
byte[] buffer = new byte[buffersize];
long start = System.currentTimeMillis();
while(in.read(buffer) != -1) { }
System.out.println(System.currentTimeMillis() - start);
}
Both methods run in about the same amount of time for some servers, such as www.wikipedia.org, but reading from the socket is much slower -- minutes as opposed to milliseconds -- for others, such as www.google.com. Can someone explain why this is, and perhaps give me some pointers as to what, if anything, I can do to speed up the reads from the socket? Thanks.

So, HTTP/1.1 turns on keepalive by default for client requests. In your socket example, you're sending HTTP/1.1 as your version string, so you're implicitly accepting that you can support keepalive, yet you're completely disregarding it.
Basically, you're blocking trying to read more from the server, despite the fact that the server is waiting for you to do something (either send another request or close the connection.)
You need to either send a header "Connection: close" or send HTTP/1.0 as your version string.

Returning CSV file from Servlet using ServletOutputStream over HTTPS in Internet Explorer

I have a Servlet which is returning a csv file that is 'working' over HTTP in both internet explorer and firefox. When I execute the same Servlet over HTTPS only firefox continues to download the csv file over HTTPS. I don't think this is necessarily an Internet 6 or 7 issue described on MSDN :
The message is:
Internet Explorer cannot download
data.csv from mydomain.com Internet
Explorer was not able to open this
Internet site. The requested site is
either unavailable or cannot be found.
Please try again later.
Please note that the site is still 'up' after this message and you can continue to browse the site, its just the download of the CSV that prompts this message. I have been able to access similar files over https on IE from other j2ee applications so I believe it is our code. Should we not be closing the bufferedOutputStream?
UPDATE
whether to close or not to close the output stream:
I asked this question on the java posse forums and the discussion there is also insightful. In the end it seems that no container should rely on the 'client' (your servlet code in this case) to close this output stream. So if your failure to close the stream in your servlet causes a problem it is more a reflection on the poor implementation of your servlet container than your code. I sited the behavior of the IDEs and tutortials from Sun, Oracle and BEA and how they are also inconsistent in whether they close the stream or not.
About IE specific behavior: In our case a separate product 'Oracle Web Cache' was introducing the additional header values which impacts Internet explorer only because of the way IE implements the 'No Cache' requirement (see the MSDN article).
The code is:
public class DownloadServlet extends HttpServlet {
public void doGet(HttpServletRequest request,
HttpServletResponse response) throws ServletException,
IOException {
ServletOutputStream out = null;
ByteArrayInputStream byteArrayInputStream = null;
BufferedOutputStream bufferedOutputStream = null;
try {
response.setContentType("text/csv");
String disposition = "attachment; fileName=data.csv";
response.setHeader("Content-Disposition", disposition);
out = response.getOutputStream();
byte[] blobData = dao.getCSV();
//setup the input as the blob to write out to the client
byteArrayInputStream = new ByteArrayInputStream(blobData);
bufferedOutputStream = new BufferedOutputStream(out);
int length = blobData.length;
response.setContentLength(length);
//byte[] buff = new byte[length];
byte[] buff = new byte[(1024 * 1024) * 2];
//now lets shove the data down
int bytesRead;
// Simple read/write loop.
while (-1 !=
(bytesRead = byteArrayInputStream.read(buff, 0, buff.length))) {
bufferedOutputStream.write(buff, 0, bytesRead);
}
out.flush();
out.close();
} catch (Exception e) {
System.err.println(e); throw e;
} finally {
if (out != null)
out.close();
if (byteArrayInputStream != null) {
byteArrayInputStream.close();
}
if (bufferedOutputStream != null) {
bufferedOutputStream.close();
}
}
}

I am really confused about your "from back through the breast into the head" write mechanism. Why not simple (the servlet output stream will be bufferend, thats container stuff):
byte[] csv = dao.getCSV();
response.setContentType("text/csv");
response.setHeader("Content-Disposition", "attachment; filename=data.csv"));
reponse.setContentLength(csv.length);
ServletOutputStream out = response.getOutputStream();
out.write(csv);
There should also be no need to flush the output stream nor to close.
The header content should not be parsed case sensitive by IE, but who knows: do not camelcase fileName. The next question is the encoding. CSV is text, so you should use getWriter() instead or getOutputStream() and set the content type to "text/csv; charset=UTF-8" for example. But the dao should provide the CSV as String instead of byte[].
The servlet code has nothing to d with HTTPS, so the protocol does not matter from the server side. You may test the servlet from localhost with HTTP i hope.
What about filters in your application? A filter may als set an HTTP header (or as footer) with cache-control for example.