We would like to create a simple JAVA proxy server that would gather info from the request headers if need be.
We have implemented the proxy server, pointed our browser to the host and port, and we are able to browse sites that use simple HTTP protocol without an issue.
We support the GET and HEAD HTTP methods.
However, sites that use HTTPS cause an issue, as these initiate a CONNECT method, and we can't get the secure connection up, we are not sure what to send in the response. (Or if we would be able to monitor further requests from that point onward.)
Any suggestions? Any 3rd party implementation would do. But we would like to support HTTP and HTTPS in the same process.
private void intercept() throws IOException {
final DataInputStream socketInput = new DataInputStream(this.socket.getInputStream());
final String httpMessage = RequestHeader.readHttpHeader(socketInput);
//Request header is our own convenience file with utility methods that parses the request header.
requestHeader = new RequestHeader(httpMessage);
try {
if ("GET".equals(requestHeader.getType()) || "HEAD".equals(requestHeader.getType())) {
final URL url = new URL(requestHeader.getUrl());
final HttpURLConnection connectionHttp = (HttpURLConnection) url.openConnection();
connectionHttp.setRequestMethod(requestHeader.getType());
// Send response back to client
final DataInputStream dis = new DataInputStream(connectionHttp.getInputStream());
// Add response header
final StringBuilder sb = new StringBuilder();
sb.append(requestHeader.getHttpVersion() + " " + connectionHttp.getResponseCode() + " "
+ connectionHttp.getResponseMessage() + "\r\n");
final Map<String, List<String>> map = connectionHttp.getHeaderFields();
for (final Map.Entry<String, List<String>> entry : map.entrySet()) {
final String key = entry.getKey();
sb.append(key + " : "
+ entry.getValue().toString().replace("[", "").replace("]", "").replace(",", " ") + "\r\n");
}
sb.append("\r\n");
// Add response content
final DataOutputStream socketOutput = new DataOutputStream(this.socket.getOutputStream());
socketOutput.write(sb.toString().getBytes(), 0, sb.toString().getBytes().length);
final byte[] data = new byte[(int) Short.MAX_VALUE];
int index = dis.read(data, 0, (int) Short.MAX_VALUE);
while (index != -1) {
socketOutput.write(data, 0, index);
index = dis.read(data, 0, (int) Short.MAX_VALUE);
}
socketOutput.flush();
// NOTE this works perfectly fine for HTTP sites. We can intercept the communication properly.
} else if ("CONNECT".equals(requestHeader.getType())) {
// TODO establish connection
// First line of header: CONNECT www.facebook.com:443 HTTP/1.1
// We have tried to send back 200 ok response, but no further requests were sent.
} else {
//Nothing else is supported
return;
}
} catch (final MalformedURLException e) {
System.out.print("MalformedURLException " + e.getMessage());
// return Response.status(Status.BAD_REQUEST).entity(e.getMessage()).build();
} catch (final IOException e) {
System.out.print("IOException " + e.getMessage());
// return Response.status(Status.BAD_REQUEST).entity(e.getMessage()).build();
} finally {
System.out.println("Finished.");
}
}
HTTPComponents supports HTTPS. Also, you don't have to write the socket logic yourself, so there's that.
If I recall right, HTTPS works out of the box, no need for any odd configuration or special ssl calls.
Related
I have simple http server on android from Android Samples. I like this server, so I also want to recive POST data from browser. How can I recive it with standart things (without external libraries) ? I try to recive it like GET, but js console show connection error.
private void handle(Socket socket) throws IOException {
BufferedReader reader = null;
PrintStream output = null;
try {
String route = null;
// Read HTTP headers and parse out the route.
reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
String line;
while (!TextUtils.isEmpty(line = reader.readLine())) {
if (line.startsWith("GET /")) {
int start = line.indexOf('/') + 1;
int end = line.indexOf(' ', start);
route = line.substring(start, end);
break;
}
}
// Output stream that we send the response to
output = new PrintStream(socket.getOutputStream());
// Prepare the content to send.
if (null == route) {
writeServerError(output);
return;
}
byte[] bytes = loadContent(route);
if (null == bytes) {
writeServerError(output);
return;
}
// Send out the content.
output.println("HTTP/1.0 200 OK");
output.println("Content-Type: " + detectMimeType(route));
output.println("Content-Length: " + bytes.length);
output.println();
output.write(bytes);
output.flush();
} finally {
if (null != output) {
output.close();
}
if (null != reader) {
reader.close();
}
}
}
full code
In general the http request is a text fragment. The type of the request is indicated in the text. So in a GET request the example you provide first finds the "GET" string. Then parses the get request. The same should be done for POST. First identify the "POST" then parse the rest of the request.
I want to get an image from a server by making http request manually . I make a connection , create a http request wait for , get the http reply from server and then i want data inside it.I separated header from data and saved it into a file with ".jpeg" extension . But my .jpeg file could not be opened. What can i do ?
Note: i cant use any plugin or libraries . I just can process on http request!
Here my code :
try (Socket socket = new Socket("ceit.aut.ac.ir", 80)) {
// send an HTTP request to the web server
DataOutputStream outToServer = new DataOutputStream(socket.getOutputStream());
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(socket.getInputStream()));
outToServer.writeBytes("GET /~94131090/CN1_Project_Files/flower.jpeg HTTP/1.1\r\n");
outToServer.writeBytes("Host: ceit.aut.ac.ir:80\r\n");
outToServer.writeBytes("Connection: Close\r\n");
outToServer.writeBytes("Content-Type:image/*\r\n");
outToServer.writeBytes("\r\n");
// Receive an HTTP reply from the web server
boolean loop = true;
StringBuilder sb = new StringBuilder();
while (loop) {
if (inFromServer.ready()) {
int i = 0;
while (i != -1) {
i = inFromServer.read();
sb.append((char) i);
}
loop = false;
}
}
//Download Image
String data = separate(sb.toString());
//???
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
Here my separate function :
public String separate(String str){
String msg = str;
int index = msg.indexOf("close");
// "close" and blank end of http response line including \r\n
// 3(close) + 2(\r) + 2 (\n) + 2 (\r) + 2 (\n) = 4 + 2 + 2 + 2 = 6
return msg.substring(index+11);
}
I am writing a web proxy and so far I can read a GET request from a client, format it, and send it to the server, I believe that I have been able to get the response back from the server, but am unsure of how to send the response to the client.
Scanner readClient = new Scanner(new InputStreamReader(client.getInputStream()));
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(client.getInputStream()));
System.out.println("Client Request: ");
String request;
String host = "";
String path = "";
String[] parts = new String[4];
while((request = bufferedReader.readLine())!= null) {
if (request.indexOf("deflate") != -1) {
break;
}
if(request.indexOf("GET") != -1){
parts = request.split(" ");
path = parts[1];
System.out.println("THIS IS THE PATH: " + path);
}
if(request.indexOf("Host") != -1){
parts = request.split(": ");
host = parts[1];
System.out.println("THIS IS THE HOST: " + host);
}
System.out.println(request);
}
Socket server = new Socket(host, 80);
System.out.println("Successfully connected to host: " + host);
PrintWriter writeServer = new PrintWriter(new DataOutputStream(server.getOutputStream()));
InputStream readServer = server.getInputStream();
writeServer.print("GET " + path + "\r\n" + "Host: " + host + "\r\n" + "Connection: close\r\n\r\n");
writeServer.flush();
OutputStream writeClient = client.getOutputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte buffer[] = new byte[1024];
for(int s; (s=readServer.read(buffer)) != -1; )
{
baos.write(buffer, 0, s);
}
byte result[] = baos.toByteArray();
System.out.println("message sent");
}
catch (Exception e) {
System.out.println("Start Exception: " + e.getMessage());
}
}
** Not sure how I am supposed to record edits made to the question, but I have changed my wording and updated my code as well as included more of it.
You just need to read and copy the input to the output, taking note of the content-length or transfer-encoding headers on the way past, and stop when you exhaust either the content-length or whatever the transfer encoding thinks is the end of the response.
What kind of errors are you trying to catch? Did some homework last term using Scanner(URL.openStream()) and for anything "not normal" that would display as an error in a browser it would throw an Exception. Here's my catch() statement with some comments, it worked for what I needed at the time.
// do we have an error?
catch (Exception ex) {
// rather than specific exceptions related to the type of
// error (network, protocol, webserver content/configuration)
// the java.net.URL.openStream(URL) seems to return
// a different message in .getMessage() that you have to
// parse to figure out what happened.
// would these messages be different in a different java/jvm implementation?
String errorMsg=ex.getMessage();
// nicer errors below
//System.out.println("Error: "+errorMsg+"\n\r");
// what makes up our URL? this lets us get the hostname
// easily as urlParts[2].
String[] urlParts=theURL.split("/");
// on DNS failure (use http://aintthere.example.com as test URL)
// Exception.getMessage() seems to return the desired hostname
if(errorMsg.indexOf(urlParts[2])==0){
System.out.println("DNS error - invalid or unknown hostname");
}
// on a 404 error (use http://www.example.com/aintthere) the
// Exception.getMessage() appears to return the URL requested.
if(errorMsg.indexOf(theURL)==0){
System.out.println("The requested URL does not exist: "+theURL);
}
// no route to host or host off line and/or denying connections
if(errorMsg.indexOf("Connection timed out")==0){
System.out.println("That host is unreachable or is not allowing connections");
}
// turns out lots of different SSL errors - invalid certs, self signed certs, mis-matched hostnames,
// all sorts of things. seems easier to parse for ".security." in the message since
// they seem to come either from java.security.cert.* or sun.security.*
if(errorMsg.indexOf(".security.")!=-1){
System.out.println("Insecure SSL connection attempt - not allowed");
}
// both 500 (Internal Server Error) and 403 (Access to Resource Forbidden)
// produce nice standard looking error messages with the error number in them, so
// we check for that. Why doesn't 404 do that?
if(errorMsg.indexOf("HTTP response code: 500")!=-1){
System.out.println("The webserver is suffering from its own issues - Internal Server Error detected");
}
if(errorMsg.indexOf("HTTP response code: 403")!=-1){
System.out.println("Access to that resource is forbidden by the webserver configuration");
}
} // end catch
I am retrieving data through an API using HTTP Basic Authentication in the following way:
for (int i = 1; i < 10000; i++) {
try {
URL newurl = new URL ("SOMEURL" + i + ".json");
HttpURLConnection newconnection = (HttpURLConnection) newurl.openConnection();
newconnection.setRequestMethod("GET");
newconnection.setDoInput(true);
newconnection.setRequestProperty ("Authorization", "Basic " + encoding);
try {
InputStream newcontent = (InputStream)newconnection.getInputStream();
BufferedReader bfReader = new BufferedReader (new InputStreamReader (newcontent));
String newline;
if ((newline = bfReader.readLine()) != null) {
out.println(newline);
}
}
catch(Exception e2) {
}
}
catch(Exception e) {
e.printStackTrace();
}
}
Question: Instead of asking for authentication 10,000 times, can I get the authentication once for the first request, keep it alive, and make the remaining 9999 requests without asking for authentication?
Purpose: Faster processing, less load on the server.
Any comments?
HTTP is designed to be stateless, so there is nothing wrong with your approach. However, you can (but not recommened with an API) you sessions (with cookies) or provide a login URL which will issue a temporary auth token which you can pass.
Code
public HttpRequest(BufferedReader from) {
String firstLine = "";
try {
firstLine = from.readLine();
} catch (IOException e) {
System.out.println("Error reading request line: " + e);
}
String[] tmp = firstLine.split(" ");
method = tmp[0];
URI = tmp[1];
version = tmp[2];
System.out.println("URI is: " + URI);
if(method.equals("POST")){
try {
String line = from.readLine();
while (line.length() != 0) {
headers += line + CRLF;
if (line.startsWith("Host:")) {
tmp = line.split(" ");
if (tmp[1].indexOf(':') > 0) {
String[] tmp2 = tmp[1].split(":");
host = tmp2[0];
port = Integer.parseInt(tmp2[1]);
} else {
host = tmp[1];
port = HTTP_PORT;
}
}
line = from.readLine();
}
headers += "Connection: close" + CRLF;
headers += CRLF;
}
catch (IOException e) {
System.out.println("Error reading from socket: " + e);
return;
}
}
else {
System.out.println("Error: Method not supported");
return;
}
System.out.println("Host to contact is: " + host + " at port " + port);
}
Problem
I am making a proxy server using Java.
The code above handles an HTTP POST Request. It successfully reads the POST header and prints it in the command prompt but the body is missing.
Can you take look at my code and see the problem? Thanks.
(NOTE: I excluded the GET part because there were no problems with that.)
Result
The problem is that you still have things to read on the InputStream. That's why when you shut down the browser, there's nothing else to read so is printed. You have to read exactly the number of bytes that is declared in "Content-Length"
Try something like this:
int cL = Integer.valueOf(contentLength);
byte[] buffer = new byte[cL];
String postData = "";
System.out.println("Reading "+ cL + "bytes");
in.read(buffer, 0, cL);
postData = new String(buffer, 0, buffer.length);
System.out.println(postData);
The body request will be in the postData string.
This is not how to write a proxy server. A proxy server only has to do the following:
Accept incoming connections. For each connection:
Read an HTTP CONNECT request.
Connect to the target host specified in the CONNECT request and send an appropriate response to the client.
If the connect succeeded, copy bytes between the upstream server and the downstream client, in both directions simultaneously.
When you read EOS in one direction, shutdown the opposite socket for output.
When you have shutdown in both directions, close both sockets.
That's it. There is no parsing of POST requests or anything else required. Not even a Reader.