Develop Reference

Is HttpServletRequest.getRequestURL() spoofable?

The question is self explanatory, I hope. I am setting up a Spring Security enviroment with a CAS-server. Because the exact same application is deployed on the same server, but the server is accessible via different host names (.de domain, .com domain, possibly more than that) and we want to deploy the same application on test systems and the local one as well, I built a dynamic service, where the service URL is derived from request URL.
public static String makeDynamicUrlFromRequest(ServiceProperties serviceProperties, HttpServletRequest request) {
String serviceUrl = "https://backup-url.de/login";
URI uri = null;
try {
uri = new URI(request.getRequestURL().toString());
} catch (URISyntaxException e) {
logger.error("Someone tried accessing a disallowed service!", e);
}
if(uri != null){
serviceUrl = uri.getScheme() + "://" + uri.getHost() + "/login";
}
return serviceUrl;
}
Is it possible to spoof this? If it is, does an additional regex-check provide me with the necessary security against this?

#developerwjk
"If they modified the request url how would you have gotten the request?"
An HTTP server is just a program that listens on a TCP port, waits for some incoming text and writes out some text as a response. (A trivial web server can be written in like 20 lines of code.) It only sees the IP address and port of whatever connected to it. That could even be a proxy, or some other sort of middle-ware. If you don't tell the program "by the way, I reached you through the URL http://my.com/myapp/servlet" then it just doesn't know e.g. how a browser will reach it.
#Schaka
I don't know about your particular setup, but for jetty9, the result of getRequestURL is determined from the request URL in the request header, and - if the former is missing - the URL in the Host parameter. That is, if you connect to my.com and send the following request:
POST http://spoofed1.tld/myapp/servlet HTTP/1.1
Host: spoofed2.tld
(Keep in mind that the Host parameter is mandatory.)
Then getRequestURL will return http://spoofed1.tld/myapp/servlet
And if you send this:
POST /myapp/servlet HTTP/1.1
Host: spoofed2.tld
Then jetty itself will respond with
HTTP/1.1 302 Found
Location: http://spoofed2.tld/myapp/servlet
Content-Length: 0
Server: Jetty(<some version number>)
So the answer is yes, HttpServletRequest.getRequestURL() is spoofable! by modifying the request URL and/or the Host request header.

Socket versus URL website access

I have a Java application which opens an existing company's website using the Socket class:
Socket sockSite;
InputStream inFile = null;
BufferedWriter out = null;
try
{
sockSite = new Socket( presetSite, 80 );
inFile = sockSite.getInputStream();
out = new BufferedWriter( new OutputStreamWriter(sockSite.getOutputStream()) );
}
catch ( IOException e )
{
...
}
out.write( "GET " + presetPath + " HTTP/1.1\r\n\r\n" );
out.flush();
I would read the website with the stream inFile and life is good.
Recently this started to fail. I was getting an HTTP 301 "site has moved" error but no moved-to link. The site still exists and responds using the same original HTTP reference and any web browser. But the above code comes back with the HTTP 301.
I changed the code to this:
URL url;
InputStream inFile = null;
try
{
url = new URL( presetSite + presetPath );
inFile = url.openStream();
}
catch ( IOException e )
{
...
}
And read the site with the original code from inFile stream and it now works again.
This difference doesn't just occur in Java but it also occurs if I use Perl (using IO::Socket::INET approach opening the website port 80, then issuing a GET fails, but using LWP::Simple method get just works). In other words, I get a failure if I open the web page first with port 80, then do a GET, but it works fine if I use a class which does it "all at once" (that just says, "get me web page with such-and-such an HTTP address").
I thought I'd try the different approaches on http://www.microsoft.com and got an interesting result. In the case of opening port 80, followed by issuing the GET /..., I received an HTTP 200 response with a page that said, "Your current user agent
In one case, I tried the "port 80" open followed by GET / on www.microsoft.com and I received an HTTP 200 response page that said, "Your current user agent appears to be from an automated process...". But if I use the second method (URL class in Java, or LWP in Perl) I simply get their web page.
So my question is: how does the URL class (in Java) or the LWP module (in Perl) do its thing under the hood that makes it different from opening the website on port 80 and issuing a GET?

Most servers require the Host: header, to allow virtual hosting (multiple domains on one IP)

If you use a packet capturing software to see what's being sent when URL is used, you'll realize that there's a lot more than just "GET /" being sent. All sorts of additional header information are included. If a server gets just a simple "GET /", it's easy to deduct that it can't be a very sophisticated client on the other end.
Also, HTTP 1.0 is "outdated", the current version is 1.1.

Java URL implementation delegates to HttpURLConnection if it starts with "http:"

PHP receiving a POST request from Java

EDIT: this is what is shown in my website logs
xx.xx.xxx.xx - - [27/Jan/2012:17:42:24 -0500] "POST /dir/addData2.php HTTP/1.1" 200 - www.mywebsites.com "-" "Java/1.7.0" "-"
I am hosting my website at 1&1, and I want to have page blank.php that should take a POST request and upload it to my database. I think I am sending my POST properly, and that somehow I am not handling it properly on my website. Because nothing is being inserted to my database. The response has content length 0, but even if i send a header with the length of the string it wont change. Another option is that the host wont allow me to do remote post requests (still waiting on reply).
I send the post request from a Java application like this:
URL url = new URL("www.mywebsite.com/blank.php");
HttpURLConnection request = (HttpURLConnection)url.openConnection();
request.setRequestProperty("Content-type","application/x-www-form-urlencoded");
request.setRequestMethod("POST");
OutputStreamWriter post = new OutputStreamWriter(request.getOutputStream());
String data = URLEncoder.encode("account", "UTF-8") + "=" + URLEncoder.encode(message[0], "UTF-8");
data += "&" + URLEncoder.encode("message", "UTF-8") + "=" + URLEncoder.encode(message[2], "UTF-8");
data += "&" + URLEncoder.encode("type", "UTF-8") + "=" + URLEncoder.encode(message[0], "UTF-8");
post.write(data);
post.flush();
/*
/ String example
/account=103&message=Feller+1391.88+0&type=103
*/
The response from the server is:
null=[HTTP/1.1 200 OK]
Date=[Fri, 27 Jan 2012 21:59:10 GMT]
Content-Length=[0]
Keep-Alive=[timeout=2, max=200]
Connection=[Keep-Alive]
Content-Type=[text/html]
Server=[Apache]
X-Powered-By=[PHP/5.2.17]
My webpage has this basic PHP code (right now, will improve/check for stuff later)
$link = mysql_connect($hostname, $username, $password);
$db_selected = mysql_select_db($database, $link);
$query = "INSERT INTO newData(account, message, type) VALUES('$_POST[account]', '$_POST[message]', '$_POST[type]')";
mysql_query($query) || die();
I want to point out that I do not have a private server, and I will probably use http://hc.apache.org/httpcomponents-core-ga/examples.html later. Right now I just want to send a string from my Java application, receive it with PHP and insert to MySQL database.

Yes, I'm certain your ISP allows both "post" and "get" HTTP server requests.
Superficially, both your Java and PHP look "OK".
STRONG SUGGESTION:
1) Check your Apache logs on the server.
Make sure the request arrived (it probably did).
Check for errors (there could well be none for your request - but you SHOULD see SOMETHING in the Apache error log.
2) Verify PHP is working.
Writing a "hello world" page with "phpinfo ();" is a good way to do this.
3) Verify that MySQL is working.
Writing a "hello world" PHP page to verify you can 1) connect, 2) make a query, and 3) echo the query back in your web browser is ideal.
4) Verify that you can successfully read the "$_POST[account]" value you THINK you're getting from the client.
5) At that point, you should be able to:
a) have the client communicate with your server
b) read the post request and write it to your database
'Hope that helps .. and please post back with any specific questions/problems you might have.

You haven't told the client code that this is a POST request. It will probably do a GET by default.
url = new URL(targetURL);
connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("POST");

It looks like you're not setting the request method on your HttpUrlConnection, and the default is GET. You'd need something like
request.setRequestMethod("POST");
If that doesn't work, I'd consider rather using the Apache HTTP client right away from the beginning, it's easiert to use than Java's standard HTTP client API.

android app failed to get content at 80 port with HttpClient

I want to develop an android app to get post.xml with HttpClient. But it failed to get content with 80 port.
If I start the web server(WEBrick here) with 3000 port, the URI is http://192.168.1.103:3000/posts.xml;
Android App can get response with correct length, like 568;
The same web files, I started them with another server (Nignx here) with 80 port, the uri is
"http://192.168.1.103/posts.xml; The Android App can NOT get content with length, it's -1 here.
This URI can be opened with browser(both PC and android emulator) correctly. Furthermore, the response is "HTTP/1.1 200 OK" with responsep.getStatusLine().
is it related with "Socket ports below 1024 can NOT access in linux like system", which is on
http://groups.google.com/group/android-developers/browse_thread/thread/660123ca64ba1229#
Any Ninja can tell me what should I do if I can to get content with 80 port?
The following is my code.
public class AndroidWorldActivity extends Activity {
/** Called when the activity is first created. */
TextView tv;
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
retreiveProjects();
}
private void retreiveProjects()
{
HttpClient httpClient = new DefaultHttpClient();
try
{
String url3000 = "http://192.168.1.103:3000/posts.xml";
String url = "http://192.168.1.103/posts.xml";
Log.d( "posts", "performing get " + url3000);
HttpGet httpGet=new HttpGet(url3000);
HttpResponse responsep=httpClient.execute(httpGet);
System.out.println(responsep.getStatusLine());
HttpEntity httpEntity = responsep.getEntity();
int length = ( int ) httpEntity.getContentLength();
// print the lenght of content
System.out.println("The content length is: "+length);
Log.d( "posts", "The content length is: " + length );

From your description, I understand that you are trying to connect from adroid to an external HTTP server attached to port 80? If so, restriction about port lower than 1024 on android has nothing to do (you are not trying to listen on port 80 on android device). I think, that you have a problem with Nginx.
Try to execute GET request from an external machine to Nginx and investigate response content (headers, payload). I would recommend to do it with some more low-level tool instead of web browser (almost all web browser nowadays are able to "repair" illegal server responses), for example curl:
curl -D - http://192.168.1.103/posts.xml

You seem to have two separate problems.
The problem on WeBrick seems to be that the UNIX / Linux won't allow your web server to bind to port 80. There are two obvious things to check:
Is the web server running with root privilege when it attempts to bind to the port? See Is there a way for non-root processes to bind to "privileged" ports on Linux? for a variety of ways to work around this problem.
Is some other application already bound to port 80? Is so, kill it and try to run your web server again.
The problem with Nignx is different. Here, the server is up and running and giving your client responses, but the client is seeing -1 as the content length.
This is normal behaviour. The getContentLength() method returns -1 if the response doesn't have a "Content-length" header, and it is perfectly legitimate (according to the HTTP specification) for a response to not have this header. You have two choices:
Change your client-side application to deal with the case where the content length is unspecified; e.g. just read the body into a buffer and count how many bytes you got.
Change the server to set the relevant header.
FOLLOWUP
I see. Your original question was hard to understand and I misinterpreted. You seemed to be saying that WEBrick wasn't working at all.
The difference between WEBrick and Nginx is that they simply implement the response differently. Both are legitimate (valid) implementation. The real problem is that your application is assuming that a web server will always set the "Content-length" header. That is an incorrect assumption.
To repeat, the problem / fault is in your client code, not in Nginx.

How can I send an HTTP Response using only standard network libraries?

I'm working on my first homework project in a web programming class, which is to write a simple web server in Java. I'm at the point where I have data being transmitted back and forth, and to the untrained eye, my baby server seems to be working fine. However, I can't find a way to send appropriate responses. (In other words, an invalid page request would show a 404-ish HTML page, but it still returns a 200 OK status when I view response headers).
I'm limited to being able to use standard network libraries for socket management and standard I/O libraries to read and write bytes and strings from an input stream. Here's some pertinent code:
From my main...
ServerSocket servSocket = new ServerSocket(port, 10); // Bind the socket to the port
System.out.println("Opened port " + port + " successfully!");
while(true) {
//Accept the incoming socket, which means that the server process will
//wait until the client connects, then prepare to handle client commands
Socket newDataSocket = servSocket.accept();
System.out.println("Client socket created and connected to server socket...");
handleClient(newDataSocket); //Call handleClient method
}
From the handleClient method...(inside a loop that parses the request method and path)
if(checkURL.compareTo("/status") == 0) { // Check to see if status page has been requested
System.out.println("STATUS PAGE"); // TEMPORARY. JUST TO MAKE SURE WE ARE PROPERLY ACCESSING STATUS PAGE
sendFile("/status.html", dataStream);
}
else {
sendFile(checkURL, dataStream); // If not status, just try the input as a file name
}
From sendFile method...
File f = new File(where); // Create the file object
if(f.exists() == true) { // Test if the file even exists so we can handle a 404 if not.
DataInputStream din;
try {
din = new DataInputStream(new FileInputStream(f));
int len = (int) f.length(); // Gets length of file in bytes
byte[] buf = new byte[len];
din.readFully(buf);
writer.write("HTTP/1.1 200 OK\r\n"); // Return status code for OK (200)
writer.write("Content-Length: " + len + "\r\n"); // WAS WRITING TO THE WRONG STREAM BEFORE!
writer.write("Content-Type: "+type+"\r\n\r\n\r\n"); // TODO VERIFY NEW CONTENT-TYPE CODE
out.write(buf); // Writes the FILE contents to the client
out.flush();
out.close();
} catch (FileNotFoundException e) {
e.printStackTrace(); // Not really handled since that's not part of project spec, strictly for debug.
}
}
else {
writer.write("HTTP/1.1 404 Not Found\r\n"); // Attempting to handle 404 as simple as possible.
writer.write("Content-Type: text/html\r\n\r\n\r\n");
sendFile("/404.html", sock);
}
Can anybody explain how, in the conditional from sendFile, I can change the response in the 404 block (Like I said before, the response headers still show 200 OK)? This is bugging the crap out of me, and I just want to use the HTTPResponse class but I can't. (Also, content length and type aren't displayed if f.exists == true.)
Thanks!

Edit It looks to me like in the 404 situation, you're sending something like this:
HTTP/1.1 404 Not Found
Content-Type: text/html
HTTP/1.1 200 OK
Content-Length: 1234
Content-Type: text/html
...followed by the 404 page. Note the 200 line following the 404. This is because your 404 handling is calling sendFile, which is outputting the 200 response status code. This is probably confusing the receiver.
Old answer that missed that:
An HTTP response starts with a status line followed (optionally) by a series of headers, and then (optionally) includes a response body. The status line and headers are just lines in a defined format, like (to pick a random example):
HTTP/1.0 404 Not Found
To implement your small HTTP server, I'd recommend having a read through the spec and seeing what the responses should look like. It's a bit of a conceptual leap, but they really are just lines of text returned according to an agreed format. (Well, it was a conceptual leap for me some years back, anyway. I was used to environments that over-complicated things.)
It can also be helpful to do things like this from your favorite command line:
telnet www.google.com 80
GET /thispagewontbefound
...and press Enter. You'll get something like this:
HTTP/1.0 404 Not Found
Content-Type: text/html; charset=UTF-8
X-Content-Type-Options: nosniff
Date: Sun, 12 Sep 2010 23:01:14 GMT
Server: sffe
Content-Length: 1361
X-XSS-Protection: 1; mode=block
...followed by some HTML to provide a friendly 404 page. The first line above is the status line, the rest are headers. There's a blank line between the status line/headers and the first line of content (e.g., the page).

The problem you are seeing is most likely related to a missing flush() on your writer. Depending on which type of Writer you use the bytes are first written to a buffer that needs to be flushed to the stream. This would explain why Content-Length and Content-Type are missing in the output. Just flush it before you write additional data to the stream.
Further you call sendFile("/404.html", sock);. You did not post the full method here - but I suppose that you call it recursively inside sendFile and thus send the 200 OK status for your file /404.html.

Based on your reported symptoms, I think the real problem is that you are not actually talking to your server at all! The evidence is that 1) you cannot get a 404 response, and 2) a 200 response does not have the content length and type. Neither of these should be possible ... if you are really talking to the code listed above.
Maybe:
you are talking to an older version of your code; i.e. something is going wrong in your build / deploy cycle,
you are (mistakenly) trying to deploy / run your code in a web container (Jetty, Tomcat, etc), or
your client code / browser is actually talking to a different server due to proxying, an incorrect URL, or something like that.
I suggest that you add some trace printing / logging at appropriate points of your code to confirm that it is actually being invoked.