I'm trying to post some data from a Java client using sockets. It talks to localhost running php code, that simply spits out the post params sent to it.
Here is Java Client:
public static void main(String[] args) throws Exception {
Socket socket = new Socket("localhost", 8888);
String reqStr = "testString";
String urlParameters = URLEncoder.encode("myparam="+reqStr, "UTF-8");
System.out.println("Params: " + urlParameters);
try {
Writer out = new OutputStreamWriter(socket.getOutputStream(), "UTF-8");
out.write("POST /post3.php HTTP/1.1\r\n");
out.write("Host: localhost:8888\r\n");
out.write("Content-Length: " + Integer.toString(urlParameters.getBytes().length) + "\r\n");
out.write("Content-Type: text/html\r\n\n");
out.write(urlParameters);
out.write("\r\n");
out.flush();
InputStream inputstream = socket.getInputStream();
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
String string = null;
while ((string = bufferedreader.readLine()) != null) {
System.out.println("Received " + string);
}
} catch(Exception e) {
e.printStackTrace();
} finally {
socket.close();
}
}
This is how post3.php looks like:
<?php
$post = $_REQUEST;
echo print_r($post, true);
?>
I expect to see an array (myparams => "testString") as the response. But its not passing post args to server.
Here is output:
Received HTTP/1.1 200 OK
Received Date: Thu, 25 Aug 2011 20:25:56 GMT
Received Server: Apache/2.2.17 (Unix) mod_ssl/2.2.17 OpenSSL/0.9.8r DAV/2 PHP/5.3.6
Received X-Powered-By: PHP/5.3.6
Received Content-Length: 10
Received Content-Type: text/html
Received
Received Array
Received (
Received )
Just a FYI, this setup works for GET requests.
Any idea whats going on here?
As Jochen and chesles rightly point out, you are using the wrong Content-Type: header - it should indeed be application/x-www-form-urlencoded. However there are several other issues as well...
The last header should be seperated from the body by a blank line between the headers and the body. This should be a complete CRLF (\r\n), in your code it is just a new line (\n). This is an outright protocol violation and I'm a little surprised you haven't just got a 400 Bad Request back from the server, although Apache can be quite forgiving in this respect.
You should specify Connection: close to ensure that you are not left hanging around with open sockets, the server will close the connection as soon as the request is complete.
The final CRLF sequence is not required. PHP is intelligent enough to sort this out by itself, but other server languages and implementations may not be...
If you are working with any standardised protocol in it's raw state, you should always start by at least scanning over the RFC.
Also, please learn to secure your Apache installs...
It looks like you are trying to send data in application/x-www-form-urlencoded format, but you are setting the Content-Type to text/html.
Use
out.write("Content-Type: application/x-www-form-urlencoded\n\n");
instead. As this page states:
The Content-Length and Content-Type headers are critical because they tell the web server how many bytes of data to expect, and what kind, identified by a MIME type.
For sending form data, i.e. data in the format key=value&key2=value2 use application/x-www-form-urlencoded. It doesn't matter if the value contains HTML, XML, or other data; the server will interpret it for you and you'll be able to retrieve the data as usual in the $_POST or $_REQUEST arrays on the PHP end.
Alternatively, you can send your data as raw HTML, XML, etc. using the appropriate Content-Type header, but you then have to retrieve the data manually in PHP by reading the special file php://input:
<?php
echo file_get_contents("php://input");
?>
As an aside, if you're using this for anything sufficiently complex, I would strongly recommend the use of an HTTP client library like HTTPClient.
Related
I am using HttpsURLConnection to call a server and return the response returned from the HttpsURLConnection from my servlet. I am copying the response from HttpssURLConnection to HttpServletresponse using streams, copying bytes from the httpconnection response input stream to the response's output stream, checking the end by seeing if read returns < 0.
Following is the code for copying the response. The variable response is of type HttpServletResponse and the variable httpCon is of type HttpsURLConnection.
InputStream responseStream = httpCon.getInputStream();
if (responseStream != null)
{
OutputStream os = response.getOutputStream();
byte[] buffer = new byte[1024];
int len;
while ((len = responseStream.read(buffer)) >= 0)
{
os.write(buffer, 0, len);
}
os.flush();
os.close();
}
On the client side, I am using python requests library to read the response.
What I am seeing that if I use the curl to test my servlet, I am getting the proper response json, response = u'{"key":"value"}'.
If i read it from the requests python, it is putting some extra characters in the response , the response looks like the following
response = u'b0\r\n{"key":"value"}\r\n0\r\n\r\n'
Both the strings are unicode. But the second one has extra characters.
Same resonse if I try from curl/Postman restclient, I am able to get it properly. But from python requests, it is not working. I tried another livetest library in python, with that also, it is not working and the response has same characters. I also tried to change the accept-encoding header but it did not have any effect.
Because of this, I am not able to parse the json.
I don't want to change the client to parse this kind of string.
Can I change something on the server so that it will work correctly?
Did the response contain the below header "Transfer-Encoding: chunked"?
The response should be in Chunked transfer encoding
https://en.wikipedia.org/wiki/Chunked_transfer_encoding.
In this case, you get \r\n0\r\n\r\n at the end of the response is as expected since it is terminating symbol of this encoding. I guest curl/Postman just help us to handle Chunked transfer encoding, so you can't find these chunked symbols.
I am trying to read bytes into chars from a server which is not maintained by me. Here am the client. My issue is am not getting the required response from the request sent to the server. From my understanding, to detect the end of a message, there are three common ways:
*Closing the connection at the end of the message.
*Putting the length of the message before the data itself
*Using a separator; some value which will never occur in the normal data
So this what I have done so far.Am using sockets to achieve writing to the server like this:
Socket outgoing = new Socket(Host, Port);
String request = "GET http://www.firtRequest.com/ HTTP/1.1\r\nHost: www.firstRequest.com\r\n\r\n" + "GET http://www.secondRequest.com/ HTTP/1.1\r\nHost: www.secondRequest.com\r\n\r\n";
outgoing.getOutputStream().write(request.getBytes());
outgoing.getOutputStream().flush();
Using getInputStream() to read from the socket server,I should get two reponses back but the second response carries a html tag which from my understanding isn't part of the resonse so am guessing am not reading till the end of the stream for the first request sent to the server or am not reading the reponses properly altogether.
Output1:
HTTP/1.1 200 OK
Date: mon,24 Aug 2015 09:02:30 GMT
Content-Type: application/json
Content-Length: 42
Output2:
<!DOCTYPE html PUBLIC "-//W3C"
....
<head>
....
</head>
<body>
...
</body>
HTTP/1.1 200 OK
Here is my read method in which am trying to detect the end of the stream using "\r\n\r\n" tag in the reponse or when the stream hits -1.
public static String ReadStream(InputStream inputStream) throws IOException {
StringBuilder builder = new StringBuilder();
while (true) {
int rdL = inputStream.read();
if (rdL == -1) {
break;
}
// Convert the bytes read into characters
builder.append((char) rdL);
if (builder.indexOf("\r\n\r\n") != -1) {
// EOS detected
break;
}
}
return builder.toString();
}
Any pointers to what am doing wrong to be getting that html tag? Thanks
there are three common ways:
There's only one 'common way' with HTTP, and that is to implement RFC 2616 correctly. You haven't made the slightest attempt here. Look it up. But there's no good reason to try to implement HTTP yourself when HttpURLConnection already exists, not to mention numerous third-party HTTP APIs.
I have developed manually a soap connection via Java sockets (it was a very simple soap request and Axis was giving lots of build problem).
To achieve this, I basically copied the HTTP header I was getting out of Soap UI when , and coded the follwing:
String hostname = "aaaaa";
int port = 11111;
InetAddress addr = InetAddress.getByName(hostname);
Socket sock = new Socket(addr, port);
sock.setSoTimeout(100000);
BufferedWriter wr = new BufferedWriter(new OutputStreamWriter(sock.getOutputStream(),"UTF-8"));
wr.write("POST " + "http://aaaa:11111/servicePath" + " HTTP/1.1\r\n");
wr.write("Host: aaaaa:11111\r\n");
wr.write("Accept-Encoding: gzip,deflate\r\n");
wr.write("Content-Length: " + soapXml.length() + "\r\n");
wr.write("Content-Type: text/xml; charset=\"UTF-8\"\r\n");
//wr.write("Connection: Keep-Alive\r\n");
wr.write("SOAPAction: \"/someSoapAction\"\r\n");
wr.write("User-Agent: Apache-HttpClient/4.1.1 (java 1.5)\r\n");
wr.write("\r\n");
wr.write(soapXml);
wr.flush();
The requests are successful, meaning I get from the service the responses I expect.
For instance, when I put in my hand written "soapXml" some invalid parameters, I get a 500 error back with XML explaining the problem, If I set everything correctly I get a 200 OK with an xml body following the header.
The problem is that the socket hangs 60 seconds before reading the HTTP body in case of a 200OK.
It basically reads the full header, then waits 60 seconds, then (I think some protocol times out and) finally reads the xml body.
Here's the code with which I read the response:
String line;
BufferedReader rd = new BufferedReader(new java.io.InputStreamReader(sock.getInputStream(),"UTF-8"));
while ((line = rd.readLine()) != null) {
i++;
LOG.debug("cycle: "+i+" -------- "+line);
//after printing the header, it hangs 60 seconds before printing the follwing XML
if ((line.length() >0) && (line.charAt(0) == '<'))
{
responseXML = line;
}
}
Here is a sample 200 OK header:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: text/xml;charset=utf-8
Content-Length: 1052
Date: Sun, 13 Jan 2013 08:33:45 GMT -- hanging 60 seconds here
-- blank line
<?xml version="1.0" encoding="UTF-8"?>................</SOAP-ENV:Envelope>
Has anybody ever faced this? Please note this is not a server problem apparently, since with soap UI the answer is immediately provided.
Thank you
The server is keeping the connection open for HTTP keep-alive, clearly for a 60 second timeout. Either change to HTTP 1.0 or else take note of the returned Content-length header and stop reading at that many bytes.
The server environment change (a new machine was provided) solved the issue.
Now response time is below 1 sec also for 200OK.
Unluckily I don't have any details regarding the tech stack of the envrionment. Anyways thanks for help.
I am trying to send a zipfile from my android application to our server and I keep getting a 411 length required error.
Here is the code that I am using to do that.
HttpPost post = new HttpPost("http://www.xyz.org/upload.json");
post.setHeader(C.constants.HTTP_CONTENT_TYPE, "application/octet-stream");
try {
FileInputStream fis = new FileInputStream("/data/data/org.myapp.appname/app_content.zip");
InputStreamEntity reqEntity = new InputStreamEntity(fis, -1);
post.setEntity(reqEntity);
String response = doPost(post);
Log.v(tag, "response from server " + response);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
What am I doing wrong here and may I also know how I can add more parameters with this post to send them to the server.
You should use a multipart MIME type, and parts for each parameters and for the files. See this old blog post of mine.
As you should know, an HTTP request (and response, too) is made up of two section: a header and a body (also called entity). In the header you specify a Content-Type so that the server script knows how to interpret the following bytes (you can also specify the encoding for textual data on the same line).
After the header, the client sends an empty line and an optional entity. A multipart entity is like a stack of parts separated by a boundary. This is what your desired HTTP request should look like on the wire:
POST /path/to/script.json HTTP/1.1
Host: localhost
Content-Type: multipart/form-data; boundary=a2ksdf0?--
a2ksdf0?--
Content-Disposition: form-data; name="profile_picture"; filename="me.png"
Content-Type: image/png
<content here>
a2ksdf0?--
Content-Disposition: form-data; name="username";
Content-Type: text/plain
Joe Black
BTW, it's the first time I see an upload script named upload.json. What do you use on the server side?
I'm working on my first homework project in a web programming class, which is to write a simple web server in Java. I'm at the point where I have data being transmitted back and forth, and to the untrained eye, my baby server seems to be working fine. However, I can't find a way to send appropriate responses. (In other words, an invalid page request would show a 404-ish HTML page, but it still returns a 200 OK status when I view response headers).
I'm limited to being able to use standard network libraries for socket management and standard I/O libraries to read and write bytes and strings from an input stream. Here's some pertinent code:
From my main...
ServerSocket servSocket = new ServerSocket(port, 10); // Bind the socket to the port
System.out.println("Opened port " + port + " successfully!");
while(true) {
//Accept the incoming socket, which means that the server process will
//wait until the client connects, then prepare to handle client commands
Socket newDataSocket = servSocket.accept();
System.out.println("Client socket created and connected to server socket...");
handleClient(newDataSocket); //Call handleClient method
}
From the handleClient method...(inside a loop that parses the request method and path)
if(checkURL.compareTo("/status") == 0) { // Check to see if status page has been requested
System.out.println("STATUS PAGE"); // TEMPORARY. JUST TO MAKE SURE WE ARE PROPERLY ACCESSING STATUS PAGE
sendFile("/status.html", dataStream);
}
else {
sendFile(checkURL, dataStream); // If not status, just try the input as a file name
}
From sendFile method...
File f = new File(where); // Create the file object
if(f.exists() == true) { // Test if the file even exists so we can handle a 404 if not.
DataInputStream din;
try {
din = new DataInputStream(new FileInputStream(f));
int len = (int) f.length(); // Gets length of file in bytes
byte[] buf = new byte[len];
din.readFully(buf);
writer.write("HTTP/1.1 200 OK\r\n"); // Return status code for OK (200)
writer.write("Content-Length: " + len + "\r\n"); // WAS WRITING TO THE WRONG STREAM BEFORE!
writer.write("Content-Type: "+type+"\r\n\r\n\r\n"); // TODO VERIFY NEW CONTENT-TYPE CODE
out.write(buf); // Writes the FILE contents to the client
out.flush();
out.close();
} catch (FileNotFoundException e) {
e.printStackTrace(); // Not really handled since that's not part of project spec, strictly for debug.
}
}
else {
writer.write("HTTP/1.1 404 Not Found\r\n"); // Attempting to handle 404 as simple as possible.
writer.write("Content-Type: text/html\r\n\r\n\r\n");
sendFile("/404.html", sock);
}
Can anybody explain how, in the conditional from sendFile, I can change the response in the 404 block (Like I said before, the response headers still show 200 OK)? This is bugging the crap out of me, and I just want to use the HTTPResponse class but I can't. (Also, content length and type aren't displayed if f.exists == true.)
Thanks!
Edit It looks to me like in the 404 situation, you're sending something like this:
HTTP/1.1 404 Not Found
Content-Type: text/html
HTTP/1.1 200 OK
Content-Length: 1234
Content-Type: text/html
...followed by the 404 page. Note the 200 line following the 404. This is because your 404 handling is calling sendFile, which is outputting the 200 response status code. This is probably confusing the receiver.
Old answer that missed that:
An HTTP response starts with a status line followed (optionally) by a series of headers, and then (optionally) includes a response body. The status line and headers are just lines in a defined format, like (to pick a random example):
HTTP/1.0 404 Not Found
To implement your small HTTP server, I'd recommend having a read through the spec and seeing what the responses should look like. It's a bit of a conceptual leap, but they really are just lines of text returned according to an agreed format. (Well, it was a conceptual leap for me some years back, anyway. I was used to environments that over-complicated things.)
It can also be helpful to do things like this from your favorite command line:
telnet www.google.com 80
GET /thispagewontbefound
...and press Enter. You'll get something like this:
HTTP/1.0 404 Not Found
Content-Type: text/html; charset=UTF-8
X-Content-Type-Options: nosniff
Date: Sun, 12 Sep 2010 23:01:14 GMT
Server: sffe
Content-Length: 1361
X-XSS-Protection: 1; mode=block
...followed by some HTML to provide a friendly 404 page. The first line above is the status line, the rest are headers. There's a blank line between the status line/headers and the first line of content (e.g., the page).
The problem you are seeing is most likely related to a missing flush() on your writer. Depending on which type of Writer you use the bytes are first written to a buffer that needs to be flushed to the stream. This would explain why Content-Length and Content-Type are missing in the output. Just flush it before you write additional data to the stream.
Further you call sendFile("/404.html", sock);. You did not post the full method here - but I suppose that you call it recursively inside sendFile and thus send the 200 OK status for your file /404.html.
Based on your reported symptoms, I think the real problem is that you are not actually talking to your server at all! The evidence is that 1) you cannot get a 404 response, and 2) a 200 response does not have the content length and type. Neither of these should be possible ... if you are really talking to the code listed above.
Maybe:
you are talking to an older version of your code; i.e. something is going wrong in your build / deploy cycle,
you are (mistakenly) trying to deploy / run your code in a web container (Jetty, Tomcat, etc), or
your client code / browser is actually talking to a different server due to proxying, an incorrect URL, or something like that.
I suggest that you add some trace printing / logging at appropriate points of your code to confirm that it is actually being invoked.