Develop Reference

Filter HTTPEntity output to include new lines

I am using org.apache.http.HttpEntity for doing a multipart/form data POST to HTTPURLConnection to upload a file.
Here is the code that I am using.
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
String part1 = "\n{\"name\":\"test.txt\",\"creationTime\":1527023510389,\"fileUri\":\"/storage/test.txt\"}";
File file = new File("/storage/test.txt");
HttpEntity entity = MultipartEntityBuilder.create()
.setMode(HttpMultipartMode.BROWSER_COMPATIBLE)
.addBinaryBody("data", part1.getBytes(), ContentType.APPLICATION_JSON, "data.txt")
.addBinaryBody("file", file, ContentType.TEXT_PLAIN, filename)
.setBoundary(boundaryString)
.build();
OutputStream os = conn.getOutputStream();
entity.writeTo(os);
I see that the body is being posted as the following.
--BOUNDARY
Content-Disposition: form-data; name="metadata"; filename="metadata.txt"
Content-Type: application/json
{"name":"test.txt","creationTime":1527023510389,"fileUri":"/storage/test.txt"}
--BOUNDARY
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=ISO-8859-1
test file contents
--BOUNDARY--
The problem is that the server requires a new line between the Content-Type and the contents of the first part. I've tried adding extra "\n" to the beginning contents (as seen but it gets erased when using HttpEntity.writeto().
The output that I want is the following:
--BOUNDARY
Content-Disposition: form-data; name="metadata"; filename="metadata.txt"
Content-Type: application/json
{"name":"test.txt","creationTime":1527023510389,"fileUri":"/storage/test.txt"}
--BOUNDARY
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=ISO-8859-1
test file contents
--BOUNDARY--
I attempted to modify rewriting the output but not sure if this is the best way to do it by storing in a temporary file. The files I will be working with will be up to 20mb if that makes any difference.
entity.writeTo(new FileOutputStream("file.tmp"));
BufferedReader reader = new BufferedReader(new FileReader("file.tmp"));
OutputStream os = conn.getOutputStream();
PrintWriter writer = new PrintWriter(new BufferedOutputStream(os));
String str;
while ((str = reader.readLine()) != null) {
writer.println(str);
if (str.contains("Content-Type: ")) {
writer.println("\n");
}
}
writer.close();
reader.close();
os.close();
conn.connect();
if (conn.getResponseCode() == HttpURLConnection.HTTP_OK) {
// It's failing when accessing the above method
}
I tried running the above code and I get the following error:
java.lang.IllegalStateException: state: 2
at com.android.okhttp.internal.http.HttpConnection.readResponse(HttpConnection.java:234)
at com.android.okhttp.internal.http.HttpTransport.readResponseHeaders(HttpTransport.java:104)
at com.android.okhttp.internal.http.HttpEngine.readNetworkResponse(HttpEngine.java:1156)
at com.android.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:976)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:509)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:438)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:567)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java)

It turns out that the HttpEntity.writeTo method is putting the necessary new lines but when I was printing the output to System.out, Android Studio's Logcat does not show plain new lines. I confirmed this by opening the file.tmp I was creating above and it had the proper new lines in there. It looks like there's some other error with the request since the body is valid for the server.
EDIT: Found the error in my request. I wasn't setting the Content-Type (I think I erased it while deleting some other code). I ended up using this to set the content type.
conn.addRequestProperty(entity.getContentType().getName(), entity.getContentType().getValue());

Extracting the body from HTTP post requestH

I have a weird problem when trying to extract the body of a given
HTTP post request.
If I try to extract only the header, it works fine. When I try to extract the body, the method blocks (even thought the stream still has data in it).
Here is my code:
private void extractHeader() throws Exception {
StringBuffer buffer = new StringBuffer();
InputStreamReader reader = new InputStreamReader(socket.getInputStream());
BufferedReader bufferedReader = new BufferedReader(reader);
boolean extractBody = false;
int bodyLength = 0;
String line;
while (!(line = bufferedReader.readLine()).equals("")) {
buffer.append(line + "");
if (line.startsWith("POST")) {
extractBody = true;
}
if (line.startsWith("Content-Length:")) {
bodyLength = Integer.valueOf(line.substring(line.indexOf(' ') + 1, line.length()));
}
}
requestHeader = buffer.toString();
if (extractBody) {
char[] body = new char[bodyLength];
reader.read(body, 0, bodyLength);
requestBody = new String(body);
}
}
And this is the request request:
POST /params_info.html HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:42.0) Gecko/20100101 Firefox/42.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://localhost:8080/index.html
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 31
firstname=Mickey&lastname=Mouse
From what I understand, the loop will go until seeing the empty string
and then stoping. At this stage, the reader can read 'Content-Length' number of bytes. So it should have no problem reading the body and finish. Instead, it blocks on the line 'reader.read(body, 0, bodyLength);'
(The reason I don't use readLine() is because body does not end with \n).
I've tried debugging it in al kinds of ways but I get nothing. Can anyone please help with this?

You're reading the header using the bufferedReader:
while (!(line = bufferedReader.readLine()).equals("")) {
but read the body using reader, which has no data available, as this has been read and buffered by the bufferedReader:
reader.read(body, 0, bodyLength);
Change that line to
bufferedReader.read(body, 0, bodyLength);

Java HttpUrlConnection POST request special characters strange behavior

i'm trying to implements a POST request with HttpURLConnection. This is my code:
private static void call(String body) throws IOException{
HttpURLConnection con = null;
con = (HttpURLConnection)new URL("http://127.0.0.1:8080").openConnection();
con.setRequestProperty("Accept-Charset", "UTF-8");
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/json; charset=utf-8");
con.setRequestProperty("Accept", "application/json; charset=utf-8");
con.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
wr.writeBytes(body);
wr.flush();
wr.close();
...
}
I post it to localhost just to sniff it with WireShark.
The problem is that when my body is a string containing characters like 'ò' 'à' 'è' 'ç' ... the request i see has le string correct with those characters replaced by dots.
example:
if body is "hèllo!" ---> the request body is "h.llo!"
Just for test i'm executing the above method in java main and i pass the parameter this way:
String pString = "{\"titlè\":\"Hèllo Wòrld!\"}";
String params = new String(pString.getBytes("UTF-8"),"UTF-8");
....
call(body);
and this is what i get in WireShark:
POST / HTTP/1.1
Accept-Charset: UTF-8
Content-Type: application/json; charset=utf-8
Accept: application/json; charset=utf-8
User-Agent: Java/1.6.0_43
Host: 127.0.0.1:8080
Connection: keep-alive
Content-Length: 24
{"titl.":"H.llo W.rld!"}
Any help would be appreciated.
Thank you

The internal string representation in Java is always UTF-16. So in your second example params = new String(pString.getBytes("UTF-8"),"UTF-8"); converts pString to a byte array with UTF-8 content and then back to UTF-16 which is stored in params.
Every encoding has to be done when strings enter or leave the VM. That means in your case you have to set the encoding when you write the body to the stream.
wr.write(body.getBytes("UTF-8"));

XMLHttpRequest java javascript

I try to communicate between javascript and java. My script javascript send a message to java and java send a response.
javascript part:
var xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange=function()
{
if (xmlhttp.readyState==4)
{
alert(xmlhttp.responseText);
}
}
var s = "LIGNE \n 2 \n il fait beau \nEND\n";
xmlhttp.open("POST","http://localhost:6020",true);
xmlhttp.send(s);
java part:
try {
serverSocket = new ServerSocket(6020);
} catch (IOException e) {
System.err.println("Could not listen on port: 6020.");
System.exit(-1);
}
serverSocket.accept()
BufferedReader br = new BufferedReader(
new InputStreamReader(socket.getInputStream()));
BufferedWriter bw = new BufferedWriter(
new OutputStreamWriter(socket.getOutputStream()));
String ligne = "";
while(!(ligne = plec.readLine()).equals("END")){
System.out.println(ligne);
}
bw.write("Il fait beau\n");
bw.flush();
bw.close();
plec.close();
socket.close();
output java :
POST / HTTP/1.1
Host: localhost:6020
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:13.0) Gecko/20100101 Firefox/13.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://localhost:8080/test.html
Content-Length: 30
Content-Type: text/plain; charset=UTF-8
Origin: http://localhost:8080
Pragma: no-cache
Cache-Control: no-cache
LIGNE
2
il fait beau
So, I receive correctly the message send by javascript but the alert his always empty. How to response at this message?
I try a lot of possiblity but they don't work. And I don't want to use the servlet, it's to heavy to do that.
Thanks.
Edit:
I did this :
bw.write("HTTP/1.1 200 OK\r\n"+
"Content-Type: text/html; charset=utf-8\r\n"+
"Content-Length: 13\r\n\r\n" +
"il fait beau\n");
and this:
String data = "il fait beau \n";
StringBuilder builder = new StringBuilder();
builder.append("HTTP/1.1 200 OK\r\n");
builder.append("Content-Type: text/html; charset=utf-8\r\n");
builder.append("Content-Length:" + data.length() + "\r\n\r\n");
builder.append(data);
bw.write(builder.toString());
But the alert remain empty. Maybe it's a problem in the javascript.

The javascript needs to see a full HTTP response. Merely sending back characters to it, makes it discard the reply as it is an invalid HTTP response.
In your java code, send back something like this
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: <length of data>
---data here---
Reference
Something like:
StringBuilder builder = new StringBuilder();
builder.append("HTTP/1.1 200 OK\r\n");
builder.append("Content-Type: text/plain; charset=utf-8\r\n");
builder.append("Content-Length:" + data.length() + "\r\n\r\n);
builder.append(data);
bw.write(builder.toString());

Try:
bw.write("HTTP/1.1 200 OK\r\n"+
"Content-Type: text/html; charset=utf-8\r\n"+
"Content-Length: 13\r\n\r\n" +
"il fait beau\n");
HTTP-Headers are separated by \r\n (CRLF). Headers and body is spearated by \r\n\r\n.
Note that you set the length to 13 because you also have to count the \n at the end of your string.
EDIT: It does not work because of the cross-domain-policy. http://localhost:6020 is not the same port as the website which executes your JavaScript and so the xmlhttprequest might not be delivered.

Soap content length

I look at the following is a sample SOAP 1.1 request to SP server, but its does not matter.
POST /_vti_bin/lists.asmx HTTP/1.1
Host: 192.168.0.25
Content-Type: text/xml; charset=utf-8
Content-Length: length
SOAPAction: "http://schemas.microsoft.com/sharepoint/soap/GetList"
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<GetList xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<listName>string</listName>
</GetList>
</soap:Body>
</soap:Envelope>
The field Content-Length: length need to be replaced with actual values. What its value? Where is I can see it? Or how to calculate it value before request?
UPD1.
I use ksoap lib
headerPropertyObj = new HeaderProperty("Content-Length", "383"); // should be calc before
headerList.add(headerPropertyObj);
transport.setUrl(URL);
request = new SoapObject(NAMESPACE, METHOD_NAME);
request.addProperty("listName", "Tasks");
envelope.setOutputSoapObject(request);
transport.call(SOAP_ACTION, envelope, headerList);

I solved this problem. I wrote myself class
public class MyHttpTransportSE extends Transport
where is I reloaded call method like below
public List call(String soapAction, SoapSerializationEnvelope envelope, List headers)
throws IOException, XmlPullParserException {
if (soapAction == null)
soapAction = "\"\"";
byte[] requestData = createRequestData(envelope);
requestDump = debug ? new String(requestData) : null;
responseDump = null;
connection = getServiceConnection();
connection.setRequestProperty("User-Agent", "kSOAP/2.0");
// connection.setRequestProperty("SOAPAction", soapAction);
// connection.setRequestProperty("Content-Type", "text/xml");
connection.setRequestProperty("Content-Type", "application/soap+xml");
connection.setRequestProperty("Connection", "close");
connection.setRequestProperty("Content-Length", "" + requestData.length);
Its helped for me, I hope this helps for smb.

The Content-Length value is the length (in bytes) of the body, which starts with the first < and ends with the last character of the body, here > or maybe a newline.

Creating the envelope the right way should be enough, without setting headers by yourself.
http://www.helloandroid.com/tutorials/using-ksoap2-android-and-parsing-output-data
SoapSerializationEnvelope envelope = new SoapSerializationEnvelope(SoapEnvelope.VER11);
envelope.setOutputSoapObject(request);
httpTransport.debug = true; //try debugging
If your problem relates to sharepoint authentication, then you should add envelope headers properly so the length keeps being calculated by the library itself. Error in authentication when subscribing to a sharepoint webservice using ksoap2-android
this could help: http://davidsit.wordpress.com/2010/03/03/creating-sharepoint-list-items-with-php/