Develop Reference

How to make Java HttpPost NOT include url in body

We have some old java code that POSTs some fields and values to a dotnet5 web api - The api is having problems dealing with the body of the POST as it includes the url/uri as the first part of the body.
The Java sends: http://127.0.0.1:5555?producerRef=GREEN&systemId=78&status=false
But the api is expecting something like: producerRef=GREEN&systemId=78&status=false
as per https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST#example. If we send a test message via Postman then the api has no problems.
This is the Java code:
List<NameValuePair> params = new ArrayList<NameValuePair>(queryParams.size());
for (Map.Entry<String, String> entry : queryParams.entrySet()) {
params.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
}
// the address is just that, there's NO parameters
HttpPost post = new HttpPost(this.cmAddress.toURI());
post.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));
post.setHeader(HttpHeaders.CONTENT_TYPE, "application/x-www-form-urlencoded");
CloseableHttpResponse response = httpClient.execute(post);
It's quite simple, but always adds the url to the start of the body of the request. If this is the only way to produce this, what could I do to produce something that looks like this: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST#example
Many Thanks.

This request seems like a GET request rather than a POST since the request params are in the URL. i don't know about the specifications of the Api you're using, but you can try OKHTTP, you can easily copy the code directly from postman
Postman Get example:

Your issue seems to be at below line
HttpPost post = new HttpPost(this.cmAddress.toURI());
This is the only place which will set the POST url ( another way is to use setURI which is not called anywhere in the code sample you have shared).
If you can use a debugger try checking the value of cmAdress variable

How can i change charset encoding in HTTP response in Java

I have to fetch some JSON object from a remote server and for that i am using this function which is working great except that for sometime some weird data is getting fetched which i believe is because it is using ASCII charset to decode.
Please find below thw method that i am using
public HttpResponse call(String serviceURL,String serviceHost,String namespace,String methodName,String payloadKey, String payloadValue) throws ClientProtocolException,IOException,JSONException
{
HttpResponse response = null;
HttpContext HTTP_CONTEXT = new BasicHttpContext();
HTTP_CONTEXT.setAttribute(CoreProtocolPNames.USER_AGENT, "Mozilla/5.0");
HttpPost httppost = new HttpPost(serviceURL);
httppost.setHeader("User-Agent",Constants.USER_AGENT_BROWSER_FIREFOX);
httppost.setHeader("Accept", "application/json, text/javascript, */*");
httppost.setHeader("Accept-Language","en-US,en;q=0.8");
httppost.setHeader("Content-Encoding", "foo-1.0");
httppost.setHeader("Content-Type", "application/json; charset=UTF-8");
httppost.setHeader("X-Requested-With","XMLHttpRequest");
httppost.setHeader("Host",serviceHost);
httppost.setHeader("X-Foo-Target", String.format("%s.%s", namespace,methodName));
/*Making Payload*/
JSONObject objectForPayload = new JSONObject();
objectForPayload.put(payloadKey, payloadValue);
StringEntity stringentity = new StringEntity(objectForPayload.toString());
httppost.setEntity(stringentity);
response = client.execute(httppost);
return response;
}
All these headers that i am passing are correct and i have verified the same via inspect element in Google chrome or Firebug plugin if you are familiar with Mozilla.
Now the problem is that most of the time i am getting the readable data but sometimes i do get unreadable data.
I debugged using eclipse and noticed that the charset under wrappedEntity is showing as "US-ASCII". I am attaching a jpg for reference
Can someone please tell me how can i change the charset from ASCII to UTF-8 of the response before i do response = client.execute(httppost); .
PS:As you have noticed that i am passing charset=utf-8 in the header and that i have already verified using firebug and google chrome that i am passing the exact headers .
Please zoom in to see the image more clearly
Thanks in advance

i was able to resolve the issue just mentioning it for people that may face similar issue.
after getting the response first get the entity by using
HttpEntity entity = response.getEntity();
and since my response was a json object convert entity to string but using "UTF-8" something like this
responseJsonObject = new JSONObject(EntityUtils.toString(entity,"UTF-8"));
previously i was just doing
responseJsonObject = new JSONObject(EntityUtils.toString(entity));

I don't think it's a problem with your headers, I think it's a problem with your string. Just having the header say it's utf-8 doesn't mean the string you write is utf-8, and that depends a lot on how the string was encoded and what's in the "payloadValue"
That said, you can always re-encode the thing correctly before sending it across the wire, for example:
objectForPayload.put(payloadKey, payloadValue);
StringEntity stringentity = new StringEntity(
new String(
objectForPayload.toString().getBytes(),
"UTF8"));
See if that works for you.

You may need to add an "Accept-Encoding"-header and set this to "UTF-8"

Just for the record: the "Content-Encoding" header field is incorrect - a correct server would reject the request as it contains an undefined content coding format.
Furthermore, attaching a charset parameter to application/json is meaningless.

bourne already answered that in the above comment though.
Changing entity = IOUtils.toString(response.getEntity().getContent())
TO entity = EntityUtils.toString(response.getEntity(),"UTF-8")
did the trick.

Java Apache HTTPClient do not encode POST entity

I have an endpoint that requires an 'authenticity_token' that is in the format like:
Iq2rNXN+OxERv+s6TSloJfKkPZVvqnWe1m0NfODB5OI=
However, sometimes it has "special" characters, such as:
E7IzeP73OgPGgXM/up295ky1mMQMio2Nb8HMLxJFyfw=
This gets encoded to:
E7IzeP73OgPGgXM%26%2347%3Bup295ky1mMQMio2Nb8HMLxJFyfw%3D
For some reason, the endpoint does not like the encoding of those special characters and will think the token is invalid. Is it possible to add a POST variable that does not encode specific values? I am currently doing something like:
HttpPost post = new HttpPost(URL + NEW_FINDING);
List <NameValuePair> nvps = new ArrayList <NameValuePair>();
nvps.add(new BasicNameValuePair("foo", foo));
nvps.add(new BasicNameValuePair("authenticity_token", authenticityToken));
post.setEntity(new UrlEncodedFormEntity(nvps));

You can always use ByteArrayEntity or StringEntity instead of UrlEncodedFormEntity and do the encoding yourself. It should look something like foo=var1&bar=var2.
You have to set Content-Type=application/x-www-form-urlencoded
You may want to find out what your endpoint expects as a charset parameter for the application/x-www-form-urlencoded value of the Content-Type header. Then pass it as a parameter to the UrlEncodedFormEntity constructor. This should be the right fix.

Trying to HttpPost a file in Android to my server

This is roughly the code I'm working with now:
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
nameValuePairs.add(new BasicNameValuePair("json", json.toString()));
nameValuePairs.add(new BasicNameValuePair("blob", file.getAbsolutePath()));
post_request.setEntity(new UrlEncodedFormEntity(nameValuePairs));
The the reply I get back from the server is good for the first add() statement but, for the second one I'm not trying to send the path, I'm trying to send the file. Taking off .getAbsolutePath() should do the trick, but It won't let me as it only accepts strings. How would I go about sending the file?

you should use a MultipartEntity, not an UrlEncodedForm one. In a Multipart body you can store objects of different mime types

Java posting to http fields/form keeps failing

Ok so I have tried a lot of stuff including sockets,httpConnection,httpClients,proxies,using cookies,and wide array of other things but this just keeps failing.What I need my program to do is send some data to html site and then recive adquate response.But resposne is always exact opposite of what I am expection.To be little more specific site I am talking about is http://hidemyass.com and I need to set the field value to "http://reddit.com" whos name is u and then press the button witch will redirect to Reddit through a proxy.So in theory response I should get is the Reddit html source or regular server response.But what I keep getting is http://hidemyass.com html source code with default values set(aka as I have not sent those values over a connection to it).
So here is what I did and it did not work.
CODE:
//I will skip used import because I think they are not of importance
//Also try catch blocks will be skipped and not written
private static UrlEncodedFormEntity entity;
public static void main(String[] args){
HttpClient client=new DefaultHttpClient();
ArrayList<BasicNameValuePair> params=new ArrayList<BasicNameValuePair>();
params.add(new BasicNameValuePair("u","http://reddit.com");
params.add(new BasicNameValuePair("hmabutton,"Hide My Ass");
entity=new UrlEncodedFormEntity(params,null)//there is no encoding on this site
HttpPost post=new HttpPost("http://hidemyass.com");
post.setEntity(entity);
HttpResponse response=client.execute(post);
HttpEntity ent=response.getEntity();
System.out.println(EntityUtils.toString(ent));
}
Can anyone help me see my error finally and get the site to do what I want?

You must post your request to the form's action URL:
HttpPost post=new HttpPost("http://hidemyass.com/process.php");