Develop Reference

Sending Query Parameters encoded in a URI on a Post which also has json

I have a server that handles a POST request with JSON. It also looks and decodes query parameters from the URI. My Java Client currently uses HTTPPost to send across the Json with ContentType application/json.
I wonder whehther URLEncodeUtil method format would be able to accomplish this. Except the documentation mentions
suitable for use as an application/x-www-form-urlencoded list of
parameters in an HTTP PUT or HTTP POST.
So my question is
1. Would this work with ContenType set to application/json.
2. Is there another way to accomplish what the Server requires, ie: have JSON as well as Query parameters encoded in the URI.

There are two official methods of posting form data via the (HTML spec). The pertinent value is application/x-www-form-urlencoded which adds a ? along with the name/value pairs encoded in the URL. If the form method is POST then it will be the first line after the HTTP POST statement.
Everything we do with HTTP in REST web services is valid HTTP, but not for HTML. So the application/json can have a combination of the application/x-www-form-urlencoded style parameters and the JSON payload.
The HTTP request will look something like this:
POST /blog/posts?myparam=Something%20Good&token=donotdothis
Accept: application/json
Content-Type: application/json
Content-Length: 57
{"title":"Hello World!","body":"This is my first post!"}
Also spelled out here: http://www.jsonrpc.org/historical/json-rpc-over-http.html
It's the ? that marks the beginning of extra parameters. So while that is technically legal, it does beg the question why everything you need to post can't be part of your JSON. The downside of this approach is that the query parameters are all part of your HTTP logs and is very visible. You definitely should not use this approach with passwords or any other personally identifiable information. Depending on privacy laws in your country, you want to minimize unnecessary records to make compliance much easier.

Spring REST Docs: special characters are not shown correctly

In my Spring MVC Test (UTF-8 encoded) we find:
this.mockMvc = MockMvcBuilders.webAppContextSetup(context).apply(springSecurity())
.apply(documentationConfiguration(restDocumentation)
.snippets().withEncoding("UTF-8")) // default
.build();
...
myRequestDTO.setValue("Größe");
ResultActions action = this.mockMvc
.perform(post("/my-service")
.content(jacksonObjectMapper.writeValueAsString(myRequestDTO))
...
action.andDo(document("docs"));
The asciidoctor file contains
HTTP Request
include::{snippets}/docs/http-request.adoc[]
After I have rendered it and open the generated HTML file (which is UTF-8 encoded, too) in my firefox browser I find
HTTP Request
POST /my-service HTTP/1.1
...
Größe
How can the special chars be displayed correctly?

The underlying problem here was with the conversion of a request's content as a byte[] into a String. Spring REST Docs uses the charset attribute of the Content-Type header to determine the Charset that should be used when creating the String. If there's no Content-Type header or its value doesn't have a charset attribute, the JVM's default Charset is used (as a result of calling new String(bytes)).
There are two ways to avoid corruption of special characters:
Specify a charset attribute in the request's Content-Type header. Use text/plain;charset=UTF-8 rather than text/plain, for example.
Configure the JVM's default Charset by setting the file.encoding system property. -Dfile.encoding=UTF8, for example.

After I have called prettyPrint() it works:
action.andDo(document("docs",
preprocessRequest(prettyPrint()),
preprocessResponse(prettyPrint())));

HttpURLConnection "enctype" POST [duplicate]

How can I set content type of HTTP Put as xxxx+xml?
I was referring to solution in this link Android, sending XML via HTTP POST (SOAP). Its fine when we set content type like this, i mean the xml is came along with the request:
httppost.setHeader("Content-Type","application/soap+xml;charset=UTF-8");
but when i change type soap to something custom, the xml disappear on the request (i saw on the wireshark), like this:
httppost.setHeader("Content-Type","application/vnd.oma-pcc+xml;charset=UTF-8");
then, i tried put the xml only, so the request is ok again:
httppost.setHeader("Content-Type","application/xml;charset=UTF-8");
I want to know what exactly the rules for the content-type than come together with the xml type so that the xml still there.
Thanks.

Assuming you're using HTTPClient of 4.1.3 or greater -
When constructing you're entity, you have the option to specify the content being used for the POST or PUT operation for certain entities.
There is a ContentType object which should be used to specify this.
Using the factory method .create() you can specify the mimetype with a charset - the ContentType will be used by the framework to properly emit the header in question.
Example API call:
ContentType.create("application/vnd.oma-pcc+xml", CharSet.forName("UTF-8"));
NOTE Editing for HttpClient 4.1.2
In the case of 4.1.2, when you create your entity for the post or put operation, set the content type on the entity not the execution (HttpPost or HttpPut) using setContentType(String). This is deprecated in 4.1.3 and beyond.

how to remove a header from URLConnection

I am talking to a file upload service that accepts post data, not form data. By default, java's HttpURLConnection sets the Content-Type header to application/x-www-form-urlencoded. this is obviously wrong if i'm posting pure data.
I (the client) don't know the content type. I don't want the Content-Type header set at all. the service has a feature where it will guess at the content type (based on the file name, reading some data from the file, etc).
How do I unset a header? There's no remove header, and setting it to null doesn't change the value and setting it to the empty string results in the header being set with no value.

I haven't tested this approach but you can try this:
Extend HttpURLConnection and try by overriding its getContentHandler() and setContentHandler(...) methods. Most probably this should work as, you will take a look at code of getContentHandler().

Use Apache HttpClient instead of URLConnection
Use fluent Request to generate your request
use removeHeader()

What do you mean "i don't want the Content-Type header to set at all"?
The browser (or other http client) sends your post request to the server, so it has to inform the server which way it encoded the parameters.
If the Content-Type header is not set, on the server side you (= your server) won't be able to understand how to parse the received data.
If you didn't set Content-Type, the default value will be used.
You browser (or other http client) MUST do two things:
Send key/value pairs.
Inform the server how the key/value pairs were encoded.
So, it is impossible to completely get rid of this header.

I just accomplished this by setting the header to null.
connection.setRequestProperty(MY_HEADER, null);

Check if a URL's mimetype is not a web page

I want to check if a URL's mimetype is not a webpage. Can I do this in Java? I want to check if the file is a rar or mp3 or mp4 or mpeg or whatever, just not a webpage.

You can issue an HTTP HEAD request and check for Content-Type response headers. You can use the HttpURLConnection.setRequestMethod("HEAD") before you issue the request. Then issue the request with URLConnection.connect() and then use URLConnection.getContentType() which reads the HTTP headers.
The bonus of using a HEAD request is that the actual resource is never transmitted/generated. You can also use a GET request and inspect the resulting stream using URLConnection.guessContentTypeFromStream() which will inspect the actual bytes and try to guess what the stream represents. I think that it looks for magic numbers or other patterns in the stream.

There's nothing inherent in a URL which will tell you what you will receive when you request it. You have to actually request the resource, and then inspect the content-type header. At that point, it's still not clear what you should do - some content types will (almost) always be handled by the browser, e.g. text/html. Some types should be handled by a browser, e.g. application/xhtml+xml. Some types may be handled by the browser, e.g. application/pdf.
Which, if any, of these you consider to be "webpage" is still not clear - you'll need to decide for yourself.
You can inspect the content-type header once you're requested the resource, using, for example, the HttpURLConnection class.

content-type:text/html represents webpage.