Develop Reference

Servlet request.getParameter() returns null in PUT but not in POST [duplicate]

Trying to get parameters from a PUT request using HttpServlet#doPut:
public void doPut(HttpServletRequest request, HttpServletResponse response) {
String name = request.getParameter("name");
// name is null
}
Using curl to send the request:
curl -X PUT \
--data "name=batman" \
--header "Content-Type: text/plain" http://localhost:8080/sample.html
works fine with using doGet and GET curl request. Am I missing something?

Based on comments and further research I realized that the Servlet cannot assume anything about the data being put onto the server and therefore, will not parse name/value pairs.
The following solution seems to be the proper way to handle any data passed via PUT, and can be parsed as XML, Name/Value, or whatever.
BufferedReader br = new BufferedReader(new InputStreamReader(req.getInputStream());
String data = br.readLine();

Unlike in doGet() and doPost() methods, we are not able to get the request parameters using the getParameter() method in doPut() and doDelete() methods. We need to retrieve them manually from the input stream.
The following method retrieves request parameters and returns them in a map:
public static Map<String, String> getParameterMap(HttpServletRequest request) {
BufferedReader br = null;
Map<String, String> dataMap = null;
try {
InputStreamReader reader = new InputStreamReader(
request.getInputStream());
br = new BufferedReader(reader);
String data = br.readLine();
dataMap = Splitter.on('&')
.trimResults()
.withKeyValueSeparator(
Splitter.on('=')
.limit(2)
.trimResults())
.split(data);
return dataMap;
} catch (IOException ex) {
Logger.getLogger(Utils.class.getName()).log(Level.SEVERE, null, ex);
} finally {
if (br != null) {
try {
br.close();
} catch (IOException ex) {
Logger.getLogger(Utils.class.getName()).log(Level.WARNING, null, ex);
}
}
}
return dataMap;
}
The example uses Google's Guava library to parse the parameters.
For a complete example containing doGet(), doPost(), doPut() and doDelete() methods, you can read my Using jsGrid tutorial.

What do you mean by doPut() is not working? As far as I know, it doesn't work like doGet() or doPost(), where you have request parameters and stuff.
PUT can be used to put something on the server. In particular, the PUT operation allows a client to place a file on the server and is similar to sending a file by FTP. Check out this example, I found on JGuru.
->GET /file.dat HTTP/1.1
<-HTTP/1.1 404 Not Found
->PUT /file.dat HTTP/1.1
Content-Length: 6
Content-Type: text/plain
Hello!
<-HTTP/1.1 200 OK
->GET /file.dat HTTP/1.1
<-HTTP/1.1 200 OK
Content-Length: 6
Content-Type: text/plain
Hello!

How to transform a curl command to HTTP POST request using java

I would like to run this specific curl command with a HTTP POST request in java
curl --location --request POST "http://106.51.58.118:5000/compare_faces?face_det=1" \
--header "user_id: myid" \
--header "user_key: thekey" \
--form "img_1=https://cdn.dnaindia.com/sites/default/files/styles/full/public/2018/03/08/658858-577200-katrina-kaif-052217.jpg" \
--form "img_2=https://cdn.somethinghaute.com/wp-content/uploads/2018/07/katrina-kaif.jpg"
I only know how to make simple POST requests by passing a JSON object, But i've never tried to POST based on the above curl command.
Here is a POST example that I've made based on this curl command:
curl -X POST TheUrl/sendEmail
-H 'Accept: application/json' -H 'Content-Type: application/json'
-d '{"emailFrom": "smth#domain.com", "emailTo":
["smth#gmail.com"], "emailSubject": "Test email", "emailBody":
"708568", "generateQRcode": true}' -k
Here is how i did it using java
public void sendEmail(String url) {
try {
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
//add reuqest header
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/json; utf-8");
con.setRequestProperty("Accept", "application/json");
con.setDoOutput(true);
// Send post request
JSONObject test = new JSONObject();
test.put("emailFrom", emailFrom);
test.put("emailTo", emailTo);
test.put("emailSubject", emailSubject);
test.put("emailBody", emailBody);
test.put("generateQRcode", generateQRcode);
String jsonInputString = test.toString();
System.out.println(jsonInputString);
System.out.println("Email Response:" + returnResponse(con, jsonInputString));
} catch (Exception e) {
System.out.println(e);
}
System.out.println("Mail sent");
}
public String returnResponse(HttpURLConnection con, String jsonInputString) {
try (OutputStream os = con.getOutputStream()) {
byte[] input = jsonInputString.getBytes("utf-8");
os.write(input, 0, input.length);
} catch (Exception e) {
System.out.println(e);
}
try (BufferedReader br = new BufferedReader(new InputStreamReader(con.getInputStream(), "utf-8"))) {
StringBuilder response = new StringBuilder();
String responseLine = null;
while ((responseLine = br.readLine()) != null) {
response.append(responseLine.trim());
}
return response.toString();
} catch (Exception e) {
System.out.println("Couldnt read response from URL");
System.out.println(e);
return null;
}
}
I've found this useful link but i can't really understand how to use it in my example.
Is it any different from my example? and if yes how can i POST the following data?
Note: Required Data
HEADERS:
user_id myid
user_key mykey
PARAMS:
face_det 1
boxes 120,150,200,250 (this is optional)
BODY:
img_1
multipart/base64 encoded image or remote url of image
img_2
multipart/base64 encoded image or remote url of image
Here is the complete documentation of the API

There are three things that your HttpURLConnection needs:
The request method. You can set this with setRequestMethod.
The headers. You can set them with setRequestProperty.
The content type. The HTML specification requires that an HTTP request containing a form submission have application/x-www-form-urlencoded (or multipart/form-data) as its body’s content type. This is done by setting the Content-Type header using the setRequestProperty method, just like the other headers.
It’s not clear what you’re trying to do here. As Boris Verkhovskiy points out, curl’s --form option includes data as a part of a multipart request. In your command, the content of that request would be the characters of the URLs themselves. If you really want to submit URLs, not the images at those locations, you could use an application/x-www-form-urlencoded request body to do it. The body itself needs to URL-encoded, as the content type indicates. The URLEncoder class exists for this purpose.
The steps look like this:
String img1 = "https://cdn.dnaindia.com/sites/default/files/styles/full/public/2018/03/08/658858-577200-katrina-kaif-052217.jpg";
String img2 = "https://cdn.somethinghaute.com/wp-content/uploads/2018/07/katrina-kaif.jpg";
con.setRequestMethod("POST");
con.setDoOutput(true);
con.setRequestProperty("user_id", myid);
con.setRequestProperty("user_key", thekey);
con.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
String body =
"img_1=" + URLEncoder.encode(img1, "UTF-8") + "&" +
"img_2=" + URLEncoder.encode(img2, "UTF-8");
try (OutputStream os = con.getOutputStream()) {
byte[] input = body.getBytes(StandardCharsets.UTF_8);
os.write(input);
}
However, if you want to submit the actual images, you will need to create a MIME request body. Java SE cannot do this, but the MimeMultipart class of JavaMail, which is part of the Java EE specification, can.
Multipart multipart = new MimeMultipart("form-data");
BodyPart part;
part = new MimeBodyPart();
part.setDataHandler(new DataHandler(new URL(img1)));
multipart.addBodyPart(part);
part = new MimeBodyPart();
part.setDataHandler(new DataHandler(new URL(img2)));
multipart.addBodyPart(part);
con.setRequestMethod("POST");
con.setDoOutput(true);
con.setRequestProperty("user_id", myid);
con.setRequestProperty("user_key", thekey);
con.setRequestProperty("Content-Type", multipart.getContentType());
try (OutputStream os = con.getOutputStream()) {
multipart.writeTo(os);
}
You should remove all catch blocks from your code, and amend your method signatures to include throws IOException (or throws IOException, MessagingException). You don’t want users of your application to think the operation was successful if in fact it failed, right?

How to extract JSON from this HTTP request in Java

The method below is hit or miss if it parses the request properly...
Here is what the request looks like:
POST /addEvent/ HTTP/1.1
Host: localhost:1234
Content-Type: multipart/form-data; boundary=Boundary+0xAbCdEfGbOuNdArY
Accept-Encoding: gzip, deflate
Content-Length: 201
Accept-Language: en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5
Accept: application/json
Connection: keep-alive
User-Agent: XXXXXXXXXXXXXXXXXX
--Boundary+0xAbCdEfGbOuNdArY
Content-Disposition: form-data; name="userInfo"
{ "user_id" : 1, "value" : "Water", "typeCode" : "Searched" }
Here is how we are extracting it now...
//Key where the request begins
String keyString = "\"userInfo\"";
//Get the index of the key
int end = bufferedJson.lastIndexOf("\"userInfo\"");
//Create substring at beginning of the json
String json = bufferedJson.substring(end+keyString.length(), bufferedJson.length());
//Convert json to feed item
Gson gson = new Gson();
Event eventItem = gson.fromJson(json, Event.class);
I get this error pretty often:
Expected BEGIN_OBJECT but was STRING at line 1 column 1
How can we parse this efficiently?

Use Apache HTTP Client 4 to read Http response body in a convenient way. If you need to marshall your json further to a java object then make use of jackson. Here is the sample code:
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
/**
* This example demonstrates the use of the {#link ResponseHandler} to simplify
* the process of processing the HTTP response and releasing associated resources.
*/
public class ClientWithResponseHandler {
public final static void main(String[] args) throws Exception {
HttpClient httpclient = new DefaultHttpClient();
try {
HttpGet httpget = new HttpGet("http://www.google.com/");
System.out.println("executing request " + httpget.getURI());
// Create a response handler
ResponseHandler<String> responseHandler = new BasicResponseHandler();
// Body contains your json stirng
String responseBody = httpclient.execute(httpget, responseHandler);
System.out.println("----------------------------------------");
System.out.println(responseBody);
System.out.println("----------------------------------------");
} finally {
// When HttpClient instance is no longer needed,
// shut down the connection manager to ensure
// immediate deallocation of all system resources
httpclient.getConnectionManager().shutdown();
}
}
}

To parse this better:
First, it seems like you're taking a "raw" HTTP POST request, and then reading it line by line using BufferedReader (your comment suggests this), this way you'll lose the new line chars; if you are going to do so, add a new line ("\n") every time you read a line to your final String, this way it doesn't lose the new lines and facilitates the things for the next step.
Now, with this final String, you can use this:
String json = null;
Pattern pattern = Pattern.compile("\n\n");
Matcher matcher = pattern.matcher(myString); // myString is the String you built from your header
if(matcher.find() && matcher.find()) {
json = myString.substring(matcher.start() + 2);
} else {
// Handle error: json string wasn't found
}
CAVEATS: this works if:
POST Request will always be multipart/form-data
There are not other parameters in the request
you STOP reading the request as soon as you find your json data
you included "\n" every time you read a line as I said in the first step
Personally I wouldn't read the raw HTTP header, I'd rather use Apache commons FileUpload or the like, but if your are going to do it this way, I think this is the less terrible solution.

You can use
gson().fromJson(request.getReader(), Event.class);
or
String json = request.getReader().readLine();
gson().fromJson(json, Event.class);

FileNotFoundException while getting the InputStream object from HttpURLConnection

I am trying to send a post request to a url using HttpURLConnection (for using cUrl in java).
The content of the request is xml and at the end point, the application processes the xml and stores a record to the database and then sends back a response in form of xml string. The app is hosted on apache-tomcat locally.
When I execute this code from the terminal, a row gets added to the db as expected. But an exception is thrown as follows while getting the InputStream from the connection
java.io.FileNotFoundException: http://localhost:8080/myapp/service/generate
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1401)
at org.kodeplay.helloworld.HttpCurl.main(HttpCurl.java:30)
Here is the code
public class HttpCurl {
public static void main(String [] args) {
HttpURLConnection con;
try {
con = (HttpURLConnection) new URL("http://localhost:8080/myapp/service/generate").openConnection();
con.setRequestMethod("POST");
con.setDoOutput(true);
con.setDoInput(true);
File xmlFile = new File("test.xml");
String xml = ReadWriteTextFile.getContents(xmlFile);
con.getOutputStream().write(xml.getBytes("UTF-8"));
InputStream response = con.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(response));
for (String line ; (line = reader.readLine()) != null;) {
System.out.println(line);
}
reader.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Its confusing because the exception is traced to the line InputStream response = con.getInputStream(); and there doesn't seem to be any file involved for a FileNotFoundException.
When I try to open a connection to an xml file directly, it doesn't throw this exception.
The service app uses spring framework and Jaxb2Marshaller to create the response xml.
The class ReadWriteTextFile is taken from here
Thanks.
Edit:
Well it saves the data in the DB and sends back a 404 response status code at the same time.
I also tried doing a curl using php and print out the CURLINFO_HTTP_CODE which turns out to be 200.
Any ideas on how do I go about debugging this ? Both service and client are on the local server.
Resolved:
I could solve the problem after referring to an answer on SO itself.
It seems HttpURLConnection always returns 404 response when connecting to a url with a non standard port.
Adding these lines solved it
con.setRequestProperty("User-Agent","Mozilla/5.0 ( compatible ) ");
con.setRequestProperty("Accept","*/*");

I don't know about your Spring/JAXB combination, but the average REST webservice won't return a response body on POST/PUT, just a response status. You'd like to determine it instead of the body.
Replace
InputStream response = con.getInputStream();
by
int status = con.getResponseCode();
All available status codes and their meaning are available in the HTTP spec, as linked before. The webservice itself should also come along with some documentation which overviews all status codes supported by the webservice and their special meaning, if any.
If the status starts with 4nn or 5nn, you'd like to use getErrorStream() instead to read the response body which may contain the error details.
InputStream error = con.getErrorStream();

FileNotFound is just an unfortunate exception used to indicate that the web server returned a 404.

To anyone with this problem in the future, the reason is because the status code was a 404 (or in my case was a 500). It appears the InpuStream function will throw an error when the status code is not 200.
In my case I control my own server and was returning a 500 status code to indicate an error occurred. Despite me also sending a body with a string message detailing the error, the inputstream threw an error regardless of the body being completely readable.
If you control your server I suppose this can be handled by sending yourself a 200 status code and then handling whatever the string error response was.

For anybody else stumbling over this, the same happened to me while trying to send a SOAP request header to a SOAP service. The issue was a wrong order in the code, I requested the input stream first before sending the XML body. In the code snipped below, the line InputStream in = conn.getInputStream(); came immediately after ByteArrayOutputStream out = new ByteArrayOutputStream(); which is the incorrect order of things.
ByteArrayOutputStream out = new ByteArrayOutputStream();
// send SOAP request as part of HTTP body
byte[] data = request.getHttpBody().getBytes("UTF-8");
conn.getOutputStream().write(data);
if (conn.getResponseCode() != HttpURLConnection.HTTP_OK) {
Log.d(TAG, "http response code is " + conn.getResponseCode());
return null;
}
InputStream in = conn.getInputStream();
FileNotFound in this case was an unfortunate way to encode HTTP response code 400.

FileNotFound in this case means you got a 404 from your server - could it be that the server does not like "POST" requests?

FileNotFound in this case means you got a 404 from your server
You Have to Set the Request Content-Type Header Parameter
Set “content-type” request header to “application/json” to send the request content in JSON form.
This parameter has to be set to send the request body in JSON format.
Failing to do so, the server returns HTTP status code “400-bad request”.
con.setRequestProperty("Content-Type", "application/json; utf-8");
Full Script ->
public class SendDeviceDetails extends AsyncTask<String, Void, String> {
#Override
protected String doInBackground(String... params) {
String data = "";
String url = "";
HttpURLConnection con = null;
try {
// From the above URL object,
// we can invoke the openConnection method to get the HttpURLConnection object.
// We can't instantiate HttpURLConnection directly, as it's an abstract class:
con = (HttpURLConnection)new URL(url).openConnection();
//To send a POST request, we'll have to set the request method property to POST:
con.setRequestMethod("POST");
// Set the Request Content-Type Header Parameter
// Set “content-type” request header to “application/json” to send the request content in JSON form.
// This parameter has to be set to send the request body in JSON format.
//Failing to do so, the server returns HTTP status code “400-bad request”.
con.setRequestProperty("Content-Type", "application/json; utf-8");
//Set Response Format Type
//Set the “Accept” request header to “application/json” to read the response in the desired format:
con.setRequestProperty("Accept", "application/json");
//To send request content, let's enable the URLConnection object's doOutput property to true.
//Otherwise, we'll not be able to write content to the connection output stream:
con.setDoOutput(true);
//JSON String need to be constructed for the specific resource.
//We may construct complex JSON using any third-party JSON libraries such as jackson or org.json
String jsonInputString = params[0];
try(OutputStream os = con.getOutputStream()){
byte[] input = jsonInputString.getBytes("utf-8");
os.write(input, 0, input.length);
}
int code = con.getResponseCode();
System.out.println(code);
//Get the input stream to read the response content.
// Remember to use try-with-resources to close the response stream automatically.
try(BufferedReader br = new BufferedReader(new InputStreamReader(con.getInputStream(), "utf-8"))){
StringBuilder response = new StringBuilder();
String responseLine = null;
while ((responseLine = br.readLine()) != null) {
response.append(responseLine.trim());
}
System.out.println(response.toString());
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (con != null) {
con.disconnect();
}
}
return data;
}
#Override
protected void onPostExecute(String result) {
super.onPostExecute(result);
Log.e("TAG", result); // this is expecting a response code to be sent from your server upon receiving the POST data
}
and call it
new SendDeviceDetails().execute("");
you can find more details in this tutorial
https://www.baeldung.com/httpurlconnection-post

The solution:
just change localhost for the IP of your PC
if you want to know this: Windows+r > cmd > ipconfig
example: http://192.168.0.107/directory/service/program.php?action=sendSomething
just replace 192.168.0.107 for your own IP (don't try 127.0.0.1 because it's same as localhost)

Please change
con = (HttpURLConnection) new URL("http://localhost:8080/myapp/service/generate").openConnection();
To
con = (HttpURLConnection) new URL("http://YOUR_IP:8080/myapp/service/generate").openConnection();

twitter streaming api - set filter in http post request (apache httpcomponents)

I'm trying out the twitter streaming api. I could succesfully filter tweets by using curl, as stated here:
curl -d #tracking http://stream.twitter.com/1/statuses/filter.json -u <user>:<pass>
where tracking is a plain file with the content:
track=Berlin
Now I tried to do the same thing in JavaSE, using Apache's HTTPComponents:
UsernamePasswordCredentials creds = new UsernamePasswordCredentials(<user>, <pass>);
DefaultHttpClient httpClient = new DefaultHttpClient();
httpClient.getCredentialsProvider().setCredentials(AuthScope.ANY, creds);
HttpPost httpPost = new HttpPost("http://stream.twitter.com/1/statuses/filter.json");
HttpParams params = new BasicHttpParams();
params = params.setParameter("track", "Berlin");
httpPost.setParams(params);
try {
HttpResponse httpResponse = httpClient.execute(httpPost);
HttpEntity entity = httpResponse.getEntity();
if (entity != null) {
InputStream instream = entity.getContent();
String t;
BufferedReader br = new BufferedReader(new InputStreamReader(instream));
while(true) {
t = br.readLine();
if(t != null) {
linkedQueue.offer(t);
}
}
}
} catch (IOException ioe) {
System.out.println(ioe.getMessage());
}
finally{
httpClient.getConnectionManager().shutdown();
}
When I run that, I get:
No filter parameters found. Expect at least one parameter: follow track
as a single entry in my linkedQueue. Seems the api wants the parameter in a different form, but cannot find any hint in the documentation. Can somebody share some experiences with the api or see any other problem with the code? Thanks!
EDIT
Putting the filter parameter into the params was a bad idea. As it's post data, it needs to be defined as an Entity before the request is being made:
StringEntity postEntity = new StringEntity("track=Berlin", "UTF-8");
postEntity.setContentType("application/x-www-form-urlencoded");
httpPost.setEntity(postEntity);
That's what I was doing wrong. Thanks Brian!

I suspect you need to post the data as the contents of your HTTP post. The man page for curl -d says:
(HTTP) Sends the specified data in a
POST request to the HTTP server, in
the same way that a browser does when
a user has filled in an HTML form and
presses the submit button. This will
cause curl to pass the data to the
server using the content-type
application/x-www-form-urlencoded.
so I believe you have to set that content type and put the contents of the tracking file in the body of your post.