I am just trying to call an external web-service from my Java AWS Lambda function. To do this I can't get the org.apache.http client to work. I have the code:
public static String get(String get) throws ClientProtocolException, IOException{
RequestConfig defaultRequestConfig = RequestConfig.custom().setExpectContinueEnabled(true).build();
HttpGet httpGetRequest = new HttpGet(get);
RequestConfig requestConfig = RequestConfig.copy(defaultRequestConfig).setSocketTimeout(5000).setConnectTimeout(5000).setConnectionRequestTimeout(5000).build();
httpGetRequest.setConfig(requestConfig);
CloseableHttpClient httpClient = HttpClientBuilder.create().build();
HttpResponse httpResponse = httpClient.execute(httpGetRequest); <<<<<
HttpEntity entity = httpResponse.getEntity();
StringBuffer all = new StringBuffer();
if (entity != null) {
InputStream inputStream = entity.getContent();
BufferedReader in = new BufferedReader(new InputStreamReader(new BufferedInputStream(inputStream)));
String read;
while ((read = in.readLine()) != null){
all.append(read);
all.append("\n");
}
httpClient.close();
}
return all.toString();
}
Once it is deployed it hangs on the line HttpResponse httpResponse = httpClient.execute(httpGetRequest); <<<<< above.
I can prove that I have connectivity from the lambda function to the internet by setting up a Socket connection e.g. Socket s = new Socket(InetAddress.getByName("bbc.co.uk"), 80); and I can retrieve data this way.
So the question is either what is wrong with the first code fragment which works locally but not when deployed? or is there a preferred way to call web-services from AWS lambda functions in Java (I have had a search but can't find a best practice) ? I would prefer not to have to hand craft the HTTP requests using Sockets if I can help it.
The issue here is the usage of Apache HttpClient which is a large library. Switching to OkHttp fixed the issue in my case.
Please try to use Rest Template to make this call.
try {
String url = "baseurl for AWS insatnce +8080/Testing/mailGenration/generate?msg=20;
logger.log("Building HttpClient......");
HttpClient client = HttpClientBuilder.create().build();
HttpGet request = new HttpGet(url);
HttpResponse response = client.execute(request);
logger.log("Response Code : " + response.getStatusLine().getStatusCode());
} catch (Exception e) {
logger.log("Exception in sendhttpclient:" + e);
}
I have started to test http client apache API. I need it because I would like to send requests and to receive responses to virustotal API. Virus total API requires to parameters in the post request:
the api key value (a unique value for each user)
the file itself as I understood from their website.
For example:
>>> url = "https://www.virustotal.com/vtapi/v2/url/scan"
>>> parameters = {"url": "http://www.virustotal.com",
... "apikey": "-- YOUR API KEY --"}
>>> data = urllib.urlencode(parameters)
>>> req = urllib2.Request(url, data)
At the moment, I am trying to do the same thing in Java instead of Python. Here is a part of my source code commented to guide throughout the steps:
CloseableHttpClient httpClient = HttpClientBuilder.create().build();
//create post request
HttpPost request = new HttpPost("https://www.virustotal.com/vtapi/v2/file/scan");
//http json header
request.addHeader("content-type", "application/json");
String str = gson.toJson(param);
String fileName = UUID.randomUUID().toString() + ".txt";
try {
//API key
StringEntity entity = new StringEntity(str);
Writer writer = new BufferedWriter(new FileWriter(fileName));
writer.write(VirusDefinitionTest.malware());
request.setEntity(entity);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
FileBody fileBody = new FileBody(new File(fileName));
builder.addTextBody("my_file", fileName);
HttpEntity entity = builder.build();
request.setEntity(entity);
HttpResponse response;
try {
response = httpClient.execute(request);
...
Unfortunately, I receive HTTP/1.1 403 Forbidden. Obviously, the error is somewhere in the entities but I cannot find how to do it. Any help would be deeply welcomed.
This worked for me with Apache 4.5.2 HttpClient:
CloseableHttpClient httpclient = HttpClients.createDefault();
try {
HttpPost httppost = new HttpPost("https://www.virustotal.com/vtapi/v2/file/scan");
FileBody bin = new FileBody(new File("... the file here ..."));
// the API key here
StringBody comment = new StringBody("5ec8de.....", ContentType.TEXT_PLAIN);
HttpEntity reqEntity = MultipartEntityBuilder.create()
.addPart("apikey", comment)
.addPart("file", bin)
.build();
httppost.setEntity(reqEntity);
System.out.println("executing request " + httppost.getRequestLine());
CloseableHttpResponse response = httpclient.execute(httppost);
try {
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
HttpEntity resEntity = response.getEntity();
if (resEntity != null) {
System.out.println("ToString:" + EntityUtils.toString(resEntity));
}
EntityUtils.consume(resEntity);
} finally {
response.close();
}
} finally {
httpclient.close();
}
The important part was the reqEntity which had to have two specifically named fields, "apikey", and "file". Running this with a valid API key gives me the expected response from the API.
The problem seems to be that first you add explicit "content-type" header which is "application/json" and at the end you send the Muiltipart entity. You need to add all the parameters and the file to the Muiltipart entity. Now the parameters are not send, because they are overwritten by Muiltipart entity:
CloseableHttpClient httpClient = HttpClientBuilder.create().build();
//create post request
HttpPost request = new HttpPost("https://www.virustotal.com/vtapi/v2/file/scan");
//http json header
request.addHeader("content-type", "application/json");
String str = gson.toJson(param);
String fileName = UUID.randomUUID().toString() + ".txt";
try {
//API key
StringEntity entity = new StringEntity(str);
Writer writer = new BufferedWriter(new FileWriter(fileName));
writer.write(VirusDefinitionTest.malware());
// --> You set parameters here !!!
request.setEntity(entity);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
FileBody fileBody = new FileBody(new File(fileName));
builder.addTextBody("my_file", fileName);
HttpEntity entity = builder.build();
// --> You overwrite the parameters here !!!
request.setEntity(entity);
HttpResponse response;
try {
response = httpClient.execute(request);
lets assume this URL...
http://www.example.com/page.php?id=10
(Here id needs to be sent in a POST request)
I want to send the id = 10 to the server's page.php, which accepts it in a POST method.
How can i do this from within Java?
I tried this :
URL aaa = new URL("http://www.example.com/page.php");
URLConnection ccc = aaa.openConnection();
But I still can't figure out how to send it via POST
Updated answer
Since some of the classes, in the original answer, are deprecated in the newer version of Apache HTTP Components, I'm posting this update.
By the way, you can access the full documentation for more examples here.
HttpClient httpclient = HttpClients.createDefault();
HttpPost httppost = new HttpPost("http://www.a-domain.example/foo/");
// Request parameters and other properties.
List<NameValuePair> params = new ArrayList<NameValuePair>(2);
params.add(new BasicNameValuePair("param-1", "12345"));
params.add(new BasicNameValuePair("param-2", "Hello!"));
httppost.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));
//Execute and get the response.
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
if (entity != null) {
try (InputStream instream = entity.getContent()) {
// do something useful
}
}
Original answer
I recommend to use Apache HttpClient. its faster and easier to implement.
HttpPost post = new HttpPost("http://jakarata.apache.org/");
NameValuePair[] data = {
new NameValuePair("user", "joe"),
new NameValuePair("password", "bloggs")
};
post.setRequestBody(data);
// execute method and handle any error responses.
...
InputStream in = post.getResponseBodyAsStream();
// handle response.
for more information check this URL: http://hc.apache.org/
Sending a POST request is easy in vanilla Java. Starting with a URL, we need t convert it to a URLConnection using url.openConnection();. After that, we need to cast it to a HttpURLConnection, so we can access its setRequestMethod() method to set our method. We finally say that we are going to send data over the connection.
URL url = new URL("https://www.example.com/login");
URLConnection con = url.openConnection();
HttpURLConnection http = (HttpURLConnection)con;
http.setRequestMethod("POST"); // PUT is another valid option
http.setDoOutput(true);
We then need to state what we are going to send:
Sending a simple form
A normal POST coming from a http form has a well defined format. We need to convert our input to this format:
Map<String,String> arguments = new HashMap<>();
arguments.put("username", "root");
arguments.put("password", "sjh76HSn!"); // This is a fake password obviously
StringJoiner sj = new StringJoiner("&");
for(Map.Entry<String,String> entry : arguments.entrySet())
sj.add(URLEncoder.encode(entry.getKey(), "UTF-8") + "="
+ URLEncoder.encode(entry.getValue(), "UTF-8"));
byte[] out = sj.toString().getBytes(StandardCharsets.UTF_8);
int length = out.length;
We can then attach our form contents to the http request with proper headers and send it.
http.setFixedLengthStreamingMode(length);
http.setRequestProperty("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");
http.connect();
try(OutputStream os = http.getOutputStream()) {
os.write(out);
}
// Do something with http.getInputStream()
Sending JSON
We can also send json using java, this is also easy:
byte[] out = "{\"username\":\"root\",\"password\":\"password\"}" .getBytes(StandardCharsets.UTF_8);
int length = out.length;
http.setFixedLengthStreamingMode(length);
http.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
http.connect();
try(OutputStream os = http.getOutputStream()) {
os.write(out);
}
// Do something with http.getInputStream()
Remember that different servers accept different content-types for json, see this question.
Sending files with java post
Sending files can be considered more challenging to handle as the format is more complex. We are also going to add support for sending the files as a string, since we don't want to buffer the file fully into the memory.
For this, we define some helper methods:
private void sendFile(OutputStream out, String name, InputStream in, String fileName) {
String o = "Content-Disposition: form-data; name=\"" + URLEncoder.encode(name,"UTF-8")
+ "\"; filename=\"" + URLEncoder.encode(filename,"UTF-8") + "\"\r\n\r\n";
out.write(o.getBytes(StandardCharsets.UTF_8));
byte[] buffer = new byte[2048];
for (int n = 0; n >= 0; n = in.read(buffer))
out.write(buffer, 0, n);
out.write("\r\n".getBytes(StandardCharsets.UTF_8));
}
private void sendField(OutputStream out, String name, String field) {
String o = "Content-Disposition: form-data; name=\""
+ URLEncoder.encode(name,"UTF-8") + "\"\r\n\r\n";
out.write(o.getBytes(StandardCharsets.UTF_8));
out.write(URLEncoder.encode(field,"UTF-8").getBytes(StandardCharsets.UTF_8));
out.write("\r\n".getBytes(StandardCharsets.UTF_8));
}
We can then use these methods to create a multipart post request as follows:
String boundary = UUID.randomUUID().toString();
byte[] boundaryBytes =
("--" + boundary + "\r\n").getBytes(StandardCharsets.UTF_8);
byte[] finishBoundaryBytes =
("--" + boundary + "--").getBytes(StandardCharsets.UTF_8);
http.setRequestProperty("Content-Type",
"multipart/form-data; charset=UTF-8; boundary=" + boundary);
// Enable streaming mode with default settings
http.setChunkedStreamingMode(0);
// Send our fields:
try(OutputStream out = http.getOutputStream()) {
// Send our header (thx Algoman)
out.write(boundaryBytes);
// Send our first field
sendField(out, "username", "root");
// Send a seperator
out.write(boundaryBytes);
// Send our second field
sendField(out, "password", "toor");
// Send another seperator
out.write(boundaryBytes);
// Send our file
try(InputStream file = new FileInputStream("test.txt")) {
sendFile(out, "identification", file, "text.txt");
}
// Finish the request
out.write(finishBoundaryBytes);
}
// Do something with http.getInputStream()
String rawData = "id=10";
String type = "application/x-www-form-urlencoded";
String encodedData = URLEncoder.encode( rawData, "UTF-8" );
URL u = new URL("http://www.example.com/page.php");
HttpURLConnection conn = (HttpURLConnection) u.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
conn.setRequestProperty( "Content-Type", type );
conn.setRequestProperty( "Content-Length", String.valueOf(encodedData.length()));
OutputStream os = conn.getOutputStream();
os.write(encodedData.getBytes());
The first answer was great, but I had to add try/catch to avoid Java compiler errors.
Also, I had troubles to figure how to read the HttpResponse with Java libraries.
Here is the more complete code :
/*
* Create the POST request
*/
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("http://example.com/");
// Request parameters and other properties.
List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("user", "Bob"));
try {
httpPost.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));
} catch (UnsupportedEncodingException e) {
// writing error to Log
e.printStackTrace();
}
/*
* Execute the HTTP Request
*/
try {
HttpResponse response = httpClient.execute(httpPost);
HttpEntity respEntity = response.getEntity();
if (respEntity != null) {
// EntityUtils to get the response content
String content = EntityUtils.toString(respEntity);
}
} catch (ClientProtocolException e) {
// writing exception to log
e.printStackTrace();
} catch (IOException e) {
// writing exception to log
e.printStackTrace();
}
A simple way using Apache HTTP Components is
Request.Post("http://www.example.com/page.php")
.bodyForm(Form.form().add("id", "10").build())
.execute()
.returnContent();
Take a look at the Fluent API
I suggest using Postman to generate the request code. Simply make the request using Postman then hit the code tab:
Then you'll get the following window to choose in which language you want your request code to be:
simplest way to send parameters with the post request:
String postURL = "http://www.example.com/page.php";
HttpPost post = new HttpPost(postURL);
List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("id", "10"));
UrlEncodedFormEntity ent = new UrlEncodedFormEntity(params, "UTF-8");
post.setEntity(ent);
HttpClient client = new DefaultHttpClient();
HttpResponse responsePOST = client.execute(post);
You have done. now you can use responsePOST.
Get response content as string:
BufferedReader reader = new BufferedReader(new InputStreamReader(responsePOST.getEntity().getContent()), 2048);
if (responsePOST != null) {
StringBuilder sb = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
System.out.println(" line : " + line);
sb.append(line);
}
String getResponseString = "";
getResponseString = sb.toString();
//use server output getResponseString as string value.
}
Using okhttp :
Source code for okhttp can be found here https://github.com/square/okhttp.
If you're writing a pom project, add this dependency
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>4.2.2</version>
</dependency>
If not simply search the internet for 'download okhttp'. Several results will appear where you can download a jar.
your code :
import okhttp3.*;
import java.io.IOException;
public class ClassName{
private void sendPost() throws IOException {
// form parameters
RequestBody formBody = new FormBody.Builder()
.add("id", 10)
.build();
Request request = new Request.Builder()
.url("http://www.example.com/page.php")
.post(formBody)
.build();
OkHttpClient httpClient = new OkHttpClient();
try (Response response = httpClient.newCall(request).execute()) {
if (!response.isSuccessful()) throw new IOException("Unexpected code " + response);
// Get response body
System.out.println(response.body().string());
}
}
}
Easy with java.net:
public void post(String uri, String data) throws Exception {
HttpClient client = HttpClient.newBuilder().build();
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(uri))
.POST(BodyPublishers.ofString(data))
.build();
HttpResponse<?> response = client.send(request, BodyHandlers.discarding());
System.out.println(response.statusCode());
Here is more information:
https://openjdk.java.net/groups/net/httpclient/recipes.html#post
Since java 11, HTTP requests can be made by using java.net.http.HttpClient with less code.
var values = new HashMap<String, Integer>() {{
put("id", 10);
}};
var objectMapper = new ObjectMapper();
String requestBody = objectMapper
.writeValueAsString(values);
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("http://www.example.com/abc"))
.POST(HttpRequest.BodyPublishers.ofString(requestBody))
.build();
HttpResponse<String> response = client.send(request,
HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
Call HttpURLConnection.setRequestMethod("POST") and HttpURLConnection.setDoOutput(true); Actually only the latter is needed as POST then becomes the default method.
I recomend use http-request built on apache http api.
HttpRequest<String> httpRequest = HttpRequestBuilder.createPost("http://www.example.com/page.php", String.class)
.responseDeserializer(ResponseDeserializer.ignorableDeserializer()).build();
public void send(){
String response = httpRequest.execute("id", "10").get();
}
I Need to upload one file from my Android using multipart and tried with the below code but with no luck.
I got a connection timeout exception and tried with different code having the same result.
try
{
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(URL);
MultipartEntityBuilder entityBuilder = MultipartEntityBuilder.create();
entityBuilder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
entityBuilder.addTextBody(USER_ID, "DFD");
entityBuilder.addTextBody(NAME, "DFD");
String filepath = Environment.getExternalStorageDirectory() + "/Download/myImage.jpg";
Log.d(MULTIPART_TAG, filepath);
File file = new File(filepath);
if(file != null)
{
entityBuilder.addBinaryBody("IMAGE", file);
}
HttpEntity entity = entityBuilder.build();
post.setEntity(entity);
HttpResponse response = client.execute(post);
HttpEntity httpEntity = response.getEntity();
String result = EntityUtils.toString(httpEntity);
Log.v("result", result);
}
catch(Exception e)
{
e.printStackTrace();
}
Here is the exception that I get:
08-06 17:49:03.006: W/System.err(24761): Caused by:
libcore.io.ErrnoException: connect failed: ETIMEDOUT (Connection timed
out)
I also tried with those solutions:
https://stackoverflow.com/a/19188010/1948785
and with the deprecated class MultipartEntity
My second question is what is the meaning of this warning (I dont know if it is related with my problem but I get it when performing the request):
08-06 17:45:53.461: W/dalvikvm(24761): VFY: unable to resolve static
field 3008 (INSTANCE) in
Lorg/apache/http/message/BasicHeaderValueParser;
The libs I am using are those:
httpclient-4.3.4.jar
httpcore-4.3.2.jar
httpmime-4.3.4.jar
Try to set timeout duration of HttpClient.
int TIMEOUT_MILLISEC = 20000; // = 20 seconds
HttpParams httpParams = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(httpParams, TIMEOUT_MILLISEC);
HttpConnectionParams.setSoTimeout(httpParams, TIMEOUT_MILLISEC);
HttpClient client = new DefaultHttpClient(httpParams);
Try this ,
DefaultHttpClient httpClient = new DefaultHttpClient();
// yourimagefile is your imagefile
HttpPost httpPostRequest = new HttpPost(URL);
// Try This
httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
MultipartEntity mpEntity = new MultipartEntity();
ContentBody cbFile = new FileBody(yourimagefile, "image/jpeg");
mpEntity.addPart("file", cbFile);
httpPostRequest.setEntity(mpEntity);
HttpResponse response = (HttpResponse) httpclient.execute(httpPostRequest);
Check the size of the image. It might be that the image is too large and it is taking a long period of time to upload and the server is issuing a timeout on that connection.
In my application, the User can upload images to a PHP server, the iOS version is working 100%, the Android version used for this tutorial to upload image:
tutorial example
And the function I'm using is this:
public static String sendPost(String url, String imagePath)
throws IOException, ClientProtocolException {
HttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION,
HttpVersion.HTTP_1_1);
HttpPost httppost = new HttpPost(url);
File file = new File(imagePath);
MultipartEntity mpEntity = new MultipartEntity();
ContentBody cbFile = new FileBody(file, "image/jpeg");
mpEntity.addPart("userfile", cbFile);
httppost.setEntity(mpEntity);
//Log.e("executing request " + httppost.getRequestLine());
HttpResponse response = httpclient.execute(httppost);
HttpEntity resEntity = response.getEntity();
//Log.e(""+response.getStatusLine());
if (resEntity != null) {
//Log.e(EntityUtils.toString(resEntity));
}
if (resEntity != null) {
resEntity.consumeContent();
}
httpclient.getConnectionManager().shutdown();
return response.toString();
}
return response.toString(); get it in org.apache.http.message.BasicHttpResponse # 406dc148
But the return of the web service is the URL where the image was saved, I need to have a string in the return of the PHP server, rather than the return I mentioned above how can I have it?
I wanted something like this (HttpURLConnection):
HttpURLConnection conn;
...
String response= "";
Scanner inStream = new Scanner(conn.getInputStream());
while(inStream.hasNextLine())
response+=(inStream.nextLine());
Log.e("resp", response);
After one hour onsegui trying to get the response from the Web service as follows:
...
byte [] responseBody = httppost.getMethod().getBytes();
Log.e("RESPONSE BODY",""+(new String(responseBody)));
...
If you want the content returned by the HTTP server, you shouldn't do this:
if (resEntity != null) {
resEntity.consumeContent();
}
... because that says "throw away the content".
Try this if the response is of type String
ResponseHandler<String> responseHandler = new BasicResponseHandler();
HttpResponse httpResponse = httpClient.execute(post, new BasicHttpContext()); // new BasicHttpContext() not necessary
// verify connection response status using httpResponse.getStatusLine().getStatusCode() then
String response = responseHandler.handleResponse(httpResponse);
if(response != mull){
Log.e("Response : "+response);
}else{
// Handle exception
}
return response;