Develop Reference

httpclient to upload file results in a corrapted file

I am using apache httpclient to upload a file to a server. I need to use basic authentication(username and password). The response in 200, however, the server logs reveal that the file in missing some data.
public static String pushdata() throws FileNotFoundException, UnsupportedEncodingException {
File file = new File("/home/mohamed/atomtest.txt");
CredentialsProvider provider = new BasicCredentialsProvider();
UsernamePasswordCredentials credentials = new UsernamePasswordCredentials("user", "pass");
provider.setCredentials(AuthScope.ANY, credentials);
HttpClient httpclient = HttpClientBuilder.create().setDefaultCredentialsProvider(provider).build();
HttpEntity data = MultipartEntityBuilder.create().setMode(HttpMultipartMode.BROWSER_COMPATIBLE).addBinaryBody("file", file).build();
HttpUriRequest request = RequestBuilder.post("link").setEntity(data).build();
ResponseHandler<String> responseHandler = response -> {
int status = response.getStatusLine().getStatusCode();
System.out.println(status);
if (status >= 200 && status < 300) {
HttpEntity entity = response.getEntity();
String entityString = EntityUtils.toString(entity);
System.out.println(entityString);
return entityString;
} else {
throw new ClientProtocolException("Unexpected response status: " + status);
}
};
String responseBody = "pushDATA";
try {
responseBody = httpclient.execute(request, responseHandler);
} catch (IOException ex) {
System.out.println(ex.toString());
}
return responseBody;
}
The file is saved as UTF-16, and must be uploaded with the same encoding.
kind regards.

How to get httpclient response as stream

I am using httpclient 4.5.5
i want to get large files upto 1 gb in response. But it seems like
CloseableHttpResponse response = httpClient.execute(httpGet);
HttpEntity entity = response.getEntity();
This returns whole response so its not good to have whole response in memory. Is there any way to get response as stream?

Apache HttpClient as of version 4.0 (as well as Apache HttpAsyncClient) supports full content streaming for both incoming and outgoing HTTP messages. Use HttpEntity to get access to the underling content input stream
CloseableHttpClient client = HttpClients.createDefault();
HttpGet httpGet = new HttpGet("http://myhost/tons-of-stuff");
try (CloseableHttpResponse response1 = client.execute(httpGet)) {
final HttpEntity entity = response1.getEntity();
if (entity != null) {
try (InputStream inputStream = entity.getContent()) {
// do something useful with the stream
}
}
}

You need Apache Async Client.
HttpAsyncClient is the ASYNC version of Apache HttpClient. Apache HttpClient construct the whole response in memory, while with HttpAsyncClient, you can define a Handler (Consumer) to process the response while receiving data.
https://hc.apache.org/httpcomponents-asyncclient-4.1.x/index.html
Here is an example from their official example code
package org.apache.http.examples.nio.client;
import java.io.IOException;
import java.nio.CharBuffer;
import java.util.concurrent.Future;
import org.apache.http.HttpResponse;
import org.apache.http.impl.nio.client.CloseableHttpAsyncClient;
import org.apache.http.impl.nio.client.HttpAsyncClients;
import org.apache.http.nio.IOControl;
import org.apache.http.nio.client.methods.AsyncCharConsumer;
import org.apache.http.nio.client.methods.HttpAsyncMethods;
import org.apache.http.protocol.HttpContext;
/**
* This example demonstrates an asynchronous HTTP request / response exchange with
* a full content streaming.
*/
public class AsyncClientHttpExchangeStreaming {
public static void main(final String[] args) throws Exception {
CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
try {
httpclient.start();
Future<Boolean> future = httpclient.execute(
HttpAsyncMethods.createGet("http://httpbin.org/"),
new MyResponseConsumer(), null);
Boolean result = future.get();
if (result != null && result.booleanValue()) {
System.out.println("Request successfully executed");
} else {
System.out.println("Request failed");
}
System.out.println("Shutting down");
} finally {
httpclient.close();
}
System.out.println("Done");
}
static class MyResponseConsumer extends AsyncCharConsumer<Boolean> {
#Override
protected void onResponseReceived(final HttpResponse response) {
}
#Override
protected void onCharReceived(final CharBuffer buf, final IOControl ioctrl) throws IOException {
while (buf.hasRemaining()) {
System.out.print(buf.get());
}
}
#Override
protected void releaseResources() {
}
#Override
protected Boolean buildResult(final HttpContext context) {
return Boolean.TRUE;
}
}
}

Use HttpURLConnection instead of httpClient.
final HttpURLConnection conn = (HttpURLConnection)url.openConnection();
final int bufferSize = 1024 * 1024;
conn.setChunkedStreamingMode(bufferSize);
final OutputStream out = conn.getOutputStream();

representing curl command in java

what is the Java equivalent of the following curl command:
curl -u <username>:<password> -F "access=#svnaccess.txt" https://svn.xxx-xxx.de/upload.cgi
i try to update the access rules of a svn repository by uploading a .txt file to this url.
any help is much appreciated!

From your curl command you are using
-u to use basic auth with the provided username and password
-F which uses http verb post
# to load the contents of the file
The Java equivalent is to use HttpClient. However most examples are out of date with the current version. So as of 4.4.1 ...
Authentication is simplify a case of building HttpClients with a CredentialsProvider
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope(server, (https ? 443:80)),
new UsernamePasswordCredentials("username", "password")
);
CloseableHttpClient httpclient = HttpClients
.custom()
.setDefaultCredentialsProvider(credsProvider)
.build();
To reproduce your use of -F to upload a file you will want to add something that implements org.apache.http.NameValuePair and you can then use UrlEncodedFormEntity with your file you want to upload.
List<NameValuePair> formparams = new ArrayList<NameValuePair>();
formparams.add(new FileNameValuePair("access", new File("./svnaccess.txt")));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formparams, Consts.UTF_8);
httppost.setEntity(entity)
Runable Example
Example.java
import java.io.*;
import java.net.*;
import java.util.*;
import org.apache.http.*;
import org.apache.http.auth.*;
import org.apache.http.client.*;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.*;
import org.apache.http.client.utils.*;
import org.apache.http.impl.client.*;
public class Example {
public static void main(String[] args) throws Exception {
String server = "svn.xxx-xxx.de";
String path = "/upload.cgi";
Boolean https = true;
curl(server, path, https);
}
private static void curl(String server, String path, Boolean https)
throws URISyntaxException, IOException, ClientProtocolException {
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope(server, (https ? 443:80)),
new UsernamePasswordCredentials("username", "password")
);
CloseableHttpClient httpclient = HttpClients
.custom()
.setDefaultCredentialsProvider(credsProvider)
.build();
URI uri = new URIBuilder()
.setScheme(https ? "https":"http")
.setHost(server)
.setPath(path)
.build();
try {
HttpPost httppost = new HttpPost(uri);
List<NameValuePair> formparams = new ArrayList<NameValuePair>();
formparams.add(new FileNameValuePair("access", new File("./svnaccess.txt")));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formparams, Consts.UTF_8);
httppost.setEntity(entity);
CloseableHttpResponse response = httpclient.execute(httppost);
System.out.println(httppost.getRequestLine());
try {
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
} finally {
response.close();
}
} finally {
httpclient.close();
}
}
}
FileNameValuePair.java
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
public class FileNameValuePair implements org.apache.http.NameValuePair
{
private String name;
private File file;
public FileNameValuePair(String name, File file)
{
this.name = name;
this.file = file;
}
public String getName() {
return name;
}
public String getValue() {
String everything = "";
try(BufferedReader br = new BufferedReader(new FileReader(file))) {
StringBuilder sb = new StringBuilder();
String line = br.readLine();
while (line != null) {
sb.append(line);
sb.append(System.lineSeparator());
line = br.readLine();
}
everything = sb.toString();
} catch (IOException e) {
e.printStackTrace();
}
return everything;
}
}

Solved it.
The #-argument indicates the upload of a file, rather than its contents.
Changed code:
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.addBinaryBody("access", new File("./svnaccess.txt"));
HttpEntity entity = builder.build();
httppost.setEntity(entity);
Thanks for pointing me in the right direction!

Bouncing between "Adapter is detached" and "No wrapped connection" with HttpClient

So as i said i'm bouncing back and forth between these two errors when trying to run HttpClient.execute(HttpPost). Getting IllegalStateException
public class NetMethods {
private static HttpClient client = new DefaultHttpClient();
public static void getStuff() {
ArrayList<Alarm> alarms = new ArrayList<Alarm>();
HttpPost post = HttpPostFactory.getHttpPost("GetStuff");
StringBuilder builder = new StringBuilder();
HttpResponse response = client.execute(post); // Exception thrown here
...
Also, my MttpPostFactory just has this
import org.apache.http.client.methods.HttpPost;
public class HttpPostFactory {
private static final String url = "http://example.com/ExampleFolder/";
public static HttpPost getHttpPost(String s) {
return new HttpPost(url + s);
}
}

This may arise from not closing the InputStream's you get from HttpClient, especially if arising from different threads...either not reading the whole content or calling the same HttpClient instance from two different threads.

I found solution from Androider's blog:
I got the log :
Invalid use of SingleClientConnManager: connection still allocated.
or
Caused by: java.lang.IllegalStateException: No wrapped connection.
or
Caused by: java.lang.IllegalStateException: Adapter is detached.
Finally got Solution:
public static DefaultHttpClient getThreadSafeClient() {
DefaultHttpClient client = new DefaultHttpClient();
ClientConnectionManager mgr = client.getConnectionManager();
HttpParams params = client.getParams();
client = new DefaultHttpClient(new ThreadSafeClientConnManager(params,
mgr.getSchemeRegistry()), params);
return client;
}

Try with this..
// Execute the asynctask with your URL
new myAsyncTask().execute("http://example.com/ExampleFolder/");
// Asynctask Callback method
private class myAsyncTask extends AsyncTask<String, Void, Void>
{
protected void onPreExecute()
{
super.onPreExecute();
}
#Override
protected Void doInBackground(String... arg0)
{
// Creating service handler class instance
ServiceHandler serhand = new ServiceHandler();
// Making a request to url and getting response
serhand.makeServiceCall(arg0[0], ServiceHandler.GET);
return null;
}
protected void onPostExecute(Void result)
{
}
}
// Service Handler class
package yourpagename
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.List;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.utils.URLEncodedUtils;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
public class ServiceHandler {
static String response = null;
public final static int GET = 1;
public final static int POST = 2;
public ServiceHandler() {
}
/**
* Making service call
* #url - url to make request
* #method - http request method
* */
public String makeServiceCall(String url, int method) {
return this.makeServiceCall(url, method, null);
}
/**
* Making service call
* #url - url to make request
* #method - http request method
* #params - http request params
* */
public String makeServiceCall(String url, int method,
List<NameValuePair> params) {
try {
// http client
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpEntity httpEntity = null;
HttpResponse httpResponse = null;
// Checking http request method type
if (method == POST) {
HttpPost httpPost = new HttpPost(url);
// adding post params
if (params != null) {
httpPost.setEntity(new UrlEncodedFormEntity(params));
}
httpResponse = httpClient.execute(httpPost);
} else if (method == GET) {
// appending params to url
if (params != null) {
String paramString = URLEncodedUtils
.format(params, "utf-8");
url += "?" + paramString;
}
HttpGet httpGet = new HttpGet(url);
httpResponse = httpClient.execute(httpGet);
}
httpEntity = httpResponse.getEntity();
response = EntityUtils.toString(httpEntity);
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return response;
}
}

How to send simple http post request with post parameters in java

I need a simple code example of sending http post request with post parameters that I get from form inputs.
I have found Apache HTTPClient, it has very reach API and lots of sophisticated examples, but I couldn't find a simple example of sending http post request with input parameters and getting text response.
Update: I'm interested in Apache HTTPClient v.4.x, as 3.x is deprecated.

Here's the sample code for Http POST, using Apache HTTPClient API.
import java.io.InputStream;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.PostMethod;
public class PostExample {
public static void main(String[] args){
String url = "http://www.google.com";
InputStream in = null;
try {
HttpClient client = new HttpClient();
PostMethod method = new PostMethod(url);
//Add any parameter if u want to send it with Post req.
method.addParameter("p", "apple");
int statusCode = client.executeMethod(method);
if (statusCode != -1) {
in = method.getResponseBodyAsStream();
}
System.out.println(in);
} catch (Exception e) {
e.printStackTrace();
}
}
}

I pulled this code from an Android project by Andrew Gertig that I have used in my application. It allows you to do an HTTPost. If I had time, I would create an POJO example, but hopefully, you can dissect the code and find what you need.
Arshak
https://github.com/AndrewGertig/RubyDroid/blob/master/src/com/gertig/rubydroid/AddEventView.java
private void postEvents()
{
DefaultHttpClient client = new DefaultHttpClient();
/** FOR LOCAL DEV HttpPost post = new HttpPost("http://192.168.0.186:3000/events"); //works with and without "/create" on the end */
HttpPost post = new HttpPost("http://cold-leaf-59.heroku.com/myevents");
JSONObject holder = new JSONObject();
JSONObject eventObj = new JSONObject();
Double budgetVal = 99.9;
budgetVal = Double.parseDouble(eventBudgetView.getText().toString());
try {
eventObj.put("budget", budgetVal);
eventObj.put("name", eventNameView.getText().toString());
holder.put("myevent", eventObj);
Log.e("Event JSON", "Event JSON = "+ holder.toString());
StringEntity se = new StringEntity(holder.toString());
post.setEntity(se);
post.setHeader("Content-Type","application/json");
} catch (UnsupportedEncodingException e) {
Log.e("Error",""+e);
e.printStackTrace();
} catch (JSONException js) {
js.printStackTrace();
}
HttpResponse response = null;
try {
response = client.execute(post);
} catch (ClientProtocolException e) {
e.printStackTrace();
Log.e("ClientProtocol",""+e);
} catch (IOException e) {
e.printStackTrace();
Log.e("IO",""+e);
}
HttpEntity entity = response.getEntity();
if (entity != null) {
try {
entity.consumeContent();
} catch (IOException e) {
Log.e("IO E",""+e);
e.printStackTrace();
}
}
Toast.makeText(this, "Your post was successfully uploaded", Toast.LENGTH_LONG).show();
}

HTTP POST request example using Apache HttpClient v.4.x
HttpClient httpClient = HttpClients.createDefault();
HttpPost httpPost = new HttpPost(url);
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.addTextBody("param1", param1Value, ContentType.TEXT_PLAIN);
builder.addTextBody("param2", param2Value, ContentType.TEXT_PLAIN);
HttpEntity multipart = builder.build();
httpPost.setEntity(multipart);
HttpResponse response = httpClient.execute(httpMethod);

http://httpunit.sourceforge.net/doc/cookbook.html
use PostMethodWebRequest and setParameter method

shows a very simple exapmle where you do post from Html page, servlet processes it and sends a text response..
http://java.sun.com/developer/onlineTraining/Programming/BasicJava1/servlet.html