Develop Reference

Apache HttpClient with POST file-uploading won't do a basic auth on mysterious reasons

I am trying to upload a file by the POST method with the Apache HttpClient library.
I used the example code for the preemptive basic authentification here:
package ahcs;
// many imports, press ctrl-o in eclipse
public class App {
static final String url = "http://127.0.0.1:64738/test/";
static final String content = "test\nfile\ndata";
static final String httpUser = "testuser";
static final String httpPasswd = "testPassword";
static final String fileUploadFieldName = "uploadData";
static final String fileName = "upload.dat";
public static void main(String[] args) {
System.err.println("Uploading to URL " + url);
CloseableHttpClient httpclient = HttpClientBuilder.create().build();
HttpPost httpPost = new HttpPost(url);
httpPost.setProtocolVersion(HttpVersion.HTTP_1_1);
MultipartEntityBuilder mpEntityBuilder =
MultipartEntityBuilder.create();
mpEntityBuilder.setMode(HttpMultipartMode.RFC6532);
mpEntityBuilder.addBinaryBody(fileUploadFieldName,
content.getBytes(), ContentType.DEFAULT_BINARY, fileName);
httpPost.setEntity(mpEntityBuilder.build());
System.err.println("executing request " + httpPost.getRequestLine());
HttpEntity resEntity = null;
try {
// Really simple HTTP Authentification, grat Apache
HttpHost httpHost = URIUtils.extractHost(new URI(url));
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(httpUser, httpPasswd));
AuthCache authCache = new BasicAuthCache();
authCache.put(httpHost, new BasicScheme());
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(credsProvider);
context.setAuthCache(authCache);
HttpResponse response = httpclient.execute(httpPost);
resEntity = response.getEntity();
System.err.println(response.getStatusLine().toString());
if (resEntity != null) {
System.err.println(EntityUtils.toString(resEntity));
}
int status = response.getStatusLine().getStatusCode();
if (status != HttpStatus.SC_OK) {
throw new HttpResponseException(status,
"Upload error! (" + status + ")");
}
EntityUtils.consume(resEntity);
httpclient.close();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
Unfortunately, it doesn't do what I want. The request what the apache httpclient gives, is this (I got this by listening from the command line with an nc -p 64738 -l command):
POST /test/ HTTP/1.1
Content-Length: 249
Content-Type: multipart/form-data; boundary=PIrvSJ07MLxTV2rC4d-5ZfoL3CvJFJdJqO4i
Host: 127.0.0.1:64738
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.4 (Java/1.8.0_151)
Accept-Encoding: gzip,deflate
--PIrvSJ07MLxTV2rC4d-5ZfoL3CvJFJdJqO4i
Content-Disposition: form-data; name="uploadData"; filename="upload.dat"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
test
file
data
--PIrvSJ07MLxTV2rC4d-5ZfoL3CvJFJdJqO4i--
As we can see, everything is okay, except that the authentification header is simply missing.
Why is it so? What is the bug?

According to RFC7617 you need only one header "Authorization" with values "Basic " + login:passord in Base64 encoding to successefuly pass Basic authorization.
Your code is correct, except one thing - when you call httpPost.execute you are not pass execution context, and AuthCache and CredentialsProvider wasn't used at all.
package ahcs;
// many imports, press ctrl-o in eclipse
public class App {
static final String url = "http://127.0.0.1:64738/test/";
static final String content = "test\nfile\ndata";
static final String httpUser = "testuser";
static final String httpPasswd = "testPassword";
static final String fileUploadFieldName = "uploadData";
static final String fileName = "upload.dat";
public static void main(String[] args) {
System.err.println("Uploading to URL " + url);
CloseableHttpClient httpclient = HttpClientBuilder.create().build();
HttpPost httpPost = new HttpPost(url);
httpPost.setProtocolVersion(HttpVersion.HTTP_1_1);
MultipartEntityBuilder mpEntityBuilder =
MultipartEntityBuilder.create();
mpEntityBuilder.setMode(HttpMultipartMode.RFC6532);
mpEntityBuilder.addBinaryBody(fileUploadFieldName,
content.getBytes(), ContentType.DEFAULT_BINARY, fileName);
httpPost.setEntity(mpEntityBuilder.build());
System.err.println("executing request " + httpPost.getRequestLine());
HttpEntity resEntity = null;
try {
// Really simple HTTP Authentification, grat Apache
HttpHost httpHost = URIUtils.extractHost(new URI(url));
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(httpUser, httpPasswd));
AuthCache authCache = new BasicAuthCache();
authCache.put(httpHost, new BasicScheme());
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(credsProvider);
context.setAuthCache(authCache);
// context was missed
HttpResponse response = httpclient.execute(httpPost, context);
resEntity = response.getEntity();
System.err.println(response.getStatusLine().toString());
if (resEntity != null) {
System.err.println(EntityUtils.toString(resEntity));
}
int status = response.getStatusLine().getStatusCode();
if (status != HttpStatus.SC_OK) {
throw new HttpResponseException(status,
"Upload error! (" + status + ")");
}
EntityUtils.consume(resEntity);
httpclient.close();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
But for Basic Auth using this API may be a bit verbose, it was designed to support many different authorization schemes.
If you know what charset server will use to decode Authorization header (suppose it UTF-8), you can write one-liner:
httpPost.setHeader("Authorization", "Basic " + Base64.getEncoder().encodeToString((httpUser + ':' + httpPasswd).getBytes‌​("UTF-8")));

Trying setting session cookie using HttpClient

Help setting cookie to HttpClient
Created a program which logins to an external web service. However, to obtain vital information from
an HTTP GET, I am unable to pass in the cookie (generated from the login).
public class ClientHelper {
private final static String PROFILE_URL =
"http://externalservice/api/profile.json";
private final static String LOGIN_URL = "http://externalservice/api/login";
public static Cookie login(final String username, final String password) {
DefaultHttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(LOGIN_URL);
HttpContext localContext = new BasicHttpContext();
client.getParams().setParameter("http.useragent", "Custom Browser");
client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION,
HttpVersion.HTTP_1_1);
List<Cookie> cookies = null;
BasicClientCookie cookie = null;
try {
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(3);
nameValuePairs.add(new BasicNameValuePair("user", username));
nameValuePairs.add(new BasicNameValuePair("passwd", password));
UrlEncodedFormEntity entity =
new UrlEncodedFormEntity(nameValuePairs, HTTP.UTF_8);
entity.setContentType("application/x-www-form-urlencoded");
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = client.execute(post, localContext);
cookies = client.getCookieStore().getCookies();
System.out.println(cookies.get(1));
cookie = new BasicClientCookie(cookies.get(1).getName(), cookies.get(1).getValue());
cookie.setVersion(cookies.get(1).getVersion());
cookie.setDomain(cookies.get(1).getDomain());
cookie.setExpiryDate(cookies.get(1).getExpiryDate());
cookie.setPath(cookies.get(1).getPath());
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line = "";
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
}
catch (Throwable e) {
e.printStackTrace();
}
return cookie;
}
public static void getProfile(Cookie cookie) {
DefaultHttpClient client = new DefaultHttpClient();
HttpContext context = new BasicHttpContext();
CookieStore cookieStore = new BasicCookieStore();
cookieStore.addCookie(cookie);
client.setCookieStore(cookieStore);
context.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
HttpGet get = new HttpGet(PROFILE_URL);
HttpResponse response;
try {
response = client.execute(get, context);
BufferedReader rd =
new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
String line = "";
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
}
catch (ClientProtocolException e) {
e.printStackTrace();
}
catch (IOException e) {
e.printStackTrace();
}
}
}
App.java (class that uses ClientHelper):
public class App {
private static final String USER = "myusername";
private static final String PASSWD = "mypassword";
public static void main(String[] args) {
Cookie cookie = ClientHelper.login(USER, PASSWD);
ClientHelper.getProfile(cookie);
}
}
When I run App, I am able to login (I see the generated JSON) but the getProfile() method returns an empty JSON object:
{}
From the command line, using curl I am trying to emulate this:
curl -b Cookie.txt http://externalservice/api/profile.json
This actually works but not my Java program.

Try by executing this part of the code:
List<Cookie> cookies = client.getCookieStore().getCookies();
for (Cookie cookie : cookies) {
singleCookie = cookie;
}
After
HttpResponse response = client.execute(post, localContext);

After changing your code to get the cookies after the login request, you actually are getting all the cookies from the request.
I suspect the problem is that whatever Cookie it is at index 1 in the CookieStore isn't the one you need, and obviously since it's not throwing an IndexOutOfBounds exception when you do that, there's at least one other Cookie in there (at index 0). Return the list of cookies and send all of them with your profile request.
Taking your code, changing all those indexes from 1 to 0 and pointing at this simple PHP script shows that it is receiving then sending the cookies:
<?php
setcookie("TestCookie", "Some value");
print_r($_COOKIE);
?>
output:
[version: 0][name: TestCookie][value: Some+value][domain: www.mydomain.org][path: /][expiry: null]
Array
(
)
Array
(
[TestCookie] => Some value
)

I figured it out... I was creating two different HTTP clients instead of using the same one.
#Brian Roach & Raunak Agarwal thank you both very much for the help!
Here's the fix:
public static HttpClient login(final String username, final String password)
{
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(LOGIN_URL);
client.getParams().setParameter("http.useragent", "Custom Browser");
client.getParams().setParameter(
CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
try
{
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(3);
nameValuePairs.add(new BasicNameValuePair("user", username));
nameValuePairs.add(new BasicNameValuePair("passwd", password));
UrlEncodedFormEntity entity =
new UrlEncodedFormEntity(nameValuePairs, HTTP.UTF_8);
entity.setContentType("application/x-www-form-urlencoded");
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = client.execute(post);
BufferedReader reader =
new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
String line = "";
while ((line = reader.readLine()) != null)
{
System.out.println(line);
}
}
catch (Throwable e) { e.printStackTrace(); }
return client;
}
public static void getProfile(HttpClient client)
{
HttpGet get = new HttpGet(PROFILE_URL);
HttpResponse response;
try
{
response = client.execute(get);
BufferedReader reader =
new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
String line = "";
while ((line = reader.readLine()) != null)
{
System.out.println(line);
}
}
catch (ClientProtocolException e) { e.printStackTrace(); }
catch (IOException e) { e.printStackTrace(); }
}

Get response from the web service for HttpResponse

In my application, the User can upload images to a PHP server, the iOS version is working 100%, the Android version used for this tutorial to upload image:
tutorial example
And the function I'm using is this:
public static String sendPost(String url, String imagePath)
throws IOException, ClientProtocolException {
HttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION,
HttpVersion.HTTP_1_1);
HttpPost httppost = new HttpPost(url);
File file = new File(imagePath);
MultipartEntity mpEntity = new MultipartEntity();
ContentBody cbFile = new FileBody(file, "image/jpeg");
mpEntity.addPart("userfile", cbFile);
httppost.setEntity(mpEntity);
//Log.e("executing request " + httppost.getRequestLine());
HttpResponse response = httpclient.execute(httppost);
HttpEntity resEntity = response.getEntity();
//Log.e(""+response.getStatusLine());
if (resEntity != null) {
//Log.e(EntityUtils.toString(resEntity));
}
if (resEntity != null) {
resEntity.consumeContent();
}
httpclient.getConnectionManager().shutdown();
return response.toString();
}
return response.toString(); get it in org.apache.http.message.BasicHttpResponse # 406dc148
But the return of the web service is the URL where the image was saved, I need to have a string in the return of the PHP server, rather than the return I mentioned above how can I have it?
I wanted something like this (HttpURLConnection):
HttpURLConnection conn;
...
String response= "";
Scanner inStream = new Scanner(conn.getInputStream());
while(inStream.hasNextLine())
response+=(inStream.nextLine());
Log.e("resp", response);
After one hour onsegui trying to get the response from the Web service as follows:
...
byte [] responseBody = httppost.getMethod().getBytes();
Log.e("RESPONSE BODY",""+(new String(responseBody)));
...

If you want the content returned by the HTTP server, you shouldn't do this:
if (resEntity != null) {
resEntity.consumeContent();
}
... because that says "throw away the content".

Try this if the response is of type String
ResponseHandler<String> responseHandler = new BasicResponseHandler();
HttpResponse httpResponse = httpClient.execute(post, new BasicHttpContext()); // new BasicHttpContext() not necessary
// verify connection response status using httpResponse.getStatusLine().getStatusCode() then
String response = responseHandler.handleResponse(httpResponse);
if(response != mull){
Log.e("Response : "+response);
}else{
// Handle exception
}
return response;

HTTP/1.1 400 Bad Request on httpGet with Base Authentication

I made an application that do a httpGet.
My application uses my own certificate and so the server requires a user login.
For certificate i used this tutorial.
For httpGet I've made this implementation:
HttpClient client = new CustClient(getApplicationContext()); // HttpClient that uses my certificates
// Example send http request
final String url = "https://ip:port/";// <--in my implementation i've a right url
HttpResponse response = null;
HttpGet httpGet = new HttpGet(url);
//login
httpGet.addHeader("Authorization", "Basic "+Base64.encodeToString("root:root".getBytes(),Base64.DEFAULT));
StringBuilder testo = null;
try {
response = client.execute(httpGet);
InputStream contenuto = response.getEntity().getContent();
BufferedReader rd = new BufferedReader(new InputStreamReader(contenuto));
String line;
// Read response until the end
while ((line = rd.readLine()) != null) {
testo.append(line);
}
} catch (Exception e) {
}
Whyen i make client.execute(httpGet) response is HTTP/1.1 400 Bad Request. Why?
Is correct to authenticate as in my code?

the problem was the Authorization header.
We have to use:
httpGet.addHeader("Authorization", "Basic "+Base64.encodeToString("root:root".getBytes(),Base64.NO_WRAP));
Instead of:
httpGet.addHeader("Authorization", "Basic "+Base64.encodeToString("root:root".getBytes(),Base64.DEFAULT));
because the DEFAULT parameter add "CR" line terminator at the end of string and it's uncorrect if you'll use it that header.

Problems connecting with a PHP: withg java.lang.Exception: HTTP error: 401

I have this code, who should connect to a php remote file and should get a String representing a XML file. But something is wrong, it is giving me error 401.
The variable url is the direction of the php:
String response=getXML("http://ficticiousweb.com/scripts/getMagazinesList.php");
If i paste the real direction (that is a ficticious direction) on the webbrowser, it works and gives me the XML.
This is my code:
public String getXML(String url){
try{
StringBuilder builder = new StringBuilder();
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet(url);
HttpResponse response = httpclient.execute(httpget);
int statuscode = response.getStatusLine().getStatusCode();
if(statuscode == 200)
{
HttpEntity entity = response.getEntity();
InputStream content = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(content));
String line;
while ((line = reader.readLine()) != null) builder.append(line);
}
else throw new Exception("HTTP error: " + String.valueOf(statuscode));
return builder.toString();
}catch(Exception e){e.printStackTrace();}
return null;
}
What is wrong with the code?
thanks

You need to login to the requested site in order to download or access the xml. This can be done by authenticated schema based upon what is supported. Normally, there are 2 types of schemas where used. Basic and Digest. Below code will help you for BASIC AUTH.
HttpClient httpclient = new DefaultHttpClient();
HttpResponse response;
String _username = "username";
String _password = "password";
try {
((AbstractHttpClient) httpclient).getCredentialsProvider().setCredentials(
new org.apache.http.auth.AuthScope(webhostname, webport)),
new org.apache.http.auth.UsernamePasswordCredentials(_username, _password));
response = httpclient.execute(new HttpGet(completeurlhere));
StatusLine statusLine = response.getStatusLine();
if(statusLine.getStatusCode() == HttpStatus.SC_OK) {
try {
InputStream is = response.getEntity().getContent();
this._data = is;
} catch(Exception ex) {
Log.e("DBF Error",ex.toString());
}
} else {
response.getEntity().getContent().close();
throw new IOException(statusLine.getReasonPhrase());
}
} catch(ClientProtocolException cpe) {
Log.e("ClientProtocolException # at FPT",cpe.toString());
} catch(Exception ex) {
Log.e("Exception at FETCHPROJECTASK",ex.toString());
}

Well a 401 means you aren't Authorized to do the GET request. You should ask the website how to Authenticate the request...
Authorization happens through the Authorization Header in the HTTP request. You should look into that and probably fill that header yourself with your credentials... (if the server accepts that)