Develop Reference

QAF: '401 Unauthorized: Access is denied' when running script with NTLM request

I already implemented the NTLMAuthClient class
rest.client.impl=com.qmetry.qaf.automation.ws.client.NTLMAuthClient
into the project properties but still getting the aforementioned error.
Here is the JSON request template I am using and I am not sure whether the structure is what causes the error:
{
    "headers":
    {
        "Authorization":
        [
            {
                "username": "${ntlm.user}",
                "password": "${ntlm.password}"
            },
            {
                "domain": "${ntlm.domain}"
            }
        ],
        "Content-Type":"application/json"
    },
    
    "endPoint": "",
    "baseUrl": "${base_url}/api/data/v4/contacts(integrationkey='${clientNumber}')",
    "method": "PATCH",
    
    "body":
    {
    "userid": ""
    }
}
And here's the response from the QMetry report:
Request
Client out-bound request PATCH
https://xxxx.com/CRMDEV2/api/data/v4/contacts(integrationkey='xxxx')
Authorization: [{username=xxxx, password=xxxx},
{domain=xxxx}] Content-Type: application/json
{"userid":""}
Reponse
Client in-bound response 401 REQ_ID:
3d391170-40d7-4ea6-80ca-2531a7fe73ff Server: WWW-Authenticate: NTLM
WWW-Authenticate: Negotiate Set-Cookie:
ReqClientId=21490c2c-bdfe-4c00-9330-401ca0bb5811; expires=Thu,
22-Oct-2071 11:59:11 GMT; path=/; secure; HttpOnly Content-Length: 49
Date: Fri, 22 Oct 2021 11:59:11 GMT Content-Type: text/plain HTTP
Error 401 - Unauthorized: Access is denied
Appreciate the help.
QAF and Selenium version I am using:
QAF version: 2.1.14
Selenium: 3.141.59

Refer NTLMAuthClient java-doc. Instead of Authorization in request call header, you should try providing properties used by NTLMAuthClient.
NTLMAuthClient class uses following properties:
ntlm.user - The user name. This should not include the domain to authenticate with. For example: "user" is correct whereas "DOMAIN\\user" is not.
ntlm.password - The password
ntlm.workstation - workstation (default is blank) The workstation the authentication request is originating from. Essentially, the computer name for this machine.
ntlm.domain- domain The domain to authenticate within (default is blank).
#register NTLMAuthClient
rest.client.impl=com.qmetry.qaf.automation.ws.client.NTLMAuthClient
set above properties in your property file and then try without authorization in request call header.

Apache HttpClient is not showing Content-Length and Content-Encoding headers of the response

I installed Apache httpcomponents-client-5.0.x and while reviewing the headers of the http response, I was shocked it doesn't show the Content-Length and Content-Encoding headers, this is the code I used for testing
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
import org.apache.hc.client5.http.impl.classic.CloseableHttpResponse;
import org.apache.hc.client5.http.impl.classic.HttpClients;
import com.sun.net.httpserver.Headers;
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet request = new HttpGet(new URI("https://www.example.com"));
CloseableHttpResponse response = httpclient.execute(request);
Header[] responseHeaders = response.getHeaders();
for(Header header: responseHeaders) {
System.out.println(header.getName());
}
// this prints all the headers except
// status code header
// Content-Length
// Content-Encoding
No matter what I try I get the same result, like this
Iterator<Header> headersItr = response.headerIterator();
while(headersItr.hasNext()) {
Header header = headersItr.next();
System.out.println(header.getName());
}
Or this
HttpEntity entity = response.getEntity();
System.out.println(entity.getContentEncoding()); // NULL
System.out.println(entity.getContentLength()); // -1
According to this question that has been asked 6 years ago, it seems like an old issue even with older versions of Apache HttpClient.
Of-course the server is actually returning those headers as confirmed by Wireshark, and Apache HttpClient logs itself
2020-04-03 07:59:09,106 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << HTTP/1.1 200 OK
2020-04-03 07:59:09,106 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Content-Encoding: gzip
2020-04-03 07:59:09,106 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Accept-Ranges: bytes
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Age: 451956
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Cache-Control: max-age=604800
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Content-Type: text/html; charset=UTF-8
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Date: Fri, 03 Apr 2020 05:59:09 GMT
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Etag: "3147526947+gzip"
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Expires: Fri, 10 Apr 2020 05:59:09 GMT
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Server: ECS (dcb/7EEB)
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Vary: Accept-Encoding
2020-04-03 07:59:09,109 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << X-Cache: HIT
2020-04-03 07:59:09,109 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Content-Length: 648
BTW, java.net.http library known as JDK HttpClient works great and show all the headers.
Is there something wrong I did, or should I report a bug that been there for years ?

HttpComponents committer here...
You did not closely pay attention what Dave G said. By default, HttpClientBuilder will enable transparent decompression and the reason why you don't see some headers anymore is here:
if (decoderFactory != null) {
response.setEntity(new DecompressingEntity(response.getEntity(), decoderFactory));
response.removeHeaders(HttpHeaders.CONTENT_LENGTH);
response.removeHeaders(HttpHeaders.CONTENT_ENCODING);
response.removeHeaders(HttpHeaders.CONTENT_MD5);
} ...
Regarding the JDK HttpClient, it will not perform any transparent decompression, therefore you see the length of the compressed stream. You have to decompress on your own.
curl committer here...
I have raised an issue too.
Update: 03 Feb. '23 The secret codez to disable automatic decompression are:
CloseableHttpClient httpclient = HttpClients.createSimple();
// OR
CloseableHttpClient httpclient = HttpClients.custom().disableContentCompression().build();

The content-length may be potentially ignored in this case.
HttpGet request = new HttpGet(new URI("https://www.example.com"));
request.setHeader("Accept-Encoding", "identity");
CloseableHttpResponse response = httpclient.execute(request);
I can see the following
HttpEntity entity = response.getEntity();
System.out.println(entity.getContentLength());
System.out.println(entity.getContentEncoding());
Output
...
2020-04-03 03:04:17.760 DEBUG 34196 --- [ main] org.apache.hc.client5.http.headers : http-outgoing-0 << Content-Length: 1256
...
1256
null
I'd like to direct your attention to this header being sent:
http-outgoing-0 >> Accept-Encoding: gzip, x-gzip, deflate
That tells the server that this client can accept gzip, x-gzip, and deflate content in response. The response is stating it is 'gzip' encoded.
http-outgoing-0 << Content-Encoding: gzip
I believe that HttpClient is transparently handling this internally and making the content available.
As stated in the other article you referenced, one of the answers indicated that the method EntityUtils.toByteArray(httpResponse.getEntity()).length could be applied to get the content length.

get youtube redirected link from short link

When i click url:
https://youtu.be/rCiqxFuRi6o?rel=0&wmode=transparent
It redirects to
https://www.youtube.com/watch?v=rCiqxFuRi6o&feature=youtu.be&rel=0&wmode=transparent
How to get programatically from android app this redirected link?
I want finally to get youtube video id.

when you connect to
https://youtu.be/rCiqxFuRi6o?rel=0&wmode=transparent
you'll get http 302 status (redirection) :
HTTP/1.1 302 Found
Date: Thu, 07 May 2015 14:25:38 GMT
Server: gwiseguy/2.0
Location: https://www.youtube.com/watch?v=rCiqxFuRi6o&feature=youtu.be
Content-Length: 0
Content-Type: text/html
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
so you just need to read the Location header from the http response header.

App-Engine directing endpoint to previous class name, now 404 not found

I've been using App-Engine as the backend for an Android and iOS application. It's been working without problem with both the local development server (over http) and actual app-engine (over https).
Then I noticed that, while renaming endpoints, I accidentally duplicated a word in the class name of an endpoint: RegionRegionIconsEndpoint instead of simply RegionIconsEndpoint. It was a 1-line fix.
public class RegionRegionIconsEndpoint {
#ApiMethod(name = "getRegionIcons", path="regionIcons", httpMethod = HttpMethod.POST)
public RegionInfoVersion.RegionIcons getRegionIcons(User user, #Named("id") String id)
throws OAuthRequestException {
...
}
}
became
public class RegionIconsEndpoint {
#ApiMethod(name = "getRegionIcons", path="regionIcons", httpMethod = HttpMethod.POST)
public RegionInfoVersion.RegionIcons getRegionIcons(User user, #Named("id") String id)
throws OAuthRequestException {
...
}
}
I generated new cloud-endpoint libraries and continued development using the local development server. All good.
When I deployed it to the real App-Engine service, however, a problem arose. When my app starts, there are a series of calls to other endpoints defined just as the one shown above; these always work fine. Then there are calls to this endpoint. A typical call looks like this:
POST https://my-app.appspot.com/_ah/api/client/v1/regionIcons?id=foo
Authorization is also provided and the expected result comes back most of the time... say 80%. The AE logs look like this:
2014-05-02 21:36:30.551 /_ah/spi/com.example.app.endpoints.RegionIconsEndpoint.getRegionIcons 200 48ms 0kb Google-HTTP-Java-Client/1.16.0-rc (gzip) module=default version=1
70.80.59.221 - - [02/May/2014:18:36:30 -0700] "POST /_ah/spi/com.example.app.endpoints.RegionIconsEndpoint.getRegionIcons HTTP/1.1" 200 149 - "Google-HTTP-Java-Client/1.16.0-rc (gzip)" "my-app.appspot.com" ms=49 cpu_ms=41 cpm_usd=0.000017 app_engine_release=1.9.4 instance=006c1b117c1b2d35341e0f407ae5785a825b65e5
The remaining times, I get a 404 Not Found response and the AE logs have this:
2014-05-02 21:36:30.852 /_ah/spi/BackendService.logMessages 204 16ms 0kb module=default version=1
10.1.0.41 - - [02/May/2014:18:36:30 -0700] "POST /_ah/spi/BackendService.logMessages HTTP/1.1" 204 0 - - "my-app.appspot.com" ms=16 cpu_ms=0 app_engine_release=1.9.4 instance=006c1b117c1b2d35341e0f407ae5785a825b65e5
E 2014-05-02 21:36:30.851
Request URL: https://my-app.appspot.com/_ah/api/client/v1/regionIcons?id=foo
Method: client.getRegionIcons
Error Code: 404
Reason: notFound
Message: service 'com.example.app.endpoints.RegionRegionIconsEndpoint' not found
2014-05-02 21:36:30.802 /_ah/spi/com.example.app.endpoints.RegionRegionIconsEndpoint.getRegionIcons 404 16ms 0kb Google-HTTP-Java-Client/1.16.0-rc (gzip) module=default version=1
70.80.59.221 - - [02/May/2014:18:36:30 -0700] "POST /_ah/spi/com.example.app.endpoints.RegionRegionIconsEndpoint.getRegionIcons HTTP/1.1" 404 166 - "Google-HTTP-Java-Client/1.16.0-rc (gzip)" "my-app.appspot.com" ms=16 cpu_ms=0 cpm_usd=0.000019 app_engine_release=1.9.4 instance=006c1b117c1b2d35341e0f407ae5785a825b65e5
You can see on the Message line that, sometimes, AE is still trying to process the call using the old class name with the duplicated word! I've done searches over my entire code-base and the generated files and I cannot find the string "RegionRegion" anywhere. I've checked the web.xml file a dozen times and it has only the new "RegionIconsEndpoint" class name.
Wondering if somehow Google's servers were keeping old information around, I deployed the new version of my app as 2-dot-my-app.appspot.com. The behavior remains exactly the same except that there are no AE log messages for the requests that fail with 404 on this version. Successful request logs are as before.
Both my Android and iPad apps are experiencing this. In addition, I've managed to reproduce it using the web and Google's API explorer on my-app.appspot.com. In this last case, a successful request shows this:
200 OK
cache-control: no-cache, no-store, max-age=0, must-revalidate
content-encoding: gzip
content-length: 171
content-type: application/json; charset=UTF-8
date: Sat, 03 May 2014 03:07:05 GMT
etag: "G170GGjYGsLnxTffzUEJmTttHzU/LUWzmydK3mjH7IeRbEc_n9J6cDQ"
expires: Fri, 01 Jan 1990 00:00:00 GMT
pragma: no-cache
server: GSE
{
"iconsVid": "foo",
"iconsVersion": 3,
"kind": "client#resourcesItem",
"etag": "\"G170GGjYGsLnxTffzUEJmTttHzU/LUWzmydK3mjH7IeRbEc_n9J6cDQ\""
}
and a failed request shows this:
404 Not Found
cache-control: private, max-age=0
content-encoding: gzip
content-length: 169
content-type: application/json; charset=UTF-8
date: Sat, 03 May 2014 03:08:34 GMT
expires: Sat, 03 May 2014 03:08:34 GMT
server: GSE
{
"error": {
"errors": [
{
"domain": "global",
"reason": "notFound",
"message": "service 'com.example.app.endpoints.RegionRegionIconsEndpoint' not found"
}
],
"code": 404,
"message": "service 'com.example.app.endpoints.RegionRegionIconsEndpoint' not found"
}
}
again clearly showing an access to the old class name. When trying to do the same to the v2 version that I deployed (2-dot-my-app.appspot.com), it's different. A success request ends like this:
200 OK
cache-control: no-cache, no-store, max-age=0, must-revalidate
content-encoding: gzip
content-length: 171
content-type: application/json; charset=UTF-8
date: Sat, 03 May 2014 03:12:08 GMT
etag: "EP5CWx59se1v4KdDnkfEx7cTkis/LUWzmydK3mjH7IeRbEc_n9J6cDQ"
expires: Fri, 01 Jan 1990 00:00:00 GMT
pragma: no-cache
server: GSE
{
"iconsVid": "foo",
"iconsVersion": 3,
"kind": "client#resourcesItem",
"etag": "\"EP5CWx59se1v4KdDnkfEx7cTkis/LUWzmydK3mjH7IeRbEc_n9J6cDQ\""
}
and a failed request ends like this:
404 Not Found
cache-control: no-cache, no-store, max-age=0, must-revalidate
content-encoding: gzip
content-length: 29
content-type: text/html; charset=UTF-8
date: Sat, 03 May 2014 03:06:10 GMT
expires: Fri, 01 Jan 1990 00:00:00 GMT
pragma: no-cache
server: GSE
Not Found
I don't know what else to try. To me, it looks like a bug in App-Engine.
So... any ideas what is going on here and how to fix or work around it?
2014-05-04: I tried changing the method from POST to GET: exact same behavior. I tried changing the path from regionIcons to regionIconsFoo: exact same behavior. I tried changing the #API version from v1 to v2: exact same behavior.
Finally, I tried changing the name of the class back to the previous (with the duplicated word): I get fewer failures (maybe 5% instead of 20%) but they still occur with the failing requests trying to access the now non-existent class name without the duplicated word.
Restoring the correct name resumes the originally described behavior with the original failure rate.

I've been struggling with similar problem. Check logs on appengine.com project site. You should see log about updating, if it has additional info about error - check it.
Sometimes AE works on local machine well but deployment process reveals some bugs.
Edit:
1. Rename the class back to old "double" name, upload it to AE and check if all requests are working without a bug, if yes, rename the class again. (if it's appengine bug it should fix it).
2. Create as simple as possible api and substitute it with your project. Update it to AE and check with api explorer is everything ok, without methods from your main project. If it's ok, once again swap "test" project with your true one and upload to AE.

This isn't an answer because it doesn't address the cause, but it is my solution.
I duplicated the working class back into the old class name.
public class RegionRegionIconsEndpoint {
#ApiMethod(name = "getRegionIconsOld", path="regionIconsOld", httpMethod = HttpMethod.POST)
public RegionInfoVersion.RegionIcons getRegionIcons(User user, #Named("id") String id)
throws OAuthRequestException {
...
}
}
Now in the log, even though I'm only ever calling getRegionIcons, I see indications that both classes are being called with the "old" version handling about 20% of the requests. It's a hack and I don't like it, but it works and the clients are happy with it.
If you can't beat 'em, join 'em.

HttpClient 4.2.2 and proxy with username/password

I have a problem with HttpClient 4.2.2 and proxy with username/password.
HttpClient works in this way:
List item
Send request without proxy stuff (however I set proxy parameters for each request)
Get response from Squid with 407 error
Send request with proxy stuff
It's very strange behavior, Is it possible to add info about proxy to each request? I've tried to add hardcoded "Proxy-Authorization" header to each request and it works fine, why HttpClient can't do the same?
Java code
DefaultHttpClient httpClient = new DefaultHttpClient();
httpClient.getParams().setParameter(PROTOCOL_VERSION, HTTP_1_1);
...
String proxyServer = getProxyServer();
int proxyPort = getProxyPort();
List<String> authpref = new ArrayList<String>();
authpref.add(AuthPolicy.BASIC);
httpClient.getParams().setParameter(AuthPNames.PROXY_AUTH_PREF, authpref);
String proxyUser = getProxyUser();
String proxyPassword = getProxyPassword();
CredentialsProvider credsProvider = httpClient.getCredentialsProvider();
credsProvider.setCredentials(new AuthScope(proxyServer, proxyPort), new UsernamePasswordCredentials(proxyUser, proxyPassword));
httpClient.setCredentialsProvider(credsProvider);
HttpHost proxy = new HttpHost(proxyServer, proxyPort, (proxyServer.indexOf("https") != 0) ? "http" : "https");
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
...
HttpPut put = new HttpPut(url);
/*** hardcoded header ***/
//put.addHeader("Proxy-Authorization", "Basic eHRlbmR4LmRuZXByOnF3ZXJ0eQ==");
/*** hardcoded header ***/
put.setEntity(entity);
httpClient.execute(put);
httpClient.getConnectionManager().shutdown();
[DefaultClientConnection] Sending request: PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg HTTP/1.1
[headers] >> PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg HTTP/1.1
[headers] >> Authorization: Basic eGRldjo0YTFmNmMwOTgyYWRkMWQ0NDg1YjRhMGE4YWMxY2JjMWNiMTA0ODc1
[headers] >> Content-Length: 2
[headers] >> Host: 172.26.27.22:8080
[headers] >> Proxy-Connection: Keep-Alive
[headers] >> User-Agent: Apache-HttpClient/4.2.2 (java 1.5)
[DefaultClientConnection] Receiving response: HTTP/1.0 407 Proxy Authentication Required
[headers] << HTTP/1.0 407 Proxy Authentication Required
[headers] << Server: squid/2.7.STABLE8
[headers] << Date: Thu, 08 Nov 2012 10:09:49 GMT
[headers] << Content-Type: text/html
[headers] << Content-Length: 1431
[headers] << X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
[headers] << Proxy-Authenticate: Basic realm="Please, enter username and password"
[headers] << X-Cache: MISS from 172.26.27.94
[headers] << X-Cache-Lookup: NONE from 172.26.27.94:3128
[headers] << Via: 1.0 172.26.27.94:3128 (squid/2.7.STABLE8)
[headers] << Connection: close
[DefaultHttpClient] Authentication required
[DefaultHttpClient] 172.26.27.94:3128 requested authentication
[ProxyAuthenticationStrategy] Authentication schemes in the order of preference: [Basic]
[DefaultHttpClient] Selected authentication options: [BASIC]
[DefaultClientConnection] Connection 0.0.0.0:63344<->172.26.27.94:3128 closed
[DefaultClientConnectionOperator] Connecting to 172.26.27.94:3128
[RequestAddCookies] CookieSpec selected: best-match
[RequestAuthCache] Re-using cached 'basic' auth scheme for http://172.26.27.22:8080
[RequestAuthCache] No credentials for preemptive authentication
[RequestProxyAuthentication] Proxy auth state: CHALLENGED
[RequestProxyAuthentication] Generating response to an authentication challenge using basic scheme
[DefaultHttpClient] Attempt 2 to execute request
[DefaultClientConnection] Sending request: PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg HTTP/1.1
[headers] >> PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg HTTP/1.1
[headers] >> Authorization: Basic eGRldjo0YTFmNmMwOTgyYWRkMWQ0NDg1YjRhMGE4YWMxY2JjMWNiMTA0ODc1
[headers] >> Content-Length: 2
[headers] >> Host: 172.26.27.22:8080
[headers] >> Proxy-Connection: Keep-Alive
[headers] >> User-Agent: Apache-HttpClient/4.2.2 (java 1.5)
[headers] >> Proxy-Authorization: Basic eHRlbmR4LmRuZXByOnF3ZXJ0eQ==
[DefaultClientConnection] Receiving response: HTTP/1.0 201 Created
[headers] << HTTP/1.0 201 Created
[headers] << Content-Length: 0
[headers] << Date: Thu, 08 Nov 2012 10:09:49 GMT
[headers] << X-Cache: MISS from 172.26.27.94
[headers] << X-Cache-Lookup: MISS from 172.26.27.94:3128
[headers] << Via: 1.1 172.26.27.94:3128 (squid/2.7.STABLE8)
[headers] << Connection: keep-alive
[headers] << Proxy-Connection: keep-alive
Squid log
1352370666.778 0 172.26.27.94 TCP_DENIED/407 1870 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg - NONE/- text/html
1352370671.429 8 172.26.27.94 TCP_MISS/201 282 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg proxyuser DIRECT/172.26.27.22 -
1352370671.474 0 172.26.27.94 TCP_DENIED/407 1882 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image2.jpg - NONE/- text/html
1352370671.486 7 172.26.27.94 TCP_MISS/201 282 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image2.jpg proxyuser DIRECT/172.26.27.22 -
With hardcoded header
There is no 407 error in the java log, and squid log
Squid log
1352370542.016 8 172.26.27.94 TCP_MISS/201 282 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg proxyuser DIRECT/172.26.27.22 -
1352370542.033 7 172.26.27.94 TCP_MISS/201 282 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image2.jpg proxyuser DIRECT/172.26.27.22 -

The authentication process with an HTTP proxy is described in RFC2616 §14.33 and §14.34 and is exactly as you see with HttpClient. It includes:
the proxy sending a 407 (Proxy Authentication Required) response with a Proxy-Authenticate header containing the challenge applicable to the requested resource,
the client issuing a new request with a Proxy-Authorization header consisting of the credentials containing the authentication information.
Implementing the preemptive authentication with HttpClient is described in this tutorial (section §4.8), and requires prepopulating the authentication cache of the client object.
Unfortunately, their code does not work when preauthenticating to a proxy server. It may be a bit tricky to understand how to get it right, but actually it is as simple as passing a parameter to the BasicScheme constructor:
AuthCache authCache = new BasicAuthCache();
AuthScheme basicAuthScheme = null;
if (isProxy) {
basicAuthScheme = new BasicScheme(ChallengeState.PROXY);
} else {
basicAuthScheme = new BasicScheme(ChallengeState.TARGET);
}
authCache.put(host, basicAuthScheme);
httpContext.setAttribute(ClientContext.AUTH_CACHE, authCache);

It's your AuthScope:
// You set proxyServer here vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
credsProvider.setCredentials(new AuthScope(proxyServer, proxyPort), new UsernamePasswordCredentials(proxyUser, proxyPassword));
httpClient.setCredentialsProvider(credsProvider);
// But here you are looking for the indexof https to determine if it is an SSL proxy
// is the String returned from getProxyServer() above a URL or a host name?
HttpHost proxy = new HttpHost(proxyServer, proxyPort, (proxyServer.indexOf("https") != 0) ? "http" : "https");
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
Buried in my comment above - is the string returned by getProxyServer() a URL or a hostname?
EDIT
I think I have figured out what is going on. Basic Auth has to be sent on every request for it to work. If you are making multiple executions you are going to need to have some mechanism to cache the authentication data. By executing the client in the fashion you are right now, there is no "context" that can store that as one is created on every request.
The other item to look at is the client tutorial sections 4.7 and 4.8. If you are looking to completely eliminate the 407 error followed by a request with the BASIC authentication, then make sure you read section 4.8.
Pay very close attention to the "localcontext" variable they have defined as this acts as a state container for your client.