I have a problem with HttpClient 4.2.2 and proxy with username/password.
HttpClient works in this way:
List item
Send request without proxy stuff (however I set proxy parameters for each request)
Get response from Squid with 407 error
Send request with proxy stuff
It's very strange behavior, Is it possible to add info about proxy to each request? I've tried to add hardcoded "Proxy-Authorization" header to each request and it works fine, why HttpClient can't do the same?
Java code
DefaultHttpClient httpClient = new DefaultHttpClient();
httpClient.getParams().setParameter(PROTOCOL_VERSION, HTTP_1_1);
...
String proxyServer = getProxyServer();
int proxyPort = getProxyPort();
List<String> authpref = new ArrayList<String>();
authpref.add(AuthPolicy.BASIC);
httpClient.getParams().setParameter(AuthPNames.PROXY_AUTH_PREF, authpref);
String proxyUser = getProxyUser();
String proxyPassword = getProxyPassword();
CredentialsProvider credsProvider = httpClient.getCredentialsProvider();
credsProvider.setCredentials(new AuthScope(proxyServer, proxyPort), new UsernamePasswordCredentials(proxyUser, proxyPassword));
httpClient.setCredentialsProvider(credsProvider);
HttpHost proxy = new HttpHost(proxyServer, proxyPort, (proxyServer.indexOf("https") != 0) ? "http" : "https");
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
...
HttpPut put = new HttpPut(url);
/*** hardcoded header ***/
//put.addHeader("Proxy-Authorization", "Basic eHRlbmR4LmRuZXByOnF3ZXJ0eQ==");
/*** hardcoded header ***/
put.setEntity(entity);
httpClient.execute(put);
httpClient.getConnectionManager().shutdown();
[DefaultClientConnection] Sending request: PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg HTTP/1.1
[headers] >> PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg HTTP/1.1
[headers] >> Authorization: Basic eGRldjo0YTFmNmMwOTgyYWRkMWQ0NDg1YjRhMGE4YWMxY2JjMWNiMTA0ODc1
[headers] >> Content-Length: 2
[headers] >> Host: 172.26.27.22:8080
[headers] >> Proxy-Connection: Keep-Alive
[headers] >> User-Agent: Apache-HttpClient/4.2.2 (java 1.5)
[DefaultClientConnection] Receiving response: HTTP/1.0 407 Proxy Authentication Required
[headers] << HTTP/1.0 407 Proxy Authentication Required
[headers] << Server: squid/2.7.STABLE8
[headers] << Date: Thu, 08 Nov 2012 10:09:49 GMT
[headers] << Content-Type: text/html
[headers] << Content-Length: 1431
[headers] << X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
[headers] << Proxy-Authenticate: Basic realm="Please, enter username and password"
[headers] << X-Cache: MISS from 172.26.27.94
[headers] << X-Cache-Lookup: NONE from 172.26.27.94:3128
[headers] << Via: 1.0 172.26.27.94:3128 (squid/2.7.STABLE8)
[headers] << Connection: close
[DefaultHttpClient] Authentication required
[DefaultHttpClient] 172.26.27.94:3128 requested authentication
[ProxyAuthenticationStrategy] Authentication schemes in the order of preference: [Basic]
[DefaultHttpClient] Selected authentication options: [BASIC]
[DefaultClientConnection] Connection 0.0.0.0:63344<->172.26.27.94:3128 closed
[DefaultClientConnectionOperator] Connecting to 172.26.27.94:3128
[RequestAddCookies] CookieSpec selected: best-match
[RequestAuthCache] Re-using cached 'basic' auth scheme for http://172.26.27.22:8080
[RequestAuthCache] No credentials for preemptive authentication
[RequestProxyAuthentication] Proxy auth state: CHALLENGED
[RequestProxyAuthentication] Generating response to an authentication challenge using basic scheme
[DefaultHttpClient] Attempt 2 to execute request
[DefaultClientConnection] Sending request: PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg HTTP/1.1
[headers] >> PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg HTTP/1.1
[headers] >> Authorization: Basic eGRldjo0YTFmNmMwOTgyYWRkMWQ0NDg1YjRhMGE4YWMxY2JjMWNiMTA0ODc1
[headers] >> Content-Length: 2
[headers] >> Host: 172.26.27.22:8080
[headers] >> Proxy-Connection: Keep-Alive
[headers] >> User-Agent: Apache-HttpClient/4.2.2 (java 1.5)
[headers] >> Proxy-Authorization: Basic eHRlbmR4LmRuZXByOnF3ZXJ0eQ==
[DefaultClientConnection] Receiving response: HTTP/1.0 201 Created
[headers] << HTTP/1.0 201 Created
[headers] << Content-Length: 0
[headers] << Date: Thu, 08 Nov 2012 10:09:49 GMT
[headers] << X-Cache: MISS from 172.26.27.94
[headers] << X-Cache-Lookup: MISS from 172.26.27.94:3128
[headers] << Via: 1.1 172.26.27.94:3128 (squid/2.7.STABLE8)
[headers] << Connection: keep-alive
[headers] << Proxy-Connection: keep-alive
Squid log
1352370666.778 0 172.26.27.94 TCP_DENIED/407 1870 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg - NONE/- text/html
1352370671.429 8 172.26.27.94 TCP_MISS/201 282 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg proxyuser DIRECT/172.26.27.22 -
1352370671.474 0 172.26.27.94 TCP_DENIED/407 1882 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image2.jpg - NONE/- text/html
1352370671.486 7 172.26.27.94 TCP_MISS/201 282 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image2.jpg proxyuser DIRECT/172.26.27.22 -
With hardcoded header
There is no 407 error in the java log, and squid log
Squid log
1352370542.016 8 172.26.27.94 TCP_MISS/201 282 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image1.jpg proxyuser DIRECT/172.26.27.22 -
1352370542.033 7 172.26.27.94 TCP_MISS/201 282 PUT http://172.26.27.22:8080/myapp/rest/2/3/29/image2.jpg proxyuser DIRECT/172.26.27.22 -
The authentication process with an HTTP proxy is described in RFC2616 §14.33 and §14.34 and is exactly as you see with HttpClient. It includes:
the proxy sending a 407 (Proxy Authentication Required) response with a Proxy-Authenticate header containing the challenge applicable to the requested resource,
the client issuing a new request with a Proxy-Authorization header consisting of the credentials containing the authentication information.
Implementing the preemptive authentication with HttpClient is described in this tutorial (section §4.8), and requires prepopulating the authentication cache of the client object.
Unfortunately, their code does not work when preauthenticating to a proxy server. It may be a bit tricky to understand how to get it right, but actually it is as simple as passing a parameter to the BasicScheme constructor:
AuthCache authCache = new BasicAuthCache();
AuthScheme basicAuthScheme = null;
if (isProxy) {
basicAuthScheme = new BasicScheme(ChallengeState.PROXY);
} else {
basicAuthScheme = new BasicScheme(ChallengeState.TARGET);
}
authCache.put(host, basicAuthScheme);
httpContext.setAttribute(ClientContext.AUTH_CACHE, authCache);
It's your AuthScope:
// You set proxyServer here vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
credsProvider.setCredentials(new AuthScope(proxyServer, proxyPort), new UsernamePasswordCredentials(proxyUser, proxyPassword));
httpClient.setCredentialsProvider(credsProvider);
// But here you are looking for the indexof https to determine if it is an SSL proxy
// is the String returned from getProxyServer() above a URL or a host name?
HttpHost proxy = new HttpHost(proxyServer, proxyPort, (proxyServer.indexOf("https") != 0) ? "http" : "https");
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
Buried in my comment above - is the string returned by getProxyServer() a URL or a hostname?
EDIT
I think I have figured out what is going on. Basic Auth has to be sent on every request for it to work. If you are making multiple executions you are going to need to have some mechanism to cache the authentication data. By executing the client in the fashion you are right now, there is no "context" that can store that as one is created on every request.
The other item to look at is the client tutorial sections 4.7 and 4.8. If you are looking to completely eliminate the 407 error followed by a request with the BASIC authentication, then make sure you read section 4.8.
Pay very close attention to the "localcontext" variable they have defined as this acts as a state container for your client.
Related
I'm trying to access rest api through my companies proxy server this requires me authenticate with the proxy.
I can access the outside world using curl with the parameters:
curl -v -s -user $kerbrosId -x $ProxyServer:$proxyPort --proxy-negotiate https://google.com
I receive the following output:
Enter proxy password for user '$user':
* Trying $proxyIP:$proxyPort...
* Connected to $proxyURL ($proxyIP) port $proxyPort (#0)
* allocate connect buffer
* Establish HTTP proxy tunnel to google.com:443
* Proxy auth using Negotiate with user 'bradsw'
> CONNECT google.com:443 HTTP/1.1
> Host: google.com:443
> Proxy-Authorization: Negotiate ${shortBase64token}
> User-Agent: curl/7.83.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 407 Proxy Authentication Required
< Proxy-Authenticate: NEGOTIATE ${longerBase64token}
< Cache-Control: no-cache
< Pragma: no-cache
< Content-Type: text/html; charset=utf-8
< Proxy-Connection: Keep-Alive
< Connection: Keep-Alive
< Content-Length: 5621
<
* Ignore 5621 bytes of response-body
* Establish HTTP proxy tunnel to google.com:443
* Proxy auth using Negotiate with user '$user'
> CONNECT google.com:443 HTTP/1.1
> Host: google.com:443
> Proxy-Authorization: Negotiate ${evenLongerBase64token}
> User-Agent: curl/7.83.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed
* schannel: disabled automatic use of client certificate
* ALPN: offers http/1.1
* ALPN: server accepted http/1.1
> GET / HTTP/1.1
> Host: google.com
> User-Agent: curl/7.83.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Location: https://www.google.com/
< Content-Type: text/html; charset=UTF-8
< Date: Thu, 29 Sep 2022 22:09:00 GMT
< Expires: Sat, 29 Oct 2022 22:09:00 GMT
< Cache-Control: public, max-age=2592000
< Server: gws
< Content-Length: 220
< X-XSS-Protection: 0
< X-Frame-Options: SAMEORIGIN
< Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
I have tried a million variations of
RestTemplate(HttpComponentsClientHttpRequestFactory(
HttpClientBuilder.create()
.setProxy(HttpHost(proxyUrl, proxyPort))
//other calls here to set things up
.build()
)
)
Nothing seems to authenticate correctly. I get back a 407 PROXY_AUTHENTICATION_REQUIRED with the companies default call the help desk if your browser doesn't work page in my stack trace.
Our systems are configured with an appropriate kbr5.config file so that should not be the issue & in other places in the code we are getting an SSO token a based on a ticket stored by kinit using the equivalent of RestTemplate().getForEntity("${sso-endpoint}", String::class.java).
I would ideally like to be able to delegate credentials to the OS via logged on user (either windows or REL based Linux systems) though I would consider simply authenticating at all a win at this point. I'm working in Kotlin but can easily translate a Java solution to Kotlin.
Any ideas appreciated?
I installed Apache httpcomponents-client-5.0.x and while reviewing the headers of the http response, I was shocked it doesn't show the Content-Length and Content-Encoding headers, this is the code I used for testing
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
import org.apache.hc.client5.http.impl.classic.CloseableHttpResponse;
import org.apache.hc.client5.http.impl.classic.HttpClients;
import com.sun.net.httpserver.Headers;
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet request = new HttpGet(new URI("https://www.example.com"));
CloseableHttpResponse response = httpclient.execute(request);
Header[] responseHeaders = response.getHeaders();
for(Header header: responseHeaders) {
System.out.println(header.getName());
}
// this prints all the headers except
// status code header
// Content-Length
// Content-Encoding
No matter what I try I get the same result, like this
Iterator<Header> headersItr = response.headerIterator();
while(headersItr.hasNext()) {
Header header = headersItr.next();
System.out.println(header.getName());
}
Or this
HttpEntity entity = response.getEntity();
System.out.println(entity.getContentEncoding()); // NULL
System.out.println(entity.getContentLength()); // -1
According to this question that has been asked 6 years ago, it seems like an old issue even with older versions of Apache HttpClient.
Of-course the server is actually returning those headers as confirmed by Wireshark, and Apache HttpClient logs itself
2020-04-03 07:59:09,106 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << HTTP/1.1 200 OK
2020-04-03 07:59:09,106 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Content-Encoding: gzip
2020-04-03 07:59:09,106 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Accept-Ranges: bytes
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Age: 451956
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Cache-Control: max-age=604800
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Content-Type: text/html; charset=UTF-8
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Date: Fri, 03 Apr 2020 05:59:09 GMT
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Etag: "3147526947+gzip"
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Expires: Fri, 10 Apr 2020 05:59:09 GMT
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Server: ECS (dcb/7EEB)
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Vary: Accept-Encoding
2020-04-03 07:59:09,109 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << X-Cache: HIT
2020-04-03 07:59:09,109 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Content-Length: 648
BTW, java.net.http library known as JDK HttpClient works great and show all the headers.
Is there something wrong I did, or should I report a bug that been there for years ?
HttpComponents committer here...
You did not closely pay attention what Dave G said. By default, HttpClientBuilder will enable transparent decompression and the reason why you don't see some headers anymore is here:
if (decoderFactory != null) {
response.setEntity(new DecompressingEntity(response.getEntity(), decoderFactory));
response.removeHeaders(HttpHeaders.CONTENT_LENGTH);
response.removeHeaders(HttpHeaders.CONTENT_ENCODING);
response.removeHeaders(HttpHeaders.CONTENT_MD5);
} ...
Regarding the JDK HttpClient, it will not perform any transparent decompression, therefore you see the length of the compressed stream. You have to decompress on your own.
curl committer here...
I have raised an issue too.
Update: 03 Feb. '23 The secret codez to disable automatic decompression are:
CloseableHttpClient httpclient = HttpClients.createSimple();
// OR
CloseableHttpClient httpclient = HttpClients.custom().disableContentCompression().build();
The content-length may be potentially ignored in this case.
HttpGet request = new HttpGet(new URI("https://www.example.com"));
request.setHeader("Accept-Encoding", "identity");
CloseableHttpResponse response = httpclient.execute(request);
I can see the following
HttpEntity entity = response.getEntity();
System.out.println(entity.getContentLength());
System.out.println(entity.getContentEncoding());
Output
...
2020-04-03 03:04:17.760 DEBUG 34196 --- [ main] org.apache.hc.client5.http.headers : http-outgoing-0 << Content-Length: 1256
...
1256
null
I'd like to direct your attention to this header being sent:
http-outgoing-0 >> Accept-Encoding: gzip, x-gzip, deflate
That tells the server that this client can accept gzip, x-gzip, and deflate content in response. The response is stating it is 'gzip' encoded.
http-outgoing-0 << Content-Encoding: gzip
I believe that HttpClient is transparently handling this internally and making the content available.
As stated in the other article you referenced, one of the answers indicated that the method EntityUtils.toByteArray(httpResponse.getEntity()).length could be applied to get the content length.
I'm using Apache HttpComponents (4.5.2) and I'm trying to request HTTPS page via proxy server (SSH Tunneling).
The problem is that according to logs Client sends first request without Proxy-Authorization header, but after Proxy respond with 407 error (Proxy Authentication Required), it retires authentication with sending Proxy-Authorization header.
I think problem in my code, I need something like enabling primitive auth , but I couldn't find any information about how to do that.
Below is logs for confirming my words.
First request:
03:12:06,643 DEBUG headers:135 - http-outgoing-0 >> CONNECT t.myhost.com:443 HTTP/1.1
03:12:06,643 DEBUG headers:138 - http-outgoing-0 >> Host: t.myhost.com
03:12:06,643 DEBUG headers:138 - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_45)
03:12:06,793 DEBUG headers:124 - http-outgoing-0 << HTTP/1.1 407 Proxy Authentication Required
03:12:06,794 DEBUG headers:127 - http-outgoing-0 << Proxy-Authenticate: Basic realm="ProxyCompany"
03:12:06,794 DEBUG headers:127 - http-outgoing-0 << Proxy-Connection: close
// then it retries request with included Proxy-Authorization header
03:12:06,795 DEBUG HttpAuthenticator:77 - Authentication required
03:12:06,795 DEBUG HttpAuthenticator:107 - 162.243.116.56:71223 requested authentication
03:12:06,795 DEBUG ProxyAuthenticationStrategy:174 - Authentication schemes in the order of preference: [Negotiate, Kerberos, NTLM, Digest, Basic]
03:12:06,795 DEBUG ProxyAuthenticationStrategy:203 - Challenge for Negotiate authentication scheme not available
03:12:06,796 DEBUG ProxyAuthenticationStrategy:203 - Challenge for Kerberos authentication scheme not available
03:12:06,796 DEBUG ProxyAuthenticationStrategy:203 - Challenge for NTLM authentication scheme not available
03:12:06,796 DEBUG ProxyAuthenticationStrategy:203 - Challenge for Digest authentication scheme not available
03:12:06,800 DEBUG HttpAuthenticator:157 - Selected authentication options: [BASIC [complete=true]]
03:12:06,800 DEBUG DefaultManagedHttpClientConnection:81 - http-outgoing-0: Close connection
03:12:06,801 DEBUG DefaultHttpClientConnectionOperator:138 - Connecting to /162.243.116.56:71223
03:12:06,942 DEBUG DefaultHttpClientConnectionOperator:145 - Connection established 192.168.0.100:13391<->162.243.116.56:71223
03:12:06,942 DEBUG HttpAuthenticator:198 - Generating response to an authentication challenge using basic scheme
03:12:06,947 DEBUG headers:135 - http-outgoing-0 >> CONNECT t.myhost.com:443 HTTP/1.1
03:12:06,947 DEBUG headers:138 - http-outgoing-0 >> Host: t.myhost.com
03:12:06,947 DEBUG headers:138 - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_45)
03:12:06,947 DEBUG headers:138 - http-outgoing-0 >> Proxy-Authorization: Basic bHVtXXXXXXXXXXXXxOTE5NTUXXXXXXRmNmRkYmI1Mjk0MA==
03:12:07,304 DEBUG HttpAuthenticator:86 - Authentication succeeded
03:12:07,305 DEBUG ProxyAuthenticationStrategy:227 - Caching 'basic' auth scheme for http://162.243.116.56:71223
And this is my code (it's Scala, but pretty easy to read):
val credProvider = {
val provider = new BasicCredentialsProvider()
provider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials("myUser", "myPass"))
provider
}
val connManager = {
val mngr = new PoolingHttpClientConnectionManager()
mngr.setDefaultMaxPerRoute(Integer.MAX_VALUE)
mngr.setMaxTotal(Integer.MAX_VALUE)
mngr
}
val client = HttpClients.custom()
.setConnectionManager(connManager)
.disableRedirectHandling()
.setDefaultCredentialsProvider(credProvider)
.setProxy(new HttpHost(162.243.116.56, 71223 ))
.build()
val requestConfig = RequestConfig.custom()
.setConnectTimeout(30000)
.setConnectionRequestTimeout(30000)
.build()
val request = new HttpGet(url)
request.setConfig(requestConfig)
val response = client.execute(request)
How I can solve this problem (cause client to always send Proxy-Authorization )?
I'm not sure if it is the same problem, but with version 4.5.2 there was a bug introduced with the SPN (HTTP/something#somerealm) and https : HTTPCLIENT-1712 (the comments are especially interesting since they show a history of what happened).
A switch to version 4.5.1 should solve it (if this is the same problem of course)
I have spent several days trying to figure this out, looking at all the info I could find on the Azure website, the Azure git hubs, and all the relevant stack overflow posts. I hope I am missing something simple.
I am using the example java code being posted around the web to obtain a token:
try {
exec = Executors.newFixedThreadPool(1);
context = new AuthenticationContext("https://login.microsoftonline.com/8e4f0713-5eea-4da0-99c0-xxxxxxxxxxxx",
true, exec);
ClientCredential cred = new ClientCredential(webClientID, clientSecret);
Future<AuthenticationResult> future = context.acquireToken("https://management.azure.com/", cred, null);
result = future.get();
} catch(Exception e) {
logger.warn("Exception " + e);
} finally {
exec.shutdown();
}
if (result == null) {
return null;
}
return result.getAccessToken();
This generates a token which I place into the request header:
Authorization: Bearer -token-
The GET https://management.azure.com/subscriptions/758ad253-cbf5-4b18-8863-xxxxxxxxxxxx/providers/Microsoft.Commerce/RateCard?api-version=2015-06-01-preview%26%24filter%3DOfferDurableId+eq+%27MS-AZR-0003p%27+and+Currency+eq+%27USD%27+and+Locale+eq+%27en-US%27+and+RegionInfo+eq+%27US%27
Returns 403 code:
Exception: java.io.IOException: Server returned HTTP response code: 403 for URL: https://management.azure.com/subscriptions/758ad253-cbf5-4b18-8863-xxxxxxxxxxxx/providers/Microsoft.Commerce/RateCard?api-version=2015-06-01-preview%26%24filter%3DOfferDurableId+eq+%27MS-AZR-0003p%27+and+Currency+eq+%27USD%27+and+Locale+eq+%27en-US%27+and+RegionInfo+eq+%27US%27
Headers:
[0] null: HTTP/1.1 403 Forbidden
[1] Cache-Control: no-cache
[2] Pragma: no-cache
[3] Content-Type: application/json; charset=utf-8
[4] Expires: -1
[5] x-ms-failure-cause: gateway
[6] x-ms-request-id: e4ad9253-e034-481d-aba0-f46902b7057f
[7] x-ms-correlation-request-id: e4ad9253-e034-481d-aba0-f46902b7057f
[8] x-ms-routing-request-id: EASTUS:20151103T205103Z:e4ad9253-e034-481d-aba0-f46902b7057f
[9] Strict-Transport-Security: max-age=31536000; includeSubDomains
[10] Date: Tue, 03 Nov 2015 20:51:02 GMT
[11] Connection: close
[12] Content-Length: 303
I did all the setup on the Azure mgmt console, to create the App in Azure AD, get the clientID & client secret, etc. The SSL/HTTPS code is:
azureURL = new java.net.URL(url);
con = (HttpsURLConnection)azureURL.openConnection();
con.disconnect();
con.setDoOutput(true);
con.setDoInput(true);
con.setUseCaches(false);
con.setSSLSocketFactory(MyUtils.getSSLSocketFactory());
con.setRequestMethod("GET");
con.setRequestProperty("x-ms-version", "2015-06-01-preview");
con.setRequestProperty("Content-Type", "application/json");
String token = getAccessTokenFromServicePrincipalCredentials();
if (token != null) {
con.setRequestProperty("Authorization", "Bearer " + token);
con.connect();
in = (InputStream)con.getContent();
InputStreamReader inr = new InputStreamReader(in);
} else {
logger.warn("unable to obtain prices");
}
Any suggestions on how to debug the problem?
I see that you're getting an access token for the Service Principal (i.e. the application itself). Please make sure that you grant this Service Principal user at least Reader role on your Azure Subscription. I believe you're getting this error because this user doesn't have any access to your Azure Subscription.
Please see this link: https://azure.microsoft.com/en-in/documentation/articles/role-based-access-control-configure/ (Manage access using the Azure Management Portal section) on how you can assign role.
As we can’t call Usage & Rate Card APIs via certificate with HTTPS requests. As it is mentioned:
All of the tasks that you do on resources using the Azure Resource Manager must be authenticated with Azure Active Directory
on Authenticating Azure Resource Manager requests.
So you got a 403 issue.
Please try to build common HTTP request for the REST APIs with the request header Content-Type and Authorization which is mentioned on Resource Usage (Preview).
Also you can test to get the info you want in HTTP request build tool, like:
And here is a similar thread How to use Management certificate based authentication for making REST API calls to Azure? for your reference.
For the file not found the reply is:
java.io.FileNotFoundException:
https://management.azure.com/subscriptions/758ad253-cbf5-4b18-8863-3eed082xxxxx/providers/Microsoft.Commerce/RateCard?api-version=2015-06-01-preview%26%24filter%3DOfferDurableId+eq+%27MS-AZR-0003p%27+and+Currency+eq+%27USD%27+and+Locale+eq+%27en-US%27+and+RegionInfo+eq+%27US%27
HTTP/1.1 404 Not Found
[1] Cache-Control: no-cache
[2] Pragma: no-cache
[3] Content-Type: application/json; charset=utf-8
[4] Expires: -1
[5] x-ms-failure-cause: gateway
[6] x-ms-request-id: 8bd5ea3a-5a5f-4eb5-86b5-bd6581f94e00
[7] x-ms-correlation-request-id: 8bd5ea3a-5a5f-4eb5-86b5-bd6581f94e00
[8] x-ms-routing-request-id: EASTUS:20151119T181954Z:8bd5ea3a-5a5f-4eb5-86b5-bd6581f94e00
[9] Strict-Transport-Security: max-age=31536000; includeSubDomains
[10] Date: Thu, 19 Nov 2015 18:19:53 GMT
[11] Content-Length: 348
Let me see what else I can get...
I am trying to get contents of the page: http://www.washingtonpost.com/wp-srv/simulation/simulation_test.json
I am able to get the contents in my java project. Here is the code:
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
HttpResponse response = client.execute(request);
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
This prints the contents in my Java project.
However, when I execute this code in my Android project, I get the following in return :
<HTML><HEAD><TITLE>Cisco Systems Inc. Web Authentication Redirect</TITLE><META http-equiv="Cache-control" content="no-cache"><META http-equiv="Pragma" content="no-cache"><META http-equiv="Expires" content="-1"><META http-equiv="refresh" content="1; URL=https://1.1.1.1/login.html?redirect=www.washingtonpost.com/wp-srv/simulation/simulation_test.json"></HEAD></HTML>
How can i solve this problem? why does it occur only in my Android project and not in my Java project?
Thanks
ref for android client
assume not useing proxy when on android connected to WIFI.
Assume that on android you are NOT using net interface for 3G/4G
Assume that you are using the android httpclient packages and not the apache httpclient.
assume that manifest has granted Web permissions.
You have to know the above in order to do much with debug.
You will get a bunch of default headers from default packages when you invoke these:
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
You will need to either turn on logging for WIRE & HEADERS OR you will need to dump your headers for the get request so that you know exactly whats in there.
Then , compare your android generated http GET ( all headers ) to the curl test script you see below.. If your android headers are reasonably close ( dont need to be exact there can be some extras in there like 'gzip' encoding ) to the curl headers , then the android will work. The curl example shows only 3 headers and i would make a wild guess that your android client , by default, is setting 7 or 8 headers and that the WP server does not like something in the cruft of the extra headers and is redirecting to the logon.
# ~$ curl --verbose http://www.washingtonpost.com/wp-srv/simulation/simulation_test.json
* About to connect() to www.washingtonpost.com port 80 (#0)
* Trying 23.204.109.48...
* connected
* Connected to www.washingtonpost.com (23.204.109.48) port 80 (#0)
> GET /wp-srv/simulation/simulation_test.json HTTP/1.1
> User-Agent: curl/7.28.1-DEV
> Host: www.washingtonpost.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: webserver
< Content-Type: text/plain
< Last-Modified: Wed, 12 Feb 2014 21:16:22 GMT
< Content-Length: 58187
< ETag: "e34b-52fbe4a6"
< Accept-Ranges: bytes
< Date: Mon, 10 Mar 2014 16:54:42 GMT
< Connection: keep-alive
<
{
"posts": [
android httpclient 4.3 sample with HEADER logs turned on...
D/ch.boye.httpclientandroidlib.headers(15360): http-outgoing-2 >> PUT /1/classes/MediaItem/XPdLazBUzV HTTP/1.1
D/ch.boye.httpclientandroidlib.headers(15360): http-outgoing-2 >> Content-Length: 90
D/ch.boye.httpclientandroidlib.headers(15360): http-outgoing-2 >> Content-Type: application/json; charset=UTF-8
D/ch.boye.httpclientandroidlib.headers(15360): http-outgoing-2 >> Host: api.parse.com
D/ch.boye.httpclientandroidlib.headers(15360): http-outgoing-2 >> Connection: Keep-Alive
D/ch.boye.httpclientandroidlib.headers(15360): http-outgoing-2 >> User-Agent: Apache-HttpClient/UNAVAILABLE (java 1.5)
D/ch.boye.httpclientandroidlib.headers(15360): http-outgoing-2 >> Accept-Encoding: gzip,deflate
D/ch.boye.httpclientandroidlib.headers(15360): http-outgoing-2 >> X-Parse-Session-Token: li9ds
D/ch.boye.httpclientandroidlib.headers(15360): http-outgoing-2 >> X-Parse-Application-Id: 3qovv
D/ch.boye.httpclientandroidlib.headers(15360): http-outgoing-2 >> X-Parse-REST-API-Key: Theoj
D/