I have an app that makes http requests to a remote server. I do this with the following code:
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("myURL");
try {
ArrayList<BasicNameValuePair> postVariables = new ArrayList<BasicNameValuePair>(2);
postVariables.add(new BasicNameValuePair("key","value"));
httpPost.setEntity(new UrlEncodedFormEntity(postVariables));
HttpResponse response = httpClient.execute(httpPost);
String responseString = EntityUtils.toString(response.getEntity());
if (responseString.contains("\"success\":true")){
//this means the request succeeded
} else {
//failed
}
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
This goes really well, but one of our customers has set up an APN that requires requests to go via a certain proxy server. If I add the following to the request this works, the request gets rerouted via the proxy to the server:
HttpHost httpHost = new HttpHost("proxyURL",8080);
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, httpHost);
So far so good, however, I use a library that makes some http requests as well. The library's code is not accesible for me, so I can't add those two lines to the code. I contacted the creators of that library, and they told me it should be possible to set up the android environment so that all requests will automatically go through the proxy. Is there something like that? I didn't find anything on google.
I'm basically looking for a way to set the above two lines as a standard for all http requests. Please note that the APN does not set the proxy as a default for the entire phone, so apps will have to do this manually (and yes that means the majority of the apps don't work on that customer's phone).
It's been a year or two since I've needed to use it, but if I remember correctly, you can use the System.setProperty(String, String) in order to set an environment-wide setting for your application to route all HTTP traffic through a proxy. The properties that you should need to set are "http.proxyHost" and "http.proxyPort" and then use your HttpClient normally without specifying a proxy because the VM will handle routing requests.
Docs for more information about what I'm talking about can be found here: ProxySelector (just so you know what keys to use) and here for documentation about the actual System.setProperty(String, String) function
If that doesn't work for you, let me know and I'll try to dig out my old code that set a system-level proxy. BTW, it's really only "system-level" since each app runs in it's own Dalvik so you won't impact other app's network communications.
Related
I'm using Java EE technology for my web application. And I use Apache Solr for my search engine. AFAIK, after I config Solr successfully, Solr is running as a Restful service.
So, under my servlet, I try to call this service as another console application I do before. Here is a sample code is used to get json data from an url:
/** Get Data Fom URL Using GET Method */
public static String getResponseFromGetRequest(String url) {
HttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(url);
/** after prepare for data. prepare for sending */
try {
/**
* HttpResponse is an interface just like HttpGet
* therefore we can't initialize them
*/
HttpResponse httpResponse = httpClient.execute(httpGet);
return parseHttpResponse(httpResponse);
} catch (ClientProtocolException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
return null;
}
But the package org.apache.http.impl.client.DefaultHttpClient is not available (under tomcat server at least). So, I think maybe there are some problems here. That we cannot call another service in servlet, right? If my guess is true, how can I fix this problem?
A brief code on using an API available in java to interact with solr(you don't need to know or care about servlets, httpclients, nothing).
HttpSolrServer server = new HttpSolrServer("your restful url goes here");
SolrQuery query = new SolrQuery();
QueryResponse response = server.query(query);
List<MyClass> beans = response.getBeans(MyClass.class);
server should probably be a spring bean, it's the java representation (proxy object) of your running solr server.
query is a query to run against the server, configure this to specify what you want from solr.
response is the answer to the request.
beans are objects which are straight from solr itself (as far as i remember, your solr fields get mapped to the java object's fields by field name, no restrictions at all).
note that server has a lot of ways to do simple operations without anything explicit, like server.addBean(myBean);
Here is how to get Solrj in maven:
<dependency>
<groupId>org.apache.solr</groupId>
<artifactId>solr-solrj</artifactId>
<version>4.5.1</version>
</dependency>
If you don't have maven:
http://mvnrepository.com/artifact/org.apache.solr/solr-solrj/4.10.3
click on "Download ( JAR )".
When I send a request using a proxy client, if I get a certain response, I would like to be able to modify the request and then send the same request again for all requests.
Normally I would do something like:
BookStore proxy = JAXRSClientFactory.create("http://books", BookStore.class);
try
{
proxy.getBook("someId");
}
catch(WebApplicationException ex)
{
Response r = ex.getResponse();
if (r.getStatusCode() == 404)
{
proxy.getBook("anotherId");
}
}
But in this case, there is a common thing I want to do for all requests: If I get a specific http code, modify some header values, and then try again (probably with a limit on the amount of retries).
I haven't seen a way that cxf proxy clients explicitly support this, how could I go about implementing it?
You need to write an interceptor to do this for every request.
here you go for sample code and documentation http://cxf.apache.org/docs/jax-rs-filters.html
I have been using HTTPClient version 4.1.2 to try to access a REST over HTTP API that requires Basic Authentication. Here is client code:
DefaultHttpClient httpClient = new DefaultHttpClient(new ThreadSafeClientConnManager());
// Enable HTTP Basic Auth
httpClient.getCredentialsProvider().setCredentials(
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
new UsernamePasswordCredentials(this.username, this.password));
HttpHost proxy = new HttpHost(this.proxyURI.getHost(), this.proxyURI.getPort());
httpClient.getParams().setParameter(ConnRouteParams.DEFAULT_PROXY, proxy);
When I construct a POST request, like this:
HttpPost request = new HttpPost("http://my/url");
request.addHeader(new BasicHeader("Content-type", "application/atom+xml; type=entry")); // required by vendor
request.setEntity(new StringEntity("My content"));
HttpResponse response = client.execute(request);
I see in Charles Proxy that there are two requests being sent. One without the Authorization: Basic ... header and one with it. The first one fails with a 401, as you would expect, but the second goes through just fine with a 201.
Does anyone know why this happens? Thanks!
EDIT:
I should make clear that I have already looked at this question, but as you can see I set the AuthScope the same way and it didn't solve my problem. Also, I am creating a new HttpClient every time I made a request (though I use the same ConnectionManager), but even if I use the same HttpClient for multiple requests, the problem still persists.
EDIT 2:
So it looks like what #LastCoder was suggesting is the way to do. See this answer to another question. The problem stems from my lack of knowledge around the HTTP spec. What I'm looking to do is called "preemptive authentication" and the HttpClient docs mention it here. Thankfully, the answer linked to above is a much shorter and cleaner way to do it.
Rather than using .setCredentials() why don't you just encode USERNAME:PASSWORD and add the authentication header with .addHeader()
This means that your server/target endpoint is creating a new session for every client request. This forces every request of yours to go through a hand-shake, which means the clients first makes the call and realizes that it needs authorization, then it follows with the authorization. What you need to do is send the authorization preemptively as follows:
httpClient.getParams().setAuthenticationPreemptive(true);
Just to understand the process you may log your client request headers, to give you an idea of what your client is sending and receiving:
See if this works.
I'm building a simple web-scraper and i need to fetch the same page a few hundred times, and there's an attribute in the page that is dynamic and should change at each request. I've built a multithreaded HttpClient based class to process the requests and i'm using an ExecutorService to make a thread pool and run the threads. The problem is that dynamic attribute sometimes doesn't change on each request and i end up getting the same value on like 3 or 4 subsequent threads. I've read alot about HttpClient and i really can't find where this problem comes from. Could it be something about caching, or something like it!?
Update: here is the code executed in each thread:
HttpContext localContext = new BasicHttpContext();
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params,
HTTP.DEFAULT_CONTENT_CHARSET);
HttpProtocolParams.setUseExpectContinue(params, true);
ClientConnectionManager connman = new ThreadSafeClientConnManager();
DefaultHttpClient httpclient = new DefaultHttpClient(connman, params);
HttpHost proxy = new HttpHost(inc_proxy, Integer.valueOf(inc_port));
httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY,
proxy);
HttpGet httpGet = new HttpGet(url);
httpGet.setHeader("User-Agent",
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
String iden = null;
int timeoutConnection = 10000;
HttpConnectionParams.setConnectionTimeout(httpGet.getParams(),
timeoutConnection);
try {
HttpResponse response = httpclient.execute(httpGet, localContext);
HttpEntity entity = response.getEntity();
if (entity != null) {
InputStream instream = entity.getContent();
String result = convertStreamToString(instream);
// System.out.printf("Resultado\n %s",result +"\n");
instream.close();
iden = StringUtils
.substringBetween(result,
"<input name=\"iden\" value=\"",
"\" type=\"hidden\"/>");
System.out.printf("IDEN:%s\n", iden);
EntityUtils.consume(entity);
}
}
catch (ClientProtocolException e) {
// TODO Auto-generated catch block
System.out.println("Excepção CP");
} catch (IOException e) {
// TODO Auto-generated catch block
System.out.println("Excepção IO");
}
HTTPClient does not use cache by default (when you use DefaultHttpClient class only). It does so, if you use CachingHttpClient which is HttpClient interface decorator enabling caching:
HttpClient client = new CachingHttpClient(new DefaultHttpClient(), cacheConfiguration);
Then, it analyzes If-Modified-Since and If-None-Match headers in order to decide if request to the remote server is performed, or if its result is returned from cache.
I suspect, that your issue is caused by proxy server standing between your application and remote server.
You can test it easily with curl application; execute some number of requests omitting proxy:
#!/bin/bash
for i in {1..50}
do
echo "*** Performing request number $i"
curl -D - http://yourserveraddress.com -o $i -s
done
And then, execute diff between all downloaded files. All of them should have differences you mentioned. Then, add -x/--proxy <host[:port]> option to curl, execute this script and compare files again. If some responses are the same as others, then you can be sure that this is proxy server issue.
Generally speaking, in order to test whether or not HTTP requests are being made over the wire, you can use a "sniffing" tool that analyzes network traffic, for example:
Fiddler ( http://fiddler2.com/fiddler2/ ) - I would start with this
Wireshark ( http://www.wireshark.org/ ) - more low level
I highly doubt HttpClient is performing caching of any sort (this would imply it needs to store pages in memory or on disk - not one of its capabilities).
While this is not an answer, its a point to ponder: Is it possible that the server (or some proxy in between) is returning you cached content? If you are performing many requests (simultaneously or near simultaneously) for the same content, the server may be returning you cached content because it has decided that the information has not "expired" yet. In fact the HTTP protocol provides caching directives for such functionality. Here is a site that provides a high level overview of the different HTTP caching mechanisms:
http://betterexplained.com/articles/how-to-optimize-your-site-with-http-caching/
I hope this gives you a starting point. If you have already considered these avenues then that's great.
You could try appending some unique dummy parameter to the URL on every request to try to defeat any URL-based caching (in the server, or somewhere along the way). It won't work if caching isn't the problem, or if the server is smart enough to reject requests with unknown parameters, or if the server is caching but only based on parameters it cares about, or if your chosen parameter name collides with a parameter the site actually uses.
If this is the URL you're using
http://www.example.org/index.html
try using
http://www.example.org/index.html?dummy=1
Set dummy to a different value for each request.
I'm working on building an android application which requires different levels of authentication, and I would like to do so using Active Directory.
From what I've read, using Kerberos is the way Microsoft suggests. How do I do this for Android? I see the javax.security.auth doc, but it doesn't tell me too much.
I also saw a note somewhere that Kerberos does not contain user groups - is this true? In that case, would I have to somehow combine LDAP as well?
EDIT
The main goal here is achieving an LDAP connection to the active directory in order to authenticate and give the user correct permissions for the enterprise Android application. The real barrier here is the fact that Google left out many of the Java Web Services API from it's port to android. (i.e. javax.naming) Also, many of the connection mechanisms in the Android jar seem to be only included as legacy code, and they in fact actually do nothing.
For that you might be better off just staying completely within LDAP and don't venture into the kerberos. Kerberos gives you advantage of Single Sign On, but since your android app doesn't have any credentials already in place it doesn't really help you. I guess google had their own reasons not to include the javax.naming into the distro. It is pretty heavy stuff.
You might be able to either port the stuff yourself from java runtime library sources, or might be better off using native LDAP library. For example this one.
Just remember to use secure LDAP connection or at least secure authentication method. More info about this is here.
I found the documentation here to be really useful when I was writing my code to authenticate with my Kerberos server. Here's how I authenticate with my kerberos server, but you might need to tweak it for yours (hence me including the link):
public static final int REGISTRATION_TIMEOUT = 30 * 1000; // ms
private static DefaultHttpClient httpClient;
private static final AuthScope SERVER_AUTH_SCOPE =
new AuthScope("urls to kerberos server", AuthScope.ANY_PORT);
public static DefaultHttpClient getHttpClient(){
if(httpClient == null){
httpClient = new DefaultHttpClient();
final HttpParams params = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(params, REGISTRATION_TIMEOUT);
HttpConnectionParams.setSoTimeout(params, REGISTRATION_TIMEOUT);
ConnManagerParams.setTimeout(params, REGISTRATION_TIMEOUT);
}
return httpClient;
}
public static boolean authenticate(String username, String password)
{
UsernamePasswordCredentials creds =
new UsernamePasswordCredentials(username, password);
DefaultHttpClient client = getHttpClient();
client.getCredentialsProvider().setCredentials(SERVER_AUTH_SCOPE, creds);
boolean authWorked = false;
try{
HttpGet get = new HttpGet(AUTH_URI);
HttpResponse resp = client.execute(get);
authWorked = resp.getStatusLine().getStatusCode() != 403
}
catch(IOException e){
Log.e("TAG", "IOException exceptions");
//TODO maybe do something?
}
return authWorked;
}
Have you looked at using JCIFS? Based on these questions [1] [2] and this site, JCIFS works under Android. The JCIFS site has a simple NTLM Authenticator example that could help get you started. However, based on this Samba list message, you will need to use LDAP and custom code to get the user's groups.
Try this tutorial from Oracle. My code likes a charm. Hopefully everything is included in Android's VM distro.