HttpSolrServer with proxy - java

Using the HttpSolrServer to connect to a Solr instance. Trying to run this through a proxy but currently the configuration doesn't look to be getting applied, it continues using the baseUrl. This is the code:
DefaultHttpClient httpClient = new DefaultHttpClient();
ModifiableSolrParams params = new ModifiableSolrParams();
params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 128);
params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 32);
params.set(HttpClientUtil.PROP_FOLLOW_REDIRECTS, false);
params.set(HttpClientUtil.PROP_CONNECTION_TIMEOUT, this.connectionTimeout);
httpClient.getCredentialsProvider().setCredentials(
AuthScope.ANY,
new UsernamePasswordCredentials(proxyUsername, proxyPassword)
);
HttpHost proxy = new HttpHost(proxyUrl, proxyPort, proxyProtocol);
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
HttpSolrServer solrServer = new HttpSolrServer(baseUrl, httpClient);
Have also tried:
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(this.connectionTimeout)
.setConnectionRequestTimeout(this.connectionTimeout)
.setSocketTimeout(this.connectionTimeout)
.build();
HttpClientBuilder httpClientBuilder = HttpClients.custom()
.setDefaultRequestConfig(requestConfig)
.setMaxConnTotal(128)
.setMaxConnPerRoute(32)
.disableRedirectHandling();
HttpHost proxy = new HttpHost(proxyUrl, proxyPort, proxyProtocol);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(
new AuthScope(proxyUrl, proxyPort, AuthScope.ANY_REALM, "basic"),
new UsernamePasswordCredentials(proxyUsername, proxyPassword));
httpClientBuilder
.setDefaultCredentialsProvider(credentialsProvider)
.setTargetAuthenticationStrategy(new ProxyAuthenticationStrategy())
.setRoutePlanner(routePlanner);
CloseableHttpClient httpClient = httpClientBuilder.build();
HttpSolrServer solrServer = new HttpSolrServer(baseUrl, httpClient);
Versions
SolrJ 4.7.2
HttpClient 4.3.1

Related

Deprecated HttpClient, now using apache-httpclient-4.3.x

I have code that works fine but its now deprecated. Everything is depricated except BasicCredentialsProvider
SSLSocketFactory sslsf = new SSLSocketFactory("TLS", null, null, keyStore, null, new TrustSelfSignedStrategy(),
new AllowAllHostnameVerifier());
Scheme https = new Scheme("https", 28181, sslsf);
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
schemeRegistry.register(https);
PoolingClientConnectionManager connectionManager = new PoolingClientConnectionManager(schemeRegistry);
connectionManager.setMaxTotal(100);
connectionManager.setDefaultMaxPerRoute(5);
DefaultHttpClient httpClient = new DefaultHttpClient(connectionManager);
BasicCredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(new AuthScope(uri.getHost(), uri.getPort(), name),
new UsernamePasswordCredentials(username, password));
httpClient.setCredentialsProvider(credsProvider);
httpClient.getParams().setIntParameter(CoreConnectionPNames.SO_TIMEOUT, 60000);
return new HttpContextRequestScopedApacheClientExecutor(httpClient);
I have tried to do it myself. First I replaced SSLSocketFactory with
SSLConnectionSocketFactory sslConnectionFactory = new SSLConnectionSocketFactory
(SSLContexts.custom().
loadTrustMaterial(keyStore, new TrustSelfSignedStrategy()).
useTLS().build(), SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
then I tried to use
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("https", sslConnectionFactory)
.build();
I couldn't fit my port there, so I after searching for quite some time I am still lost on how to do it. And for the rest I have no solution yet. Any help on this would be apriciated.
I think I have managed to do it, it's working so far.
HttpClientBuilder builder = HttpClientBuilder.create();
KeyStore keyStore = initSSL();
SSLConnectionSocketFactory sslConnectionFactory = new SSLConnectionSocketFactory(SSLContexts.custom().loadTrustMaterial(keyStore, new
TrustSelfSignedStrategy()).useTLS().build(), SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", sslConnectionFactory)
.build();
PoolingHttpClientConnectionManager ccm = new PoolingHttpClientConnectionManager(registry);
ccm.setMaxTotal(100);
ccm.setDefaultMaxPerRoute(5);
builder.setConnectionManager(ccm);
BasicCredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(new AuthScope(uri.getHost(), uri.getPort(), name),
new UsernamePasswordCredentials(username, password));
builder.setDefaultCredentialsProvider(credsProvider);
RequestConfig.Builder requestBuilder = RequestConfig.custom();
requestBuilder.setSocketTimeout(60000);
builder.setDefaultRequestConfig(requestBuilder.build());
return new HttpContextRequestScopedApacheClientExecutor(builder.build());

i'm trying to send a https request through a proxy with apache httpclient,but i can't find the headers on the proxy side

i'm trying to send a https request through a proxy with apache httpclient,but i can't find the headers on the proxy side
HttpClient httpClient =new DefaultHttpClient();
HttpHost proxy = new HttpHost("10.1.1.100", 8080);
httpClient.getParams().setParameter(ConnRouteParams.DEFAULT_PROXY,proxy);
HttpGet get = new HttpGet(uri);
get.addHeader("Proxy-Authorization", "222222");
HttpResponse hr = defaultHttpClient.execute(get);
the proxy side only find proxy-connection and user-agent:
Proxy-Connection:[Keep-Alive] User-Agent:[Apache-HttpClient/4.3.6 (java 1.5)]
First, that's not how you authenticate to a proxy. Second, those headers are added to the get request (not to the proxy). Finally, this is based on an example the HttpClient examples - specifically ClientProxyAuthentication and updated to use try-with-resources (and modified to use an URL)
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(new AuthScope("10.1.1.100", 8080),
new UsernamePasswordCredentials("username", "password"));
try (CloseableHttpClient httpclient = HttpClients.custom()
.setDefaultCredentialsProvider(credsProvider).build()) {
URL url = new URL(uri);
HttpHost target = new HttpHost(url.getHost(), url.getPort(),
url.getProtocol());
HttpHost proxy = new HttpHost("10.1.1.100", 8080);
RequestConfig config = RequestConfig.custom().setProxy(proxy)
.build();
HttpGet httpget = new HttpGet(url.getPath());
httpget.setConfig(config);
System.out.println("Executing request " + httpget.getRequestLine()
+ " to " + target + " via " + proxy);
try (CloseableHttpResponse response = httpclient.execute(target,
httpget)) {
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
EntityUtils.consume(response.getEntity());
}
} catch (IOException e1) {
e1.printStackTrace();
}

Apache HttpClient 4.3.5 set proxy

It seems that I can specify the proxy when I construct new HttpClient with:
HttpHost proxy = new HttpHost("someproxy", 8080);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();
taken from http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d5e475
Is it possible to modify existing client's proxy settings.
You can create your own implementation of HttpRoutePlanner that will allow change of the HttpHost.
public class DynamicProxyRoutePlanner implements HttpRoutePlanner {
private DefaultProxyRoutePlanner defaultProxyRoutePlanner = null;
public DynamicProxyRoutePlanner(HttpHost host){
defaultProxyRoutePlanner = new DefaultProxyRoutePlanner(host);
}
public void setProxy(HttpHost host){
defaultProxyRoutePlanner = new DefaultProxyRoutePlanner(host);
}
public HttpRoute determineRoute(HttpHost target, HttpRequest request, HttpContext context) {
return defaultProxyRoutePlanner.determineRoute(target,request,context);
}
}
Then you can use this DynamicProxyRoutePlanner in your code
HttpHost proxy = new HttpHost("someproxy", 8080);
DynamicProxyRoutePlanner routePlanner = new DynamicProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();
//Any time change the proxy
routePlanner.setProxy(new HttpHost("someNewProxy", 9090));

http client 4.3 not sending credentials

I am trying to send a get request using apache http client 4.3 (to a client using self sign cert), however I get back the error "Requires Authentication" everytime. In a web browser it works just fine so the username / password / url is correct. Is this not the way to pass username/password using http client 4.3?
public static String sendJsonHttpGetRequest(
String host,
String path,
String username,
String password,
int socketTimeout,
int connectionTimeout,
int connectionRequestTimeout
) throws Exception
{
String responseBody = null;
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
SSLContextBuilder builder = new SSLContextBuilder();
builder.loadTrustMaterial(null, new TrustStrategy(){
#Override
public boolean isTrusted(java.security.cert.X509Certificate[] chain, String authType)
throws java.security.cert.CertificateException
{
return true;
}
});
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(builder.build());
CloseableHttpClient httpclient = HttpClients.custom().setSSLSocketFactory(sslsf).setDefaultCredentialsProvider(credsProvider).build();
URIBuilder uriB = new URIBuilder().setScheme("https").setHost(host).setPath(path);
HttpGet _http = new HttpGet( uriB.build() );
RequestConfig _requestConfig = RequestConfig.custom().
setSocketTimeout(socketTimeout).
setConnectTimeout(connectionTimeout).
setConnectionRequestTimeout(connectionRequestTimeout).build();
_http.addHeader("Content-Type", "application/json");
_http.addHeader("Accept","application/json, text/xml;q=9, /;q=8");
_http.setConfig(_requestConfig);
// ###########################
ResponseHandler<String> response = new BasicResponseHandler();
responseBody = httpclient.execute(_http, response);
return responseBody;
}
turns out now with http 4+ you have to provide it in two locations for it to work,
second is
authCache.put(host, basicAuth);
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
HttpClientContext _context = HttpClientContext.create();
_context.setAuthCache(authCache);
_context.setCredentialsProvider(credentialsProvider);
responseBody = httpclient.execute(_http, response, _context);
I don't use this library myself, but have you tried the HttpClient class?
HttpClient client = new HttpClient();
client.getParams().setAuthenticationPreemptive(true);
client.getState().setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
GetMethod method = new GetMethod(uri);
client.executeMethod(method);
You still have to build the uri and set timeouts, but it could be an option.

Apache HttpClient 4.1 - Proxy Settings

I am trying to POST some parameters to a server, but I need to set up the proxy. can you help me to to sort it "setting the proxy" part of my code ?
HttpHost proxy = new HttpHost("xx.x.x.xx");
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter("3128",proxy);
HttpPost httpost = new HttpPost(url);
List<NameValuePair> nvps = new ArrayList<NameValuePair>();
nvps.add(new BasicNameValuePair("aranan", song));
httpost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));
HttpResponse response = httpclient.execute(httpost);
HttpEntity entity = response.getEntity();
System.out.println("Request Handled?: " + response.getStatusLine());
in = entity.getContent();
httpclient.getConnectionManager().shutdown();
Yes I sorted out my own problem,this line
httpclient.getParams().setParameter("3128",proxy);
should be
httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY,proxy);
Complete Example of a Apache HttpClient 4.1, setting proxy can be found below
HttpHost proxy = new HttpHost("ip address",port number);
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY,proxy);
HttpPost httpost = new HttpPost(url);
List<NameValuePair> nvps = new ArrayList<NameValuePair>();
nvps.add(new BasicNameValuePair("param name", param));
httpost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.ISO_8859_1));
HttpResponse response = httpclient.execute(httpost);
HttpEntity entity = response.getEntity();
System.out.println("Request Handled?: " + response.getStatusLine());
InputStream in = entity.getContent();
httpclient.getConnectionManager().shutdown();
Non deprecated way of doing it (also in 4.5.5 version) is:
HttpHost proxy = new HttpHost("proxy.com", 80, "http");
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();
This is quick way I use to set the proxy:
import org.apache.http.HttpHost;
import org.apache.http.client.HttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
...
HttpHost proxy = new HttpHost("www.proxy.com", 8080, "http");
HttpClient httpClient = HttpClientBuilder.create().setProxy(proxy).build();
When I use apache httpclient v4.5.5,I found HttpClient.getParams() is deprecated in v4.3,we should use org.apache.http.client.config.RequestConfig instead.
Code sample
shows that:
HttpHost target = new HttpHost("httpbin.org", 443, "https");
HttpHost proxy = new HttpHost("127.0.0.1", 8080, "http");
RequestConfig config = RequestConfig.custom()
.setProxy(proxy)
.build();
HttpGet request = new HttpGet("/");
request.setConfig(config);
CloseableHttpResponse response = httpclient.execute(target, request);

Categories

Resources