Develop Reference

Migration from httpclient4 to httpclient5

in our project we switched from apache httpclient 4 to httpclient 5 now we have a ssl problem in one module. The code in httpclient 4 was
private void buildHttpClient() throws MalformedURLException {
try {
URL aURL = new URL(BASE_URL);
String host = aURL.getAuthority();
SSLSocketFactory socketFactory = (SSLSocketFactory) SSLSocketFactory.getDefault();
HostnameVerifier defaultHostnameVerifier = SSLConnectionSocketFactory.getDefaultHostnameVerifier();
SSLConnectionSocketFactory systemSocketFactory = new SSLConnectionSocketFactory(socketFactory, defaultHostnameVerifier);
CredentialsProvider provider = new BasicCredentialsProvider();
provider.setCredentials(
new AuthScope(new HttpHost(host)),
new UsernamePasswordCredentials(USER, PASSWORD));
httpclient = HttpClients.custom()
.setDefaultCredentialsProvider(provider)
.setSSLSocketFactory(systemSocketFactory)
.build();
} catch (MalformedURLException e) {
throw new MalformedURLException(BASE_URL);
}
}
the new code is
private void buildClient() throws MalformedURLException {
URL aURL = new URL(BASE_URL);
String host = aURL.getAuthority();
final BasicCredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope(host, 443),
new UsernamePasswordCredentials(USER, PASSWORD.toCharArray()));
SSLContext ctx = SSLContexts.createDefault();
final SSLConnectionSocketFactory sslSocketFactory = SSLConnectionSocketFactoryBuilder.create()
.setSslContext(ctx)
.build();
final HttpClientConnectionManager cm = PoolingHttpClientConnectionManagerBuilder.create()
.setSSLSocketFactory(sslSocketFactory)
.build();
httpclient = HttpClients.custom()
.setConnectionManager(cm)
.setDefaultCredentialsProvider(credsProvider)
.build();
}
httpclient is CloseableHttpClient in both cases. Testin locally never got a problem but testing on the customer server shows now PKIX path building failed: com.ibm.security.cert.IBMCertPathBuilderException: unable to find valid certification path to requested target
When i roll back to an old deployment it works on the server but the new one doesn't work. The keystore deployed by the customer should be correct and I don't want to use an own keystore.
Can somebody describe where I should look, or what is the problem with the ssl?

You are getting a certification error. You can add following static block in order to close SSL verification.
static {
// this part is needed cause Lebocoin has invalid SSL certificate, that
// cannot be normally processed by Java
TrustManager[] trustAllCertificates = new TrustManager[] { new X509TrustManager() {
#Override
public X509Certificate[] getAcceptedIssuers() {
return null; // Not relevant.
}
#Override
public void checkClientTrusted(X509Certificate[] certs, String authType) {
// Do nothing. Just allow them all.
}
#Override
public void checkServerTrusted(X509Certificate[] certs, String authType) {
// Do nothing. Just allow them all.
}
} };
HostnameVerifier trustAllHostnames = (String hostname, SSLSession session) -> true // Just
// allow
// them
// all.
;
try {
System.setProperty("jsse.enableSNIExtension", "false");
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCertificates, new SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
HttpsURLConnection.setDefaultHostnameVerifier(trustAllHostnames);
} catch (GeneralSecurityException e) {
throw new ExceptionInInitializerError(e);
}
}

Disable SSL on HttpClient with PoolingNHttpClientConnectionManager

I am trying to disable SSL on my CloseableHttpAsyncClient. Below is the code I use:
private synchronized CloseableHttpAsyncClient getCloseableClient() throws Exception {
if (closeableHttpAsyncClient == null) {
logger.info("New Async Client created ");
closeableHttpAsyncClient = HttpAsyncClientBuilder.create()
.setDefaultRequestConfig(createConnConfig())
.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
.setSSLContext(new SSLContextBuilder().loadTrustMaterial(null, new TrustStrategy() {
#Override
public boolean isTrusted(java.security.cert.X509Certificate[] x509Certificates, String s) throws CertificateException {
return true;
}
}).build())
// .setKeepAliveStrategy(kepAliveStrategy) TODO
.setConnectionManager(createPoolingConnManager()).build();
closeableHttpAsyncClient.start();
}
return closeableHttpAsyncClient;
}
However when I run the above I still get:
javax.net.ssl.SSLHandshakeException: General SSLEngine problem
What could be wrong here, I assume I have to modify my PoolingNHttpClientConnectionManager as well. However I am not aware of how to do this. Any pointers would be helpful

The problem with your code is that #setConnectionManager call overrides #setSSLContext rendering the SSLContext instance ineffective.
Either let HttpAsyncClientBuilder create a connection manager internally
SSLContext sslContext = new SSLContextBuilder()
.loadTrustMaterial(null, (x509Certificates, s) -> true)
.build();
HttpAsyncClientBuilder.create()
.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
.setSSLContext(sslContext)
.build();
Or pass the SSLContext instance to PoolingNHttpClientConnectionManager as a constructor argument.
SSLContext sslContext = new SSLContextBuilder()
.loadTrustMaterial(null, (x509Certificates, s) -> true)
.build();
Registry<SchemeIOSessionStrategy> registry = RegistryBuilder.<SchemeIOSessionStrategy>create()
.register("http", NoopIOSessionStrategy.INSTANCE)
.register("https", new SSLIOSessionStrategy(sslContext, NoopHostnameVerifier.INSTANCE))
.build();
DefaultConnectingIOReactor ioReactor = new DefaultConnectingIOReactor();
HttpAsyncClientBuilder.create()
.setConnectionManager(new PoolingNHttpClientConnectionManager(ioReactor, registry))
.build();

Deprecated HttpClient, now using apache-httpclient-4.3.x

I have code that works fine but its now deprecated. Everything is depricated except BasicCredentialsProvider
SSLSocketFactory sslsf = new SSLSocketFactory("TLS", null, null, keyStore, null, new TrustSelfSignedStrategy(),
new AllowAllHostnameVerifier());
Scheme https = new Scheme("https", 28181, sslsf);
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
schemeRegistry.register(https);
PoolingClientConnectionManager connectionManager = new PoolingClientConnectionManager(schemeRegistry);
connectionManager.setMaxTotal(100);
connectionManager.setDefaultMaxPerRoute(5);
DefaultHttpClient httpClient = new DefaultHttpClient(connectionManager);
BasicCredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(new AuthScope(uri.getHost(), uri.getPort(), name),
new UsernamePasswordCredentials(username, password));
httpClient.setCredentialsProvider(credsProvider);
httpClient.getParams().setIntParameter(CoreConnectionPNames.SO_TIMEOUT, 60000);
return new HttpContextRequestScopedApacheClientExecutor(httpClient);
I have tried to do it myself. First I replaced SSLSocketFactory with
SSLConnectionSocketFactory sslConnectionFactory = new SSLConnectionSocketFactory
(SSLContexts.custom().
loadTrustMaterial(keyStore, new TrustSelfSignedStrategy()).
useTLS().build(), SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
then I tried to use
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("https", sslConnectionFactory)
.build();
I couldn't fit my port there, so I after searching for quite some time I am still lost on how to do it. And for the rest I have no solution yet. Any help on this would be apriciated.

I think I have managed to do it, it's working so far.
HttpClientBuilder builder = HttpClientBuilder.create();
KeyStore keyStore = initSSL();
SSLConnectionSocketFactory sslConnectionFactory = new SSLConnectionSocketFactory(SSLContexts.custom().loadTrustMaterial(keyStore, new
TrustSelfSignedStrategy()).useTLS().build(), SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", sslConnectionFactory)
.build();
PoolingHttpClientConnectionManager ccm = new PoolingHttpClientConnectionManager(registry);
ccm.setMaxTotal(100);
ccm.setDefaultMaxPerRoute(5);
builder.setConnectionManager(ccm);
BasicCredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(new AuthScope(uri.getHost(), uri.getPort(), name),
new UsernamePasswordCredentials(username, password));
builder.setDefaultCredentialsProvider(credsProvider);
RequestConfig.Builder requestBuilder = RequestConfig.custom();
requestBuilder.setSocketTimeout(60000);
builder.setDefaultRequestConfig(requestBuilder.build());
return new HttpContextRequestScopedApacheClientExecutor(builder.build());

HttpSolrServer with proxy

Using the HttpSolrServer to connect to a Solr instance. Trying to run this through a proxy but currently the configuration doesn't look to be getting applied, it continues using the baseUrl. This is the code:
DefaultHttpClient httpClient = new DefaultHttpClient();
ModifiableSolrParams params = new ModifiableSolrParams();
params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 128);
params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 32);
params.set(HttpClientUtil.PROP_FOLLOW_REDIRECTS, false);
params.set(HttpClientUtil.PROP_CONNECTION_TIMEOUT, this.connectionTimeout);
httpClient.getCredentialsProvider().setCredentials(
AuthScope.ANY,
new UsernamePasswordCredentials(proxyUsername, proxyPassword)
);
HttpHost proxy = new HttpHost(proxyUrl, proxyPort, proxyProtocol);
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
HttpSolrServer solrServer = new HttpSolrServer(baseUrl, httpClient);
Have also tried:
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(this.connectionTimeout)
.setConnectionRequestTimeout(this.connectionTimeout)
.setSocketTimeout(this.connectionTimeout)
.build();
HttpClientBuilder httpClientBuilder = HttpClients.custom()
.setDefaultRequestConfig(requestConfig)
.setMaxConnTotal(128)
.setMaxConnPerRoute(32)
.disableRedirectHandling();
HttpHost proxy = new HttpHost(proxyUrl, proxyPort, proxyProtocol);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(
new AuthScope(proxyUrl, proxyPort, AuthScope.ANY_REALM, "basic"),
new UsernamePasswordCredentials(proxyUsername, proxyPassword));
httpClientBuilder
.setDefaultCredentialsProvider(credentialsProvider)
.setTargetAuthenticationStrategy(new ProxyAuthenticationStrategy())
.setRoutePlanner(routePlanner);
CloseableHttpClient httpClient = httpClientBuilder.build();
HttpSolrServer solrServer = new HttpSolrServer(baseUrl, httpClient);
Versions
SolrJ 4.7.2
HttpClient 4.3.1

Apache HttpClient 4.3.5 set proxy

It seems that I can specify the proxy when I construct new HttpClient with:
HttpHost proxy = new HttpHost("someproxy", 8080);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();
taken from http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d5e475
Is it possible to modify existing client's proxy settings.

You can create your own implementation of HttpRoutePlanner that will allow change of the HttpHost.
public class DynamicProxyRoutePlanner implements HttpRoutePlanner {
private DefaultProxyRoutePlanner defaultProxyRoutePlanner = null;
public DynamicProxyRoutePlanner(HttpHost host){
defaultProxyRoutePlanner = new DefaultProxyRoutePlanner(host);
}
public void setProxy(HttpHost host){
defaultProxyRoutePlanner = new DefaultProxyRoutePlanner(host);
}
public HttpRoute determineRoute(HttpHost target, HttpRequest request, HttpContext context) {
return defaultProxyRoutePlanner.determineRoute(target,request,context);
}
}
Then you can use this DynamicProxyRoutePlanner in your code
HttpHost proxy = new HttpHost("someproxy", 8080);
DynamicProxyRoutePlanner routePlanner = new DynamicProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();
//Any time change the proxy
routePlanner.setProxy(new HttpHost("someNewProxy", 9090));