I’m a newbie in ES and I have a task in my new job to upgrade from 6.4.2 to 7.4.1 – From TCP client to Rest High Level API.
Previously we built the client like this:
Settings settings = Settings.builder()
.put("xpack.security.user", String.format("%s:%s",esJavaUser,esJavaPassword))
.put("cluster.name", esClusterName)
.put("xpack.security.transport.ssl.enabled", xpackSecurityTransportSslEnabled)
.put("xpack.ssl.certificate_authorities", xpackSslCertificateAuthorities)
.build();
client = new PreBuiltXPackTransportClient(settings);
Now, in rest API, it’s changed to this:
final CredentialsProvider credentialsProvider =
new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(esJavaUser, esJavaPassword));
RestClientBuilder restClientBuilder = RestClient.builder(hosts)
.setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder
.setDefaultCredentialsProvider(credentialsProvider));
restHighLevelClient = new RestHighLevelClient(restClientBuilder);
With this build I set ES user and password by CredentialsProvider but what about ssl.enabled and certificate_authorities”? how should I provided them with rest API?
I got an answer from ES forum (didn't thought to ask there first..)
Because, as developer, I always looking for answer here, in stackoverflow, I decide to not delete this question and copy TimV answer:
The documentation you are looking for is here: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.4/_encrypted_communication.html
SSL is automatically enabled (or not) based on the scheme (protocol) in the HttpHost objects you pass to the builder.
RestClient.builder(hosts)
If you are using SSL, you want to pass "https" as the scheme (3rd argument) when you construct the HttpHost objects (hosts).
Unfortunately there is no simple means to pass certificate_authorities to the Rest client, you need to turn those certificates into a standard Java truststore.
You can probably find some sample code on the web ("convert PEM certificates to Java truststore"), but the gist of it is:
Open the certificate authority files as an InputStream
Create a X.509 certificate factory: java.security.cert.CertificateFactory.getInstance("X.509")
Call generateCertificates on the certificate factory to read those certificate files into java Certificate objects
Construct an empty KeyStore object
Add the loaded certificates as trusted entries
Pass that to SSLContextBuilder.loadTrustMaterial
Link: https://discuss.elastic.co/t/es-7-4-1-authentication-rest-api/211969
Related
I know my way around Java but I'm fairly inexperienced when it comes to the topic of SSL certificates. So my whole approach may be complete and utter nonsense.
What I'm trying to achieve is the following: I have a webservice build with Apache CXF in a Spring Boot application. That webservice is called a x509 client certificate and I want to use that certificate to get a JWT from a Keycloak instance which I have already configured. Getting the token from keycloak works when I do it like this:
var keystore = KeyStore.getInstance("PKCS12");
keystore.load(new ClassPathResource("keystore.pfx").getInputStream(), "myPassphrase".toCharArray());
HttpClient client = HttpClients.custom().setSSLContext(new SSLContextBuilder()
.loadTrustMaterial(new ClassPathResource("truststore.jks").getFile(), "trustStorePW".toCharArray())
.loadKeyMaterial(keystore, "myPassphrase".toCharArray())
.build()
).setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE).build();
var postRequest = new HttpPost("https://localhost:8443/realms/master/protocol/openid-connect/token");
postRequest.addHeader("Content-Type", "application/x-www-form-urlencoded");
var entity = new StringEntity("grant_type=password&client_id=my-client&client_secret=my-secret");
postRequest.setEntity(entity);
var response = client.execute(postRequest);
That's fine and tells me that my keycloak setup is correct. What I'm trying to do now is to extract the certificate from the SOAP request and forward it to the keycloak call. I've done this with an interceptor and pull the certificate from the request like this:
if (httpRequest instanceof HttpServletRequest request
&& request.getAttribute("javax.servlet.request.ssl_session_mgr") instanceof SSLSupport sslSupport
&& sslSupport.getLocalCertificateChain() != null
) {
for (var cert : sslSupport.getLocalCertificateChain()) {
var keystore = KeyStore.getInstance("PKCS12");
keystore.load(null, "passphrase".toCharArray());
keystore.setCertificateEntry("1", cert);
// then use the same code as before for calling keycloak
}
}
But now I get {"error_description":"X509 client certificate is missing.","error":"invalid_request"}. Looking into it I realize that the main difference between the two keystores is that the one created from the HttpRequest does not contain the private key, so I suspect that's the reasons it doesn't work. The keystore.pfx I used in the first example contains only the client certificate btw.
Is there a way to make it work like this or is my whole approach completely wrong? Because that's what I'm starting to suspect. And if so, how could I solve this?
I am trying to access Azure Key Vault local by using Service Principle credentials from local for development perspective.
But it seems that Azure SDK is always checking IMDS connectivity ("169.254.169.254")
Code I used to retrieve secret:
SecretClient secretClient = new SecretClientBuilder()
.vaultUrl(keyVaultUri)
.credential(new DefaultAzureCredentialBuilder().build())
.buildClient();
I also added below variables as env variables:
AZURE_CLIENT_ID
AZURE_CLIENT_SECRET
AZURE_TENANT_ID
Can somebody help me with how can we access azure resources like key vault from our local using Service Principle in java
To use service principal to auth locally, just use ClientSecretCredential.
Sample:
import com.azure.identity.ClientSecretCredential;
import com.azure.identity.ClientSecretCredentialBuilder;
import com.azure.security.keyvault.secrets.SecretClient;
import com.azure.security.keyvault.secrets.SecretClientBuilder;
import com.azure.security.keyvault.secrets.models.KeyVaultSecret;
public class vacate {
public static void main(String[] args) {
String clientId="xxxxxx";
String clientSecret="xxxxxx";
String tenantId="xxxxxx";
ClientSecretCredential credential1 = new ClientSecretCredentialBuilder()
.tenantId(tenantId)
.clientId(clientId)
.clientSecret(clientSecret)
.build();
SecretClient secretClient = new SecretClientBuilder()
.vaultUrl("<your-key-vault-url>")
.credential(credential1)
.buildClient();
//do other things
}
}
Actually, I think DefaultAzureCredential you used should also work, it tries to create a valid credential in the following order, if you have already set the environment variable correctly, it should work, if not, just use the ClientSecretCredential like above, it will work.
The way it fixed my problem (and may be will help others as well):
Indeed as mentioned in Joy's answer, you need to use ClientSecretCredential or you can also use Azure Toolkit for IntelliJ for authentication
I was using old azure identity which was going to old authentication end point login.microsoftonline.com/{{tenant_id}} which got fixed after upgrading to latest version (1.2.3). Now it goes to new end point of login.microsoftonline.com/common/oauth2
For me, I was also getting a lot of SSL errors. To fix it adding below certificates to trusted certificates worked :
DigiCert Global Root CA
DigiCert SHA2 Secure Server CA
If your network is behind a proxy, you also need to configure proxy and added corresponding CA Root certificate to your keystore and truststore.
My goal: I am working on an integration of Jetty Embedded that would make it simple to use. The interface would allow, among others, for integration of external sources for TLS certificates, without the use of the Java KeyStore.
This would allow for greater flexibility when building distributed web services (in my case an experimental, self-hosted CDN).
However, I am having problems building the integration. The stub implementation is in this repository.
What I've tried: I have tried replacing the key manager and the trust manager and set break points to every function in it. However, when trying to access the server, these break points are never triggered. Instead, I'm encountering this error:
javax.net.ssl.SSLHandshakeException: no cipher suites in common
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1478)
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:813)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.fill(SslConnection.java:621)
at org.eclipse.jetty.server.HttpConnection.fillRequestBuffer(HttpConnection.java:322)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:231)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:112)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:261)
at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:150)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:112)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590)
at java.lang.Thread.run(Thread.java:748)
I have tried analyzing the "standard" Jetty setup which a certificate from a keystore, but without much luck. I'm failing to find the point where Jetty is obtaining the cipher / certificate information that I should override.
My question: How can I get Jetty to use my own certificate source instead of the Java KeyStore and TrustStore?
#EJP pointed me in the right direction, so here's how to do it:
Here's how it needs to be done.
First, set up Jetty for TLS:
HttpConfiguration https = new HttpConfiguration();
https.addCustomizer(new SecureRequestCustomizer());
SslContextFactory sslContextFactory = new JettySslContextFactory(configuration.getSslProviders());
ServerConnector sslConnector = new ServerConnector(
server,
new SslConnectionFactory(sslContextFactory, "http/1.1"),
new HttpConnectionFactory(https)
);
sslConnector.setPort(httpsPort);
Note the class JettySslContextFactory. This class extends the built-in X509ExtendedKeyManager and needs to override the protected KeyManager[] getKeyManagers(KeyStore keyStore) throws Exception method in order to provider a custom KeyManager, like this:
#Override
protected KeyManager[] getKeyManagers(KeyStore keyStore) throws Exception {
return new KeyManager[] {
new JettyX509ExtendedKeyManager(certificateProviders)
};
}
In addition to that, the following steps are run in every connection:
The SNI matcher is consulted with the SNI host name. This seems to be the only place where the SNI host name is even available.
The key manager is consulted to get the alias (sort of a key ID) for a certain key type (EC or RSA). Here we need grab the host name from the SNI matcher, because otherwise we wouldn't know which host name to match on.
Based on the alias (key ID) we can then return the private key and certificate.
At least this is what I gathered from debugging this issue. The full code is online here.
I am working on a project that would like to be able to use certificates or keys as a method of authentication for SNMPv3. We are using the java library SNMP4J.
During my research I have found that SNMP uses TLS/DTLS for message encryption and supposedly also for authentication. Source 1 | Source 2 | Source 3
Looking into the little documentation SNMP4J has, I found that it allows the usage of TLS certificates for encrypting traffic. But I am not sure how the authentication is done, if possible, using a public/private key pair. TLS Traffic Encryption Example | SNMP4J Documentation
Any help would be appreciated.
I was able to authenticate using a similar method as described in the example TLS Traffic Encryption Example.
So as one would expect from the example, I can confirm that SNMP4J uses the keystore set in the Java Property javax.net.ssl.keystore, javax.net.ssl.keyStorePassword, javax.net.ssl.trustStore, and javax.net.ssl.trustStorePassword.
Below are the changes I made to the example to make it work.
The alias (or security name in the documentation) needs to be set in the CertifiedTarget constructor so it knows which certificate to use.
CertifiedTarget ct = new CertifiedTarget(new OctetString(alias));
The security level must be set or the SNMP agent will complain and fail authentication.
ct.setSecurityLevel(SecurityLevel.AUTH_PRIV);
The SecurityCallback subject DN must match the server certificate subject EXACTLY the way it wants otherwise it will deny all responses.
securityCallback.addAcceptedSubjectDN("EMAILADDRESS=admin#net-snmp.org, CN=snmpagent, OU=Development, O=Net-SNMP, L=Davis, ST=CA, C=US");
Lastly, you must register the server public certificate alias (Security Name) with the address.
securityCallback.addLocalCertMapping(ct.getAddress(), "snmpagent");
It comes together to look something like this.
// Set java keystore manually
System.setProperty("javax.net.ssl.keyStore", KEYSTORE_DIR);
System.setProperty("javax.net.ssl.keyStorePassword", "changeit");
System.setProperty("javax.net.ssl.trustStore", KEYSTORE_DIR);
System.setProperty("javax.net.ssl.trustStorePassword", "changeit");
// create the TLS transport mapping:
TLSTM transport = new TLSTM();
// set the security callback (only required for command responder,
// but also recommended for command generators) -
// the callback will be configured later:
DefaultTlsTmSecurityCallback securityCallback = new DefaultTlsTmSecurityCallback();
((TLSTM) transport).setSecurityCallback(securityCallback);
MessageDispatcher md = new MessageDispatcherImpl();
// we need MPv3 for TLSTM:
MPv3 mpv3 = new MPv3();
md.addMessageProcessingModel(mpv3);
Snmp snmp = new Snmp(md, transport);
// create and initialize the TransportSecurityModel TSM:
SecurityModels.getInstance().addSecurityModel(new TSM(new OctetString(mpv3.getLocalEngineID()), false));
// do not forget to listen for responses:
snmp.listen();
CertifiedTarget ct = new CertifiedTarget(new OctetString("alias"));
ct.setVersion(SnmpConstants.version3);
ct.setSecurityModel(SecurityModel.SECURITY_MODEL_TSM);
ct.setAddress(GenericAddress.parse(myAddress));
ct.setSecurityLevel(SecurityLevel.AUTH_PRIV);
securityCallback.addAcceptedSubjectDN("EMAILADDRESS=admin#net-snmp.org, CN=snmpagent, OU=Development, O=Net-SNMP, L=Davis, ST=CA, C=US");
securityCallback.addLocalCertMapping(ct.getAddress(), "snmpagentalias");
PDU pdu = new ScopedPDU();
pdu.add(new VariableBinding(new OID(someOid)));
pdu.setType(PDU.GET);
ResponseEvent response = snmp.send(pdu, ct);
You also have to make sure all the certificates are properly configured so that it actually takes them.
As a side-note, in the discovery of this my team and I discovered several bugs in the TLS handling by SNMP4J, mostly in the transport layer. It seems to be a timing issue (race condition maybe?) where it will get the SNMP data but then ignore it. We were able to get around it by setting the CertifiedTarget timeout and retries really high. We will officially report on this when we have more information.
I am looking into OCSP support in Java.
I see that there are some APIs offered in plain java for this(i.e. without using some third-party library e.g. Bouncy Castle).
Now I know, that OCSP can be implemented via Direct Trust Model and Delegated Trust Model.
From various posts I get the impression that the direct trust model is supported e.g I have seen something like:
X509Certificate ocspCert = getCertFromFile(OCSP_SERVER_CERT);
certSet.add(ocspCert);
CertStoreParameters storeParams = new CollectionCertStoreParameters(certSet);
CertStore store = CertStore.getInstance("Collection", storeParams);
// init PKIX parameters
PKIXParameters params = null;
params = new PKIXParameters (trustedCertsSet);
params.addCertStore(store);
and as I understand the code, the OSCP server certificate is being passed to the framework for validation.
My question is the following: Is the Delegated Trust Model supported as well? If some example was provided, it would be very helpful.
Thanks
RedHat has an open source OCSP Tools written in Java.