For my personal pet-project I wrote on Java application, which pulls code from GIT repository through https and performs some analysis. However when I started testing it against code store in corporate repository I got “PKIX path building failed” and “unable to find valid certification path to requested target” errors.
Quick googling showed that the main cause of the problem is incorrect SSL settings in the infrastructure like self-signed certificate, for example.
I found possible workarounds to avoid this error:
Add host certificate manually into JRE trusted certificates.
Dynamically add the certificate to trusted certificates from applications resource files during its startup.
During startup of the app disable SSL validations at all, but this makes application vulnerable to "man in the middle" attacks.
But that's not the question.
When I was working with this corporate repository with standard GIT client (through https too) I've never got any issues with SSL or certificates. So the question is - how does official GIT client avoids this problem with self signed certificate in the chain?
Git itself does not do anything with any of this.
Git uses helpers—external programs and libraries—when talking to other Git providers. Those programs and libraries are responsible for all of this. So if you use git fetch or git push—note that git pull uses git fetch—and you use these with a URL that begins with https://, Git uses a system-provided HTTPS library,1 usually called "libcurl" or just curl. If you use an ssh:// URL, Git uses the system-provided ssh.
The Pro Git Book includes a chapter on Git Internals, with one section dedicated to Environment Variables. When using libcurl, these variables are particularly useful:
GIT_CURL_VERBOSE prints out the same stuff that curl -v prints.
GIT_SSL_NO_VERIFY turns off SSL certificate verification. So the default must be to do that verification, which (of course) will have issues with self-signed certificates.
When I was working with this corporate repository with standard GIT client (through https too) I've never got any issues with SSL or certificates.
That's interesting and curious; perhaps invoking Git with GIT_CURL_VERBOSE set will show what's going on here, or perhaps not. Or, perhaps the libcurl that was used to build your particular Git uses some CA files that your Java libraries do not.
1The details for this will depend on how your Git binaries were built. If you like, you can get the source code for Git and build it yourself, and thus have control over which library is used.
Related
I have been surprised to observe that the code ClientBuilder.newBuilder().connectTimeout(10, TimeUnit.SECONDS).readTimeout(10, TimeUnit.SECONDS).build().target("https://www.oracle.com/java/technologies/naming-conventions.html").request().get(); throws a “Read timed out” exception using the Jersey JAX-RS client implementation. (When I tried to report this, I was told that it is not a bug.)
I expected that Jersey would use the default Java truststore, and that the default Java truststore would include CA certificates that permit to trust Oracle’s (and many other) website.
Indeed, running keytool -list -cacerts, I see that my truststore contains 166 certificates. I suppose that some of them permit to validate Oracle’s website, but I do not know how I can check that (well, apart from doing exactly what I am trying to do with Jersey).
I realize that I can download Oracle’s certificate and add it to my truststore, but I obviously do not want to do this for every classical web site out there to which I would like to connect using SSL. I would rather like to understand which web sites are supposed to be trusted by default in a “normal” Java installation (is there an official documentation about this somewhere?); how I can check whether my specific installation has some problem that prevents Oracle’s web site to be trusted; or, if it is normal, whether I can tell Java once and for all to trust the web sites that, say, Firefox or Chromium would trust by default.
This answer, for example, suggests that OpenJdk (which is what I use) should trust by default the same web sites as Firefox. And Firefox obviously trusts Oracle.
What am I missing?
Edit I realize it seems relevant to specify that I use Debian stable, as this issue might be OS-specific. I wonder if there’s something going wrong on my installation. “As far as I'm aware every distro patches OpenJDK to use its own list, which I guess is why this issue hasn't got much attention on GNU/Linux systems” -- Andrew Haley on OpenJDK ML. “Debian has tooling to create a cacert file from the system‘s keystore. There is a hook system that updates the cacert every time the system‘s keystore is changed. lib/security/cacert is actually a symlink to that file. (…) In case of Debian, it‘s identical with Mozilla‘s list.” -- aahlenst on GH Adoptium issue. Indeed: ls -l /usr/lib/jvm/default-java/lib/security/cacerts reveals that it links to
/etc/ssl/certs/java/cacerts.
I am using Git with Intellij in Windows 10 environment. For some reason my git credentials are messed up. When I try to fetch/clone, Intellij is not asking me the credentials to enter. It appears as if it is trying to use already stored git credentials. I am getting the following error when I try to fetch :
7:58:31.180: git -c core.quotepath=false fetch origin --progress
--prune
java.io.IOException: Authentication failed: at
org.jetbrains.git4idea.ssh.SSHMain.authenticate(SSHMain.java:298) at
org.jetbrains.git4idea.ssh.SSHMain.start(SSHMain.java:172) at
org.jetbrains.git4idea.ssh.SSHMain.main(SSHMain.java:137) Caused by:
java.io.IOException: Publickey authentication failed. at
com.trilead.ssh2.auth.AuthenticationManager.authenticatePublicKey(AuthenticationManager.java:262)
at
com.trilead.ssh2.Connection.authenticateWithPublicKey(Connection.java:443)
at org.jetbrains.git4idea.ssh.SSHMain.tryPublicKey(SSHMain.java:352)
at org.jetbrains.git4idea.ssh.SSHMain.authenticate(SSHMain.java:231)
... 2 more Caused by: java.io.IOException: Invalid PEM structure,
'-----BEGIN...' missing at
com.trilead.ssh2.crypto.PEMDecoder.parsePEM(PEMDecoder.java:138) at
com.trilead.ssh2.crypto.PEMDecoder.decode(PEMDecoder.java:313) at
com.trilead.ssh2.auth.AuthenticationManager.authenticatePublicKey(AuthenticationManager.java:171)
... 5 more fatal: Could not read from remote repository. Please make
sure you have the correct access rights and the repository exists.
So, I googled and found various approaches and tried all of them only to find none is working for me.
I tried in the following ways:
1)Removing the credentials stored in Windows Credentials Manager
2)Change remote repository credentials (authentication) on Intellij IDEA 14
3)Git credential helper - update password
and so on.. Any other thoughts to fix the issue so that the Intellij would ask me the credentials before I fetch/clone my repo?
TL;DR Update to the latest version (2018.3 or above), it uses Native SSH by default and if everything works in the terminal, it should work in the IDE.
A bit more detailed answer below:
Git credential helpers work for HTTP-base connections. Judging by the error, you are using SSH, so no credential helper is involved so any manipulations around it will not help.
The error message also belongs to the IntelliJ Built-in SSH client, and the reason is either wrong ssh-key in use or unsupported key format - and builtin client is known to not support new OpenSSH key format - see https://youtrack.jetbrains.com/issue/IDEA-151116
The solution to this is to configure IDE to use Native SSH, which is, in fact, telling git to use system default SSH client, or the one bundled with GitForWindows in case it is used. Native SSH is supposed to support the key format in use.
Prior to 2018.3, IntelliJ was not able to handle interactive prompts for passphrase or host verification, so ssh-agent had to be used. Stating from 2018.3, IntelliJ can handle those prompts and Native SSH is the default. So updating should solve the issue should it be the key format, or some specific .ssh config not supported by the builtin SSH client.
I was running into some other issues when I configured Native SSH, but downgrading git version to 2.17.0 from 2.20.1 has solved the problem
I am quite new to working with certificates and security, so pardon me if this is a no-brainer to others. I have followed this guide to set up my Grails application to run on HTTPS with self-signed certificates.
I am trying to establish 2-way SSL with another HTTPS network (a Nifi standalone instance) running on the same machine. I can get the Nifi instance to talk to Grails over HTTPS, but I am having issues with Grails talking to Nifi (specifically to a ListenHTTP processor).
I was hoping someone could advise how to use certificate authentication in Grails when posting over HTTPS.
Nifi uses certificate authentication; however per the above guide Grails only specifies a single keystore (for receiving requests?) so I'm a bit thrown off. I can successfully CURL to Nifi's REST API by specifying the --cert and --key properties, but since the final product will be a WAR on a client machine I want to set this up the 'right way', and I believe leaving those files on the client machine is a really big no-no for security.
During early development RestBuilder was sufficient for 2-way comms over HTTP, however, I am unable to find any mention of using it with certificate authentication (only basic authentication is covered in the documentation?).
HTTPBuilder shows up a lot when I looked for alternatives, however looking at the relevant documentation (line 139 'certificate()') it states that it takes a whole keystore JKS and password. I think this is close but not quite what I am looking for considering I only have one keystore; I am open to correction here.
Please note that I will be unavailable to respond until at least the day after this question was posted.
When making an outgoing HTTPS connection, if the remote endpoint (in this case Apache NiFi) requires client certificate authentication, the originating endpoint (Grails) will attempt to provide a certificate. The certificate that Grails is using to identify itself as a service is fine to use in this scenario, provided:
The certificate either does not have the ExtendedKeyUsage extension set, or if it is set, both ServerAuth and ClientAuth values are present. If ClientAuth is missing, the system will not allow this certificate to be used for client authentication, which is the necessary role in this exchange.
The certificate has a valid SubjectAlternativeName value which matches the hostname it is running on. RFC 6125 prescribes that SAN values should be used for certificate identity rather than Distinguished Name (DN) and Common Name (CN). So if the Grails app is running on https://grails.example.com, the SAN must contain values for grails.example.com or *.example.com.
The certificate must be imported into NiFi's truststore in order to allow NiFi to authenticate a presenter of this certificate.
NiFi must have ACL permissions in place for this "user". This can be done through the UI or by modifying the conf/authorizers.xml file before starting NiFi for the first time. See NiFi Admin Guide - Authorizers Configuration for more information.
Your concern for leaving the cert.pem and key.key files on the client machine is understandable, but the sensitive information contained therein is the same data that's in your keystore. At some point, the private key must be accessible by the Grails app in order to perform HTTPS processes, so having it in the keystore is functionally equivalent (you don't mention having a password on the *.key file, but obviously you should have a password on the keystore).
I am using mailgun to send email but from last day it shows error during POST request on email send api as below.I am using java/spring/AWS to my web app.I tried to restart app and to check if certificate is already in the trust store but we never had generate trust store keys.
com.mashape.unirest.http.exceptions.UnirestException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at com.mashape.unirest.http.HttpClientHelper.request(HttpClientHelper.java:143)
at com.mashape.unirest.request.BaseRequest.asJson(BaseRequest.java:68)
On 1/22, we updated our SSL certificates due to Symantec's PKI infrastructure set to become untrusted. Some older versions of Java do not have the "DigiCert Global Root G2" CA.
There are several options:
Import the "DigiCert Global Root G2" CA into your "cacerts" file.
Upgrade your JRE to 8u91 (or higher), which includes this root.
To import the "DigiCert Global Root G2" You can download the root from https://www.digicert.com/digicert-root-certificates.htm. Ensure you are downloading the correct root certificate.
Once the certificate is downloaded, you'll need to import it into with a command like the following:
keytool -import -trustcacerts -keystore /path/to/cacerts -storepass changeit -noprompt -alias digicert-global-root-g2 -file /path/to/digicert.crt
You will need to set the path to your Java Keystore and the location of the root certificate you downloaded.
Same issue for us (java).
Here is what Mailgun support said:
Hello,
Thank you for reaching out to us in regards to our SSL certification renewal, >which are now fully completed. This update was pushed out to correct numerous >certificates that did not comply with the industry-developed CA/Browser Forum >Baseline Requirements by Symantec’s PKI business. You can read more about this >here.
Now that the updates have been completed, if you’re still having difficulties >sending, we can provide some suggestions to get your application back up and >running.
If you’re utilizing PHP-
1) Download the latest CA certificates from (https://curl.haxx.se/docs/caextract.html) This is an extract of a list of certificates vetted by Mozilla.
2) Set the path to the CA certificates in the .htaccess file with a directive like the following:
php_value curl.cainfo "/path/to/cacert.pem"
If you have access to your php.ini file, you can modify this value globally by setting curl.cainfo = "/path/to/cacert.pem"
If you’re utilizing cURL-
_Ubuntu users:_https://askubuntu.com/questions/646594/how-to-fix-ca-cert-issues-with-curl-in-ubuntu-14-04
RedHat users: https://serverfault.com/questions/394815/how-to-update-curl-ca-bundle-on-redhat
Be sure to restart your sending application after updating.
If a restart of your application does not clear the issue, the certificate cache may need to be cleared. Here is some additional information as to how this might be accomplished for you:
http://unitstep.net/blog/2009/05/05/using-curl-in-php-to-access-https-ssltls-protected-sites/
http://stackoverflow.com/questions/7219989/java-and-ssl-certificates
Because every sending application is be different, the exact method for updating the certifications/restarting your sending application may vary for you. If none of these options worked, some independent research might be needed for the update/restart method applicable to you.
While these suggestions are meant to help resolve the issues you are experiencing, making certain changes to your environment can always cause further issues with your application. We highly recommend making these changes in a non-production environment prior to releasing into production. These are only intended as suggested fixes and in no-way guarantee that this will fix the issue, further steps may need to be taken depending on your specific environment. As always Mailgun will always do our best to answer any questions that you may have, but there are limitations in the support we can provide.
Thanks,
Keith#MG
Restarting the application did not fix the issue. I'm trying to find which CA they are using to see if we can import it in the trustore.
I have a Java web app that has been running fine for several months. It integrates with the Box API (https://upload.box.com/api/2.0) to save files to the cloud service. Out of the blue, we started receiving the dreaded javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated exception while trying to make a REST API call to the Box web service.
There are tons of posts out there that tell you how to manually import the cert into your key store using the keytool utility. I understand that I can do this to fix the problem. But is this really the correct fix?
The fact that my application has been running fine for months leads me to believe something in the certificate at https://upload.box.com changed. Looking at the cert in my web browser, the certificate seems valid and was only renewed a few weeks ago. What is missing from my keystore?
Is it the Root CA certificate that is missing from my keystore? If that is the case, could I just copy the cacerts file from newer version of Java? My app is currently running JDK 1.6.0_33.
I am just trying to understand why this would suddenly stop working and what the "real" fix should be. It doesn't seem like modifying the JDK keystore is the correct thing to do.
I'll just assume you're using Apache HTTP Client 4.x, before 4.2.6, 4.3 Beta2, in which case javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated is most likely to come from a certificate that isn't trusted (otherwise it means the server didn't send a cert, which is a different problem, more details in this answer).
The current cert for server you're trying to access seems to have been issued on 07/04/2014, so this indicates that the certificate, and perhaps its chain has changed recently indeed.
I don't have a JDK 1.6.0_33 at hand, but it's possible that some of these CA certs were not part of the default bundle. In any case, it's worth updating cacerts, even on older JREs (if only to remove CA certs that should no longer be trusted, for example). The JSSE Reference Guide clearly states (admittedly in the middle of a fairly long document, but it's worth searching for "important note"...):
IMPORTANT NOTE: The JDK ships with a limited number of trusted root certificates in the <java-home>/lib/security/cacerts file. As documented in keytool, it is your responsibility to maintain (that is, add/remove) the certificates contained in this file if you use this file as a truststore.
Depending on the certificate configuration of the servers you contact, you may need to add additional root certificate(s). Obtain the needed specific root certificate(s) from the appropriate vendor.
If you can't upgrade your JRE (Java 6 is in general out of support), updating the cacerts file from a more recent version is certainly a sensible compromise.
Besides the various fixes in Java 7, Java 7+ would also allow you to connect to hosts that require SNI (although this doesn't seem to be the case for this particular host).