Develop Reference

CSV file download from secured site

I am trying to download a CSV file through Spring Boot App listed here
https://github.com/koushikkothagal/coronavirus-tracker/
"https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_19-covid-Confirmed.csv";
Getting exception as listed below.
If I change to http - no errors but also file not found.
Please guide.
Result of executing java SSLPoke raw.githubusercontent.com 443
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:384)
at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:289)
at java.base/sun.security.validator.Validator.validate(Validator.java:264)
at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:222)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1308)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1199)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1146)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:177)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1180)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1091)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:721)
at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:998)
at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:970)
at SSLPoke.main(SSLPoke.java:31)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:379)
... 20 more

Solved it! Here are the Steps...
Download & Compile InstallCert.java
Location
https://github.com/escline/InstallCert
Then, run
java InstallCert <host_url>:443
Accept all certificates.
Once done - a new jssecacerts file would be generated
Copy the newly generated jssecacerts file to your $JAVA_HOME\jre\lib\security folder.
That's it.

You need to manually import target site SSL certificate to your trust store. The certificate can be added to Java central trust store or your custom trust store and provide the trust store in jvm argument.

How to fix java.security.cert.CertPathValidatorException: timestamp check failed

My mac has a TFS build agent connecting to a TFS server. That works fine. But when fetching the source code it fails with the following error:
An error occurred: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed
When I on the mac, remove the old certificate for the TFS server from the Java Truststore I get the following error:
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
If I add a new certificate for the server, into the Java truststore (the cacerts file) with good new valid time stamps, I get this error:
PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed
Thus, I know that I am making changes to the correct cacerts file, since I'am getting different output based on the two scenarios above.
I've been scanning through the cacert file, trying to find if any expired cert for the TFS server is still there. Can't find any.
I've tried importing both the root cert and the site cert, but no difference there.
No problems connecting to the server site with Safari or using curl. The certificate on the server is renewed and has valid timestamps.
Any help would be greatly appreciated.

I managed to get by the issue by installing a brand new, latest version build agent.

OpenJDK 10 rejects all SSL certificates

I installed OpenJDK 10. What I did was just unpack it, set the PATH variable and the JAVA_HOME variable.
Java even rejects the certificate of google.de (and all others I tried). I tested this by some dummy class that tries to connect. Outcome is:
C:\Users\Alexander\Downloads>java SSLPoke google.de 443
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385)
at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:290)
at java.base/sun.security.validator.Validator.validate(Validator.java:264)
at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:343)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:226)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:133)
at java.base/sun.security.ssl.ClientHandshaker.checkServerCerts(ClientHandshaker.java:1947)
at java.base/sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1777)
at java.base/sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:264)
at java.base/sun.security.ssl.Handshaker.processLoop(Handshaker.java:1098)
at java.base/sun.security.ssl.Handshaker.processRecord(Handshaker.java:1026)
at java.base/sun.security.ssl.SSLSocketImpl.processInputRecord(SSLSocketImpl.java:1137)
at java.base/sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1074)
at java.base/sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at java.base/sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1402)
at java.base/sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:733)
at java.base/sun.security.ssl.AppOutputStream.write(AppOutputStream.java:67)
at java.base/sun.security.ssl.AppOutputStream.write(AppOutputStream.java:81)
at SSLPoke.main(SSLPoke.java:31)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380)
... 18 more
Found a lot on the net how to add special certs to java but i think in my java installation something is wrong in general.
So my question is what could cause such a thing???

You should verify root certificates inside the cacerts keystore. The file is stored in JAVA_HOME/jre/lib/security/cacerts (or JAVA_HOME/lib/security/cacerts in newer Java versions). As per OpenJDK 10 Now Includes Root CA Certificates post you can use keytool command to count them:
>jdk-10\bin\keytool -cacerts -list | find "Certificate" /c
Enter keystore password: changeit
80
Most likely your Java installation is corrupted and you should reinstall. Do note that OpenJDK is provided by multiple vendors and it could be that you are using an installer which doesn't install the root certificates.

Ended up installing oracle jdk 10 (also 10.0.2) and that works
C:\Users\Alexander\Downloads>java "SSLPoke" google.de 443
Successfully connected

How to solve unable to find valid certification path to requested target on production?

I have a JKS which I need for authentication to start making use of a SOAP webservice.
On my local machine (windows) all is working fine. For add the following to my startup parameters for Tomcat in Eclipse:
-Dsun.security.ssl.allowUnsafeRenegotiation=true -Djavax.net.ssl.keyStore="path\to\keystore\keystore.jks" -Djavax.net.ssl.keyStorePassword=passwordC -Djavax.net.ssl.keyStoreType=jks
In production (ubuntu) we have the following:
-Dsun.security.ssl.allowUnsafeRenegotiation=true -Djavax.net.ssl.keyStore=/path/to/keystore/keystore.jks -Djavax.net.ssl.keyStorePassword=password -Djavax.net.ssl.keyStoreType=jks"
I'm using Java 1.7.0.80 and Tomcat 7.0.61
When I want to connect with the SOAP webservice I get the following Stacktrace:
Caused by: javax.net.ssl.SSLHandshakeException: SSLHandshakeException invoking https://www.correosexpress.com/wsp/services/GrabacionEnvio: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1359)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1343)
at org.apache.cxf.io.CacheAndWriteOutputStream.postClose(CacheAndWriteOutputStream.java:56)
at org.apache.cxf.io.CachedOutputStream.close(CachedOutputStream.java:215)
at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:652)
at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
... 121 more
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
I checked the Java versions, tomcat versions.
I also checked if the path to the JKS is correct, which it is.
On my local machine, I did not install any other certificates manually. i.e I don't have any entry in the cacerts which is related to the content in the jks.

Check with java keytool that your certificate is present in certificates list.
Check that your java home and your jre are the same. In ubuntu it possible to have java_home with one version of java and default jre with other version(check this command - update-alternatives --config java)
If you added certificate to the correct version of java and it's present in certs list check that you use correct version of certificate.

PKIX path building failed in Java application

I have been struggling for almost one week to get my applications up running after moving my applications from Windows 2000 to Windows 2008 R2 Server.
The procedure:
Installed Java JDK 1.7.0_25
Set system environment variable JAVA_HOME to C:\Progra~1\Java\jdk1.7.0_25\
Imported the certificate into cacerts with keytool
Ensured that the certificate exists in keytool with -list.
I have tried to repeat step 3 with InstallCert to ensure that i havent messed anything up.
The above methods did not solve my problem, so i tried to do it programmatically:
System.setProperty("javax.net.ssl.trustStore",
"C:/Progra~1/Java/jdk1.7.0_25/jre/lib/security/cacerts");
System.setProperty("javax.net.ssl.trustStorePassword", "changeit");
Still without any luck. I am stuck and not quite sure which direction to go from here.
Stack trace:
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:804)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:515)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
at util.SMS.send(SMS.java:93)
at domain.ActivationSMSSenderMain.sendActivationMessagesToCustomers(ActivationSMSSenderMain.java:80)
at domain.ActivationSMSSenderMain.<init>(ActivationSMSSenderMain.java:44)
at domain.ActivationSMSSenderMain.main(ActivationSMSSenderMain.java:341)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:126)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1323)
... 14 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380)
... 20 more
UPDATE:
Both
System.out.println(System.getProperty("javax.net.ssl.trustStore"));
and
System.out.println(System.getProperty("javax.net.ssl.keyStore"));
returns null.

I ran into similar issues whose cause and solution turned out both to be rather simple:
Main Cause: Did not import the proper cert using keytool
NOTE: Only import root CA (or your own self-signed) certificates
NOTE: don't import an intermediate, non certificate chain root cert
Solution Example for imap.gmail.com
Determine the root CA cert:
openssl s_client -showcerts -connect imap.gmail.com:993
in this case we find the root CA is Equifax Secure Certificate Authority
Download root CA cert.
Verify downloaded cert has proper SHA-1 and/or MD5 fingerprints by comparing with info found here
Import cert for javax.net.ssl.trustStore:
keytool -import -alias gmail_imap -file Equifax_Secure_Certificate_Authority.pem
Run your java code

You've imported the certificate into the truststore of the JRE provided in the JDK, but you are running the java.exe of the JRE installed directly.
EDIT
For clarity, and to resolve the morass of misunderstanding in the commentary below, you need to import the certificate into the cacerts file of the JRE you are intending to use, and that will rarely if ever be the one shipping inside the JDK, because clients won't normally have a JDK. Anything in the commentary below that suggests otherwise should be ignored as not expressing my intention here.
A far better solution would be to create your own truststore, starting with a copy of the cacerts file, and specifically tell Java to use that one via the system property javax.net.ssl.trustStore.
You should make building this part of your build process, so as to keep up to date with changes I the cacerts file caused by JDK upgrades.

If you are using Eclipse just cross check in Eclipse Windows--> preferences---->java---> installed JREs is pointing the current JRE and the JRE where you have configured your certificate. If not remove the JRE and add the jre where your certificate is installed

Per your pastebin, you need to add the proxy.tkk.com certificate to the truststore.

On Windows you can try these steps:
Download a root CA certificate from the website.
Find a file jssecacerts in the directory /lib/security with JRE (you can use a comand System.out.println(System.getProperty("java.home"); to find the folder with the current JRE). Make a backup of the file.
Download a program portecle.
Open the jssecacerts file in portecle.
Enter the password: changeit.
Import the downloaded certificate with porticle (Tools > Import Trusted Certificate).
Click Save.
Replace the original file jssecacerts.

In my case the issue was resolved by installing Oracle's official JDK 10 as opposed to using the default OpenJDK that came with my Ubuntu. This is the guide I followed: https://www.linuxuprising.com/2018/04/install-oracle-java-10-in-ubuntu-or.html