Develop Reference

Reading encrypted private key in PKCS#8 format through bouncycastle, Java failing in docker container

I am trying to read a PKCS#8 private key which looks like following:
key.k8 --> (Sample key. Passphrase - 123456):
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIFLTBXBgkqhkiG9w0BBQ0wSjApBgkqhkiG9w0BBQwwHAQILbKY9hPxYSoCAggA
MAwGCCqGSIb3DQIJBQAwHQYJYIZIAWUDBAEqBBCvaGt2Hmm2NpHpxbLvHKyOBIIE
0IQ7dVrAGXLZl0exYIvyxLAu6zO00jL6b3sb/agTcCFOz8JU6fBanxY0d5aYO4Dn
mynQG7BoljU470s0zIwW/wk0MmdUFl4nXWBX/4qnG0sZqZ9KZ7I8R/WrBkmpX8C/
4pjdVhu8Ht8dfOYbkbjMBTohDJz8vJ0QwDIXi9yFjjef+QjwrFOl6kAeDJFVMGqc
s7K/wOnhsL1XxfW9uTulPiZh5YTZKcatMkeGDR7c+cg5I+Mutim92diWuCekhNoa
uvhUy1M3cbs7Azp1Mhz+V0CDKklI95EvN4u23WhiJPCjAofC/e45/heOP3Dwm7WZ
zHEY1C/X8PsTl6MEEIF3ZJP+4Vr0corAs1L2FqE6oOng8dFFYmF5eRyBx6bxFd05
iYbfOH24/b3qtFKPC689kGEd0gWp1dwES35SNNK+cJqVRTjgI0oKhOai3rhbGnmp
tx4+JqploQgTorj4w9asbtZ/qZA2mYSSR/Q64SHv7LfoUCI9bgx73MqRQBgvI5yS
b4BoFBnuEgOduZLaGKGjKVW3m5/q8oiDAaspcSLCJMIrdOTYWJB+7mfxX4Xy0vEe
5m2jXpSLQmrfjgpSTpHDKi/3b6OzKOcHjSFBf8IoiHuLc5DVvLECzDUxxaMrTZ71
0YXvEPwl2R9BzEANwwR9ghJvFg1Be/d5W/WA1Efe6cNQNBlmErxD6l+4KDUgGjTr
Aaksp9SZAv8uQAsg7C57NFHpTA5Hznr5JctL+WlO+Gk0cAV6i4Py3kA6EcfatsnS
PqP2KbxT+rb2ATMUZqgWc20QvDt6j0CTA1BuVD1PNhnAUFvb2ocyEEXOra22DPPS
UPu6jirSIyFcjqFjJ9A1FD9L4/UuX2UkDSLqblFlYB1+G55KZp+EKz8SZoN5qXy1
LyMtnacEP5OtRDrOjopzVNiuV1Uv63M9QVi1hZlVLJEomgjWuvuyEuIwDaY2uryW
vx+jJEZyySFkb1JwAbrm+p6sCTFnbQ/URKC2cit/FJyKqNim6VQvGL8Sez34qV3z
D13QJgTZfsy+BaZoaQ6cJTXtJ8cN0IcQciOiDNBKMW66zO6ujS8G+KNviNQypDm6
h4sOgjMqLaZ4ezPEdNj/gaxV7Y15nVRu0re8dVkaa5t9ft/sh6A+yeTD5tS5hHkf
NI7uJPTaTXVoz7xq2PAJUTWujMLMZKtmNOzNqYvxWRy3tCOFobBQkMxqEBEwHd+x
SA+gFcJKJ+aNfCGZJ5fFr8rNlhtOF6uMwOAlfiUlP/pCUDUCKPjZVj4K95yNc8Io
jSZSPb5tGPe0HqXgc6IAfQarlUZt90oVtzL0OfOfTxe1bEzS2ccNadbx/6vjLBc4
q5UuUBppl3rXpbuZ7J1Rp3n2byF4APxFdT2LHKq+MYMfWUToau/TCMT4lFIM9tM8
7TuuyUT2PKzf/xlsl4iScw96z9xxGPQrXn7IA2W5iL+0eCLztJdjNRX1FisdfIBL
PraOVlmF8jHKbFdRZ8Yi8pApbQjvHi24g7dX7u/cq1FH/VE+nJ0O8YVCYVDw13CW
h0p7yD7BuB0R+0WnR0yvkp30vK4/rtCB+Ob8bH/+HvAZrAU5X8jq/wsQbLkrLHZV
6A6GGfX8+hy5AoaXsH1BHnMyXkaF6Mv29z8JcslDJxX/
-----END ENCRYPTED PRIVATE KEY-----
Following code is being used to parse the private key:
InputStream privateKeyInputStream = getPrivateKeyInputStream(); // reads the key file from classpath and share as DataStream
logger.info("InputStreamExists --> {} ", privateKeyInputStream.available());
PEMParser pemParser = new PEMParser(new InputStreamReader(privateKeyInputStream));
Object pemObject = pemParser.readObject();
if (pemObject instanceof PKCS8EncryptedPrivateKeyInfo) {
// Handle the case where the private key is encrypted.
PKCS8EncryptedPrivateKeyInfo encryptedPrivateKeyInfo = (PKCS8EncryptedPrivateKeyInfo) pemObject;
InputDecryptorProvider pkcs8Prov =
new JceOpenSSLPKCS8DecryptorProviderBuilder().build(passphrase.toCharArray());
privateKeyInfo = encryptedPrivateKeyInfo.decryptPrivateKeyInfo(pkcs8Prov); // fails here
}
InputStream resourceAsStream = null;
if ("local".equals(privateKeyMode)) {
resourceAsStream = this.getClass().getResourceAsStream(privateKeyPath);
} else {
File keyFile = new File(privateKeyPath);
logger.info(
"Key file found in {} mode. FileName : {}, Exists : {}",
privateKeyMode,
keyFile.getName(),
keyFile.exists());
try {
resourceAsStream = new DataInputStream(new FileInputStream(keyFile));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
When I am running this code through intelliJ on windows, the code works fine but when I run it through docker container I am getting following exception:
org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: failed to construct sequence from byte[]: Extra data detected in stream
snowflake-report-sync | at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[bcpkix-jdk15on-1.64.jar!/:1.64.00.0]
snowflake-report-sync | at com.optum.snowflakereportsync.configuration.SnowFlakeConfig.getPrivateKey(SnowFlakeConfig.java:103) ~[classes!/:na]
snowflake-report-sync | at com.optum.snowflakereportsync.configuration.SnowFlakeConfig.getConnectionProperties(SnowFlakeConfig.java:67) ~[classes!/:na]
Following is Dockerfile used:
FROM adoptopenjdk/openjdk11-openj9:latest
COPY build/libs/snowflake-report-sync-*.jar snowflake-report-sync.jar
RUN mkdir /encryption-keys
COPY encryption-keys/ /encryption-keys/ #keys are picked from docker filesystem when running in container
EXPOSE 8080
CMD java -Dcom.sun.management.jmxremote -noverify ${JAVA_OPTS} -jar snowflake-report-sync.jar
Options tried:
Ensured that key file is being read while running in container. Logger "InputStreamExists --> {}" gives number of bytes
Ran dos2unix on key.k8 just to make sure there are no Window's "^M" characters which be could be causing issue as container is linux one : FROM adoptopenjdk/openjdk11-openj9:latest
Not sure what I am doing wrong but any help or pointers would be appreciated.

Like #Bragolgirith suspected, BouncyCastle seems to have problems with OpenJ9. I guess it is not a Docker issue, because I can reproduce it on GitHub Actions, too. It is also not limited to BouncyCastle 1.64 or 1.70, it happens in both versions. It also happens on OpenJ9 JDK 11, 14, 17 on Windows, MacOS and Linux, but for the same matrix of Java and OS versions it works on Adopt-Hotspot and Zulu.
Here is an example Maven project and a failed matrix build. So if you select another JVM type, you should be fine. I know that #Bragolgirith already suggested that, but I wanted to make the problem reproducible for everyone and also provide an MCVE, in case someone wants to open a BC or OpenJ9 issue.
P.S.: It is also not a character set issue with the InputStreamReader. This build fails exactly the same as before after I changed the constructor call.
Update: I have created BC-Java issue #1099. Let's see what the maintainers can say about this.
Update 2: The solution to your problem is to explicitly set the security provider to BC for your input decryptor provider. Thanks to David Hook for his helpful comment in #1099.
BouncyCastleProvider securityProvider = new BouncyCastleProvider();
Security.addProvider(securityProvider);
// (...)
InputDecryptorProvider pkcs8Prov = new JceOpenSSLPKCS8DecryptorProviderBuilder()
// Explicitly setting security provider helps to avoid ambiguities
// which otherwise can cause problems, e.g. on OpenJ9 JVMs
.setProvider(securityProvider)
.build(passphrase.toCharArray());
See this commit and the corresponding build, now passing on all platforms, Java versions and JVM types (including OpenJ9).
Because #Bragolgirith mentioned it in his answer: If you want to avoid the explicit new JceOpenSSLPKCS8DecryptorProviderBuilder().setProvider(securityProvider), the call Security.insertProviderAt(securityProvider, 1) instead of simply Security.addProvider(securityProvider) would in this case also solve the problem. But this holds true only as long as no other part of your code or any third-party library sets another provider to position 1 afterwards, as explained in the Javadoc. So maybe it is not a good idea to rely on that.

Edit:
On second thought, when creating the JceOpenSSLPKCS8DecryptorProviderBuilder, you're not explicitly specifying the provider:
new JceOpenSSLPKCS8DecryptorProviderBuilder()
.setProvider(BouncyCastleProvider.PROVIDER_NAME) // add this line
.build(passphrase.toCharArray());
It seems OpenJ9 uses a different provider/algo selection mechanism and selects the SunJCE's AESCipher class as CipherSpi by default, while Hotspot selects BouncyCastleProvider's AES class.
Explicitly specifying the provider should work in all cases.
Alternatively, when adding the BouncyCastleProvider you could insert it at the first preferred position (i.e. Security.insertProviderAt(new BouncyCastleProvider(), 1) instead of Security.addProvider(new BouncyCastleProvider())) so that it gets selected.
(It's still unclear to me why the provider selection mechanism differs between the different JVMs.)
Original post:
I've managed to reproduce the issue and at this point I'd say it's an incompatibility issue with the OpenJ9 JVM.
Starting from a Hotspot base image instead, e.g.
FROM adoptopenjdk:11-jre-hotspot
makes the code work.
(Not yet entirely sure whether the fault lies with the Docker image itself, the OpenJ9 JVM or BouncyCastle)

bouncycastle fails NTRU generation

There are three types of default security levels of NTRU, implemented in bouncy-castle:
1. NTRUSigningKeyGenerationParameters.TEST157
2. NTRUSigningKeyGenerationParameters.APR2011_439
3. NTRUSigningKeyGenerationParameters.APR2011_743
First two are generated normally, but when I try to generate the tird one, I get the next Exception:
SEVERE: Servlet.service() for servlet [mvc-dispatcher] in context with path [] threw exception [Request processing failed; nested exception is java.lang.IllegalStateException: Signing failed: too many retries (max=100)] with root cause
java.lang.IllegalStateException: Signing failed: too many retries (max=100)
Here is piece of my code:
NTRUSigningPrivateKeyParameters ntruSigningPrivateKeyParameters1 = null;
NTRUSigner ntruSigner = new NTRUSigner(ntruSigningKeyGenerationParameters.getSigningParameters());
try {
ntruSigningPrivateKeyParameters1 = new NTRUSigningPrivateKeyParameters(ntruSigningPrivateKeyParameters.getEncoded(), ntruSigningKeyGenerationParameters);
} catch (IOException e) {
e.printStackTrace();
}
ntruSigner.init(true, ntruSigningPrivateKeyParameters);
byte [] res = ntruSigner.generateSignature();
Calling ntruSigner.generateSignature() with the third set of parameters leads to a such Exception.
Does anyone knows how to solve it?

Currently, it's a bug, so there are two solutions:
use another library - tbuktu's github project (bouncy-castle is using it with some modifications, as I see)
download sources, catch the bug of this generation parameter, solve it and pack into library for a project

It's not really a bug in the code. The problem is that the norm bound in the APR2011_743 and APR2011_743_PROD parameter sets is too low which means that the signer is unable to generate a valid signature.
For N=743, q=2048 and beta=0.127 you should choose a norm bound of around 545 (see equation 10 in J. Hoffstein et al, Performance improvements and a baseline parameter generation algorithm for NTRUSign) but the parameter sets in BouncyCastle use normBound=405. Changing this solves the issue.

Updating the normBound does appear to fix the issue, however I should point out the NTRUSigner class is now deprecated in Bouncy Castle. The NTRU signing algorithm was shown to be badly broken just over a year ago. See:
http://www.di.ens.fr/~ducas/NTRUSign_Cryptanalysis/DucasNguyen_Learning.pdf
for details.

'java.lang.NoSuchFieldError: ignoreMethodBodies' Exception while creating AST

While passing a java source code to the below mentioned code, I get an exception and have no idea how to fix this. Kindlly let me know where I am wrong/ any alternative solution...
Source Code:
char[] source = resultEntryIndustry.text.toCharArray();
ASTParser parser = ASTParser.newParser(AST.JLS3);
parser.setSource(source);
Map options = JavaCore.getOptions();
JavaCore.setComplianceOptions(JavaCore.VERSION_1_5, options);
parser.setCompilerOptions(options);
CompilationUnit cUnit = (CompilationUnit) parser.createAST(null);
Exception thrown:
Exception in thread "main" java.lang.NoSuchFieldError: ignoreMethodBodies
at org.eclipse.jdt.core.dom.CompilationUnitResolver.parse(CompilationUnitResolver.java:491)
at org.eclipse.jdt.core.dom.ASTParser.internalCreateAST(ASTParser.java:1194)
at org.eclipse.jdt.core.dom.ASTParser.createAST(ASTParser.java:801)
at itjava.model.CompilationUnitStoreIndustry.createCompilationUnitFacadeList(CompilationUnitStoreIndustry.java:71)
at itjava.presenter.WordInfoPresenterIndustry.SetCompilationUnitListAndAccessRepository(WordInfoPresenterIndustry.java:162)
at itjava.industry.code2String.main(code2String.java:31)
ERROR: JDWP Unable to get JNI 1.2 environment, jvm->GetEnv() return code = -2
JDWP exit error AGENT_ERROR_NO_JNI_ENV(183): [../../../src/share/back/util.c:820]
The source code parsed is a simple java file used to connect to database using JDBC.

I was facing an almost similar problem and found the fix. Whatever environment you're running the code in, it has some component in its classpath which is not compatible with jdt core.

BouncyCastle does not find algorithms that it provides?

I'm playing around with BouncyCastle 1.46 To my surprise, the catch-block in the snippet below is tripped quite often.
Security.addProvider(new BouncyCastleProvider());
final Set<String> found = new HashSet<String>();
final Set<String> missing = new HashSet<String>();
final DefaultSignatureAlgorithmIdentifierFinder finder = new DefaultSignatureAlgorithmIdentifierFinder();
for (Service service : new BouncyCastleProvider().getServices()) {
if ("Signature".equals(service.getType())) {
final String algorithm = service.getAlgorithm();
try {
finder.find(algorithm);
found.add(algorithm);
} catch (IllegalArgumentException ex) {
missing.add(algorithm);
}
}
}
System.out.println("Found: " + found);
System.out.println("Missing: " + missing);
I appear to be unable to use most of the algorithms through the Finder, even though Services exist that provide those algorithms. What am I doing wrong?
Update I've changed the code a little to illustrate the issue better. What might be of interest is that I am using the JDK1.5 version of BouncyCastle. The code above gives this output:
Found: [RIPEMD256WithRSAEncryption, MD5WithRSAEncryption, MD2WithRSAEncryption, SHA384WithRSAEncryption, SHA224WITHECDSA, SHA384WITHDSA, SHA256WITHDSA, SHA512WithRSAEncryption, SHA512WITHDSA, RIPEMD160WithRSAEncryption, SHA224WithRSAEncryption, SHA256WITHECDSA, RIPEMD128WithRSAEncryption, SHA384WITHECDSA, SHA256WithRSAEncryption, SHA512WITHECDSA, SHA1WithRSAEncryption, SHA224WITHDSA]
Missing: [SHA1WITHECNR, NONEwithECDSA, ECDSA, SHA512withRSA/PSS, RIPEMD160WITHECDSA, RSA, GOST3410, SHA256WITHECNR, MD5withRSA/ISO9796-2, SHA1WITHCVC-ECDSA, SHA384withRSA/PSS, SHA1withRSA/PSS, MD4WithRSAEncryption, RSASSA-PSS, SHA512WITHECNR, SHA256WITHCVC-ECDSA, SHA1withRSA/ISO9796-2, SHA224withRSA/PSS, SHA224WITHCVC-ECDSA, RAWRSASSA-PSS, SHA256withRSA/PSS, NONEWITHDSA, SHA384WITHECNR, RIPEMD160withRSA/ISO9796-2, DSA, ECGOST3410, SHA224WITHECNR, 1.2.840.113549.1.1.10]

I think that DefaultSignatureAlgorithmIdentifierFinder is part of the bcmail API. It returns algorithm identifiers recognized by this API. (Check Cryptographic Message Syntax) On the other hand the bouncy castle provider provides more algorithms. You may check the source of DefaultSignatureAlgorithmIdentifierFinder where the recognized algorithms are hardcoded:
algorithms.put("MD2WITHRSAENCRYPTION", PKCSObjectIdentifiers.md2WithRSAEncryption);
algorithms.put("MD2WITHRSA", PKCSObjectIdentifiers.md2WithRSAEncryption);
algorithms.put("MD5WITHRSAENCRYPTION", PKCSObjectIdentifiers.md5WithRSAEncryption);
algorithms.put("MD5WITHRSA", PKCSObjectIdentifiers.md5WithRSAEncryption);
algorithms.put("SHA1WITHRSAENCRYPTION", PKCSObjectIdentifiers.sha1WithRSAEncryption);
algorithms.put("SHA1WITHRSA", PKCSObjectIdentifiers.sha1WithRSAEncryption);
algorithms.put("SHA224WITHRSAENCRYPTION", PKCSObjectIdentifiers.sha224WithRSAEncryption);
algorithms.put("SHA224WITHRSA", PKCSObjectIdentifiers.sha224WithRSAEncryption);
algorithms.put("SHA256WITHRSAENCRYPTION", PKCSObjectIdentifiers.sha256WithRSAEncryption);
algorithms.put("SHA256WITHRSA", PKCSObjectIdentifiers.sha256WithRSAEncryption);
algorithms.put("SHA384WITHRSAENCRYPTION", PKCSObjectIdentifiers.sha384WithRSAEncryption);
algorithms.put("SHA384WITHRSA", PKCSObjectIdentifiers.sha384WithRSAEncryption);
algorithms.put("SHA512WITHRSAENCRYPTION", PKCSObjectIdentifiers.sha512WithRSAEncryption);
algorithms.put("SHA512WITHRSA", PKCSObjectIdentifiers.sha512WithRSAEncryption);
algorithms.put("SHA1WITHRSAANDMGF1", PKCSObjectIdentifiers.id_RSASSA_PSS);
algorithms.put("SHA224WITHRSAANDMGF1", PKCSObjectIdentifiers.id_RSASSA_PSS);
algorithms.put("SHA256WITHRSAANDMGF1", PKCSObjectIdentifiers.id_RSASSA_PSS);
algorithms.put("SHA384WITHRSAANDMGF1", PKCSObjectIdentifiers.id_RSASSA_PSS);
algorithms.put("SHA512WITHRSAANDMGF1", PKCSObjectIdentifiers.id_RSASSA_PSS);
algorithms.put("RIPEMD160WITHRSAENCRYPTION", TeleTrusTObjectIdentifiers.rsaSignatureWithripemd160);
algorithms.put("RIPEMD160WITHRSA", TeleTrusTObjectIdentifiers.rsaSignatureWithripemd160);
algorithms.put("RIPEMD128WITHRSAENCRYPTION", TeleTrusTObjectIdentifiers.rsaSignatureWithripemd128);
algorithms.put("RIPEMD128WITHRSA", TeleTrusTObjectIdentifiers.rsaSignatureWithripemd128);
algorithms.put("RIPEMD256WITHRSAENCRYPTION", TeleTrusTObjectIdentifiers.rsaSignatureWithripemd256);
algorithms.put("RIPEMD256WITHRSA", TeleTrusTObjectIdentifiers.rsaSignatureWithripemd256);
algorithms.put("SHA1WITHDSA", X9ObjectIdentifiers.id_dsa_with_sha1);
algorithms.put("DSAWITHSHA1", X9ObjectIdentifiers.id_dsa_with_sha1);
algorithms.put("SHA224WITHDSA", NISTObjectIdentifiers.dsa_with_sha224);
algorithms.put("SHA256WITHDSA", NISTObjectIdentifiers.dsa_with_sha256);
algorithms.put("SHA384WITHDSA", NISTObjectIdentifiers.dsa_with_sha384);
algorithms.put("SHA512WITHDSA", NISTObjectIdentifiers.dsa_with_sha512);
algorithms.put("SHA1WITHECDSA", X9ObjectIdentifiers.ecdsa_with_SHA1);
algorithms.put("ECDSAWITHSHA1", X9ObjectIdentifiers.ecdsa_with_SHA1);
algorithms.put("SHA224WITHECDSA", X9ObjectIdentifiers.ecdsa_with_SHA224);
algorithms.put("SHA256WITHECDSA", X9ObjectIdentifiers.ecdsa_with_SHA256);
algorithms.put("SHA384WITHECDSA", X9ObjectIdentifiers.ecdsa_with_SHA384);
algorithms.put("SHA512WITHECDSA", X9ObjectIdentifiers.ecdsa_with_SHA512);
algorithms.put("GOST3411WITHGOST3410", CryptoProObjectIdentifiers.gostR3411_94_with_gostR3410_94);
algorithms.put("GOST3411WITHGOST3410-94", CryptoProObjectIdentifiers.gostR3411_94_with_gostR3410_94);
algorithms.put("GOST3411WITHECGOST3410", CryptoProObjectIdentifiers.gostR3411_94_with_gostR3410_2001);
algorithms.put("GOST3411WITHECGOST3410-2001", CryptoProObjectIdentifiers.gostR3411_94_with_gostR3410_2001);
algorithms.put("GOST3411WITHGOST3410-2001", CryptoProObjectIdentifiers.gostR3411_94_with_gostR3410_2001);
Cheers!

Did you add BouncyCastle to the security providers? You can do that with this line:
Security.addProvider(new BouncyCastleProvider());

This answer is not directly related to BouncyCastle. But, I thought this would be useful for others:
In my case, I was using SpongyCastle. I got a similar problem:
org.e.h.p: cannot create signer: Provider SC does not provide SHA256WITHRSA
at org.e.h.a.a.a(SourceFile:101)
It turned out that proguard was removing some of the required classes. After adding the following in proguard config file:
-keep class org.spongycastle.** { *; }
the problem was resolved.

You can add Bouncy Castle to security providers on Your java platform in two steps:
1. Copy BC librarys (currently bcpkix-jdk15on-149.jar, bcprov-jdk15on-149.jar) to directory $JAVA_HOME/jre/lib/ext/
2. Register BC provider: edit file $JAVA_HOME/jre/lib/security/java.security and under line
security.provider.1=sun.security.provider.Sun
add Your BC provider
security.provider.2=org.bouncycastle.jce.provider.BouncyCastleProvider
Change numbers of rest providers. The whole block of providers should be similar to:
security.provider.1=sun.security.provider.Sun
security.provider.2=org.bouncycastle.jce.provider.BouncyCastleProvider
security.provider.3=sun.security.rsa.SunRsaSign
security.provider.4=sun.security.ec.SunEC
security.provider.5=com.sun.net.ssl.internal.ssl.Provider
security.provider.6=com.sun.crypto.provider.SunJCE
security.provider.7=sun.security.jgss.SunProvider
security.provider.8=com.sun.security.sasl.Provider
security.provider.9=org.jcp.xml.dsig.internal.dom.XMLDSigRI
security.provider.10=sun.security.smartcardio.SunPCSC

Problem validating against an XSD with Java5

I'm trying to validate an Atom feed with Java 5 (JRE 1.5.0 update 11). The code I have works without problem in Java 6, but fails when running in Java 5 with a
org.xml.sax.SAXParseException: src-resolve: Cannot resolve the name 'xml:base' to a(n) 'attribute declaration' component.
I think I remember reading something about the version of Xerces bundled with Java 5 having some problems with some schemas, but i cant find the workaround. Is it a known problem ? Do I have some error in my code ?
public static void validate() throws SAXException, IOException {
List<Source> schemas = new ArrayList<Source>();
schemas.add(new StreamSource(AtomValidator.class.getResourceAsStream("/atom.xsd")));
schemas.add(new StreamSource(AtomValidator.class.getResourceAsStream("/dc.xsd")));
// Lookup a factory for the W3C XML Schema language
SchemaFactory factory = SchemaFactory.newInstance("http://www.w3.org/2001/XMLSchema");
// Compile the schemas.
Schema schema = factory.newSchema(schemas.toArray(new Source[schemas.size()]));
Validator validator = schema.newValidator();
// load the file to validate
Source source = new StreamSource(AtomValidator.class.getResourceAsStream("/sample-feed.xml"));
// check the document
validator.validate(source);
}
Update : I tried the method below, but I still have the same problem if I use Xerces 2.9.0. I also tried adding xml.xsd to the list of schemas (as xml:base is defined in xml.xsd) but this time I have
Exception in thread "main" org.xml.sax.SAXParseException: schema_reference.4: Failed to read schema document 'null', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
Update 2: I tried to configure a proxy with the VM arguments -Dhttp.proxyHost=<proxy.host.com> -Dhttp.proxyPort=8080 and now it works. I'll try to post a "real answer" from home.
and sorry, I cant reply as a comment : because of security reasons XHR is disabled from work ...

Indeed, people have been mentioning the Java 5 Sun provided SchemaFactory is giving troubles.
So: did you include Xerces in your project yourself?
After including Xerces, you need to ensure it is being used. If you like to hardcode it (well, as a minimal requirement you'd probably use some application properties file to enable and populate the following code):
String schemaFactoryProperty =
"javax.xml.validation.SchemaFactory:" + XMLConstants.W3C_XML_SCHEMA_NS_URI;
System.setProperty(schemaFactoryProperty,
"org.apache.xerces.jaxp.validation.XMLSchemaFactory");
SchemaFactory factory =
SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
Or, if you don't want to hardcode, or when your troublesome code would be in some 3rd party library that you cannot change, set it on the java command line or environment options. For example (on one line of course):
set JAVA_OPTS =
"-Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema
=org.apache.xerces.jaxp.validation.XMLSchemaFactory"
By the way: apart from the Sun included SchemaFactory implementation giving trouble (something like com.sun.org.apache.xerces.internal.jaxp.validation.xs.schemaFactoryImpl), it also seems that the "discovery" of non-JDK implementations fails in that version. If I understand correctly than, normally, just including Xerces would in fact make SchemaFactory#newInstance find that included library, and give it precedence over the Sun implementation. To my knowledge, that fails as well in Java 5, making the above configuration required.

I tried to configure a proxy with the VM arguments -Dhttp.proxyHost=<proxy.host.com> -Dhttp.proxyPort=8080 and now it works.
Ah, I didn't realize that xml.xsd is in fact the one referenced as http://www.w3.org/2001/xml.xsd or something like that. That should teach us to always show some XML and XSD fragments as well. ;-)
So, am I correct to assume that 1.) to fix the Java 5 issue, you still needed to include Xerces and set the system property, and that 2.) you did not have xml.xsd available locally?
Before you found your solution, did you happen to try using getResource rather than getResourceAsStream, to see if the exception would then have showed you some more details?
If you actually did have xml.xsd available (so: if getResource did in fact yield a URL) then I wonder what Xerces was trying to fetch from the internet then. Or maybe you did not add that schema to the list prior to adding your own schemas? The order is important: dependencies must be added first.
For whoever gets tot his question using the search: maybe using a custom EntityResolver could have indicated the source of the problem as well (if only writing something to the log and just returning null to tell Xerces to use the default behavior).

Hmmm, just read your "comment" -- editing does not alert people for new replies, so time to ask your boss for some iPhone or some other gadget that is connected to the net directly ;-)
Well, I assume you added:
schemas.add(
new StreamSource(AtomValidator.class.getResourceAsStream("/xml.xsd")));
If so, is xml.xsd actually to be found on the classpath then? I wonder if the getResourceAsStream did not yield null in your case, and how new StreamSource(null) would act then.
Even if getResourceAsStream did not yield null, the resulting StreamSource would still not know where it was loaded from, which may be a problem when trying to include references. So, what if you use the constructor StreamSource(String systemId) instead:
schemas.add(new StreamSource(AtomValidator.class.getResource("/atom.xsd")));
schemas.add(new StreamSource(AtomValidator.class.getResource("/dc.xsd")));
You might also use StreamSource(InputStream inputStream, String systemId), but I don't see any advantage over the above two lines. However, the documentation explains why passing the systemId in either of the 2 constructors seems good:
This constructor allows the systemID to be set in addition to the input stream, which allows relative URIs to be processed.
Likewise, setSystemId(String systemId) explains a bit:
The system identifier is optional if there is a byte stream or a character stream, but it is still useful to provide one, since the application can use it to resolve relative URIs and can include it in error messages and warnings (the parser will attempt to open a connection to the URI only if there is no byte stream or character stream specified).
If this doesn't work out, then maybe some custom error handler can give you more details:
ErrorHandlerImpl errorHandler = new ErrorHandlerImpl();
validator.setErrorHandler(errorHandler);
:
:
validator.validate(source);
if(errorHandler.hasErrors()){
LOG.error(errorHandler.getMessages());
throw new [..];
}
if(errorHandler.hasWarnings()){
LOG.warn(errorHandler.getMessages());
}
...using the following ErrorHandler to capture the validation errors and continue parsing as far as possible:
import org.xml.sax.helpers.DefaultHandler;
private class ErrorHandlerImpl extends DefaultHandler{
private String messages = "";
private boolean validationError = false;
private boolean validationWarning = false;
public void error(SAXParseException exception) throws SAXException{
messages += "Error: " + exception.getMessage() + "\n";
validationError = true;
}
public void fatalError(SAXParseException exception) throws SAXException{
messages += "Fatal: " + exception.getMessage();
validationError = true;
}
public void warning(SAXParseException exception) throws SAXException{
messages += "Warn: " + exception.getMessage();
validationWarning = true;
}
public boolean hasErrors(){
return validationError;
}
public boolean hasWarnings(){
return validationWarning;
}
public String getMessages(){
return messages;
}
}