Develop Reference

Trouble connecting sam local api to secrets manager

I'm trying to set up AWS SAM locally so I don't have to deploy every time I make a code change. But I'm having trouble getting the secrets out of Secrets Manager. I've created a new SAM project using sam init --runtime java
I then created a new secret in Secret Manager, and changed the code in the HelloWorldFunction to try to retrieve the secret.
package helloworld;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.util.Base64;
import java.util.HashMap;
import java.util.Map;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.secretsmanager.AWSSecretsManager;
import com.amazonaws.services.secretsmanager.AWSSecretsManagerClientBuilder;
import com.amazonaws.services.secretsmanager.model.*;
/**
* Handler for requests to Lambda function.
*/
public class App implements RequestHandler<Object, Object> {
public Object handleRequest(final Object input, final Context context) {
Map<String, String> headers = new HashMap<>();
headers.put("Content-Type", "application/json");
headers.put("X-Custom-Header", "application/json");
try {
String secretName = "testsecret";
String region = "us-west-2";
// Create a Secrets Manager client
AWSSecretsManager client = AWSSecretsManagerClientBuilder.standard()
.withRegion(region)
.build();
String secret, decodedBinarySecret;
GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest()
.withSecretId(secretName);
GetSecretValueResult getSecretValueResult = null;
try {
getSecretValueResult = client.getSecretValue(getSecretValueRequest);
} catch (DecryptionFailureException e) {
// Secrets Manager can't decrypt the protected secret text using the provided KMS key.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (InternalServiceErrorException e) {
// An error occurred on the server side.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (InvalidParameterException e) {
// You provided an invalid value for a parameter.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (InvalidRequestException e) {
// You provided a parameter value that is not valid for the current state of the resource.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (ResourceNotFoundException e) {
System.out.println(e.getMessage());
StringWriter outError = new StringWriter();
e.printStackTrace(new PrintWriter(outError));
System.out.println(outError.toString());
// We can't find the resource that you asked for.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
}
// Decrypts secret using the associated KMS CMK.
// Depending on whether the secret is a string or binary, one of these fields will be populated.
if (getSecretValueResult.getSecretString() != null) {
secret = getSecretValueResult.getSecretString();
return new GatewayResponse(secret, headers, 200);
}
else {
decodedBinarySecret = new String(Base64.getDecoder().decode(getSecretValueResult.getSecretBinary()).array());
return new GatewayResponse(decodedBinarySecret, headers, 200);
}
} catch (Exception e) {
return new GatewayResponse("{}", headers, 500);
}
}
}
When I run sam local start-api and navigate to http://127.0.0.1:3000/hello, I get this error:
Secrets Manager can’t find the specified secret. (Service: AWSSecretsManager; Status Code: 400; Error Code: ResourceNotFoundException; Request ID: 6881467f-d968-4f4e-ae60-7e3128124cc5)
com.amazonaws.services.secretsmanager.model.ResourceNotFoundException: Secrets Manager can’t find the specified secret. (Service: AWSSecretsManager; Status Code: 400; Error Code: ResourceNotFoundException; Request ID: 6881467f-d968-4f4e-ae60-7e3128124cc5)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1632)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.doInvoke(AWSSecretsManagerClient.java:2024)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.invoke(AWSSecretsManagerClient.java:2000)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.executeGetSecretValue(AWSSecretsManagerClient.java:878)
at com.amazonaws.services.secretsmanager.AWSSecretsManagerClient.getSecretValue(AWSSecretsManagerClient.java:853)
at helloworld.App.handleRequest(App.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at lambdainternal.EventHandlerLoader$PojoMethodRequestHandler.handleRequest(EventHandlerLoader.java:259)
at lambdainternal.EventHandlerLoader$PojoHandlerAsStreamHandler.handleRequest(EventHandlerLoader.java:178)
at lambdainternal.EventHandlerLoader$2.call(EventHandlerLoader.java:888)
at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:293)
at lambdainternal.AWSLambda.<clinit>(AWSLambda.java:64)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at lambdainternal.LambdaRTEntry.main(LambdaRTEntry.java:114)
But, it's the same code to get the secret that was given in the secrets manager. Is it not possible to connect to real AWS services from sam local? I had a similar issue with DynamoDB, but was able to get it working with by using DynamoDB Local.
Any suggestions on how to either connect to the real secrets manager or to fake it locally somehow?

When you run DynamoDB Local, it is actually running a mock DDB server in a thread (or as a local process depending on how you start it) within you running test process. Unfortunately, Secrets Manager and other AWS services do not offer an equivalent testing solution.
However, if you are getting back ResourceNotFoundException it seems likely that you were able to successfully connect to Secrets Manager. It may be possible that the connection to secrets manager is using a different account than the one in which you stored the secret. One way you can check which credentials the code is using is to use the STS get caller identity call.

Mysterious ClasscastException from apache httpclient code

We are using Apache's commons-httpclient-3.1 library in our project. However, due to legacy and other project specific dependencies we are not using any of the implementations of the SecureProtocolSocketFactory interface provided by Apache (like SSLProtocolSocketFactory,StrictSSLProtocolSocketFactory) We have created our own SocketFactory called HttpClient3xSocketFactory which implements SecureProtocolSocketFactory interface.This factory is registered with Apache by calling the Protocol.registerProtocol method of org.apache.commons.httpclient.protocol.Protocol.java. Given below snippet of our code
public class HttpClient3xSocketFactory implements SecureProtocolSocketFactory {
. . .
private static final ProtocolSocketFactory socketFactory = new HttpClient3xSocketFactory();
private static final Protocol httpsProto = new Protocol("https",
socketFactory, 443);
And registered like below
Protocol.registerProtocol("https", httpsProto);
However this generates a mysterious ClasscastException from tunnelCreated() method of Apache's HttpConnection from the last line
if (usingSecureSocket) {
throw new IllegalStateException("Already using a secure socket");
}
if (LOG.isDebugEnabled()) {
LOG.debug("Secure tunnel to " + this.hostName + ":" + this.portNumber);
}
SecureProtocolSocketFactory socketFactory =
(SecureProtocolSocketFactory) protocolInUse.getSocketFactory();
The exception says --
Caused by: java.lang.ClassCastException: com.vmware.vdi.common.saml.HttpClient3xSocketFactory cannot be cast to org.apache.commons.httpclient.protocol.SecureProtocolSocketFactory
at org.apache.commons.httpclient.HttpConnection.tunnelCreated(HttpConnection.java:791)
at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(HttpMethodDirector.java:528)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:404)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:178)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:408)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:327)
But com.vmware.vdi.common.saml.HttpClient3xSocketFactory DOES implement org.apache.commons.httpclient.protocol.SecureProtocolSocketFactory interface. Have verified using GetClassLoader method that both are loaded using the same class loader. Also used reflection at run time to check that com.vmware.vdi.common.saml.HttpClient3xSocketFactory actually does implement org.apache.commons.httpclient.protocol.SecureProtocolSocketFactory interface. So I am at a loss to understand as to why the ClassCaseException is generated.

Using Eclipse To Connect To Network

I want to connect my Eclipse plug-in to an HTTPS URL, but have a problem because the user would need to accept the certificate. Of course there are a couple of tutorials for how to do this in plain Java, but that might be hard to do inside an Eclipse plug-in and I think I'd reinvent the wheel that way.
Because Eclipse has some built in tooling to connect to sites with different network protocols. An example would be the "Install new Software..." action. The tooling even has a preference page that lists HTTPS separately.
According to the Eclipse Help, the KeyStore is used "as a repository for Certificates used for trust decisions [...] when making SSL connections". Yet I couldn't figure out how to use it.
So my question is: How do I use the Eclipse's build in facilities to connect to my HTTPS site?

Based on this answer here I build my own plug-in which loads just the one certificate I need (lucky me) in its EarlyStartup:
public class EarlyStartup implements IStartup {
private static final String ALIAS = "ACME";
#Override
public void earlyStartup() {
final char[] passphrase = "changeit".toCharArray();
final char separator = File.separatorChar;
final File dir = new File(System.getProperty("java.home") + separator + "lib" + separator + "security");
final File file = new File(dir, "cacerts");
try (InputStream certIn = getClass().getResourceAsStream("acme.org.crt");
final InputStream localCertIn = new FileInputStream(file);) {
final KeyStore keystore = KeyStore.getInstance(KeyStore.getDefaultType());
keystore.load(localCertIn, passphrase);
if (keystore.containsAlias(ALIAS)) {
return;
}
final CertificateFactory cf = CertificateFactory.getInstance("X.509");
final Certificate cert = cf.generateCertificate(certIn);
keystore.setCertificateEntry(ALIAS, cert);
try (OutputStream out = new FileOutputStream(file)) {
keystore.store(out, passphrase);
}
} catch (final Exception e) {
e.printStackTrace();
}
}
}

xades4j.UnsupportedAlgorithmException: Unsupported transform on XML Signature provider

I'm trying to create a XAdES-BES signature for a given blob. For this signature, I'd need to add two transforms on the content before it is signed: Base64 (http://www.w3.org/2000/09/xmldsig#base64) & a custom one (called optional-deflate).
The problems lies with that optional transform. I'm trying to figure out how to implement a custom Transform, register it, and finally have Xades4J use it.
So far I figured a lot (thanks Google and a lot of time), so I got roughly to this: I've got a Provider-class that, in the constructor, puts the new TransformService; In my main code I add my Provider to the Security instance; then, I try to add the transform to my actual to-be-signed object.
Unfortunately, I always get the same error:
Exception in thread "main" xades4j.UnsupportedAlgorithmException: Unsupported transform on XML Signature provider (urn:xml:sig:transform:optional-deflate)
at xades4j.production.DataObjectDescsProcessor.processTransforms(DataObjectDescsProcessor.java:194)
at xades4j.production.DataObjectDescsProcessor.process(DataObjectDescsProcessor.java:87)
at xades4j.production.SignerBES.sign(SignerBES.java:173)
at xades4j.production.SignerBES.sign(SignerBES.java:122)
at com.mycompany.Test.createXades(Test.java:199)
at com.mycompany.Test.main(Test.java:47)
Caused by: org.apache.xml.security.transforms.TransformationException: Unknown transformation. No handler installed for URI urn:xml:sig:transform:optional-deflate
Original Exception was org.apache.xml.security.transforms.InvalidTransformException: Unknown transformation. No handler installed for URI urn:xml:sig:transform:optional-deflate
at org.apache.xml.security.transforms.Transforms.addTransform(Unknown Source)
at xades4j.production.DataObjectDescsProcessor.processTransforms(DataObjectDescsProcessor.java:185)
... 5 more
So, my code looks like this (abbreviated to what I think is necessary here):
TransformService class:
package com.mycompany.security;
import java.io.OutputStream;
import java.security.InvalidAlgorithmParameterException;
import java.security.spec.AlgorithmParameterSpec;
import javax.xml.crypto.Data;
import javax.xml.crypto.MarshalException;
import javax.xml.crypto.XMLCryptoContext;
import javax.xml.crypto.XMLStructure;
import javax.xml.crypto.dsig.TransformService;
import javax.xml.crypto.dsig.TransformException;
import javax.xml.crypto.dsig.spec.TransformParameterSpec;
public class OptionalDeflateTransform extends TransformService {
public AlgorithmParameterSpec getParameterSpec() {
return null;
}
public Data transform(Data data, XMLCryptoContext context) throws TransformException {
return null;
}
public Data transform(Data data, XMLCryptoContext context, OutputStream os) throws TransformException {
return null;
}
public boolean isFeatureSupported(String feature) {
return false;
}
public void init(TransformParameterSpec params) throws InvalidAlgorithmParameterException {}
public void marshalParams(XMLStructure parent, XMLCryptoContext context) throws MarshalException {}
public void init(XMLStructure parent, XMLCryptoContext context) throws InvalidAlgorithmParameterException {}
}
Provider subclass:
package com.mycompany.security;
import java.security.Provider;
public final class OptionalDeflateProvider extends Provider {
private static final long serialVersionUID = 8849833178389029123L;
public OptionalDeflateProvider() {
super("OptionalDeflate", 1.0, "OptionalDeflate provider 1.0 implementing the OptionalDeflate transform algorithm.");
put("TransformService.urn:xml:sig:transform:optional-deflate", "com.mycompany.security.OptionalDeflateTransform");
}
}
And finally, my main Test class, which contains the actual signing. Without that transform, it works (but well, doesn't add the transform, which is necessary). So Base64 works.
protected static void createXades(String content) throws Exception {
/*Get certificate & private key*/
Certificates c = new Certificates();
c.initSession(); //some helper class where I can get my certificate & private key for signing
/*Create a document*/
DocumentBuilderFactory docFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder docBuilder = docFactory.newDocumentBuilder();
Document doc = docBuilder.newDocument();
Element objectElement = doc.createElement("object");
doc.appendChild(objectElement);
Element requestElement = doc.createElement("request");
requestElement.appendChild(doc.createTextNode(content));
requestElement.setAttribute("ID", UUID.randomUUID().toString());
objectElement.appendChild(requestElement);
/*Key provider, signing profile & signer itself*/
KeyingDataProvider kp = new CustomKeyingDataProvider(c.getCertificate(), c.getPrivateKey());
XadesSigningProfile p = new XadesBesSigningProfile(kp);
p.withAlgorithmsProviderEx(new ProviderEx());
XadesSigner signer = p.newSigner();
/*Add the optional deflate provider*/
Security.addProvider(new OptionalDeflateProvider());
System.out.println("--- installed providers ---");
for (Provider pr : Security.getProviders())
System.out.println(pr.getName());
System.out.println("---");
/*Test if we can get the transformservice-instance*/
TransformService ts = TransformService.getInstance("urn:xml:sig:transform:optional-deflate", "DOM");
System.out.println(ts.getAlgorithm());
System.out.println("---");
/*Signed data*/
DataObjectDesc flatFile = new DataObjectReference("#" + requestElement.getAttribute("ID"))
.withTransform(new GenericAlgorithm("http://www.w3.org/2000/09/xmldsig#base64"))
.withTransform(new GenericAlgorithm("urn:xml:sig:transform:optional-deflate"));
SignedDataObjects dataObjs = new SignedDataObjects(flatFile);
/*Actual signing*/
signer.sign(dataObjs, objectElement);
log(objectElement.getLastChild());
}
As you can see I print some things out. I for instance logged that the installation works fine and I also logged the installed providers. I get this as output:
--- installed providers ---
SUN
SunRsaSign
SunEC
SunJSSE
SunJCE
SunJGSS
SunSASL
XMLDSig
SunPCSC
SunMSCAPI
OptionalDeflate
---
urn:xml:sig:transform:optional-deflate
---
So as far as I can see, the provider has succesfully been registered, the transformservice can be loaded without a problem, ... So I don't really see what's going on?
I've checked the source code of Xades4j as well, and what's happening internally on the line .withTransform(new GenericAlgorithm("urn:xml:sig:transform:optional-deflate")) is pretty straight forward:
import org.apache.xml.security.transforms.Transforms;
...
private Transforms processTransforms(DataObjectDesc dataObjDesc, Document document) throws UnsupportedAlgorithmException {
Collection<Algorithm> dObjTransfs = dataObjDesc.getTransforms();
if (dObjTransfs.isEmpty()) {
return null;
}
Transforms transforms = new Transforms(document);
for (Algorithm dObjTransf : dObjTransfs) {
try {
List<Node> transfParams = this.algorithmsParametersMarshaller.marshalParameters(dObjTransf, document);
if (null == transfParams) {
transforms.addTransform(dObjTransf.getUri());
} else {
transforms.addTransform(dObjTransf.getUri(), DOMHelper.nodeList(transfParams));
}
} catch (TransformationException ex) {
throw new UnsupportedAlgorithmException("Unsupported transform on XML Signature provider", dObjTransf.getUri(), ex);
}
}
return transforms;
}
The exact line throwing up the error is transforms.addTransform(dObjTransf.getUri()). This transforms object is a 'standard' apache object (org.apache.xml.security.transforms.Transforms object). So I'd guess it should be able to get the same TransformService as I do in code litteraly two lines up higher? But it isn't?
Anyone that can point me out what I'm missing? I'll be eternally grateful.

Apparently, Apache Santuario loads transforms from a internal map. There is a register
method that you probably can use to register your custom transform.

KeyManagementException: FIPS mode: only SunJSSE TrustManagers may be used

i use custom DummySocketFactory and DummyTrustMAnager to connect to smtp over TLS.
DummySocketFactory:
package XMailMessenger;
public class DummySSLSocketFactory extends SSLSocketFactory {
private SSLSocketFactory factory;
public DummySSLSocketFactory() {
try {
SSLContext sslcontext = SSLContext.getInstance("TLS");
//Security.removeProvider("SunJSSE");
sslcontext.init(null,
new TrustManager[] { new DummyTrustManager()},
null );
factory = (SSLSocketFactory)sslcontext.getSocketFactory();
} catch(Exception ex) {
System.out.println(ex.toString());
}
}
public static SocketFactory getDefault() {
SocketFactory a = new DummySSLSocketFactory();
if ( a == null ) { System.out.println("1"); }
return a;
}
...
DummyTrustManager:
public class DummyTrustManager implements X509TrustManager{
public void checkClientTrusted(X509Certificate[] cert, String authType) {
// everything is trusted
}
public void checkServerTrusted(X509Certificate[] cert, String authType) {
// everything is trusted
}
public X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[0];
//return null;
}
}
in sending e-mail i receive exception as in subject, this exception goes from function sslcontext.init in DummySSLSocketFactory. I debug it and noticed , that in code:
private X509TrustManager chooseTrustManager(TrustManager[] tm)
throws KeyManagementException {
// We only use the first instance of X509TrustManager passed to us.
for (int i = 0; tm != null && i < tm.length; i++) {
if (tm[i] instanceof X509TrustManager) {
if (SunJSSE.isFIPS() &&
!(tm[i] instanceof X509TrustManagerImpl)) {
throw new KeyManagementException
("FIPS mode: only SunJSSE TrustManagers may be used");
}
if (tm[i] instanceof X509ExtendedTrustManager) {
return (X509TrustManager)tm[i];
} else {
return new AbstractTrustManagerWrapper(
(X509TrustManager)tm[i]);
}
}
}
// nothing found, return a dummy X509TrustManager.
return DummyX509TrustManager.INSTANCE;
}
exception occures in if (SunJSSE.isFIPS() &&
!(tm[i] instanceof X509TrustManagerImpl)) expression.
I suppose that tm[i] contains my DummyTrustManager , it can not be extended from X509TrustManagerImpl so my question is : How to disable Fips in SunJSSE ?

SunJSSE can be configured to run on FIPS-140 compliant mode as long as it uses a FIPS-140 certified cryptographic hardware or software provider that implements all cryptographic algorithms required by JSSE (ex. Network Security Services – NSS, Sun Cryptographic Accelerator 6000, nCipher, etc).
To enable FIPS mode, edit the file ${java.home}/lib/security/java.security and modify the line that lists com.sun.net.ssl.internal.ssl.Provider and associate the name of the FIPS-140 cryptographic provider (ex. SunPKCS11-NSS). The name of the provider is a string that concatenates the prefix SunPKCS11- with the name of the specified PKCS#11 provider in its configuration file.
security.provider.4=com.sun.net.ssl.internal.ssl.Provider
SunPKCS11-NSS
In case of using NSS as cryptographic software token (Make use of NSS 3.1.1. or above), assuming the libraries are located under the /opt/nss/lib directory and its key database files (with the suffix .db) are under the /opt/nss/fipsdb directory, the sample configuration for representing NSS will be as follows:
# Use NSS as a FIPS-140 compliant cryptographic token
# SunPKCS11-NSS
name = NSS
nssLibraryDirectory = /opt/nss/lib
nssSecmodDirectory = /opt/nss/fipsdb
nssModule = fips
In FIPS mode, SunJSSE will perform SSL/TLS 1.0 based communication and cryptographic operations including symmetric and asymmetric encryption, signature generation and verification, message digests and message authentication codes, key generation and key derivation, random number generation, etc.

To anyone having a giant headache when you need to install a tomcat webapp on a third party server, I lost 1 hour trying to bypass this damn thing...
I solved in this way, without touching anything in the webapp.
Add this java parameter:
-Djava.security.disableSystemPropertiesFile=true
Source:
https://access.redhat.com/documentation/en-us/openjdk/8/pdf/configuring_openjdk_8_on_rhel_with_fips/OpenJDK-8-Configuring_OpenJDK_8_on_RHEL_with_FIPS-en-US.pdf
Also, if the app needs to connect to a Windows Server, you might want to disable FIPS there too:
In Control Panel, click Administrative Tools -> Local Security Policy.
In Security Settings -> Local Policies -> Security Options.
Under Policy in the right pane, double-click System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing, and then click Disabled.
Reboot the server
(bonus)
If you want to uninstall FIPS from the server, follow this giude (I didn't test it):
https://www.bggofurther.com/2021/02/disable-fips-mode-on-centos-7/