The idea is that user send me an api-key and I return him a name of his brand. The api key it's a GUID so I can't store it as a secret name.
I am looking for a way to retrieve a secret name by value from a keyvault. That's what my team came up with but it very slow and dramatically increases when adding new secrets to key vault.
public String getClientApiKeyName(final String apiKeyValue) {
for (SecretProperties secretProperties : secretClient.listPropertiesOfSecrets()) {
if (secretProperties.isEnabled()) {
final KeyVaultSecret secretWithValue = secretClient.getSecret(
secretProperties.getName(),
secretProperties.getVersion()
);
if (secretWithValue.getValue().equals(apiKeyValue)) {
return secretWithValue.getName();
}
}
}
return null;
}
Key Vault is designed for your own services secrets. If you are storing your customers' secrets (especially for high-throughput key storage scenarios), consider putting the keys in a database or storage account with encryption, and storing just the master key in Azure Key Vault.
Throttling
If you are seeing error code 429, you should the following are best practices you should implement:
Reduce the number of operations per request.
Reduce the frequency of requests.
Avoid immediate retries.
All requests accrue against your usage limits.
Backoff code example
SecretClientOptions options = new SecretClientOptions()
{
Retry =
{
Delay= TimeSpan.FromSeconds(2),
MaxDelay = TimeSpan.FromSeconds(16),
MaxRetries = 5,
Mode = RetryMode.Exponential
}
};
var client = new SecretClient(new Uri("https://keyVaultName.vault.azure.net"), new DefaultAzureCredential(),options);
//Retrieve Secret
secret = client.GetSecret(secretName);
https://learn.microsoft.com/en-us/azure/key-vault/general/overview-throttling
Related
well that´s the question, It´s possible to retrieve or refresh a password by secretmanagerid in AWS?, the password has been rotated by a policy every five minutes. I don´t want to restart my microservice to retrieve the pass, I was looking for a solution and I found something like this:
<groupId>com.amazonaws.secretsmanager</groupId>
<artifactId>aws-secretsmanager-jdbc</artifactId>
<version>1.0.5</version>
spring:
datasource:
url: jdbc-secretsmanager:mysql://database-host:3306/rotate_db
username: secret/rotation
driver-class-name: com.amazonaws.secretsmanager.sql.AWSSecretsManagerMySQLDriver
But I don't want to use the configuration in the application.yml or .properties, I want to keep these values in the parameter store as secrets, currently my code looks like this:
#Bean
public DataSource dataSource() {
AwsSecrets secrets = getSecret();
if(Objects.nonNull(secrets)){
log.info("Getting parameters: host: {}, port: {}, Db: {}, user: {}, pass: {}", secrets.getHost(), secrets.getPort(), secrets.getDatabase(),secrets.getUsername(), secrets.getPassword());
DataSource dataSource = DataSourceBuilder
.create()
.url("jdbc:postgresql://" + secrets.getHost() + ":" + secrets.getPort() + "/" + secrets.getDatabase())
.username(secrets.getUsername())
.password(secrets.getPassword())
.build();
return new TracingDataSource(dataSource);
}
log.debug("Unable to get secrets");
return null;
}
#Bean
public Filter tracingFilter() {
return new AWSXRayServletFilter("back-microservice");
}
private AwsSecrets getSecret() {
AWSSecretsManager client = AWSSecretsManagerClientBuilder.standard()
.withRegion(amazonRegion)
.withCredentials(dynamoDBConfig.accountAmazonAWSCredentials())
.build();
String secret;
GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest()
.withSecretId(secretmanagerId);
GetSecretValueResult getSecretValueResult = null;
try {
getSecretValueResult = client.getSecretValue(getSecretValueRequest);
} catch (Exception e) {
log.debug("Unable to get secrets values");
throw e;
}
if (getSecretValueResult.getSecretString() != null) {
secret = getSecretValueResult.getSecretString();
return gson.fromJson(secret, AwsSecrets.class);
}
return null;
}
Yes, it's possible to do. The exact implementation will vary by framework/ORM, but you would need to cache the credentials, but also check if a new password is needed every time a new connection is acquired.
In your code, you are setting the password once statically:
DataSource dataSource = DataSourceBuilder
.create()
.url("jdbc:postgresql://" + secrets.getHost() + ":" + secrets.getPort() + "/" + secrets.getDatabase())
.username(secrets.getUsername())
.password(secrets.getPassword())
.build();
The getPassword() function is only ever called once when you build the datasource and is reused for additional connections from that data source. Instead, you need to retrieve (from cache/secretsmanger) every time a connection is created.
AWS provides a java caching client for AWS secretsmanager (clients for other languages are also available). You can adapt that into your data source to use that to retrieve the password for every connection. You can read the official guidance documentation on that here: Rotate database credentials without restarting containers.
In the context of Spring, that means implementing this pattern in your data source driver, which is exactly what the com.amazonaws.secretsmanager.sql.AWSSecretsManagerMySQLDriver class provides. Because you are rotating the secret so often, you probably want to configure the secret cache to refresh more often than the default of 1 hour or adjust your rotation frequency.
If you really need to be rotating the password that often and you're using RDS for your database, you might consider just using IAM-based authentication instead. I can't imagine why you would want to have a password that rotates so frequently, keeping in mind you are billed for secret retrieval API calls.
I have a problem for a backend solution developpement. I need someone to guide me.
I'm developping a backend solution with SpringBoot.
My backend should :
Connect to an Active Directory
Use a token system for the security part
Give special access for each request (based on AD group)
After a lot of tests and research, I already have this :
Connect to an Active Directory
For the connection to AD part, I did this :
Inside a class that extends WebSecurityConfigurerAdapter
#Override
public void configure(AuthenticationManagerBuilder auth) {
ActiveDirectoryLdapAuthenticationProvider adProvider
= new ActiveDirectoryLdapAuthenticationProvider("domain.com", "ldap", "ou,dc");
adProvider.setConvertSubErrorCodesToExceptions(true);
adProvider.setUseAuthenticationRequestCredentials(true);
adProvider.setUserDetailsContextMapper(userDetailsContextMapper());
auth.authenticationProvider(adProvider);
}
userDetailsContextMapper() simply maps the info to an user class that contains the username, the lastname and the list memberOf of the user (AD groups).
Using token
I use this method for the token. It works too.
public SignedJWT getToken(String username) {
JWSSigner signer = null;
SecureRandom random = new SecureRandom();
byte[] sharedSecret = new byte[32];
random.nextBytes(sharedSecret);
signer = new MACSigner(sharedSecret);
JWTClaimsSet claimsSet = new JWTClaimsSet.Builder()
.subject(username)
.issuer("domain")
.expirationTime(new Date(new Date().getTime() + 60 * 1000))
.build();
SignedJWT signedJWT = new SignedJWT(new JWSHeader(JWSAlgorithm.HS256), claimsSet);
signedJWT.sign(signer);
String serializedToken = signedJWT.serialize();
return signedJWT;
}
Access per request
For the third part I want to give access to a specific request only for a given AD group.
Imagine I have two correctly mapped request :
domain.com/sales/2022/all : Return all the sales values for the year 2022
domain.com/IT/inventory/all: Return all the IT inventory
I want only the people that are from the group IT that can get a result from the IT's request and for the sale's request, only someone who is a member of the groupe sales.
But first of all, I want only people who are authenticated and have a valid token who can call a request.
I searched and tested different solutions on the internet but I did not find anything compatible with my two previous method.
Can you help me ? I'm not specially looking for a solution, a link to a good documentation or some example are enough for me.
I was wondering what the best way is for me to refresh an access token that is obtained through the client credentials flow within OAuth 2.0. I've read over the spec, but I can't seem to be able to find an answer for my particular situation.
For my specific case, I am using the Spotify Web API for my Android app in order to search for content (tracks, albums, and/or artists). In order to perform a search, I need an access token. Since I'm not interested in a Spotify user's data, I can use the client credentials flow to obtain the access token, which is explain in Spotify's terms here.
Because the access token can eventually expire, I had to figure out a way to refresh it once expiration occurred. What I'm ultimately wondering is if my approach is valid and if there's any concern with how I've approached this.
First and foremost, I stored my access token within SharedPreferences. Within onCreate(), I have the following:
#Override
protected void onCreate(Bundle savedInstanceState) {
// A bunch of other stuff, views being initialized, etc.
mAccessToken = getAccessToken();
// If the access token is expired, then we will attempt to retrieve a new one
if (accessTokenExpired()) {
retrieveAccessToken();
}
}
I've defined accessTokenExpired() and retrieveAccessToken() as follows:
private boolean accessTokenExpired() {
// If mAccessToken hasn't yet been initialized, that means that we need to try to retrieve
// an access token. In this case, we will return true;
if (mAccessToken == null) {
return true;
}
SharedPreferences preferences = getPreferences(Context.MODE_PRIVATE);
long timeSaved = preferences.getLong(PREFERENCES_KEY_TOKEN_RESPONSE_TIME_SAVED, 0L);
long expiration = preferences.getLong(PREFERENCES_KEY_TOKEN_RESPONSE_EXPIRATION, 0L);
long now = System.currentTimeMillis()/1000;
long timePassed = Math.abs(now - timeSaved);
if (timePassed >= expiration) {
return true;
} else {
return false;
}
}
One thing worth noting about retrieveAccessToken() is that I'm using Retrofit for my HTTP request:
private void retrieveAccessToken() {
// First, we obtain an instance of SearchClient through our ClientGenerator class
mClient = ClientGenerator.createClient(SearchClient.class);
// We then obtain the client ID and client secret encoded in Base64.
String encodedString = encodeClientIDAndSecret();
// Finally, we initiate the HTTP request and hope to get the access token as a response
Call<TokenResponse> tokenResponseCall = mClient.getAccessToken(encodedString, "client_credentials");
tokenResponseCall.enqueue(new Callback<TokenResponse>() {
#Override
public void onResponse(Call<TokenResponse> call, Response<TokenResponse> response) {
Log.d(TAG, "on Response: response toString(): " + response.toString());
TokenResponse tokenResponse = null;
if (response.isSuccessful()) {
tokenResponse = response.body();
Log.d(TAG, tokenResponse.toString());
mAccessToken = tokenResponse.getAccessToken();
saveAccessToken(tokenResponse);
}
}
#Override
public void onFailure(Call<TokenResponse> call, Throwable t) {
Log.d(TAG, "onFailure: request toString():" + call.request().toString());
mAccessToken = "";
}
});
}
Finally, saveAccessToken(tokenResponse) is sort of the complement of accessTokenExpired(), where I'm saving the values from the token response into SharedPreferences rather than retrieving them.
Are there any concerns with how I'm doing this? I got the idea from this SO post and slightly modified it. Spotify doesn't provide a refresh token in their access token response. Therefore, I can't make use of it here to reduce the number of access token requests I make.
Any input on this would be greatly appreciated!
Two considerations are:
you probably want some error handling around the requests you make using the access token that can handle the token expiring and do retries. The two situations where this will help are
when the token expires between checking if it's valid and your usage of it
when in the cycle of check the token is valid -> make some requests with the token -> repeat, you spend over an hour using the token. Another way you can do it is to calculate now + expected_api_request_time > token_expiration_time where expected_api_request_time would be a constant you set, but I think handling token expiry as an exception is better practice (you probably want to be able to make retries anyway in cases of network instability).
you can perform the calculations to work out when the token expires either when you retrieve the timeSaved and expiration from your local storage, or just calculate the time the token will expire initially and save that. This is relatively minor, both this and the way you've done it are fine I think.
I am looking for some help on how to add an attachment on CosmosDB. Here is the little background.
Our application is currently on IBM Bluemix and we are using CloudantDB. We use CloudanDB to store attachments (PDF file). We are no moving to Azure PaaS App Service and planning to use CosmosDB. I am looking for help on how to create an attachment on CosmosDB using Java API. What API do I need to use? I want to do a small POC.
Thanks,
Well Personally i feel In Azure, if you go want to put files into documentDb, you will pay high for the query cost. Instead it would be normal practice to use Azure blob and save the link in a field, and then return url if its public or binary data if you want it to be secured.
However, You could store it using
var myDoc = new { id = "42", Name = "Max", City="Aberdeen" }; // this is the document you are trying to save
var attachmentStream = File.OpenRead("c:/Path/To/File.pdf"); // this is the document stream you are attaching
var client = await GetClientAsync();
var createUrl = UriFactory.CreateDocumentCollectionUri(DatabaseName, CollectionName);
Document document = await client.CreateDocumentAsync(createUrl, myDoc);
await client.CreateAttachmentAsync(document.SelfLink, attachmentStream, new MediaOptions()
{
ContentType = "application/pdf", // your application type
Slug = "78", // this is actually attachment ID
});
WORKING WITH ATTACHMENTS
I have answered a similar question here
What client API I can use?
You could follow the cosmos db java sdk to CRUD attachment.
import com.microsoft.azure.documentdb.*;
import java.util.UUID;
public class CreateAttachment {
// Replace with your DocumentDB end point and master key.
private static final String END_POINT = "***";
private static final String MASTER_KEY = "***";
public static void main(String[] args) throws Exception, DocumentClientException {
DocumentClient documentClient = new DocumentClient(END_POINT,
MASTER_KEY, ConnectionPolicy.GetDefault(),
ConsistencyLevel.Session);
String uuid = UUID.randomUUID().toString();
Attachment attachment = getAttachmentDefinition(uuid, "application/text");
RequestOptions options = new RequestOptions();
ResourceResponse<Attachment> attachmentResourceResponse = documentClient.createAttachment(getDocumentLink(), attachment, options);
}
private static Attachment getAttachmentDefinition(String uuid, String type) {
return new Attachment(String.format(
"{" +
" 'id': '%s'," +
" 'media': 'http://xstore.'," +
" 'MediaType': 'Book'," +
" 'Author': 'My Book Author'," +
" 'Title': 'My Book Title'," +
" 'contentType': '%s'" +
"}", uuid, type));
}
}
In the documentation it says, total file size we can store is 2GB.
"Azure Cosmos DB allows you to store binary blobs/media either with
Azure Cosmos DB (maximum of 2 GB per account) " Is it the max we can
store?
Yes.The size of attachments is limited in document db. However, there are two methods for creating a Azure Cosmos DB Document Attachment.
1.Store the file as an attachment to a Document
The raw attachment is included as the body of the POST.
Two headers must be set:
Slug – The name of the attachment.
contentType – Set to the MIME type of the attachment.
2.Store the URL for the file in an attachment to a Document
The body for the POST include the following.
id – It is the unique name that identifies the attachment, i.e. no two attachments will share the same id. The id must not exceed 255 characters.
Media – This is the URL link or file path where the attachment resides.
The following is an example
{
"id": "device\A234",
"contentType": "application/x-zip-compressed",
"media": "www.bing.com/A234.zip"
}
If your files are over limitation , you could try to store them with second way. More details, please refer to blog.
In addition, you could notice that cosmos db attachments support
garbage collect mechanism,it ensures to garbage collect the media when all of the outstanding references are dropped.
Hope it helps you.
I use shiro in application for the authenticate. I use hashed password with a salt and I store them in my database like this :
private User createUserWithHashedPassword(String inName, String inFirstName, String inLastName, String inPassword){
ByteSource salt = randomNumberGenerator.nextBytes(32);
byte[] byteTabSalt = salt.getBytes();
String strSalt = byteArrayToHexString(byteTabSalt);
String hashedPasswordBase64 = new Sha256Hash(inPassword, salt, 1024).toBase64();
return new User(inName,inFirstName,inLastName,hashedPasswordBase64,strSalt);
}
I store the salt with a String in my database. Now in my realm I want to get back my datas from the database, I use a transactionnal service for this. But my salt is a Strong so I want it to turn back as ByteSource type with the static method :
ByteSource byteSourceSalt = Util.bytes(salt); //where the salt is a String
But when I create my SaltedAuthenticationInfo it doesn't auth.
I think my problem is from my convert method :
private String byteArrayToHexString(byte[] bArray){
StringBuffer buffer = new StringBuffer();
for(byte b : bArray) {
buffer.append(Integer.toHexString(b));
buffer.append(" ");
}
return buffer.toString().toUpperCase();
}
Thanks for your help.
As mentioned in the excellent answer https://stackoverflow.com/a/20206115/603901, Shiro's DefaultPasswordService already generates unique salts for each password.
However, there is no need to implement a custom PasswordService to add a private salt (sometimes called "pepper") to the per-user salts. Private salt can be configured in shiro.ini:
[main]
hashService = org.apache.shiro.crypto.hash.DefaultHashService
hashService.hashIterations = 500000
hashService.hashAlgorithmName = SHA-256
hashService.generatePublicSalt = true
# privateSalt needs to be base64-encoded in shiro.ini but not in the Java code
hashService.privateSalt = myVERYSECRETBase64EncodedSalt
passwordMatcher = org.apache.shiro.authc.credential.PasswordMatcher
passwordService = org.apache.shiro.authc.credential.DefaultPasswordService
passwordService.hashService = $hashService
passwordMatcher.passwordService = $passwordService
Java code for generating a matching password hash:
DefaultHashService hashService = new DefaultHashService();
hashService.setHashIterations(HASH_ITERATIONS); // 500000
hashService.setHashAlgorithmName(Sha256Hash.ALGORITHM_NAME);
hashService.setPrivateSalt(new SimpleByteSource(PRIVATE_SALT)); // Same salt as in shiro.ini, but NOT base64-encoded.
hashService.setGeneratePublicSalt(true);
DefaultPasswordService passwordService = new DefaultPasswordService();
passwordService.setHashService(hashService);
String encryptedPassword = passwordService.encryptPassword("PasswordForThisUser");
The resulting hash looks like this:
$shiro1$SHA-256$500000$An4HRyqMJlZ58utACtyGDQ==$nKbIY9Nd9vC89G4SjdnDfka49mZiesjWgDsO/4Ly4Qs=
The private salt is not stored in the database, which makes it harder to crack the passwords if an adversary gains access to a database dump.
This example was created using shiro-1.2.2
Thanks to https://github.com/Multifarious/shiro-jdbi-realm/blob/master/src/test/resources/shiro.ini for help with the syntax for shiro.ini
Have you looked at PasswordMatcher / PasswordService?
This already has all of the encoding/decoding/compare logic built-in. To use it:
Storing password in database:
PasswordService service = new DefaultPasswordService(); // or use injection or shiro.ini to populate this
private User createUserWithHashedPassword(String inName, String inFirstName, String inLastName, String inPassword){
String hashedPasswordBase64 = service.encryptPassword(inPassword);
return new User(inName,inFirstName,inLastName,hashedPasswordBase64,strSalt);
}
Then you can simply use PasswordMatcher as the matcher in your realm.
realm.setCredentialsMatcher(new PasswordMatcher());
or in shiro.ini:
matcher = org.apache.shiro.authc.credential.PasswordMatcher
realm.credentialsMatcher = $matcher
The DefaultPasswordService implementation automatically adds a random salt to each encryptPassword call. That "public" salt will be stored within the "hashedPasswordBase64" that you receive from "encryptPassword".
Because the "public" salt is individually generated for each hashed password one cannot "simply" generate a rainbow table and brute-force all your hashed passwords at once. For each hashed password the attacker would have to generate an own, unique rainbow table because of the unique "public" salt. So far you do not need to put an extra salt into the database.
To make your stored hashed passwords even more secure you can furthermore add a "private" salt that should be stored anywhere else - as long as not in the database. By using a "private" salt you could protect the hashed passwords against a brute-force rainbow-table attack, because the attacker does not know the "private" salt and cannot gain the "private" salt from the database entries.
This is a very basic example how to create a PasswordService that utilizes a "private" salt provided as a constant string and that works as CredentialsMatcher:
public class MyPrivateSaltingPasswortService extends DefaultPasswordService
{
public MyPrivateSaltingPasswortService()
{
super();
HashService service = getHashService();
if (service instanceof DefaultHashService)
{
((DefaultHashService) service).setPrivateSalt(
new SimpleByteSource("MySuperSecretPrivateSalt"));
}
}
}
you then could use your own implementation in shiro.ini:
[main]
saltedService = com.mycompany.MyPrivateSaltingPasswortService
matcher = org.apache.shiro.authc.credential.PasswordMatcher
matcher.passwordService = $saltedService
realm.credentialsMatcher = $matcher
This example was created using shiro-1.2.2
I change my type for the save of my salt. Now I'm using a byte[] instead of a String.
ByteSource salt = randomNumberGenerator.nextBytes(32);
byte[] byteTabSalt = salt.getBytes();
And I stock the byteTabSalt in my database.