According to the Doppler AWS Sync docs, Doppler Secrets are synced to AWS Secrets “one-to-one”. This means one Doppler Config becomes one secret in AWS.
My company uses MWAA (AWS Airflow) extensively and we’d like to use Doppler as a single place for all of our secrets.
However, when you set up MWAA to use AWS Secrets Manager, it expects one single secret in AWS for each Airflow Connection. This means that with the current Doppler AWS Sync I’d have to create one config in Doppler per Airflow connection, which is not a good flow (an Airflow instance can easily have 20+ connections).
The other issue is that the Doppler AWS Sync appends /doppler at the end of the secret, which doesn’t integrate well with the expected MWAA flow (the name of the secret in AWS Secrets impacts the name of the connection inside Airflow).
Therefore, I’d like to know if it’s possible to add two new functionalities:
- Allow a single Doppler config to generate many secrets in AWS Secrets Manager. An example would be an AWS Secret for each key-value-pair inside a Doppler config. Ideally, I’d be able to group the KVPs according to my needs.
- Allow those secrets to have whatever names I want them to have (without appending anything).
With Open Souce Airflow you can technically add whatever secrets backend you want, but since this is hosted in AWS we’d prefer to stick with the internal MWAA <> AWS Secrets integration to save us some headache.
Ref doc for Secrets with MWAA: Configuring an Apache Airflow connection using a AWS Secrets Manager secret - Amazon Managed Workflows for Apache Airflow
Thank you, let me know if you need further clarification.
