Use Key Vault Secrets In Data Factory

⟶

Sometimes it’s useful to get your key vault secret in a Data Factory pipeline (in a secure way).

Key Vault Permissions

There are two permission models for Azure Key Vault. Depending on the permission model of your Key Vault, execute the following steps:

• Role-based access control model: in the “Access control (IAM)” tab, set the built-in role “Key Vault Secrets User” to your Data Factory to grant reading permissions on secret contents.
• Vault access policy model: in the “Access policies” tab, under “Secret permissions”, select “List” and “Get”, then select your Data Factory principal by filling in the Data Factory name in the “Select principal” section.

Data Factory Pipeline

Create a new pipeline called “Get Secret” and add a web activity with the following properties:

1. General
   • Secure output: check
2. Settings
   • URL (dynamic content): https://@{pipeline().parameters.keyvault_name}.vault.azure.net/secrets/@{pipeline().parameters.secret_name}/@{pipeline().parameters.secret_version}?api-version=@{pipeline().parameters.api_version}
   • Method: GET
   • Authentication: Managed Identity
   • Resource: https://vault.azure.net

Make sure that the parameters are added to the pipeline:

• keyvault_name: <blank>
• secret_name: <blank>
• secret_version: <blank> (this parameter is optional)
• api_version: 7.0

Running the Pipeline

Pass the name of your Key Vault, and the name of the secret you’d like to read.

When you inspect the activity output, it merely states:

{
    "SecureOutput": "**********"
}

To use the secret in your pipeline, enter the following in the dynamic content of your activity:

@{activity('Get Secret').output.value}

At the time of writing it is not possible to get the “Execute Pipeline” activity output information, except for the run id (which you can use to fetch the pipeline run information from the Azure API). Still, I like to have a full-fledged utility pipeline available that I can just copy and use whenever I need it, and run whenever I don’t understand it.