Configure Privacy Request Webhooks

What is a Privacy Request webhook?

A privacy request webhook is an HTTPS callback that you've defined on an execution policy to trigger an external REST API endpoint either before or after a privacy request executes.

Webhooks can be one_way, where the API is pinged and the privacy request continues, or two_way, where Fides will wait for the webhook to respond before continuing.

A two_way webhook can return derived values which will be saved and used to locate other user information. For example, a webhook might take a known email to find a corresponding phone_number, i.e., a derived_identity.

Additionally, two_way webhooks can pause the request's execution by providing a halt flag in their response—allowing you to wait for other systems or complete other tasks before continuing.

Webhooks are also classified by their timing, both of which can be defined as one_way or two_way:

• PolicyPreWebhooks: Run before the privacy request
• PolicyPostWebhooks: Run after the privacy request

Both pre and post-execution webhooks run after the privacy requests have been approved. A pre-approval webhook (opens in a new tab) (PreApprovalWebhook) runs prior to approval, i.e., as soon as a privacy request is received.

Configuration

The process below defines an example https Connection with the details for an API endpoint, and then creates a PolicyPreWebhook or a PolicyPostWebhook for a specific execution policy using that Connection.

Create an HTTPS Connection

The information that describes how to connect to your API endpoint is represented by a Fides Connection.

[
    {
      "name": "My Webhook Connection Configuration",
      "key": "test_webhook_connection_config",
      "connection_type": "https",
      "access": "read"
    }
]

Add your Connection secrets

The credentials needed to access your API endpoint are defined by making a PUT to the Connection Secrets endpoint. These credentials are encrypted in the Fides app database.

    {
        "url": "https://www.example.com",
        "authorization": "test_authorization"
    }

Fides will use your authorization value to construct the Authorization header to call your API endpoint, for example, Authorization: test_authorization.

You can also prefix your auth value with Basic, Bearer, or another auth scheme.

The below secrets will result in Fides using an Authorization: Bearer mysecrettoken header:

    {
        "url": "https://www.example.com",
        "authorization": "Bearer mysecrettoken"
    }

Create pre-execution or post-execution webhooks

After you've defined a new Connection, you can create a list of webhooks to run before (PolicyPreWebhooks) or after (PolicyPostWebhooks) a privacy request is executed.

You must provide the policy key of your DSR policy when applying a webhook to that policy. You can retrieve a list of DSR policies and their policy keys by making this request:

GET /v1/dsr/policy

Next, you can update a list of webhooks for a given policy. In this example, we'll create pre-execution webhooks by adding a list of PolicyPreWebhooks.

PUT /v1/policy/{policy_key}/webhook/post_execution

When defining webhooks, they should be included in the request body in the desired order that you'd like them to execute. Any webhooks on the execution policy not included in the request will be removed from the policy.

[
    {
        "connection_config_key": "test_webhook_connection_config",
        "direction": "one_way",
        "key": "wake_up_snowflake_db",
        "name": "Wake up Snowflake DB Webhook"
    },
     {
        "connection_config_key": "test_webhook_connection_config",
        "direction": "two_way",
        "key": "prep_systems_webhook",
        "name": "Prep Systems Webhook"
    }
]

This request creates two webhooks that are run sequentially according to the order returned in the response body before a privacy request runs.

Similarly, to update your list of post-execution webhooks on a policy, use the following endpoint:

PUT /v1/policy/{policy_key}/webhook/post_execution

Update a single webhook

To update a single webhook, send a PATCH request to update selected attributes. Updates to order can likewise update the order of related webhooks.

The following example will update the PolicyPreWebhook with key wake_up_snowflake_db to be two_way instead of one_way, and will update its order from 0 to 1. Because we've defined two PolicyPreWebhooks, this causes the webhook at position 1 to move to position 0.

{
    "direction": "two_way",
    "order": 1
}

Because this PATCH request updated the order of other webhooks, a reordered summary is included under the new_order attribute:

{
    "resource": {
        "direction": "two_way",
        "key": "wake_up_snowflake_db",
        "name": "Wake up Snowflake DB Webhook",
        "connection_config": "<TRUNCATED>",
        "order": 1
    },
    "new_order": [
        {
            "key": "prep_systems_webhook",
            "order": 0
        },
        {
            "key": "wake_up_snowflake_db",
            "order": 1
        }
    ]
}

Similarly, to update a post-execution webhook on an execution policy, use the following endpoint:

PATCH /v1/policy/{policy_key}/webhook/post_execution/{post_execution_key}

Webhook request format

Before and after running access or erasure requests, Fides will send requests to any configured webhooks in sequential order with the following request body:

{
  "privacy_request_id": "pri_029832ba-3b84-40f7-8946-82aec6f95448",
  "direction": "one_way | two_way", 
  "callback_type": "pre | post", // a pre or post-execution webhook
  "identity": {
    "email": "customer-1@example.com",
    "phone_number": "+15555555555" 
  }
}

The identity attributes should exist in Fides before the webhook was created. In this example, the email field represents the user's email that was used to issue the privacy request. Other known identities for that user will also be embedded in the request (e.g., phone_number).

Webhook response format

Your webhook should respond immediately. If more processing time is needed, either make sure it is configured as a one-way webhook, or reply with halt=True if you want to pause execution and wait to finish processing.

Note that only pre-execution webhooks can pause execution.

Responses are not expected from one-way webhooks, but two-way webhooks should respond with the following:

{
  "derived_identity": {
    "email": "customer-1@gmail.com", // updated email address
    "phone_number": "+15555555555" // phone number returned in E.164 format  
  },
  "halt": "true | false" // set to true to pause execution; false to proceed
}

The example provides a derived_identity of derived identities, i.e., identities that are derived from the original identifier used for the privacy request. Values returned in the derived_identity will replace currently known identities — in this case, email and phone number — when fulfilling the privacy request. The derived_identity field is optional.

A two-way webhook will also respond with reply-to headers, allowing you to pause the request's execution while any additional processing takes place:

HTTP Headers 
  "reply-to": "/privacy-request/{privacy_request_id}/resume",
  "reply-to-token": "<jwe_token>"

Resuming request execution

Once a paused webhook finishes processing, send a request to the reply-to URL appended to your Fides domain (e.g. <your-fides-instance-domain>/privacy-request/{privacy_request_id}/resume) , along with the reply-to-token as the Bearer token in the Authorization header.

{
  "derived_identity": {
    "email": "customer-1@gmail.com",
    "phone_number": "+15555555555"
  }
}

If there are no derived identities, send an empty {} request body.

The reply-to-token is a JWE containing the current webhook ID, scopes to access the callback endpoint, and the datetime the token is issued. The token is valid for the duration defined in your privacy delay timeout, and will expire when your Redis cache expires, as represented by privacy_request_delay_timeout and default_ttl_seconds respectively in your Fides configuration file. Read how to set Fides configuration variables here.

When a request expires, it will be given an error status and require resubmission. Once the Redis cache expires, Fides no longer has the original identity data, and the privacy request should be resubmitted.