Skip to content
Storage destinations

Configure Storage Destinations

What is a storage destination?

Access requests produce a package of returned data upon completion. This data will need to be uploaded to a storage destination (e.g. an S3 bucket) in order to be returned to the user.

Fides never stores privacy request results locally. At least one storage destination must be configured if you wish to process access requests.

Storage destinations are associated to privacy request policies in their rules, allowing multiple storage destinations to be configured per privacy request policy.

Storage Destinations

Create a storage destination

Configure your storage method

To configure a Storage destination, first choose a method to store your results. Fides currently supports the following methods of storage:

  • local - This saves upload packages locally, generating a fides_uploads directory at the root of your project. This destination type should only be used for testing purposes, and not to process real-world access requests.
  • S3 - Files are uploaded to an S3 bucket of your choosing upon completion of an access request. Use S3 if you need a place to store those files.

Create your storage destination

Storage destinations are created and managed via the API. To create a new Storage destination, use the following endpoint:

    "destinations": [
        "name": str,
        "key": FidesKey (optional),
        "type": str,
        "format": str
        "details": {
          # s3
          "auth_method": str,
          "bucket": str,
          "naming": str,

Destination attributes

nameA unique user-friendly name for your storage destination.
keyA unique key used to manage your storage destination. This is auto-generated from name if left blank. Accepted values are alphanumeric, _, and ..
typeType of storage destination. Supported types include s3, and local. You may configure multiple destinations of the same type.
formatThe format of uploaded data. Supported formats include json and csv.

Additional attributes for s3 buckets

auth_methodThe authentication method for creating a session with S3. Either automatic or secret_keys.
bucketThe name of the bucket in S3.
namingThis defines how the uploaded files will be named. Currently, Fides only supports upload file naming by request_id. Use this value for all your storage destinations.

Additional attributes for local storage

namingThis defines how the uploaded files will be named. Currently, Fides supports upload file naming by request_id. Use this value for all your storage destinations.

On success, the response from the above endpoint will include a storage_key for each destination, which can be used when defining privacy request policy rules.

Example response
    "items": [
            "id": "sto_fe4e4dc0-b5d3-4ac1-bfcd-86e60e9891b9",
            "name": "s3 storage 2",
            "type": "s3",
            "details": {
                "auth_method": "secret_keys",
                "bucket": "my-bucket",
                "naming": "request_id",
                "object_name": "requests"
            "key": "s3_storage_2"
    "total": 1,
    "page": 1,
    "size": 1

Authenticate with your destination

Fides requires authenticated access to update and erase/mask data in your storage destination.

Use storage_key returned during your storage creation to provide access credentials:

PUT {host}/api/v1/storage/config/{storage_key}/secret
    # s3
    "aws_access_key_id": str,
    "aws_secret_access_key": str

Additional attributes for S3 buckets

Fides supports automatically creating a session for S3. If your auth_method is set to automatic, no secrets need to be provided. Boto3 will look for credentials on the server.
aws_access_key_idAWS access key id, obtained from AWS console.
aws_secret_access_keyAWS secret access key, obtained from AWS console.

Secrets are not saved if credentials fail authentication with the given storage destination.

Test your storage connection

To test that your storage destination works correctly, you can call the upload endpoint directly. Specify a request_id in the path with an arbitrary string:

PUT {host}/api/v1/storage/{request_id}
    "storage_key": {storage_key},
    "data": {
      #data here
storage_keyThe key associated with the storage destination.
dataA dictionary of arbitrary data you wish to upload to storage destination.


Fides can be extended to support additional storage destinations by:

  1. Adding destination-specific enums in src/fides/ops/schemas/storage/
  2. Implementing an authenticator in src/fides/ops/service/storage/
  3. Implementing the uploader in src/fides/ops/service/storage/