Create a Dataset YAML from scratch

Dataset declaration

Datasets are an abstract identifier of your data store. For example, if you are working with structured tabular data, this is usually generated from a schema.

postgres_dataset.yml

dataset:
- fides_key: postgres_example_test_dataset
  organization_fides_key: default_organization
  tags: null
  name: Postgres Example Test Dataset
  description: Example of a Postgres dataset containing a variety of related tables
    like customers, products, addresses, etc.

Descriptions for dataset keys

Key	Value
`dataset`	Declares the type of resource to Fides, this can be an `organization`, `system`, `dataset` or `policy`. This guide focuses on `dataset`.
`fides_key`	The unique identifier for your dataset.
`organization_fides_key`	A unique identifier for organizations. Your organization's key can be found in the Organization tab.
`tags`	Optionally add a list of `string` tags to provide additional context or group resources.
`name`	The user-friendly label for the dataset presented in reports and Fides Admin UI.
`description`	A description of the dataset, useful for providing additional context in review/reporting.

Collections and fields

Collections are a container for fields. For example, if you are working with structured tabular data, containers are usually generated from tables, and fields are generated from columns.

postgres_dataset.yml

collections:
  - name: customer
    fields:
    - name: id
      data_categories: 
      - user.unique_id
      fides_meta:
        primary_key: True
    - name: name
      data_categories:
      - user.name
      fides_meta:
        data_type: string
        length: 40
    - name: email
      data_categories:
      - system.operations
    fides_meta:
      identity: email
      data_type: string

Descriptions for collection and field keys

Key	Value
`collections`	Declares a collection in a Fides dataset.
`collections.name`	The user-friendly label for the collection presented in reports and Fides Admin UI.
`fields`	Declares a list of fields to attach to the parent object.
`fields.name`	The user-friendly label for a field presented in reports and Fides Admin UI.
`data_categories`	The data category label taken from the FidesLang Taxonomy (opens in a new tab) to describe the personal data found in this field.
`fides_meta`	Fides metadata attributes, used for defining entity relationships between collections and tables for privacy requests.
`data_type`	Specify a required data type for type checking.
`length`	Where a data type may require length, set the string length.

Creating datasets via the CLI Data mapping annotations