Using defer in dbt Cloud

Defer is a powerful feature that allows developers to only build and run and test models they've edited without having to first run and build all the models that come before them (upstream parents). dbt powers this by using a production manifest for comparison, and resolves the {{ ref() }} function with upstream production artifacts.

Both the dbt Cloud IDE and the dbt Cloud CLI enable users to natively defer to production metadata directly in their development workflows.

Use 'defer' to modify end-of-pipeline models by pointing to production models, instead of running everything upstream.

By default, dbt follows these rules:

• dbt uses the production locations of parent models to resolve {{ ref() }} functions, based on metadata from the production environment.
• If a development version of a deferred model exists, dbt preferentially uses the development database location when resolving the reference.
• Passing the --favor-state flag overrides the default behavior and always resolve refs using production metadata, regardless of the presence of a development relation.

For a clean slate, it's a good practice to drop the development schema at the start and end of your development cycle.

If you require additional controls over production data, create a Staging environment and dbt will use that, rather than the Production environment, to resolve {{ ref() }} functions.

Required setup

• You must select the Production environment checkbox in the Environment Settings page.
  • This can be set for one deployment environment per dbt Cloud project.
• You must have a successful job run first.

When using defer, it compares artifacts from the most recent successful production job, excluding CI jobs.

Defer in the dbt Cloud IDE

To enable defer in the dbt Cloud IDE, toggle the Defer to production button on the command bar. Once enabled, dbt Cloud will:

1. Pull down the most recent manifest from the Production environment for comparison
2. Pass the --defer flag to the command (for any command that accepts the flag)

For example, if you were to start developing on a new branch with nothing in your development schema, edit a single model, and run dbt build -s state:modified — only the edited model would run. Any {{ ref() }} functions will point to the production location of the referenced models.

Note: The Defer to staging/production toggle is currently not available when running dbt Semantic Layer commands in the dbt Cloud IDE. To use defer for Semantic layer commands in the IDE, toggle the button on and manually add the --defer flag to the command. This is a temporary workaround and will be available soon.

Select the 'Defer to production' toggle on the bottom right of the command bar to enable defer in the dbt Cloud IDE.

Defer in dbt Cloud CLI

One key difference between using --defer in the dbt Cloud CLI and the dbt Cloud IDE is that --defer is automatically enabled in the dbt Cloud CLI for all invocations, compared with production artifacts. You can disable it with the --no-defer flag.

The dbt Cloud CLI offers additional flexibility by letting you choose the source environment for deferral artifacts. You can manually set a defer-env-id key in either your dbt_project.yml or dbt_cloud.yml file. By default, the dbt Cloud CLI will prefer metadata from the project's "Staging" environment (if defined), otherwise "Production."

dbt_cloud.yml

context:
  active-host: ...
  active-project: ...
  defer-env-id: '123456'

dbt_project.yml

dbt-cloud:
  defer-env-id: '123456'