Pat
Esteemed Contributor

Not sure if I understand correctly, but the issue is that you are using .sql file that have hardcoded env?

Use bundle substitutions/variables inside the SQL file itself

Databricks Asset Bundles support variable substitution not only in YAML but also in SQL files referenced by tasks, via the same {{var.name}} syntax. The pattern:
databricks.yml:

bundle:
  name: my-bundle

variables:
  env:
    default: dev
  table_suffix:
    default: _dev

targets:
  dev:
    default: true
    variables:
      env: dev
      table_suffix: _dev
  prod:
    variables:
      env: prod
      table_suffix: ""

SQL:

CREATE OR REPLACE MATERIALIZED VIEW IDENTIFIER(
  :env || '_gold.schema_name.mv_name'
) AS
SELECT *
FROM IDENTIFIER(
  :env || '_silver.schema_name.src_table_name'
);

job.yml:

resources:
  jobs:
    mv-job:
      name: mv-job
      tasks:
        - task_key: mv_task
          sql_task:
            file:
              path: ./sql/mv_example.sql   # committed in repo
              source: WORKSPACE
            warehouse_id: abc123
            parameters:
              env: dev