Hi all,
I am trying to run my first databricks asset bundle but it always fails with
[UC_NOT_ENABLED] Unity Catalog is not enabled on this cluster. SQLSTATE: 56038
Does anyone ecountered this error when running asset bundles or does anyone know how to fix it?
the yml goes like
bundle:
name: hello_world
workspace:
host: https://...
variables:
spark_version:
description: "The version of Spark to use."
default: 15.4.x-scala2.12
resources:
jobs:
some-task:
name: "[${bundle.target}] some_task"
tasks:
- task_key: task_loop
for_each_task:
inputs: "[{\"par1\": \"hello\", \"par2\": \"world\"}, {\"par1\": \"spam\", \"par2\": \"eggs\"}]"
task:
task_key: some_task_key
job_cluster_key: job-cluster
notebook_task:
notebook_path: notebooks/nb
base_parameters: {
"par1": "{{input.par1}}",
"par2": "{{input.par2}}"
}
job_clusters:
- job_cluster_key: job-cluster
new_cluster:
spark_version: ${var.spark_version}
num_workers: 1
node_type_id: e2-highmem-2
spark_conf:
"spark.databricks.catalog.enabled": "true"
targets:
dev:
mode: development
default: true
running the bundle with
databricks bundle run -t dev some_task_key
Any help appreciated.