Databricks Community

I define all clusters as variable in separate files, so I can re-use them. Then I am accessing them in jobs as: The issue is that I want to change just the custom_tags in the cluster when instancing it for a job, cause my tags are different for each ...

I've got a fact streaming table, which is updated by SCD2 records from the CDF of a silver table. The join is on pk  (hash key generated from dimensions business keys) and factory_code (60 unique values). On each incremental processing, it reads all ...

As the subject states, my question is:Is it required to run the Ingestion Pipeline in Lakeflow Connect on Serverless compute? Cause I try to define my own cluster in the DAB, but it raises an error:`Error: cannot create pipeline: You cannot provide c...