DevOps CICD for Databricks Schema

[Mot]

Does anyone know how to do DevOps CICD pipeline to deploy changes to a Databricks Catalog? I've added a couple tables in DEV and need to deploy in Prod. Could'nt find any info on this anywhere. 

[Mot]

Thanks @Retired_mod , the links refer mostly deployments of workspace objects rather than changes in database objects, eg. a new schema, table, or new column added to a table etc. I'll keep looking...

[szymon_dybczak · Accepted solution]

Hi @Mot ,

You're looking for state based schema migration. You can achieve this using external tools like liquibase

https://medium.com/dbsql-sme-engineering/advanced-schema-management-on-databricks-with-liquibase-190...

Also take a look on below discussion at reddit, they are mentioning terraform as well:

https://www.reddit.com/r/dataengineering/s/k5ng2Qw84T