Databricks Community

I need to add PII tags at both the table and column levels for a streaming table created using Spark Declarative Pipelines.I tried applying Unity Catalog tags with the following code inside the SDP Python pipeline:spark.sql(f"""ALTER TABLE {table_nam...

I'm running a Lakeflow Declarative Pipeline that reads data from a bronze table ingested by Auto Loader and writes it to a silver table with simple transformations.The source data contains struct columns with many deeply nested fields. The table curr...

I need to execute stored procedures in my notebook, to do that I created an init script that installs ODBC driver to my job cluster. But the script stops working after some time and I can't figure out why, so the cluster can't start. Can someone send...