ABAC tag support for for Streaming tables (Spark Lakeflow Declarative Pipelines)?

excavator-matt
Contributor III

Hi!

We're using Spark Lakeflow Declarative Pipelines for ingesting data from various data sources. However, in order to achieve compliance with GDPR, we are planning to start using ABAC tagging.

However, I don't understand how we are supposed to use this implement this on streaming tables with version control. The documentation is doesn't mention it.

I think you can tag the tables manually, but that might risk getting lost as tables are recreated. I also consider simply not allowing free access in bronze and apply tagging in version controlled models, but that seems harsh.

Have I missed something or is the support lacking here?