Databricks Spark Declarative Pipelines (SDP) makes handling SCD easy with its Auto CDC feature. I had a customer who had to explode out an array as part of their pipeline in their streaming table and wanted to work through their pipeline like:
Source...
