We are experiencing a compatibility issue between the Databricks Runtime version and the Delta Live Tables (DLT) Runtime. We need to upgrade to Databricks Runtime 15.2 to enable the auto schema evolution feature. However, Databricks Runtime 15.2 requires Python 3.11, which introduces the new StrEnum class, causing a breaking change.
Our DLT pipeline uses Runtime 14.1, which runs on Python 3.10, and does not support StrEnum. Since the DLT cluster uses a specific Spark version that cannot be overridden by users, upgrading the Databricks Runtime for our job cluster would break the DLT pipeline.
Could you share the release plan for DLT and provide a timeline for when version 15.2 will be available for DLT?
Relevant links:
