Hi @eimis_pacheco, The participant's statement refers to two specific limitations when using Delta Live Tables (DLT) with Unity Catalog:
1. DLT not being able to follow the medallion architecture: The Medallion architecture is a data management strategy that organizes data into tiers (bronze, silver, gold) based on the level of transformation and refinement. The limitation here implies that DLT, when used with Unity Catalog, may not support this tiered data management approach effectively.
2. No support for third party libraries: This means that DLT, in combination with Unity Catalog, does not currently support the integration or use of third-party libraries. This could limit the functionality and flexibility of your data workflows, as you may not be able to utilize specific tools or features provided by these libraries. As for why DLT struggles with the Unity Catalog, the specifics are not detailed in the provided sources. However, it could be due to technical constraints, compatibility issues, or architectural differences between DLT and Unity Catalog.
Other limitations to consider when creating a workflow with DLT include:
- Only tables and views stored in a metastore can be shared using DLT.
- Only tables in Delta format are supported.
- View sharing is supported only in Databricks-to-Databricks sharing.
- Shareable views must be defined on Delta tables or other shareable views.
- There are limits on the number of files in metadata allowed for a shared table.
- Schemas named information_schema cannot be imported into a metastore because that schema name is reserved.
