When are DLT going to support multiple targets
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-12-2024 02:53 PM
Due to the limitations with all output data needing to be stored in one target we have stopped using DLT until more flexibility is added.
If anyone has a workaround we are open to suggestions.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-26-2024 09:22 AM
Hi Zavi,
One potential workaround is to establish multiple DLT pipelines, with each pipeline specifically configured to point to a unique target. This approach effectively allows for a diverse range of output data to be stored across various targets.
To manage these multiple pipelines efficiently, you can utilize Databricks Workflows. Workflows can orchestrate the execution of these pipelines, ensuring they operate in harmony and maintain the integrity of your data processes.
Best regards, Rafael Ribeiro
