These are awesome suggestions. To expand on our setup, we also have Informatica Cloud - IICS (CMI, CDI, etc.) connected to the entire setup generally used for bringing data from a source (PaaS, SaaS, On-prem SQL, Flat Files or streaming devices) to Databricks zones and were thinking to have a similar workaround whereby Tableau Prep outputs to a CSV storage location and have our Informatica Cloud toolset pick up the files for ingestion at a 5mins frequency. We are also wondering if it may be best to stand up a dedicated Azure SQL or a standard MS SQL server for these types of workflows and native Tableau Prep support, then connect the Azure SQL or MS SQL Server tables to Databricks zones for ingestion. Essentially going back and forth on which workaround approach may be the most self-service and with fewer toolsets in between for this data movement.
