Tableau Prep Save Output to Databricks

[vabadzhiev]

Has anyone run into use cases where your data scientist/data engineer end users build Tableau Prep Flows and steps in Tableau Prep Flow require saving output back into Databricks? There appears to be no native support for this in Tableau Prep if the target is Databricks (see screenshot attached). We are running Databricks on Azure. May be able to leverage other tools to ingest the output back into Databricks...but

I was wondering if anyone could share creative ways for a workaround solution, suggestions are much appreciated!

[Kaniz_Fatma]

Hi @vabadzhiev, While Tableau Prep doesn’t natively support saving output directly to Databricks, there are a few workarounds you can explore:

Export to Excel and Union in Tableau:

• You can export the data from Tableau Prep to an Excel file.
• Keep each output file in a separate folder.
• Use Tableau to union these Excel files by connecting to the folder where they are stored.
• Set up a scheduled refresh to keep your data model up-to-date with new data
• Custom Scripting:
  • If you’re comfortable with scripting, you can create a custom solution.
  • Use a Python or Scala script in Tableau Prep to write the data to a location accessible by Databricks (e.g., Azure Blob Storage or ADLS Gen2).
  • Set up a Databricks job to read from that location and process the data further.

Databricks Connection in Tableau:

• Although this doesn’t directly address Tableau Prep, you can set up a Databricks connection in Tableau.
• Create a datasource that joins tables from different catalogs using the same Databricks connection.
• This approach allows you to combine data from different Databricks tables within Tableau itself

[vabadzhiev]

These are awesome suggestions. To expand on our setup, we also have Informatica Cloud - IICS (CMI, CDI, etc.) connected to the entire setup generally used for bringing data from a source (PaaS, SaaS, On-prem SQL, Flat Files or streaming devices) to Databricks zones and were thinking to have a similar workaround whereby Tableau Prep outputs to a CSV storage location and have our Informatica Cloud toolset pick up the files for ingestion at a 5mins frequency. We are also wondering if it may be best to stand up a dedicated Azure SQL or a standard MS SQL server for these types of workflows and native Tableau Prep support, then connect the Azure SQL or MS SQL Server tables to Databricks zones for ingestion. Essentially going back and forth on which workaround approach may be the most self-service and with fewer toolsets in between for this data movement.