https://community.databricks.com/t5/machine-learning/what-is-the-use-case-of-having-azure-synapse-dwh-and-delta-lake/m-p/31925#M1679 <P>Thank you, so for a large workload, where we need lot of optimization we might need Synapse, but for a small/medium workload, we might have to stick to Delta Table</P> Tue, 13 Sep 2022 09:41:21 GMT Shuvi 2022-09-13T09:41:21Z https://community.databricks.com/t5/machine-learning/what-is-the-use-case-of-having-azure-synapse-dwh-and-delta-lake/m-p/31923#M1677 <P>The curated zone is pushed to cloud data warehouse such as Synapse Dedicated SQL Pools which then acts as a serving layer for BI tools and analyst.</P><P></P><P>I believe we can have models in gold layer and have BI connect to this layer or we can have serverless infra for adhoc querying</P> Tue, 13 Sep 2022 09:25:49 GMT https://community.databricks.com/t5/machine-learning/what-is-the-use-case-of-having-azure-synapse-dwh-and-delta-lake/m-p/31923#M1677 Shuvi 2022-09-13T09:25:49Z https://community.databricks.com/t5/machine-learning/what-is-the-use-case-of-having-azure-synapse-dwh-and-delta-lake/m-p/31924#M1678 <P>In the past I would have said: yes because running queries on a spark cluster is not that fast.</P><P>But right now, with Databricks SQL I don't immediately see the added value.</P><P>A use case could be proprietary login method (SQL Server login f..e.), or more tuning options (table type f.e.).</P><P>It can also be that Synapse is cheaper/faster than Databricks SQL for your use case.</P><P>So in general I'd say that there is no reason to use Synapse, but it might be possible that for your use case it is a good choice.</P> Tue, 13 Sep 2022 09:37:48 GMT https://community.databricks.com/t5/machine-learning/what-is-the-use-case-of-having-azure-synapse-dwh-and-delta-lake/m-p/31924#M1678 -werners- 2022-09-13T09:37:48Z https://community.databricks.com/t5/machine-learning/what-is-the-use-case-of-having-azure-synapse-dwh-and-delta-lake/m-p/31925#M1679 <P>Thank you, so for a large workload, where we need lot of optimization we might need Synapse, but for a small/medium workload, we might have to stick to Delta Table</P> Tue, 13 Sep 2022 09:41:21 GMT https://community.databricks.com/t5/machine-learning/what-is-the-use-case-of-having-azure-synapse-dwh-and-delta-lake/m-p/31925#M1679 Shuvi 2022-09-13T09:41:21Z https://community.databricks.com/t5/machine-learning/what-is-the-use-case-of-having-azure-synapse-dwh-and-delta-lake/m-p/31926#M1680 <P>Not necessarily. Databricks SQL can also be very fast. So I'd do a comparison on performance/cost for one of your 'heavy' workloads.</P><P>And do not forget to take the cost of copying data into account (which you do not have to do in case of databricks SQL).</P> Tue, 13 Sep 2022 09:46:18 GMT https://community.databricks.com/t5/machine-learning/what-is-the-use-case-of-having-azure-synapse-dwh-and-delta-lake/m-p/31926#M1680 -werners- 2022-09-13T09:46:18Z