topic Re: Migrate Azure Synapse Analytics data to Databricks in Warehousing & Analytics

Migrate Azure Synapse Analytics data to Databricks

Akshay_Petkar — Tue, 17 Sep 2024 03:51:15 GMT

I have to migrate the data from Azure Synapse Analytics to Databricks. Could anyone share the different approaches to migrate data, and from those, which is the best approach to use?

Re: Migrate Azure Synapse Analytics data to Databricks

Gail207Martinez — Tue, 17 Sep 2024 05:26:30 GMT

Hello!

Migrating data from Azure Synapse Analytics to Databricks can be done using several approaches. You can configure a pipeline in Azure Data Factory (ADF) to copy data from Synapse SQL to Azure Data Lake Storage (ADLS) and then load it into Databricks. Another method is to use Azure Databricks to directly query and load data from Azure Synapse using built-in connectors, which is efficient for real-time processing. Creating Delta tables in Databricks and using Delta Lake for storing and managing data is also a good option, providing ACID transactions and efficient queries. Alternatively, you can write custom TellTheBell ETL scripts in Python or Scala in Databricks to read data from Synapse and write it to Databricks. The best approach depends on your specific requirements, but for large data volumes and frequent transfers, using Azure Data Factory with Delta Lake in Databricks is often recommended due to its scalability and efficiency.

Re: Migrate Azure Synapse Analytics data to Databricks

thelogicplus — Mon, 25 Nov 2024 05:25:57 GMT

You can use Travinto Technologies code converter tool to migrate data , ETL, and report from one technology to others. we have migrated Azure Synapse Analytics data to Databricks using their services without worry for many customer. They have 50000+ adaptor that can help you to migrate any thing to any things.

Re: Migrate Azure Synapse Analytics data to Databricks

tracyberg — Wed, 25 Dec 2024 11:28:46 GMT

@thelogicplus wrote:
You can use Travinto Technologies code converter tool to migrate data , ETL, and report from one technology to others. we have migrated Azure Synapse Analytics data to Databricks using their services without worry for many customer. They have 50000+ adaptor that can help you to migrate any thing to any things.

very useful information

Re: Migrate Azure Synapse Analytics data to Databricks

raushan_kumar — Wed, 08 Jan 2025 15:19:25 GMT

Can we mount the Syanpse storage account ADLS Gen2 directly to Databricks and then we can only migrate pipeline/code to databricks

Re: Migrate Azure Synapse Analytics data to Databricks

thelogicplus — Wed, 08 Jan 2025 16:22:14 GMT

@raushan_kumar Yes, you can mount an Azure Data Lake Storage Gen2 (ADLS Gen2) account directly to Databricks using thdbutils.fs.mount command. This allows you to access the storage account like a file system within Databricks. Please connect travinto.com