topic Re: Databricks Runtime compatibility error with latest version while reading from (ADLS) Dynamic 365 . in Data Engineering

Databricks Runtime compatibility error with latest version while reading from (ADLS) Dynamic 365 .

nitinsingh1 — Wed, 22 Jun 2022 15:19:30 GMT

We are trying to establish ingestion from dynamic 365 >> ADLS >> Databricks, While reading information we need to use databricks runtime 6.4 to read the raw data from ADLS into Databricks.

Latest databricks runtime couldn’t be used, Need your help to related to this compatibility error and also, requires help in order to pull CDM utilities function in databricks.

And when reading from 6.4 runtime - we are only able to read base table from dynamic-finance and operation but we need to use entity table to ingest into bronze layer.

Is there any reference where we can use databricks to perform end to end ETL dynamic 365 >> ADLS >> DeltaLake

Re: Databricks Runtime compatibility error with latest version while reading from (ADLS) Dynamic 365 .

Hubert-Dudek — Thu, 23 Jun 2022 19:20:11 GMT

In Databricks in Partner Connect first option is Fivetran.

Fivetran has Dynamics 365 connector https://www.fivetran.com/connectors/microsoft-dynamics-365.

Re: Databricks Runtime compatibility error with latest version while reading from (ADLS) Dynamic 365 .

jose_gonzalez — Fri, 29 Jul 2022 22:46:03 GMT

Hi @nitin singh,

Just a friendly follow-up. Did you see Humbert's response? did you give it a try? do you still need help?

Re: Databricks Runtime compatibility error with latest version while reading from (ADLS) Dynamic 365 .

nitinsingh1 — Mon, 01 Aug 2022 08:51:36 GMT

@Hubert Dudek @Jose Gonzalez Thank you guys, We are already partner with Fivetran APAC team. Unfortunately customer is looking into databricks as of now.

Good New - We are able to ingest and load data from dynamic 365 >> ADLS >> Databricks (without any connector ) and load into Delta format successfully. Thank you again. Great day !!

Re: Databricks Runtime compatibility error with latest version while reading from (ADLS) Dynamic 365

BobBubble2000 — Wed, 09 Aug 2023 08:20:19 GMT

Hi @nitinsingh1

Thank you for bringing up this topic, I'm also currently looking into how to ingest exported Dynamics 365 FO data (csv files with CDM) from ADLS into Databricks. Could you share how you achieved this? I'd be very curious to see your approach.

Thanks a lot!

Bob

Re: Databricks Runtime compatibility error with latest version while reading from (ADLS) Dynamic 365

sheridan06 — Wed, 31 Jan 2024 22:09:50 GMT

Hi @nitinsingh1 ! Same question as @BobBubble2000 ... how did you achieve this without a 3rd party connector tool? More specifically, I'm trying to figure out how to connect to a Dynamics 365 Business Central ERP set of APIs to ingest data into the Databricks lakehouse.

thanks!