topic Re: I thought that Azure Data Factory is built on spark but now when I crushed it I see that is build directly on databricks :-) in Data Engineering

I thought that Azure Data Factory is built on spark but now when I crushed it I see that is build directly on databricks :-)

Hubert-Dudek — Mon, 06 Dec 2021 17:57:22 GMT

I thought that Azure Data Factory is built on spark but now when I crushed it I see that is build directly on databricks 🙂

jose_gonzalez — Tue, 07 Dec 2021 00:08:10 GMT

Yes, it seems like the error stack trace is coming from it.

-werners- — Tue, 07 Dec 2021 08:21:06 GMT

correct. Because Data Flows were available before their own (MS) spark pools were available.

But let's be honest: that is only a good thing 🙂