topic Re: external table not being written to data lake in Data Engineering

external table not being written to data lake

mridultuteja — Mon, 05 May 2025 00:38:37 GMT

I was following a tutorial to learn databricks from https://youtu.be/7pee6_Sq3VY
Great video btw

I am stuck here at 2:52:24
I am trying to create an external table directly to data lake but i am facing some weird issue saying no such location exists.
I have already tried list and it was able to access the path and list all my files on data lake

Re: external table not being written to data lake

Brahmareddy — Mon, 05 May 2025 04:22:51 GMT

Hi mridultuteja,

How are you doing today?, As per my understanding, If you’re getting an error like “no such location exists” when trying to create an external table, even though dbutils.fs.ls() works and lists your files, the issue is usually related to how paths are referenced when using Unity Catalog. When creating an external table, especially with CREATE TABLE ... LOCATION, Unity Catalog expects the path to be registered as an external location with proper permissions. Just because the notebook can read the path doesn't mean Unity Catalog has the right access. To fix this, go to the Databricks UI, register your external location under Data > External Locations, and make sure your storage credential has the correct access to the path. Once that’s done, try creating the table again using that external location. Let me know if you want help setting up the location registration—it’s a one-time thing, and then it works smoothly!

Regards,

Brahma

Re: external table not being written to data lake

Takuya-Omi — Mon, 05 May 2025 05:13:40 GMT

@mridultuteja

Hi,

In addition to Brahma's excellent response, I’ll also attach some helpful links.
This issue has come up multiple times in the community in the past.

https://community.databricks.com/t5/data-governance/no-parent-external-location-for-path-when-browsing-our-metastore/td-p/93165

https://community.databricks.com/t5/databricks-free-trial-help/getting-error-while-creating-external-delta-table-in-databricks/td-p/106499

Re: external table not being written to data lake

mridultuteja — Wed, 07 May 2025 19:29:34 GMT

I still need help in this

Re: external table not being written to data lake

Brahmareddy — Wed, 07 May 2025 19:43:38 GMT

Hi,

Did you try out my suggestion?

Regards,

Brahma

Re: external table not being written to data lake

mridultuteja — Wed, 07 May 2025 19:49:45 GMT

I need help in registering the location as you have guided.
Thanks in advance.

Re: external table not being written to data lake

Isi — Thu, 08 May 2025 00:18:30 GMT

Hey @mridultuteja

To register an external location, you have to first create a Storage Credential, and then create the External Location.

This process allows Databricks to securely access data stored in Azure Data Lake Storage Gen2 (ADLS Gen2), while maintaining fine-grained access control through Unity Catalog. External locations are used to govern access to files and directories outside of the managed Delta Lake tables.

There are different approaches to creating this through SQL or the UI, and I’m sharing a video that could help you understand it more easily 🙂

https://www.youtube.com/watch?v=KBB14qcELD4

Hope this helps, 🙂

Isi