cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Unknown Location of files for tables created using DLT

ankitmit
New Contributor III

Hi all,

I created catalog and schema using managed location 

ankitmit_0-1738140152440.png

 

But I donโ€™t see any catalogs directory within the s3 bucket path mentioned in the image above. 

ankitmit_1-1738140152445.png

 

Also, I created a schema with managed location, and I expected all the tables created within this schema using DLT to be stored within the storage location path but this is not the case.

ankitmit_2-1738140152449.png

We're creating our tables using DLT

I am not sure, whatโ€™s going wrong here, and we still donโ€™t see any files related to our tables in s3 buckets. We donโ€™t know where exactly the tables files are being stored on our s3 buckets, even though we mentioned the path in catalog and schema levels.

Looking forward to hearing some feedbacks from you.

With Kindest Regards,
Ankit Mittal.

2 REPLIES 2

VZLA
Databricks Employee
Databricks Employee

Hello, thank you for your question.

Could you confirm that your issue is regarding the location of files for tables created using Delta Live Tables (DLT) when utilizing managed storage locations at the catalog and schema levels? Specifically, it seems that:

  1. You have created a catalog and schema with managed locations in an S3 bucket.
  2. Despite the managed locations being specified, you do not see the expected directory or files in the S3 bucket for tables created using DLT.

Am I correct in understanding your issue? If so, I can help clarify why this might be happening and suggest some troubleshooting steps. Let me know if additional context is needed!

ankitmit
New Contributor III

Hi @VZLA ,
Yes, that's exactly what is happening!! 
I am trying to understand where the files are being created in s3 buckets for tables using DLT and figure out if we can can somehow specify where the files should be created for these tables so that we can have more control over it, that's why I was tried to create a catalog and schema with managed location.

Please suggest some troubleshooting steps to help resolve this issue.

With Kindest Regards,
Ankit Mittal.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group