cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

column "storage_sub_directory" is now always NULL in system.information_schema.tables

GeKo
New Contributor III

Hello,

I am running a job that depends on the information provided in column storage_sub_directory in system.information_schema.tables .... and it worked until 1-2 weeks ago.

Now I discovered in the doc that this column is deprecated and always null , was there an announcement about that change (and if yes, where) ?

1 ACCEPTED SOLUTION

Accepted Solutions

NandiniN
Databricks Employee
Databricks Employee

Hi @GeKo ,

 

 

You can get it from the UC's List tables (GET /api/2.1/unity-catalog/tables) API for now. That API response's payload has the following field:
"storage_location": "string"
 
In future, this should be supported by Databricks as storage path column (currently there was some issue because of which it was deprecated), I do not have an ETA on that, but in future you can issue a support ticket to get the status of SC-107840 or monitor the release documentation.

View solution in original post

6 REPLIES 6

JessicaS
New Contributor II

Same issue here - we were using this to join to our S3 inventory files to determine the top tables to optimize and vacuum. Started failing about 2 weeks ago

NandiniN
Databricks Employee
Databricks Employee

Hello,

Linking the documentation - https://docs.databricks.com/en/sql/language-manual/information-schema/tables.html#definition

STORAGE_SUB_DIRECTORY

STRING

Yes

No

Deprecated. Always NULL.

GeKo
New Contributor III

Hello @NandiniN ,

many thanks for your reply.

The official doc I already know and I am aware that *suddenly* the storage_sub_directory is always NULL.

But now I need a solution from where I can get this information ? is the storage_sub_directory in another system.information_schema table/view ... or from where to extract this information now ??

GeKo
New Contributor III

any news on that @NandiniN  ?

where has this info gone...?!?!

NandiniN
Databricks Employee
Databricks Employee

Hi @GeKo ,

 

 

You can get it from the UC's List tables (GET /api/2.1/unity-catalog/tables) API for now. That API response's payload has the following field:
"storage_location": "string"
 
In future, this should be supported by Databricks as storage path column (currently there was some issue because of which it was deprecated), I do not have an ETA on that, but in future you can issue a support ticket to get the status of SC-107840 or monitor the release documentation.

GeKo
New Contributor III

Many thanks for the update @NandiniN 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group