cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

can I use volume for external table location?

PaulineX
New Contributor III

Hello,

I have a parquet file test.parquet in the volume volume_ext_test. Tried to create an external table as below, it failed and says it "is not a valid URI".

create table catalog_managed.schema_test.tbl_vol as
select * from parquet.`/Volumes/catalog_managed/schema_test/volume_ext_test/test.parquet`
 
but I have no problem to create a managed table as below:
create table catalog_managed.schema_test.tbl_vol as
select * from parquet.`/Volumes/catalog_managed/schema_test/volume_ext_test/test.parquet`
 
Am I doing something wrong? Or it is the limitation?
 
Thanks.
2 ACCEPTED SOLUTIONS

Accepted Solutions

AmanSehgal
Honored Contributor III

Hi @PaulineX 

As per the documentation, you cannot use volume for storing table data. It's for loading, storing and accessing files.

 

You cannot use volumes as a location for tables. Volumes are intended for path-based data access only. Use tables for storing tabular data with Unity Catalog.

It's bit ironic here, as delta table data is stored in files as well.

Anyway, as per the documentation, answer to your question is no.

 

 

View solution in original post

PaulineX
New Contributor III

@AmanSehgal Thank you for the reply. This is exactly what I want to confirm!

Too bad, I hoped to avoid full path in notebooks.

View solution in original post

3 REPLIES 3

PaulineX
New Contributor III

sorry pasted the wrong statement. the external table statement is :

CREATE TABLE px_catalog_managed.schema_test.test_tbl2
USING parquet
OPTIONS (
  path '/Volumes/catalog_managed/schema_test/volume_ext_test/test.parquet',
  inferSchema 'true'
)
 
which gives me error

AmanSehgal
Honored Contributor III

Hi @PaulineX 

As per the documentation, you cannot use volume for storing table data. It's for loading, storing and accessing files.

 

You cannot use volumes as a location for tables. Volumes are intended for path-based data access only. Use tables for storing tabular data with Unity Catalog.

It's bit ironic here, as delta table data is stored in files as well.

Anyway, as per the documentation, answer to your question is no.

 

 

PaulineX
New Contributor III

@AmanSehgal Thank you for the reply. This is exactly what I want to confirm!

Too bad, I hoped to avoid full path in notebooks.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group