cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Creating External Table from partitioned parquet table

data_loader
New Contributor II

I am trying to create an external table in catalog using parquet where the parquet file is partitioned

I have tried using the below syntax,

%sql

CREATE TABLE table_name

(

col1 type1, 

col2 type2, 

col3 type3, 

)

USING parquet

 PARTITIONED BY ( col4 type4)

LOCATION "s3://<bucket-name>/<folder-name>/parquet_file/"

 

which gives success in creating the table

but no data is shown when i try to select the table

 

Can someone let me know what am i doing wrong.

2 REPLIES 2

MichTalebzadeh
Valued Contributor

This is my guess


Your code (LOCATION "s3://<bucket-name>/<folder-name>/parquet_file/) is likely pointing to a single Parquet file instead of the entire folder containing partitioned data.
try this

CREATE TABLE table_name
(
  col1 type1,
  col2 type2,
  col3 type3,
)
USING parquet
PARTITIONED BY (col4 type4)
LOCATION "s3://<bucket-name>/<folder-name>/"  -- Include trailing slash for the folder
Mich Talebzadeh | Technologist | Data | Generative AI | Financial Fraud
London
United Kingdom

view my Linkedin profile



https://en.everybodywiki.com/Mich_Talebzadeh



Disclaimer: The information provided is correct to the best of my knowledge but of course cannot be guaranteed . It is essential to note that, as with any advice, quote "one test result is worth one-thousand expert opinions (Werner Von Braun)".

Chizzy
New Contributor II

@data_loader , How were you able to fix this problem? I am having same issue now

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group