03-26-2024 11:37 AM
Hi I am new to databricks and need some inputs.
I am trying to create Delta External table in databricks using existing path which contains csv files.
What i observed is below code will create EXTERNAL table but provider is CSV.
-------------------------------
---------------------------
Below code will create MANAGED table and here provider is delta.
------------------------------
---------------------------
However i am trying to create EXTERNAL table where provider is delta where it uses existing path as location.
Can anybody guide or provide syntax on it if its possible.
03-26-2024 12:56 PM
@tajinder123 - could you please try this?
CREATE TABLE employee123 (id STRING, first_name STRING, last_name STRING, email STRING, gender STRING, salary DOUBLE, team STRING)
USING DELTA
LOCATION '/path/to/existing/delta/files';
03-26-2024 11:46 AM
@tajinder123 - can you please modify the syntax as below to create as a delta table
CREATE TABLE employee123
USING DELTA
LOCATION '/path/to/existing/delta/files';
03-26-2024 12:01 PM - edited 03-26-2024 12:04 PM
Thankyou, it worked but i am getting below. Can you please tell what need to be added along with it to read column names from header. Its in first line.
Also please tell how can we add schema manually. Thanks,
You are trying to read a Delta table `spark_catalog`.`default`.`employee1234` that does not have any columns.
03-26-2024 12:56 PM
@tajinder123 - could you please try this?
CREATE TABLE employee123 (id STRING, first_name STRING, last_name STRING, email STRING, gender STRING, salary DOUBLE, team STRING)
USING DELTA
LOCATION '/path/to/existing/delta/files';
03-26-2024 01:12 PM - edited 03-26-2024 01:13 PM
@shan_chandra THankyou, I just figured out same but i am glad you replied. Below i wrote and now i came and saw same thig. One thing I observed is that if files are already there in that location, create table wont work. It need to be empty, although we can use Insert into table statement and put data inside. Let me know if i am wrong here.
03-27-2024 09:49 AM
@tajinder123 - can you please try the following
1. dropping the table first and followed by
2. dbutils.fs.rm to remove the files recursively from the dbfs location and
3. create or replace command to recreate the table.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group