topic Azure data bricks delta tables .Issue in Data Engineering

Azure data bricks delta tables .Issue

RohitKulkarni — Fri, 02 Sep 2022 06:40:33 GMT

Hello Team,

I have written Spark SQL Query in data bricks :

DROP TABLE IF EXISTS Salesforce.Location;

CREATE EXTERNAL TABLE Salesforce.Location (

Id STRING,

OwnerId STRING,

IsDeleted bigint,

Name STRING,

CurrencyIsoCode STRING,

CreatedDate bigint,

CreatedById STRING,

LastModifiedDate bigint,

LastModifiedById STRING,

SystemModstamp bigint,

LocationType STRING,

Description STRING,

TimeZone STRING,

ParentLocationId STRING,

IsMobile boolean,

IsInventoryLocation boolean,

VisitorAddressId STRING,

RootLocationId STRING,

LocationLevel bigint,

ExternalReference STRING

)

USING data_source

LOCATION 'abfss://xyz@xyz.dfs.core.windows.net/Bronze/Salesforce/Location'

Hello Team,

But I am getting error as :

Error in SQL statement: ClassNotFoundException:

Failed to find data source: data_source.

Please advise.

There mutliple files are present in the below mentioned path :

https://xyz.blob.core.windows.net/xyz/Bronze/Salesforce/Location

Regards

Re: Azure data bricks delta tables .Issue

AmanSehgal — Fri, 02 Sep 2022 07:09:29 GMT

You need to provide one of the following value for 'data_source':

TEXT
AVRO
CSV
JSON
PARQUET
ORC
DELTA

eg: USING PARQUET

If you skip USING clause, then the default data source is DELTA

https://docs.databricks.com/sql/language-manual/sql-ref-syntax-ddl-create-table-using.html

Re: Azure data bricks delta tables .Issue

RohitKulkarni — Fri, 02 Sep 2022 07:19:12 GMT

I am new to databricks.Please can you explain to me with example.

Re: Azure data bricks delta tables .Issue

AmanSehgal — Sat, 03 Sep 2022 06:55:00 GMT

CREATE TABLE table_name (

column_name_1 datatype,

column_name_2 datatype,

column_name_n datatype,

)

USING data_source

LOCATION <path>

Re: Azure data bricks delta tables .Issue

jose_gonzalez — Fri, 09 Sep 2022 23:25:17 GMT

Hi @Rohit Kulkarni,

Just a friendly follow-up. Did you have time to test AmanSehgal's sample code? do you still have issues? please let us know