Cannot get past Query Data tutorial for Azure Databricks

philipkd — Fri, 23 Feb 2024 06:15:26 GMT

I created a new workspace on Azure Databricks, and I can't get past this first step in the tutorial:

DROP TABLE IF EXISTS diamonds; CREATE TABLE diamonds USING CSV OPTIONS (path "/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv", header "true")

error:

UnityCatalogServiceException: [RequestId=1a282f93-a134-4639-ac88-65798d67c924 ErrorClass=INVALID_PARAMETER_VALUE] GenerateTemporaryPathCredential uri /databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv is not a valid URI. Error message: INVALID_PARAMETER_VALUE: Missing cloud file system scheme.

adding dbfs: didn't fix my issue, as suggested here. instead I got:

AnalysisException: [UC_FILE_SCHEME_FOR_TABLE_CREATION_NOT_SUPPORTED] Creating table in Unity Catalog with file scheme dbfs is not supported. Instead, please create a federated data source connection using the CREATE CONNECTION command for the same table provider, then create a catalog based on the connection with a CREATE FOREIGN CATALOG command to reference the tables therein.

I also browsed the file system (with ls commands) and saw the file there.

Note: I got this working properly on AWS Databricks

Re: Cannot get past Query Data tutorial for Azure Databricks

dollyb — Fri, 21 Jun 2024 13:22:35 GMT

Struggling with this as well. So using dbfs:/ with CREATE TABLE statement works on AWS, but not Azure?

topic Re: Cannot get past Query Data tutorial for Azure Databricks in Data Engineering

Cannot get past Query Data tutorial for Azure Databricks

Re: Cannot get past Query Data tutorial for Azure Databricks