cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Help - Can't create table from tutorial. Is my setup wrong?

TimW
New Contributor

Trying out databricks for the first time and followed the Get Started steps. I managed to successfully create a cluster and ran the simple sql tutorial to query data from a notebook. However, got the following error:

Query:

DROP TABLE IF EXISTS diamonds;

CREATE TABLE diamonds USING CSV OPTIONS (path "/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv", header "true")

Error returned:

Error in SQL statement: UnityCatalogServiceException: [RequestId=68d811cd-a1e8-4d03-b9f3-8777d8010f59 ErrorClass=INVALID_PARAMETER_VALUE] GenerateTemporaryPathCredential uri /databricks-datasets/Rdatasets/data-001/csv/ggplot2/presidential.csv is not a valid URI. Error message: INVALID_PARAMETER_VALUE: Missing cloud file system scheme.
com.databricks.backend.common.rpc.SparkDriverExceptions$SQLExecutionException: com.databricks.sql.managedcatalog.UnityCatalogServiceException: [RequestId=68d811cd-a1e8-4d03-b9f3-8777d8010f59 ErrorClass=INVALID_PARAMETER_VALUE] GenerateTemporaryPathCredential uri /databricks-datasets/Rdatasets/data-001/csv/ggplot2/presidential.csv is not a valid URI. Error message: INVALID_PARAMETER_VALUE: Missing cloud file system scheme.
	at com.databricks.managedcatalog.ErrorDetailsHandler.wrapServiceException(ErrorDetailsHandler.scala:33)
	at com.databricks.managedcatalog.ErrorDetailsHandler.wrapServiceException$(ErrorDetailsHandler.scala:23)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.wrapServiceException(ManagedCatalogClientImpl.scala:139)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.recordAndWrapException(ManagedCatalogClientImpl.scala:3696)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.generateTemporaryPathCredentials(ManagedCatalogClientImpl.scala:3277)
	at com.databricks.sql.managedcatalog.ManagedCatalogCommon.generateTemporaryPathCredentials(ManagedCatalogCommon.scala:1487)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.$anonfun$generateTemporaryPathCredentials$2(ProfiledManagedCatalog.scala:564)
	at org.apache.spark.sql.catalyst.MetricKeyUtils$.measure(MetricKey.scala:399)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.$anonfun$profile$1(ProfiledManagedCatalog.scala:55)
	at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.profile(ProfiledManagedCatalog.scala:54)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.generateTemporaryPathCredentials(ProfiledManagedCatalog.scala:564)
	at com.databricks.unity.CredentialScopeSQLHelper$.checkPathOperations(CredentialScopeSQLHelper.scala:95)
	at com.databricks.unity.CredentialScopeSQLHelper$.register(CredentialScopeSQLHelper.scala:142)
	at com.databricks.unity.CredentialScopeSQLHelper$.registerCreateTableAccess(CredentialScopeSQLHelper.scala:526)
	at com.databricks.sql.managedcatalog.CredentialScopeTableCredentialHandler.injectCredential(ResolveWithCredential.scala:466)
	at com.databricks.sql.managedcatalog.ResolveWithCredential.com$databricks$sql$managedcatalog$ResolveWithCredential$$maybeDecorateCatalogTable(ResolveWithCredential.scala:75)

 I checked the Catalog and saw that my workspace is created. Every resource was just created using the Quick Start and the Get Started documents. Anyone has an idea of where my setup might be wrong? Thanks

1 ACCEPTED SOLUTION

Accepted Solutions

Scott_in_Zurich
New Contributor III

Adding 'dbfs:' got me past that error. Now onto debugging a PARSE SYNTAX error....

View solution in original post

3 REPLIES 3

Emil_Kaminski
Contributor

Hi TimW, 

I will do a bit of guessing as I don't have such problem as you. If you are following this tutorial - https://www.databricks.com/notebooks/gcp-qs-notebook.html, the issue might be, very, very old version of runtime. Try to create and use a cluster with modern version, which will be 13+. 

If you already have a modern version of cluster, and still face this issue, try to change path to the file to start with dbfs:. Like  - path "dbfs:/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv".

Good luck

** You might also want to subscribe to Warsaw Databricks YT channel: https://www.youtube.com/channel/UC1-u_2nI97cNHtu_FQ3HP_A

Scott_in_Zurich
New Contributor III

Adding 'dbfs:' got me past that error. Now onto debugging a PARSE SYNTAX error....

patwilliams
New Contributor II

It sounds like you're on the right track with your Databricks setup, but this kind of error could be related to a few potential issues. Based on the details youโ€™ve shared, here are some things you might want to check:

  1. Cluster Configuration: Ensure that your cluster is properly configured and running. Sometimes, issues arise from the cluster not being active or not having enough resources to handle the query. Double-check that your cluster has sufficient memory and processing power allocated.

  2. Database or Table Access: If you're querying from a pre-existing database or table, verify that it exists and that you have the appropriate permissions. You might need to adjust the database connection string or check if the table was created successfully in the first place.

  3. SQL Query Syntax: Sometimes, the issue could be as simple as a syntax error in the SQL query you're running. Make sure the query matches the schema of the data youโ€™re working with, and try running simpler queries to see if they work.

  4. Dependencies or Libraries: If your notebook relies on external libraries or packages, ensure that they are installed and up to date. An outdated library or missing dependency could prevent the table creation from succeeding.

  5. Databricks Runtime Version: Depending on the version of Databricks you're using, certain functionalities might behave differently. Itโ€™s worth checking if the runtime youโ€™re using is compatible with the tutorial youโ€™re following.

Lastly, if you're experimenting with Databricks and looking for something interesting, you should definitely check out Magis Apple TV. Itโ€™s a great way to experience seamless integration and might offer insights into handling similar data-driven applications in a different context.

Let me know if you need further help!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group