Help - Can't create table from tutorial. Is my setup wrong?

[TimW]

Trying out databricks for the first time and followed the Get Started steps. I managed to successfully create a cluster and ran the simple sql tutorial to query data from a notebook. However, got the following error:

Query:

DROP TABLE IF EXISTS diamonds;

CREATE TABLE diamonds USING CSV OPTIONS (path "/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv", header "true")

Error returned:

Error in SQL statement: UnityCatalogServiceException: [RequestId=68d811cd-a1e8-4d03-b9f3-8777d8010f59 ErrorClass=INVALID_PARAMETER_VALUE] GenerateTemporaryPathCredential uri /databricks-datasets/Rdatasets/data-001/csv/ggplot2/presidential.csv is not a valid URI. Error message: INVALID_PARAMETER_VALUE: Missing cloud file system scheme.
com.databricks.backend.common.rpc.SparkDriverExceptions$SQLExecutionException: com.databricks.sql.managedcatalog.UnityCatalogServiceException: [RequestId=68d811cd-a1e8-4d03-b9f3-8777d8010f59 ErrorClass=INVALID_PARAMETER_VALUE] GenerateTemporaryPathCredential uri /databricks-datasets/Rdatasets/data-001/csv/ggplot2/presidential.csv is not a valid URI. Error message: INVALID_PARAMETER_VALUE: Missing cloud file system scheme.
	at com.databricks.managedcatalog.ErrorDetailsHandler.wrapServiceException(ErrorDetailsHandler.scala:33)
	at com.databricks.managedcatalog.ErrorDetailsHandler.wrapServiceException$(ErrorDetailsHandler.scala:23)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.wrapServiceException(ManagedCatalogClientImpl.scala:139)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.recordAndWrapException(ManagedCatalogClientImpl.scala:3696)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.generateTemporaryPathCredentials(ManagedCatalogClientImpl.scala:3277)
	at com.databricks.sql.managedcatalog.ManagedCatalogCommon.generateTemporaryPathCredentials(ManagedCatalogCommon.scala:1487)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.$anonfun$generateTemporaryPathCredentials$2(ProfiledManagedCatalog.scala:564)
	at org.apache.spark.sql.catalyst.MetricKeyUtils$.measure(MetricKey.scala:399)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.$anonfun$profile$1(ProfiledManagedCatalog.scala:55)
	at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.profile(ProfiledManagedCatalog.scala:54)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.generateTemporaryPathCredentials(ProfiledManagedCatalog.scala:564)
	at com.databricks.unity.CredentialScopeSQLHelper$.checkPathOperations(CredentialScopeSQLHelper.scala:95)
	at com.databricks.unity.CredentialScopeSQLHelper$.register(CredentialScopeSQLHelper.scala:142)
	at com.databricks.unity.CredentialScopeSQLHelper$.registerCreateTableAccess(CredentialScopeSQLHelper.scala:526)
	at com.databricks.sql.managedcatalog.CredentialScopeTableCredentialHandler.injectCredential(ResolveWithCredential.scala:466)
	at com.databricks.sql.managedcatalog.ResolveWithCredential.com$databricks$sql$managedcatalog$ResolveWithCredential$$maybeDecorateCatalogTable(ResolveWithCredential.scala:75)

 I checked the Catalog and saw that my workspace is created. Every resource was just created using the Quick Start and the Get Started documents. Anyone has an idea of where my setup might be wrong? Thanks

[Emil_Kaminski]

Hi TimW, 

I will do a bit of guessing as I don't have such problem as you. If you are following this tutorial - https://www.databricks.com/notebooks/gcp-qs-notebook.html, the issue might be, very, very old version of runtime. Try to create and use a cluster with modern version, which will be 13+. 

If you already have a modern version of cluster, and still face this issue, try to change path to the file to start with dbfs:. Like  - path "dbfs:/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv".

Good luck

[Scott_in_Zurich · Accepted solution]

Adding 'dbfs:' got me past that error. Now onto debugging a PARSE SYNTAX error....

[patwilliams]

It sounds like you're on the right track with your Databricks setup, but this kind of error could be related to a few potential issues. Based on the details you’ve shared, here are some things you might want to check:

1. Cluster Configuration: Ensure that your cluster is properly configured and running. Sometimes, issues arise from the cluster not being active or not having enough resources to handle the query. Double-check that your cluster has sufficient memory and processing power allocated.

2. Database or Table Access: If you're querying from a pre-existing database or table, verify that it exists and that you have the appropriate permissions. You might need to adjust the database connection string or check if the table was created successfully in the first place.

3. SQL Query Syntax: Sometimes, the issue could be as simple as a syntax error in the SQL query you're running. Make sure the query matches the schema of the data you’re working with, and try running simpler queries to see if they work.

4. Dependencies or Libraries: If your notebook relies on external libraries or packages, ensure that they are installed and up to date. An outdated library or missing dependency could prevent the table creation from succeeding.

5. Databricks Runtime Version: Depending on the version of Databricks you're using, certain functionalities might behave differently. It’s worth checking if the runtime you’re using is compatible with the tutorial you’re following.

Lastly, if you're experimenting with Databricks and looking for something interesting, you should definitely check out Magis Apple TV. It’s a great way to experience seamless integration and might offer insights into handling similar data-driven applications in a different context.

Let me know if you need further help!