Help - Can't create table from tutorial. Is my setup wrong?

[TimW]

Trying out databricks for the first time and followed the Get Started steps. I managed to successfully create a cluster and ran the simple sql tutorial to query data from a notebook. However, got the following error:

Query:

DROP TABLE IF EXISTS diamonds;

CREATE TABLE diamonds USING CSV OPTIONS (path "/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv", header "true")

Error returned:

Error in SQL statement: UnityCatalogServiceException: [RequestId=68d811cd-a1e8-4d03-b9f3-8777d8010f59 ErrorClass=INVALID_PARAMETER_VALUE] GenerateTemporaryPathCredential uri /databricks-datasets/Rdatasets/data-001/csv/ggplot2/presidential.csv is not a valid URI. Error message: INVALID_PARAMETER_VALUE: Missing cloud file system scheme.
com.databricks.backend.common.rpc.SparkDriverExceptions$SQLExecutionException: com.databricks.sql.managedcatalog.UnityCatalogServiceException: [RequestId=68d811cd-a1e8-4d03-b9f3-8777d8010f59 ErrorClass=INVALID_PARAMETER_VALUE] GenerateTemporaryPathCredential uri /databricks-datasets/Rdatasets/data-001/csv/ggplot2/presidential.csv is not a valid URI. Error message: INVALID_PARAMETER_VALUE: Missing cloud file system scheme.
	at com.databricks.managedcatalog.ErrorDetailsHandler.wrapServiceException(ErrorDetailsHandler.scala:33)
	at com.databricks.managedcatalog.ErrorDetailsHandler.wrapServiceException$(ErrorDetailsHandler.scala:23)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.wrapServiceException(ManagedCatalogClientImpl.scala:139)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.recordAndWrapException(ManagedCatalogClientImpl.scala:3696)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.generateTemporaryPathCredentials(ManagedCatalogClientImpl.scala:3277)
	at com.databricks.sql.managedcatalog.ManagedCatalogCommon.generateTemporaryPathCredentials(ManagedCatalogCommon.scala:1487)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.$anonfun$generateTemporaryPathCredentials$2(ProfiledManagedCatalog.scala:564)
	at org.apache.spark.sql.catalyst.MetricKeyUtils$.measure(MetricKey.scala:399)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.$anonfun$profile$1(ProfiledManagedCatalog.scala:55)
	at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.profile(ProfiledManagedCatalog.scala:54)
	at com.databricks.sql.managedcatalog.ProfiledManagedCatalog.generateTemporaryPathCredentials(ProfiledManagedCatalog.scala:564)
	at com.databricks.unity.CredentialScopeSQLHelper$.checkPathOperations(CredentialScopeSQLHelper.scala:95)
	at com.databricks.unity.CredentialScopeSQLHelper$.register(CredentialScopeSQLHelper.scala:142)
	at com.databricks.unity.CredentialScopeSQLHelper$.registerCreateTableAccess(CredentialScopeSQLHelper.scala:526)
	at com.databricks.sql.managedcatalog.CredentialScopeTableCredentialHandler.injectCredential(ResolveWithCredential.scala:466)
	at com.databricks.sql.managedcatalog.ResolveWithCredential.com$databricks$sql$managedcatalog$ResolveWithCredential$$maybeDecorateCatalogTable(ResolveWithCredential.scala:75)

 I checked the Catalog and saw that my workspace is created. Every resource was just created using the Quick Start and the Get Started documents. Anyone has an idea of where my setup might be wrong? Thanks

[Emil_Kaminski]

Hi TimW, 

I will do a bit of guessing as I don't have such problem as you. If you are following this tutorial - https://www.databricks.com/notebooks/gcp-qs-notebook.html, the issue might be, very, very old version of runtime. Try to create and use a cluster with modern version, which will be 13+. 

If you already have a modern version of cluster, and still face this issue, try to change path to the file to start with dbfs:. Like  - path "dbfs:/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv".

Good luck

[Scott_in_Zurich · Accepted solution]

Adding 'dbfs:' got me past that error. Now onto debugging a PARSE SYNTAX error....

[patwilliams]

It sounds like you're on the right track with your Databricks setup, but this kind of error could be related to a few potential issues. Based on the details you’ve shared, here are some things you might want to check:

1. Cluster Configuration: Ensure that your cluster is properly configured and running. Sometimes, issues arise from the cluster not being active or not having enough resources to handle the query. Double-check that your cluster has sufficient memory and processing power allocated.

2. Database or Table Access: If you're querying from a pre-existing database or table, verify that it exists and that you have the appropriate permissions. You might need to adjust the database connection string or check if the table was created successfully in the first place.

3. SQL Query Syntax: Sometimes, the issue could be as simple as a syntax error in the SQL query you're running. Make sure the query matches the schema of the data you’re working with, and try running simpler queries to see if they work.

4. Dependencies or Libraries: If your notebook relies on external libraries or packages, ensure that they are installed and up to date. An outdated library or missing dependency could prevent the table creation from succeeding.

5. Databricks Runtime Version: Depending on the version of Databricks you're using, certain functionalities might behave differently. It’s worth checking if the runtime you’re using is compatible with the tutorial you’re following.

Lastly, if you're experimenting with Databricks and looking for something interesting, you should definitely check out Magis Apple TV. It’s a great way to experience seamless integration and might offer insights into handling similar data-driven applications in a different context.

Let me know if you need further help!

[patwilliams]

It seems as though you're doing great with your Databricks arrangement, however this sort of mistake could be connected with a couple of expected issues. In light of the subtleties you've shared, here are a few things you should check:

Group Setup: Guarantee that your bunch is appropriately arranged and running. At times, issues emerge from the bunch not being dynamic or not having an adequate number of assets to deal with the inquiry. Twofold check that your group has adequate memory and handling power distributed.

Data set or Table Access: In the event that you're questioning from a previous data set or table, check that it exists and that you have the proper consents. You could have to change the data set association string or check in the event that the table was made effectively in any case.

SQL Question Linguistic Structure: Once in a while, the issue could be basically as straightforward as a grammar blunder in the SQL inquiry you're running. Ensure the question coordinates the pattern of the information you're working with, and have a go at running less difficult inquiries to check whether they work.

Conditions or Libraries: Assuming your note pad depends on outside libraries or bundles, guarantee that they are introduced and exceptional. An obsolete library or missing reliance could keep the table creation from succeeding.

Databricks Runtime Form: Contingent upon the variant of Databricks you're utilizing, certain functionalities could act in an unexpected way. It merits checking if the runtime you're utilizing is viable with the instructional exercise you're following.

Finally, assuming that you're exploring different avenues regarding Databricks and searching for something intriguing, you ought to look at Magis Apple television. It's an extraordinary method for encountering consistent mix and could offer experiences into taking care of comparable information-driven applications in an alternate setting.

Inform me as to whether you want further assistance!