topic Re: Unable to run spark sql commands from ipywidget button click event in Data Governance

Unable to run spark sql commands from ipywidget button click event

129876 — Mon, 25 Jul 2022 17:48:16 GMT

I'm unable to run any command that queries data from the unity catalog within a function that executes in the event of an ipywidget button click. Code block below. I cannot do queries such as spark.sql(f"SHOW SCHEMAS;") or spark.sql(f"select * from database.table")

[Start code block]

import ipywidgets as widgets

button_01 = widgets.Button(

description='Submit',

disabled=False,

tooltip='Submit'

)

out = widgets.Output()

def get_data(x):

with out:

out.clear_output()

print(spark.sql(f"SHOW SCHEMAS;").collect())

button_01.on_click(get_data)

display(button_01, out)

[End code block]

The error it gives is "Py4JJavaError: An error occurred while calling o332.sql.

: java.lang.IllegalStateException: No api token found in Credential Context"

Also to note, I am able to run these spark sql commands alone in a cell, just not in a function with ipwidgets

Re: Unable to run spark sql commands from ipywidget button click event

Atanu — Mon, 25 Jul 2022 18:53:47 GMT

can you try to println out?

val databricksApiTokenKey = CredentialContext.INHERITED_PROPERTY_DATABRICKS_API_TOKEN

val databricksApiCredentialOpt = CredentialContext.getCredential(databricksApiTokenKey)

val rawUrlProp = spark.sparkContext.getLocalProperty(SparkContext.API_URL_PROPERTY)

Re: Unable to run spark sql commands from ipywidget button click event

129876 — Mon, 25 Jul 2022 19:00:16 GMT

I'm getting an error running those lines:

error: not found: value CredentialContext