Hi,
I have a simple python notebook with below code -
---
query = "select table_catalog, table_schema, table_name from system.information_schema.tables where table_type!='VIEW' and table_catalog='TEST' and table_schema='TEST'"
test = spark.sql(query)
display(test)
---
Running the above in my workspace returns a dataframe with appropriate results. But running the same as a task in a job which runs as a service principle is not returning any results and gives the output as "Query returned no results". I ensured service principal has select grant on system.information_schema.tables.
What am I missing here? Appreciate the help.
THank you
