Hi,

I am working on an ML project and I need to access the data in tables hosted in my Databricks cluster through a notebook that I am running locally. This has been very easy while I run the notebooks in Databricks but I cannot figure out how to do this in my local notebook.

I want to accomplish the same functionality in the snippet below.

spark_dataframe = spark.read.table(table_name)

How can I get data frame like this when I am not in the Databricks environment?

Thanks in advance 🙂