topic Re: Connect to spark session and uc tables in python file in Data Engineering

Connect to spark session and uc tables in python file

ajay_wavicle — Tue, 03 Feb 2026 12:23:47 GMT

How to Connect to spark session and uc tables in python file. I want to read uc tables in python modules in databricks workspace. How to access the current sparksession

Re: Connect to spark session and uc tables in python file

szymon_dybczak — Tue, 03 Feb 2026 13:09:49 GMT

Hi @ajay_wavicle ,

Azure Databricks automatically creates a SparkContext for each compute cluster, and creates an isolated SparkSession for each notebook or job executed against the cluster.

So following should work in python module in Databricks Workspace (use spark variable):

col_df = spark.sql("SELECT * FROM dev_factoring.information_schema.columns") display(col_df)

Another approach would be to create session explicityl using following:

from pyspark.sql import SparkSession spark_session = SparkSession.builder.getOrCreate() col_df = spark_session.sql("SELECT * FROM dev_factoring.information_schema.columns") display(col_df)

Re: Connect to spark session and uc tables in python file

ajay_wavicle — Tue, 03 Feb 2026 13:27:30 GMT

hi @szymon_dybczak , I am talking about connecting in plain python file and not databricks notebook dbc

Re: Connect to spark session and uc tables in python file

szymon_dybczak — Tue, 03 Feb 2026 13:59:08 GMT

Look carefully @ajay_wavicle , this is plain old python file. Do you see any notebook cells on screenshot?