how to access the catalog and schema from my program

Subhas1729 — Wed, 29 Apr 2026 23:38:14 GMT

I am using the SDP editor. I have set the catalog and schema in setting. how to access those variables values in my program. I am doing as follows: catalog = spark.conf.get("catalog") and it is similar for schema. When I try to use those variables, I find error as "dc is not set".

In general how to access the variables in settings.

Regards

Subhas

Re: how to access the catalog and schema from my program

szymon_dybczak — Thu, 30 Apr 2026 06:40:25 GMT

Hi @Subhas1729 ,

Default location for data assets section of the pipeline configuration UI sets the default catalog and schema for a pipeline. This default catalog and schema are used for all dataset definitions and table reads, unless overridden within the query. So if you don't specify it in your code it will still work.

If you just want to know what catalog/schema is currently active in your session, query it directly:

current_catalog = spark.sql("SELECT current_catalog()").collect()[0][0] current_schema = spark.sql("SELECT current_schema()").collect()[0][0]

The approach you used would work if you set pipeline configrations. Then you can refer for key/value pairs defined there using following code:

spark.conf.get("your_key")

So in your case you can set catalog and schema config and read it with above code.

If the answer was helpful, please consider marking it as solution

topic how to access the catalog and schema from my program in Data Engineering

how to access the catalog and schema from my program

Re: how to access the catalog and schema from my program