- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-19-2025 11:28 AM - edited 05-19-2025 11:28 AM
Hi @sensanjoy
Corrected Option 4: Hybrid Approach with SQL Variables
param_notebook:
dbutils.widgets.text("catalog", "catalog_de")
spark.conf.set("catalog.name", dbutils.widgets.get("catalog"))
dbutils.widgets.text("schema", "emp")
spark.conf.set("schema.name", dbutils.widgets.get("schema"))
# Add these lines to set SQL session variables
spark.sql(f"SET VAR catalog_name = '{dbutils.widgets.get('catalog')}'")
spark.sql(f"SET VAR schema_name = '{dbutils.widgets.get('schema')}'")
SQL notebooks (minimal change):
%run ../../config/param_notebook
-- Replace ${catalog.name} with ${VAR.catalog_name}
SELECT ${VAR.catalog_name} as catalog, ${VAR.schema_name} as schema;
USE IDENTIFIER(${VAR.catalog_name});
SELECT * FROM IDENTIFIER(CONCAT(${VAR.schema_name}, '.emp_details'));