https://community.databricks.com/t5/data-engineering/passing-variables-from-python-to-sql-in-a-notebook-using/m-p/81516#M36337 <P>I've got a notebook that I've written that's going to execute some python code to parse the workspace id to figure out which of my environments that I'm in and set a value for it. I then want to take that value, and pass it through to a code block of SQL that will execute, using the set value as a part of the table structure names that I'm executing DML on.&nbsp;</P><P>I was able to do this using standard shared compute cluster, using spark.conf.set() to create a parameter, and then calling that parameter within the SQL code using ${myparam} syntax (ie. SELECT * FROM {$myparam}_schema.MyTable). But in testing with Serverless, access to the spark.conf.set() function isn't available.</P><P>Does anyone have any suggestions on how I might be able to accomplish the same thing in Serverless Compute?</P> Thu, 01 Aug 2024 14:54:36 GMT emorgoch 2024-08-01T14:54:36Z https://community.databricks.com/t5/data-engineering/passing-variables-from-python-to-sql-in-a-notebook-using/m-p/81516#M36337 <P>I've got a notebook that I've written that's going to execute some python code to parse the workspace id to figure out which of my environments that I'm in and set a value for it. I then want to take that value, and pass it through to a code block of SQL that will execute, using the set value as a part of the table structure names that I'm executing DML on.&nbsp;</P><P>I was able to do this using standard shared compute cluster, using spark.conf.set() to create a parameter, and then calling that parameter within the SQL code using ${myparam} syntax (ie. SELECT * FROM {$myparam}_schema.MyTable). But in testing with Serverless, access to the spark.conf.set() function isn't available.</P><P>Does anyone have any suggestions on how I might be able to accomplish the same thing in Serverless Compute?</P> Thu, 01 Aug 2024 14:54:36 GMT https://community.databricks.com/t5/data-engineering/passing-variables-from-python-to-sql-in-a-notebook-using/m-p/81516#M36337 emorgoch 2024-08-01T14:54:36Z https://community.databricks.com/t5/data-engineering/passing-variables-from-python-to-sql-in-a-notebook-using/m-p/83968#M37085 <P>Thanks Kaniz, this is a great suggestion. I'll look into it and how it can work for my projects.</P> Thu, 22 Aug 2024 19:25:40 GMT https://community.databricks.com/t5/data-engineering/passing-variables-from-python-to-sql-in-a-notebook-using/m-p/83968#M37085 emorgoch 2024-08-22T19:25:40Z