cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How to pass task name as parameter in scheduled job that will be used as a schema name in query

jeroaranda
New Contributor II

I want to run a parametrized sql query in a task. Query:

 

select * from {{client}}.catalog.table

 

with client value being {{task.name}}.

if client is a string parameter, it is replaced with quotes which throws an error.
if table is a dropdown list parameter, it throws: The following parameter values are invalid: [{'key': 'client', 'value': 'randomclient'}]. 

How can I solve?

1 REPLY 1

Kaniz_Fatma
Community Manager
Community Manager

Hi @jeroaranda

  1. When you use a string parameter in your SQL query, itโ€™s important to ensure that the parameter value is properly quoted. If youโ€™re directly substituting the parameter value into the query, you might encounter issues with quotes. To avoid this, you can use placeholders and bind the parameter values securely.
  2. Dropdown list parameters can be tricky because they have predefined values. If youโ€™re getting an error like โ€œThe following parameter values are invalid,โ€ it means that the value youโ€™re trying to set for the client parameter doesnโ€™t match any of the valid options.

     

    To resolve this issue:

    • Double-check the available options for the client parameter. Make sure youโ€™re using a valid value.
    • If the value is dynamically generated (e.g., based on another parameter), ensure that the logic for populating the dropdown list is correct.
    • If youโ€™re using both client and table parameters, make sure youโ€™re handling them correctly in your Databricks task. Youโ€™ll need to:

      • Retrieve the selected client value (e.g., from a dropdown or user input).
      • Construct the SQL query dynamically by substituting the client value into the query template.

If you encounter any specific errors or need further assistance, feel free to ask! ๐Ÿ˜Š

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group