Walter_C
Databricks Employee
Databricks Employee

Can you confirm if this are the steps being followed:

  • Create the SQL Task: Ensure your SQL task is correctly set up and produces the desired output. For example:

    SELECT customer_name, market FROM example_customers;
  • Reference the SQL Task Output in ForEach Task:

    • Use the correct syntax to reference the rows output. The expression should be in the format {{ tasks.sqlTaskName.output.rows }}.
    • Ensure that the SQL task name (sqlTaskName) matches the task key you have defined in your workflow.
  • Configure the ForEach Task:

    • Set the inputs field of the ForEach task to {{ tasks.sqlTaskName.output.rows }}.
    • Define the nested task within the ForEach task to process each row. For example:
      {
        "task_key": "process_customers_iteration",
        "parameters": {
          "customer_name": "{{ input.customer_name }}",
          "market": "{{ input.market }}"
        },
        "sql_task": {
          "file": {
            "path": "/path/to/your/sql/file.sql"
          }
        }
      }