Hi @maikel ,

1. In order to pass dynamic parameters between Python script tasks:
   1. In the upstream task (named "task_1"), set the dynamic parameter via dbutils:
      

      from databricks.sdk.runtime import *
      dbutils.jobs.taskValues.set(key = "fave_food", value = "beans")​

   2. In the downstream task, set the input parameter from the upstream task, as explained here:
      
   3. In the downstream Python task itself, the dynamic parameter is passed as command-line argument:
      

      import argparse
      
      p = argparse.ArgumentParser()
      p.add_argument("-input_dynamic_param")
      args = p.parse_args()
      print(args.input_dynamic_param)​

      
      
      
2. A typical integration test of the workflow would be:
   1. Deploy the workflow via Databricks Asset Bundles (to a separate integration/staging workspace, or to a separate target in the DAB definition).
   2. Run the workflow on a subset of data.
   3. Output the result into a separate catalog / schema.
   4. Optionally, add an additional step to the workflow to compare results with the ground truth. 
   5. Ensure that the workflow deployment, input and output data are isolated from other workloads.
3. There is no general recommendation on whether to choose Python scripts or Notebooks - it all depends on your team's habits and overall practices:
   1. Notebooks give richer experience (Markdown, widgets, magic commands).
   2. Note also that you can save Notebooks as plain python scripts and run them locally (if the code doesn't depend on some rich experience).
   3. Also, you can run Databricks notebooks directly via your local IDE with Databricks connect.
   4. Please note that for Lakeflow Spark Declarative Pipelines it's different. Python files are strongly recommended over Notebooks in that case.

Hope it helps.

Best regards,