agallard
Contributor

Hi @PabloCSD,

If the workflows are configured within a single Databricks job, you can use depends_on to ensure the second workflow waits for the completion of the generate-token task. This works well for cases where both workflows are in the same job context.

Another option is if the workflows are in separate Databricks Jobs and cannot be configured in the same context, you could schedule the second workflow to start a few minutes after the first, estimating the time needed to generate the token.

  • First Workflow: Scheduled to run every hour or at specific intervals.
  • Second Workflow: Scheduled to start 5–10 minutes after the first, ensuring the token is ready.

These strategies allow the second workflow to explicitly wait for the token generation to complete, reducing risks of errors and improving efficiency in loading libraries and other components that depend on the token.

Try and comment!Regards.

Alfonso Gallardo
-------------------
 I love working with tools like Databricks, Python, Azure, Microsoft Fabric, Azure Data Factory, and other Microsoft solutions, focusing on developing scalable and efficient solutions with Apache Spark