What's the difference between %run vs dbutils.notebook.run

aladda
Databricks Employee
Databricks Employee

‎06-08-2021 01:14 PM

 
Labels:
0 Kudos

User16783855117
Databricks Employee
Databricks Employee

‎06-08-2021 07:54 AM

dbutils.notebook.run will actually run the notebook that you have referenced as a 'Job' on the current cluster, which is why a separate hyperlink is generated when using this API. Because this is technically a 'Job' invocation, some functionality is not supported, like AAD Credential Passthrough. Both of these approaches have their use cases so hopefully that is helpful!
0 Kudos

aladda
Databricks Employee
Databricks Employee

‎06-19-2021 08:29 PM

  • %run is copying code from another notebook and executing it within the one its called from. All variables defined in the notebook being called are therefore visible to the caller notebook
  • dbutils.notebook.run() is more around executing different notebooks in a workflow, an orchestration of sorts. Each notebook runs in an isolated spark session and passing parameters and return values is through a strictly defined interface.

View solution in original post

0 Kudos