topic Re: What's the difference between %run vs dbutils.notebook.run in Data Engineering

What's the difference between %run vs dbutils.notebook.run

aladda — Tue, 08 Jun 2021 20:14:12 GMT

Re: What's the difference between %run vs dbutils.notebook.run

User16783855117 — Tue, 08 Jun 2021 14:54:00 GMT

dbutils.notebook.run will actually run the notebook that you have referenced as a 'Job' on the current cluster, which is why a separate hyperlink is generated when using this API. Because this is technically a 'Job' invocation, some functionality is not supported, like AAD Credential Passthrough. Both of these approaches have their use cases so hopefully that is helpful!

Re: What's the difference between %run vs dbutils.notebook.run

aladda — Sun, 20 Jun 2021 03:29:47 GMT

%run is copying code from another notebook and executing it within the one its called from. All variables defined in the notebook being called are therefore visible to the caller notebook
dbutils.notebook.run() is more around executing different notebooks in a workflow, an orchestration of sorts. Each notebook runs in an isolated spark session and passing parameters and return values is through a strictly defined interface.