Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-27-2022 07:10 AM
dbutils.notebook.run starts a new job, that's why it takes this time and test yo can start multiple concurrently using ThreadPool or other async libraries. Probably with the better server, it could be 10 seconds, not 20.
%RUN executes other notebooks like it would be code of notebook which we run (like include in some languages).
Maybe you could plan a different architecture way (but it depends on the use case) as this way is instead for no-time critical projects (analytics). Stream and delta live tables should be used instead if it is a project when that 20s delay is too much.
My blog: https://databrickster.medium.com/