I am trying to understand why running a job takes longer than running the notebook manually.And if I try to run jobs concurrently using workflow or threads then is there a way to reduce job init time ?
tried using-dbutils.notebook.run(notebook.path, notebook.timeout, notebook.parameters)but it takes 20 seconds to start new session. %run uses same session but cannot figure out how to use it to run notebooks concurrently.
This does help, but can you help me understand why job run time takes more time than actual run time of notebook? And is there a way to reduce the time?
Hello, one thing I am not able to understand is, while developing low latency application where even 5 seconds cannot be tolerated, how to create jobs? Because each job takes 20 seconds to init.
Hello, yes responses were helpful, so I tried workflow but stuck with sending data across notebooks using workflow. Also, trying delta live tables. What I would like to know is is there threshold about how many records delta live tables can hold at a...
