so i'm still a bit confused. Perhaps because it seems like databricks and spark mean different things when they say "Job".

In my case, `spark-submit` is not any part of the process at all, unless it's being abstracted by the notebook. I'm aware of how to use `spark-submit` to start a ... "job". But what I'm trying to do is invoke some code in a library which results in a streaming query, entirely from within a notebook.

Is there any documentation that can clarify the relationship between the notebook and the spark contexts? and the spark context and a streaming query?