I have the exact same question. Databricks seems nice for some initial exploration / playing around but once it goes into professionalization being able to only use notebooks sets severe limits...