Scheduling a Complete Python Project in Databricks

naineel
New Contributor

Hi everyone,

I have a simple Python project with the following structure:

root/  
│── src/  
│   ├── package_name/  
│   │   ├── __init__.py  
│   │   ├── main.py  
│   │   ├── submodules1/  
│   │   │   ├── __init__.py  
│   │   │   ├── base1.py  
│   │   ├── submodules2/  
│   │   │   ├── __init__.py  
│   │   │   ├── base2.py  
│── pyproject.toml  

My primary concern is that while I can schedule a notebook or a single Python file in Databricks, I am unable to schedule an entire project that depends on main.py. The project functions as a CLI application and requires arguments to run.

I need guidance on how to properly schedule this complete Python project in Databricks. Any suggestions or best practices would be greatly appreciated!

Thanks in advance.