How do I define & run jobs that execute scripts that are copied inside a custom DataBricks container?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-12-2023 03:35 AM
Hi all, we are building custom Databricks containers (https://docs.databricks.com/clusters/custom-containers.html). During the container build process we install dependencies and also python source code scripts. We now want to run some of these scripts as jobs, ideally also providing command line arguments. However, when creating jobs, there doesn't seen a way to reference code that is inside the container? Any ideas?
Labels:
- Labels:
-
Container
-
Custom Docker Image