โ03-14-2025 06:34 AM
I am facing the above issue when using the Python Databricks SDK. I retreive the job-definition by "
โ11-04-2025 04:42 AM
The error "Only absolute paths are currently supported. Paths must begin with '/'" in the context of the Databricks Python SDK means that when creating a job, the notebook path you provide must be an absolute workspace path (like /Users/username/notebook) rather than a relative path or a Git reference.โ
The Databricks Jobs API (used via client.jobs.create()) expects the notebook path in the job definition to be an absolute path in the Databricks workspace filesystem, not a Git URL or a relative path.โ
Even if you're working with Git-linked notebooks, when defining a job, the "notebook_path" field must always use an absolute path that exists in the workspace (e.g., /Repos/username/repo-name/path/to/notebook).โ
Make sure that when you migrate a job between workspaces, the notebook referenced in the job is present in the destination workspace, and specify its absolute path in the job definition.
If you're using Git integration, use the absolute workspace path of the notebook synced from Git, not the Git reference or a relative path.โ
Example:
job_settings = {
"notebook_task": {
"notebook_path": "/Repos/username/repo-name/path/to/notebook"
},
# other job settings...
}
client.jobs.create(**job_settings)
If your workflow requires Git-based deployment (to keep test and production in sync), you must automate a step that pulls or syncs the notebooks from Git into the workspace and then reference those absolute paths in the job configuration.
At this time, the Databricks Jobs API does not support specifying jobs directly referencing a Git branch/tag/commit for the taskโjobs work off workspace objects, not directly from Git links.โ
| Solution Step | Details |
|---|---|
| Use absolute notebook paths | Path must begin with '/', referencing the workspace location of the notebook |
| Sync notebooks from Git before deploy | Ensure the needed notebook exists in the destination workspace |
| No direct Git refs in job creation | The Jobs API does not accept Git refs/tagsโonly absolute workspace paths allowed |
Following these steps should resolve the "Only absolute paths are currently supported" error when using the Databricks Python SDK to migrate or create jobs with notebooks.โ
โ12-01-2025 07:30 AM
@AxelM I see this was a while ago, what was your solution? In my experience, either the `source` field under notebook task is incorrectly set to "WORKSPACE" instead of "GIT", or there's a legitimate issue, such as:
> The Jobs API does not accept Git refs/tagsโonly absolute workspace paths allowed
This is not accurate.
From the docs, we see the jobs create API takes a git source option: https://docs.databricks.com/api/workspace/jobs/create#git_source
When that is specified, then other tasks, such as notebook task, require a relative reference, as OP correctly pointed out.
The notebook task parameter specifies that you must use relative paths if using git-source: https://docs.databricks.com/api/workspace/jobs/create#tasks-notebook_task-notebook_path
โ12-01-2025 10:59 AM
from databricks.sdk import WorkspaceClient
from databricks.sdk.service.jobs import Task, NotebookTask, Source, GitSource
w = WorkspaceClient()
j = w.jobs.create(
name="Migrated job",
git_source=GitSource(
git_url="https://github.com/<org>/<repo>.git",
git_provider="gitHub", # or "azureDevOpsServices", "bitbucketCloud", ...
git_branch="main" # or set git_commit
),
tasks=[
Task(
task_key="my_task",
# Path is relative to the repo root:
notebook_task=NotebookTask(
notebook_path="path/to/notebook.py",
source=Source.GIT # or omit source to default to GIT when job.git_source is set
),
# ... compute settings ...
),
# more tasks...
]
)