Re: Retrieve workspace instance name on E2 archite...

Deepak_Bhutada · ‎10-29-2021

I have a databricks job on E2 architecture in which I want to retrieve the workspace instance name within a notebook running in a Job cluster context so that I can use it further in my use case.

While the call

dbutils.notebook.entry_point.getDbutils().notebook().getContext().tags().apply("browserHostName")

works in an All-Purpose cluster in an interactive way, this key does not exist when running in a Job cluster context.

Deepak_Bhutada · ‎10-29-2021

it is not possible to get the workspace URL/Shard name from MT/E2 because we don't inject the user-visible URL, but the driver reachable one.

But as a workaround to get the shardname in our job, we can write a file with our shard name in a particular dbfs location and read it inside the job as shown below:

dbutils.fs.put("/FileStore/Deepak-test/ShardName.txt", "https://cse2.cloud.databricks.com/")

When you will run a job, you can read that file and get the shard name from there.

dbutils.fs.head("/FileStore/Deepak-test/ShardName.txt")

Or you can create a dictionary as well which you can use to get shardname inside a job

WORKSPACES = {
"111": "workspace1",
"222": "workspace1"
}
 
org_id = get_notebook_context().tags().apply("orgId")
 
print(f"https://{WORKSPACES[org_id]}.cloud.databricks.com")

Thomas_B_ · ‎04-28-2022

Actually is anybody aware of a fix for Azure Databricks, too? (My use case is to send out status messages from a batch run which includes a link to the job cluster page or run page to allow operators to click for details.)

Thomas_B_ · ‎05-25-2022

Found workaround for Azure Databricks question above: dbutils.notebook.getContext().apiUrl will return the regional URI, but this forwards to the workspace-specific one if the workspace id is specified with o=.

Retrieve workspace instance name on E2 architecture (multi-tenant) in notebook running on job cluster