Init Script not 'Absolute'
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-22-2023 11:17 PM
Hello,
I am trying to use an init script from my repo workspace.
The cluster is a 'personal compute' cluster with single user access mode with the following runtime version: '13.1 (includes Apache Spark 3.4.0, Scala 2.12)'
I have 'Files in Repos' set to 'DBR 8.4+' in my workspace settings.
I used the 'workspace' option (not dbfs, or s3) in the destination settings for the init script dropdown.
Unfortunately I am getting the following error on Cluster Startup.
```json
{
"reason": {
"code": "INIT_SCRIPT_FAILURE",
"type": "CLIENT_ERROR",
"parameters": {
"instance_id": "i-0f585b1a60a28a8b1",
"databricks_error_message": "BAD_REQUEST: requirement failed: /Workspace/Repos/services+databricks@umccr.org/data-portal-analytics/notebooks/RPackagesToInstall.sh is not absolute."
}
},
"add_node_failure_details": {
"failure_count": 1,
"resource_type": "container",
"will_retry": false
}
}
```
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-23-2023 01:59 AM
Hi @Debayan
> Looks like it is not able to get the package from the repo.
Not sure what you mean by this?
I managed to resolve the issue (also couldn't find the log files after setting the logging to dbfs://cluster-logs/ which was strange).
In this case, I think had a sneaky space character before '/Workspace' which caused my issue.
After that resolved, I got an error saying that this path didn't exist.
My guess was that by selecting workspace in the destination settings dropdown menu, I was already in the /workspace directory.
Hence by using `/Repos/services+databricks@umccr.org/data-portal-analytics/notebooks/RPackagesToInstall.sh` instead I was able to run the init script successfully.
Is there any examples of init scripts when using repos available? I couldn't find any. This could be quite useful for others!
Alexis
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-23-2023 12:05 AM
Hi, could you please enable init script logging and provide the elaborated logs here? Looks like it is not able to get the package from the repo.
https://docs.databricks.com/clusters/init-scripts.html#init-script-logging
Also, please tag @Debayan Mukherjee with your next update so that I will get notified.