- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-22-2024 08:20 AM
Morning Louis,
I do have the global configuration set to use the scheduler pool currently, but the local property is not mirroring the global property for something reason. The global configuration is setting the scheduler pool to low, but the local property is showing a string of integers similar to: '1944239804544138415', with the value being different for every notebook. I had initially thought of using an init script to set the value, but because init scripts load before Spark is running, I'm unable to set any local properties. I'm unaware of anything we are doing on our side to set dynamically set local values, but if you could please verify that the local settings normally mirror the global settings by default, I'll try checking back with our Databricks team to make sure this isn't something due to our end.