cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Erik
Valued Contributor III
since ‎09-20-2021
Saturday

User Stats

  • 64 Posts
  • 0 Solutions
  • 16 Kudos given
  • 49 Kudos received

User Activity

Use unity catalog access connector for autoloader file notification events

2 weeks ago
We have a databricks access connector, and we have granted it access to file events.  But how do we now use that access connector in cloudfiles/autoloader with file-notifications? If I provide the id in the "cloudFiles.clientId" option, I am asked to...

Managing streaming checkpoints with unity catalog

3 weeks ago
This is partly a question, partly a feature request: How do you guys handle streaming checkpoints in combination with unity catalog managed tables?It seems like the only way is to create a volume, and manually specify paths in it as streaming checkpo...

Feature request: Run cluster in a different region

12-22-2023
We are one of those which unfortunately decided (earlier) to set up our infastructure in one of the sub-par Azure regions, and we happen to dont have any GPU clusters available.We would like to test out some GPU enhanced ML in databricks, but at the ...

Run driver on spot instance

11-18-2023
The traditional advice seems to be to run the driver on "on demand", and optionally the workers on spot. And this is indeed what happends if one chooses to run with spot instances in Databricks. But I am interested in what happens if we run with a dr...

Why not enable "decommissioning" in spark?

11-18-2023
You can enable "decommissioning" in spark, which causes it to remove work from a worker when it gets a notification from the cloud that the instance goes away (e.g. SPOT instances). This is dissabled by default, but it seems like such a no-brainer to...
View more