Databricks Community

alonisser

Since we've moved from azure to aws, a specific job has extremely long vacuum runs, is there a specific flag/configuration for the s3 storage that is needed to support faster vacuum.How can I research what's going on?Note, it's not ALL jobs, but a sp...

alonisser · 04-10-2025

In the last few days, I've encountered in Azure (and before that also in AWS, but a bit different) this message about failing to start a cluster"run failed with error message Cluster '0410-173007-1pjmdgi1' was terminated. Reason: INVALID_ARGUMENT (CL...

alonisser · 06-01-2024

or even the logs don't contain the error line I see (downloaded all logs file from the UI and checked them)How can I see the stacktrace? it's essential to debug certain issues

alonisser · 08-27-2023

and it seems that older data (From the "replaced" table) isn't being removed, long after the retention period I'd be glad for clues on how to handle this

alonisser · 11-13-2022

is there an API for that? as I couldn't find a way to do this through the UIClassic serving for now (didn't get access to the new "serverless" offering)

alonisser · 04-10-2025

I see "Fleet instances do not support GPU instances" so in this case it's a no-op

alonisser · 04-10-2025

Thanks @BigRoux ,To be precise1. It's not generally constrained, just for spots, as removing spots but with the same instance type works2. it's a very small cluster, so no point in autoscaling, if it can't take 2 workers, then it won't take more .....

alonisser · 02-09-2025

Happened to me too, and pyenv update command is missing , any workarounds here?

alonisser · 06-03-2024

Thanks for the answer, but i fail to see what it has to do with my questions. it's not a "general python error", I run lots of jobs with python on Databricks clusters and know how to run python jobs and dependencies, I'm pointing to a specific issue ...

alonisser · 09-23-2023

Thanks, I know that, but the table history shows 30 days, but the actual data size and number of files and all other indicators , correlate to 170 days.

Databricks Community

User Stats

User Activity

Very long vacuum on s3

misbehavior of spots with fallback to on demand on job clusters

Since moving to dbr 14.3 with python jobs I don't see the stack trace for exceptions

Trying to vacuum a table that is constantly being "createdOrReplaced"

ML inference server (with REST api) how can I specify a GPU cluster.?

Re: misbehavior of spots with fallback to on demand on job clusters

Re: misbehavior of spots with fallback to on demand on job clusters

Re: Error when creating model env using 'virtualenv' with DBR 14.3

Re: Since moving to dbr 14.3 with python jobs I don't see the stack trace for exceptions

Re: Trying to vacuum a table that is constantly being "createdOrReplaced"