Databricks Community

It looks like a workspace-level Serverless eligibility issue rather than a missing prerequisite. Since the CLI specifically says the workspace is no longer eligible, I’d check the Serverless Compute status in the Databricks account console. If all pr...

Hi @Khasim_1, I would keep the PII in Silver within your regulatory and internal retention limits and control access with Unity Catalog's row filters and column masks, rather than redacting permanently. The reason is the one you already gave. A perma...

Quick question, being that my account is an individual account, how do I go about this "Contact your Databricks account team to request access" 

Agree with @ThomazNeto  - only thing I would add is your point 3 requirement: you could also consider using Terraform instead DABs or job for managing the ABAC policies across multiple environments. It does not support the beta features i.e. metastor...

Hi Khasim,Most of this is answered directly in the Databricks docs, so let me point to what they say.1. Planning time vs. runtime. The docs don't describe a planning-time penalty; the cost is in what the optimizer is allowed to do. Both table-level r...

Hi Khasim,No secret value ever goes into databricks.yml. Bundle variables get resolved at deploy time and rendered into the job definition in the workspace, so anyone who can open the job can read them. Only names and references live in the bundle.Se...

Hi @Gilk, Before the how, can I ask what is driving the wish to turn it off? For most pipeline tables predictive optimization is doing useful work, running OPTIMIZE, VACUUM, and ANALYZE on serverless compute so file sizes, storage, and statistics sta...

For a standard connector, the gateway continuously extracts snapshots, change logs, and metadata, while a separately scheduled serverless pipeline applies staged data (connector components). Changing the serverless pipeline schedule therefore does no...

Hi @kabiromohd,   Welcome to Databricks, and a great choice for starting your Data Engineering journey!   The short answer is that Databricks Free Edition is forever free. There is no trial period or expiration date. You can use it indefinitely as lo...

This lines up well with what Databricks' own docs and the 2026 refresh of their partitioning guidance say — a few things worth adding as confirmation/extra data points rather than pushback:On the key-count cap: confirmed at 4 clustering keys max. Wor...

Thank you for the response. From the driver logs I gather that the REPL has crashed, but this isn't the case only with Hive. I have noticed this pattern with Oracle library installation as well.