Databricks Community

When looking at a data fabric vs data lake  strategy, the data lake approach shines when you need a high-performance landing zone for vast amounts of structured, semi-structured, and unstructured data. It gives data science and engineering teams tota...

P.S. My configuration has the following required updates:Compute: Job Compute, due to serverless limitations at my companyPipeline environment > Dependencies:/Workspace/Users/{username}/lakeflow_connect/microsoft_sharepoint_pipeline_test2/Workspace/U...

My Notebook, which previously worked fine, is now scaling all plots to `repr.plot.width` by `repr.plot.width`.

@van45678 -- @balajij8 's diagnosis matches what your Spark UI shows, so I will just add three things from the operational side, one for the postmortem and two for making sure the next occurrence costs you hours instead of five days.(1) Driver logs a...

Good question, and the answer is usually simpler than it looks: those secrets are almost always YOUR OWN credentials for whatever third-party service the app talks to — not something the developer sends you.Here's the model: Marketplace apps aren't a...

I ran some tests.**Answer — verified at source level and across six runs, Spark 3.5.2 → 4.2.0(evidence dossier attached):**There are two independent rejections here sharing one error class, which iswhat made this confusing:1. `CAST(struct AS VARIANT)...

Hi @yutmurakami — I can now answer your question with data instead of hypotheses. I ran your exact scenario on my own Free Edition workspace (also AWS, different workspace than yours), twice.Test 1: existing app on my Free Edition workspace — Running...

We hit exactly this, offline feature store on Redshift, ~200-column denormalized table fed by domain models. Adding a feature meant NULLs for history, and full recompute was too expensive, so we landed on: recompute the domain for the backfill range ...