Databricks Community

If you need 15 minute CDC interval - Migrate those specific jobs to the integrated CDC pipeline in continuous speed-optimized mode. If the 15 minute jobs involves fewer than 50 tables, it works directly. If not, split them into multiple pipelines. Mo...

Really nice project this looks genuinely useful for anyone working with Databricks costs and system tables I’ll take some time to go through the repo and see where I can contribute or help improve things

Yeah, this is the classic Photon fallback trap. Flat runtime with higher cost almost always means one unsupported operator is dragging most of your plan back onto Spark, so you're paying the Photon premium on a plan that's barely touching Photon. Fir...

Hello @JoaoPigozzo , I took a look at both internal and external documentation and here is what I found. There's no separate enablement toggle for overrides. What the docs describe is a prerequisite, and it's easy to miss. The Add override control li...

Hello @Fortinbras , I took a look at both internal and external documentation and here is what I found. Short answer: your instinct is right. There's no AppKit plugin that hands you optimistic concurrency or audit history as a feature. Those belong i...

Hello @Hariharan_0510 , I took a look at both internal and external documentation and here is what I found. Building on @balajij8's reply, the serverless egress policy is the right place to start, but two things before you go hunting further. First, ...

Do you feel you have a good understanding of what is happening and how to get around your specific issue?

I'd love to be able to update this, because I pressed reply too early

Hi @HOguns,I am not an engineer, I cover the delivery side of Genie. However, I passed it to one of our internal teams. Let's see if they pick it up. I am not sure how engineers pick it up from here  In the meantime, can you also please send it to da...

Hi,Great question. This is where most Medallion implementations either scale or get messy. Here's the pattern that has worked well, based on Kimball conformed dimensions mapped onto Unity Catalog.Quick note on naming: in Databricks, "Zerobus" is a pr...

Hi,Short answer: No. There's no built-in function or parameter that returns the viewer's identity while a dashboard runs on the publisher's credentials.Why: When you publish with Share data permissions (the default, formerly called "Embed credentials...

I'll answer the size question directly first: no, we don't have a GB threshold, and I've come to think a number there doesn't survive contact with reality.The reason is that the same 5 GB behaves completely differently depending on shape. Five gigaby...