Databricks Community

Hi @ram_11 ! The extractor_sql-server-conn_cdc_sink is the shared CDC sink that the SQL Server CDC extractor job inside the IG pipeline writes to.Example screenshot below:     What actually happens: The IG extractor (the job/process) reads change ro...

Hello @priya9896, I took a look at both internal and external documentation and here is what I found. Straight answer: don't plan on the Glue Data Catalog as a federation layer over Azure Databricks. Glue's Delta integration and the Glue Catalog itse...

Great discussion. I think there is another architectural consideration worth exploring before changing the gateway configuration.Does the 15-minute freshness requirement apply to all 80 tables, or only to a subset of business-critical tables?If only ...

Hi Ben, The stdout logging-handler trick works on a classic/standard cluster, but on serverless it will not surface in the notebook cell, and that is expected rather than a problem with the helper. Here is what is going on. applyInPandas runs your Py...

HTML outputs in dark mode was recently improved. Please see https://docs.databricks.com/aws/en/notebooks/notebook-ui#html-outputs-in-dark-mode and try it out!

HTML outputs in dark mode was recently improved. Please see https://docs.databricks.com/aws/en/notebooks/notebook-ui#html-outputs-in-dark-mode and try it out!

Great post !We are currently evaluating a similar pattern using DQX as a quality gate in a Databricks data engineering workflow.Our use case involves several SharePoint Lists ingested through the native Databricks SharePoint connector into a RAW laye...

Hello @AnhPT, I took a look at both internal and external documentation and here is what I found. @Satyasai pointed you at the right page for dashboards; the reason it felt thin is that the time-window question doesn't apply to most of what's on it. ...

Hello @Capri, I took a look at both internal and external documentation and here is what I found. You're not alone, and nothing on your side caused this. @SamuelHarris reported the identical failure on Sep 30 on this thread (https://community.databri...

@Brahmareddy I haven't used this exact pattern at multi-million-row scale, but I agree that the tradeoffs start to change as the dataset and query volume grow.I would probably keep Delta as the source of truth and separate the serving layer based on ...

The ai_query route is the way to go, nothing to add there. Just a few thingsthat will bite you once you point it at a whole catalog.The big one is in the docs for the AI comments feature itself: saving a commentfires an ALTER, and that can disrupt ru...

Hi Carl,Short answer: yes, but it is a discount rather than a free voucher, and it doesnot come from the Partner Academy portal itself. The documented partner benefitis free access to Databricks Academy courses plus a 50% discount oncertification exa...

Greetings @alexgfowler_ld, I did some digging and here is what I found. First, this is one of the most carefully built-out reports I've seen on this forum. Your A/B test already rules out the usual suspects: same role, credential, data, and region wo...